Remote Procedure Calls - 50x lower latency and 70x higher bandwidth than FastAPI, implementing JSON-RPC & 🔜 REST over io_uring and SIMDJSON ☎️
APACHE-2.0 License
Published by ashvardanian over 1 year ago
interface
is a macro in MSVC. (bed3f79)winsock
includes to support windows. (d0314b1)iovecs
are required only for uring
. (9d850be);
is a problem on MSVC (578e741)MSVC
specific flags. (18a8276)pyproject
and setup.py
for Windows (2f23d56)Published by ashvardanian over 1 year ago
Published by ashvardanian over 1 year ago
ujrpc_ssl_context_t
corruption on move. (50ec686)malloc
. (3ee198d)std::error_code
produced an UB. (aded2a9)std::error_code
produced an UB. (0c72246)closed
check. (8e92515)iovecs
from posix engine. (4ccb182)batch_response_t
. (824fb5c)_make_socket
(421df2e)send_reply
(3a56ba1)sum
to login
in lunch.json (22564cf)Published by ashvardanian over 1 year ago
We all ❤️ cURL for its CLI. It's the fastest way to test a REST API server.
How about JSON-RPC? Now, there is a way 🥳
ujrpc vectorize description='Product description' -i image=./local/path.png
To address a remote server:
ujrpc vectorize description='Product description' -i image=./local/path.png --uri 0.0.0.0 -p 8545
No additional installations are needed. Simply upgrade UJRPC to v0.3 with the following code:
pip install --upgrade ujrpc==0.3
An interactive version, colored outputs, and profiling are in the works!
But first, we'll add a pinch of TLS! Stay tuned 😉
Published by ashvardanian over 1 year ago
Published by ashvardanian over 1 year ago
TCP
for Client
instead requests
(fcede79)Refactor
(abf84ed)VERSION
file. (33c1973)Client
type mismatching. (554cc11)Client
use Content-Length
for full data. (152ab58)Content-Length
(e87ad96)pack
(de12e57)create_user
with FastAPI (75b42b9)id
. (1b81e22)package.json
to workflows. (15020bd)_
to internal functions (86bc25e)Published by ashvardanian over 1 year ago
The three of us - @ishkhan42, @VoVoR, and @ashvardanian have added support for NumPy arrays and Pillow images as RPC arguments, adding a small Client
class along the way! The following example shows how to serve UForm multi-modal vision-language transformers in just a few lines of Python, connecting dozens of SIMD-accelerated libraries under the hood for ultimate performance!
from ujrpc.rich_posix import Server
import ufrom
server = Server()
model = uform.get_model('unum-cloud/uform-vl-multilingual')
@server
def vectorize(description: str, photo: PIL.Image.Image) -> numpy.ndarray:
image = model.preprocess_image(photo)
tokens = model.preprocess_text(description)
joint_embedding = model.encode_multimodal(image=image, text=tokens)
return joint_embedding.cpu().detach().numpy()
A new UKV release and the public rollout of UNSW are also right around the corner!