Back to explore

Jev-compatible public API: fast parallel System One generation with SGLang

Inspired by TypeSafe AI, the author released a Jev-compatible public API running a comparable open model Qwen3.6-35B-A3B, using SGLang radix cache for prefill reuse and fast parallel System One generation, completing 64 tasks in under 1 second.

Inspired by @typesafeai , here is a Jev-compatible public API to play with It runs a comparable open model (Qwen3.6-35B-A3B), and just uses SGLang radix cache to preserve the prefill reuse / really fast parallel systemone generation - 64 tasks in <1s. https://github.com/ekzhang/openjev-sglang…

· 977 likesOpen on X