12 points unikcc 3 hours ago 1 comments
Decision models need their own inference engine.
So we developed Flide, a high-performance and exact inference engine for Jev-like decision models.
Flide achieves up to a 9.5x speedup over vLLM and SGLang.
It now supports 30+ popular local Jev-like decision models.
Start to make your decision immediately:
pip install flide
unikcc 2 hours ago | parent
Hi HN! I'm one of the Flide team.
We built Flide from scratch to make Jev-like decision model inference up to 9.5x faster than vLLM and SGLang, while keeping exactness.
Here is the source code: https://github.com/OmniJev/Flide
Look forward to hearing your feedback and ideas!