PHANCY (06682) and Ruisi Zhilian jointly launch the open-source one-stop production-grade AI deployment platform "ModelSphere".

On October 16, PHANCY (06682) and Beijing Ruisi Zhilian Technology Co., Ltd. ("Ruisi Zhilian") jointly announced the official launch of the one-stop production-grade AI deployment platform "ModelSphere." The platform is free and open-source for enterprise users, integrating compute management, model deployment, and runtime optimization into a unified platform, helping enterprises lower the barriers to deployment and operations and more efficiently transform compute into secure, stable AI services usable for real business. As large models enter the stage of large-scale commercialization, enterprises generally face issues such as low resource utilization, fragmented chip ecosystems, high deployment barriers, and difficulty in tuning production environments when it comes to compute coordination, model deployment, and performance optimization. ModelSphere is dedicated to integrating these tasks into a unified deployment and management process, reducing enterprises' repetitive investment in underlying system adaptation, resource scheduling, and daily operations and maintenance, so that enterprises can embed various open-source large models into core business processes without needing deep expertise in complex work such as driver compilation and hardware adaptation, allowing technical teams to focus more on application development and business needs. ModelSphere has four core advantages: 1. Simplified deployment, ensuring stable operation of model services: ModelSphere provides a graphical operation panel and preset deployment capabilities, reducing the writing of environment configuration code, helping developers quickly build inference services for large models, and simplifying [the process across] different chip clusters. The platform has built-in enterprise-grade capabilities such as automatic elastic scaling, PD-disaggregated deployment, three-level KV caching, high-availability load balancing, and automatic continuation after service interruption, supporting stable operation of model services in production environments. 2. Continuous automated tuning, improving model runtime efficiency: The platform has a built-in AutoTune intelligent model scheduling strategy, as well as adaptive load balancing scheduling combined with hardware acceleration capabilities, optimizing the model's ability to process requests and first-response speed. The system's peak throughput efficiency is greatly improved by about 285% compared with the baseline, and performance jumps to 4 times the original, truly achieving "full-power" release of compute. 3. Unified management of multiple chips, reducing repetitive adaptation: The platform integrates the EngineX inference engine, supports and conducts deep tuning for multiple compute chips, and can uniformly and mixedly schedule domestic compute resources such as Huawei Ascend, ILUVATAR COREX, Moore Threads, and Hygon DCU, so users do not need to rewrite underlying logic code for different chips. 4. Continuous optimization and adaptation for mainstream models, accelerating deployment and application: The ModelSphere team continuously follows iterations in the open-source community and has adapted and optimized mainstream models such as DeepSeek-V4/V4-Flash, Qwen, Kimi, MiniMax, and GLM, supporting rapid deployment after model release and performance tuning to production environments, helping enterprises bring new models into actual business. ModelSphere's positioning in the AI technology stack At the technical level, ModelSphere works in conjunction with inference engines such as vLLM and SGLang: inference engines are responsible for improving the computational efficiency of a single server, while ModelSphere is responsible for cluster architecture-level management and optimization, including load- and cache-aware request routing, a unified KV cache pool, QoS-based automatic scaling, automatic continuation after service interruption, automated performance tuning based on online data, runtime monitoring, and service management, ensuring high performance and high availability of the system in production environments. ModelSphere is fully open-source, opening the engineering capabilities of large model deployment and optimization to more enterprises and developers, while gathering industry strength to continuously improve support for different chips and models, reduce hardware adaptation barriers, lower the threshold for large model applications, help enterprises make good use of existing compute, and efficiently unleash the commercial potential of heterogeneous compute. ModelSphere adopts the permissive Apache 2.0 license and is now available on GitHub: https://github.com/modelsphere/modelsphere