sgl-project/sglang

Docs | Cookbook | Website | Blog | Slack

SGLang is an open-source inference framework for large language, vision-language, and diffusion models, optimized for agentic workloads, large-scale serving, and RL rollouts. SGLang Diffusion is its built-in image and video generation engine, included in this repository and the sglang Python package.

👋 Get started below, or meet the community at SGLang Events, including meetups, developer meetings, workshops, and office hours.

Pull the Docker image, which includes SGLang and its dependencies:

docker pull lmsysorg/sglang:latest

Alternatively, install SGLang in an activated Python environment with uv:

uv pip install --prerelease=allow sglang

Next, launch your model:

• Quickstart: Run your first model and send a request.
• Cookbook: Choose your model and hardware to get a ready-to-run launch command.

SGLang supports a wide range of GPUs, TPUs, NPUs, CPUs, and Apple Silicon platforms.

Integrations in progress: AWS Trainium, Alibaba T-Head PPU, Cambricon MLU, Qualcomm QAIC, MetaX, Hygon HCU/DCU, Iluvatar CoreX, and more.

See the Cookbook and platform guides for model compatibility and setup.

Contributions are welcome, from bug fixes and documentation to model support and performance improvements.

Start from the lmsysorg/sglang:dev Docker image, which provides development tools and most dependencies. Clone or mount your SGLang checkout inside the container, then install it in editable mode from the repository root so tests use your local Python changes:

pip install -e "python"

In an activated virtual environment, you can use uv pip install --prerelease=allow -e "python" instead. See the development guide for container setup and testing.

1. Fork the repository and create a branch for your changes. For larger changes, discuss your proposal in a GitHub issue or on Slack.
2. Make your changes, run the relevant tests, and add regression coverage for fixes or new behavior. Run pre-commit run --all-files before submitting.
3. Open a pull request describing the change and how you tested it. Include benchmarks or accuracy evaluations when relevant.

See the contributor guide for formatting, testing, and pull request instructions. Documentation contributors can start with the docs guide.

SGLang is hosted by LMSYS, a non-profit open-source organization.

• Community discussions: Join Slack for technical questions and development discussions.
• Events: Find meetups, workshops, and office hours on SGLang Events.
• Updates: Follow X and LinkedIn for project updates, and the LMSYS Blog for release announcements and technical articles.
• Project resources: Explore the documentation, Cookbook, roadmap, release notes, issue tracker, and contributor guide.
• Contact Us: For enterprise adoption and deployment, technical consulting, sponsorship, or partnership inquiries, please contact sglang@lmsys.org.
• Contributor sponsorship: Long-term active SGLang contributors are eligible for coding agent sponsorship, including Cursor, Claude Code, or OpenAI Codex. To apply, email sglang@lmsys.org with links to your key commits or pull requests.

SGLang serves production workloads across AI labs, cloud platforms, enterprises, and universities.

We learned the design and reused code from the following projects: Guidance, vLLM, LightLLM, FlashInfer, Outlines, and LMQL.