1.0 KiB
| name | description |
|---|---|
| dgx-spark-sglang | Install and use SGLang on DGX Spark — on NVIDIA DGX Spark. Use when setting up sglang on Spark hardware. |
SGLang for Inference
Install and use SGLang on DGX Spark
SGLang is a fast serving framework for large language models and vision language models that makes your interaction with models faster and more controllable by co-designing the backend runtime and frontend language. This setup uses the optimized NVIDIA SGLang NGC Container on a single NVIDIA Spark device with Blackwell architecture, providing GPU-accelerated inference with all dependencies pre-installed.
Outcome: You'll deploy SGLang in both server and offline inference modes on your NVIDIA Spark device, enabling high-performance LLM serving with support for text generation, chat completion, and vision-language tasks using models like DeepSeek-V2-Lite.
Full playbook: /home/runner/work/dgx-spark-playbooks/dgx-spark-playbooks/nvidia/sglang/README.md