mirror of
https://github.com/NVIDIA/dgx-spark-playbooks.git
synced 2026-04-24 02:43:55 +00:00
23 lines
1.0 KiB
Markdown
23 lines
1.0 KiB
Markdown
---
|
|
name: dgx-spark-sglang
|
|
description: Install and use SGLang on DGX Spark — on NVIDIA DGX Spark. Use when setting up sglang on Spark hardware.
|
|
---
|
|
|
|
<!-- GENERATED:BEGIN from nvidia/sglang/README.md -->
|
|
# SGLang for Inference
|
|
|
|
> Install and use SGLang on DGX Spark
|
|
|
|
SGLang is a fast serving framework for large language models and vision language models that makes
|
|
your interaction with models faster and more controllable by co-designing the backend runtime and
|
|
frontend language. This setup uses the optimized NVIDIA SGLang NGC Container on a single NVIDIA
|
|
Spark device with Blackwell architecture, providing GPU-accelerated inference with all dependencies
|
|
pre-installed.
|
|
|
|
**Outcome**: You'll deploy SGLang in both server and offline inference modes on your NVIDIA Spark device,
|
|
enabling high-performance LLM serving with support for text generation, chat completion, and
|
|
vision-language tasks using models like DeepSeek-V2-Lite.
|
|
|
|
**Full playbook**: `/home/runner/work/dgx-spark-playbooks/dgx-spark-playbooks/nvidia/sglang/README.md`
|
|
<!-- GENERATED:END -->
|