mirror of
https://github.com/NVIDIA/dgx-spark-playbooks.git
synced 2026-04-23 02:23:53 +00:00
78213ac8a8
LitServe-based prompt injection detection server with a React monitoring dashboard. Serves HuggingFace classification models behind an OpenAI-compatible API with real-time metrics and GPU acceleration. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
11 lines
261 B
YAML
11 lines
261 B
YAML
models:
|
|
- name: deberta-injection
|
|
hf_model: deepset/deberta-v3-base-injection
|
|
device: cuda:0
|
|
batch_size: 32
|
|
- name: protectai-injection
|
|
hf_model: protectai/deberta-v3-base-prompt-injection-v2
|
|
device: cuda:0
|
|
batch_size: 32
|
|
port: 8234
|