| .. | ||
| README.md | ||
Open WebUI with Ollama
Install Open WebUI and use Ollama to chat with models on your Spark
Table of Contents
Overview
Basic idea
Open WebUI is an extensible, self-hosted AI interface that operates entirely offline. This playbook shows you how to deploy Open WebUI with an integrated Ollama server on your DGX Spark device using NVIDIA Sync. The setup creates a secure SSH tunnel that lets you access the web interface from your local browser while the models run on Spark's GPU.
What you'll accomplish
You will have a fully functional Open WebUI installation running on your DGX Spark, accessible through your local web browser via NVIDIA Sync's managed SSH tunneling. The setup includes integrated Ollama for model management, persistent data storage, and GPU acceleration for model inference.
What to know before starting
- How to use NVIDIA Sync to connect to your DGX Spark device
Prerequisites
- DGX Spark device is set up and accessible
- NVIDIA Sync installed and connected to your DGX Spark
- Enough disk space for the Open WebUI container image and model downloads
Time & risk
- Duration: 15-20 minutes for initial setup, plus model download time (varies by model size)
- Risks:
- Docker permission issues may require user group changes and session restart
- Large model downloads may take significant time depending on network speed
Instructions
Step 1. Configure Docker permissions
To easily manage containers without sudo, you must be in the docker group. If you choose to skip this step, you will need to run Docker commands with sudo.
Open a new terminal and test Docker access. In the terminal, run:
docker ps
If you see a permission denied error (something like permission denied while trying to connect to the Docker daemon socket), add your user to the docker group so that you don't need to run the command with sudo .
sudo usermod -aG docker $USER
newgrp docker
Step 2. Verify Docker setup and pull container
Pull the Open WebUI container image with integrated Ollama:
docker pull ghcr.io/open-webui/open-webui:ollama
Step 3. Start the Open WebUI container
Start the Open WebUI container by running:
docker run -d -p 8080:8080 --gpus=all \
-v open-webui:/app/backend/data \
-v open-webui-ollama:/root/.ollama \
--name open-webui ghcr.io/open-webui/open-webui:ollama
This will start the Open WebUI container and make it accessible at http://localhost:8080. You can access the Open WebUI interface from your local web browser.
Application data will be stored in the open-webui volume and model data will be stored in the open-webui-ollama volume.
Step 4. Create administrator account
This step sets up the initial administrator account for Open WebUI. This is a local account that you will use to access the Open WebUI interface.
In the Open WebUI interface, click the "Get Started" button at the bottom of the screen.
Fill out the administrator account creation form with your preferred credentials.
Click the registration button to create your account and access the main interface.
Step 5. Download and configure a model
This step downloads a language model through Ollama and configures it for use in Open WebUI. The download happens on your DGX Spark device and may take several minutes.
Click on the "Select a model" dropdown in the top left corner of the Open WebUI interface.
Type gpt-oss:20b in the search field.
Click the "Pull 'gpt-oss:20b' from Ollama.com" button that appears.
Wait for the model download to complete. You can monitor progress in the interface.
Once complete, select "gpt-oss:20b" from the model dropdown.
Step 6. Test the model
This step verifies that the complete setup is working properly by testing model inference through the web interface.
In the chat textarea at the bottom of the Open WebUI interface, enter:
Write me a haiku about GPUs
Press Enter to send the message and wait for the model's response.
Step 8. Cleanup and rollback
Steps to completely remove the Open WebUI installation and free up resources:
Warning
These commands will permanently delete all Open WebUI data and downloaded models.
Stop and remove the Open WebUI container:
docker stop open-webui
docker rm open-webui
Remove the downloaded images:
docker rmi ghcr.io/open-webui/open-webui:ollama
Remove persistent data volumes:
docker volume rm open-webui open-webui-ollama
Step 9. Next steps
Try downloading different models from the Ollama library at https://ollama.com/library.
You can monitor GPU and memory usage through the DGX Dashboard available in NVIDIA Sync as you try different models.
If Open WebUI reports an update is available, you can update the container image by running:
docker pull ghcr.io/open-webui/open-webui:ollama
Setup Open WebUI on Remote Spark with NVIDIA Sync
Tip
If you haven't already installed NVIDIA Sync, learn how here.
Step 1. Configure Docker permissions
To easily manage containers using NVIDIA Sync, you must be able to run Docker commands without sudo.
Open the Terminal app from NVIDIA Sync to start an interactive SSH session and test Docker access. In the terminal, run:
docker ps
If you see a permission denied error (something like permission denied while trying to connect to the Docker daemon socket), add your user to the docker group so that you don't need to run the command with sudo .
sudo usermod -aG docker $USER
newgrp docker
Test Docker access again. In the terminal, run:
docker ps
Step 2. Verify Docker setup and pull container
Open a new Terminal app from NVIDIA Sync and pull the Open WebUI container image with integrated Ollama on your DGX Spark:
docker pull ghcr.io/open-webui/open-webui:ollama
Once the container image is downloaded, continue to setup NVIDIA Sync.
Step 3. Open NVIDIA Sync Settings
- Click on the NVIDIA Sync icon in your system tray or taskbar to open the main application window.
- Click the gear icon in the top right corner to open the Settings window.
- Click on the "Custom" tab to access Custom Ports configuration.
Step 4. Add Open WebUI custom port configuration
Setting up a new Custom port will XXXX
Click the "Add New" button on the Custom tab.
Fill out the form with these values:
Name: Open WebUI
Port: 12000
Auto open in browser at the following path: Check this checkbox
Start Script: Copy and paste this entire script:
#!/usr/bin/env bash
set -euo pipefail
NAME="open-webui"
IMAGE="ghcr.io/open-webui/open-webui:ollama"
cleanup() {
echo "Signal received; stopping ${NAME}..."
docker stop "${NAME}" >/dev/null 2>&1 || true
exit 0
}
trap cleanup INT TERM HUP QUIT EXIT
## Ensure Docker CLI and daemon are available
if ! docker info >/dev/null 2>&1; then
echo "Error: Docker daemon not reachable." >&2
exit 1
fi
## Already running?
if [ -n "$(docker ps -q --filter "name=^${NAME}$" --filter "status=running")" ]; then
echo "Container ${NAME} is already running."
else
# # Exists but stopped? Start it.
if [ -n "$(docker ps -aq --filter "name=^${NAME}$")" ]; then
echo "Starting existing container ${NAME}..."
docker start "${NAME}" >/dev/null
else
# # Not present: create and start it.
echo "Creating and starting ${NAME}..."
docker run -d -p 12000:8080 --gpus=all \
-v open-webui:/app/backend/data \
-v open-webui-ollama:/root/.ollama \
--name "${NAME}" "${IMAGE}" >/dev/null
fi
fi
echo "Running. Press Ctrl+C to stop ${NAME}."
## Keep the script alive until a signal arrives
while :; do sleep 86400; done
Click the "Add" button to save configuration to your DGX Spark.
Step 5. Launch Open WebUI
Click on the NVIDIA Sync icon in your system tray or taskbar to open the main application window.
Under the "Custom" section, click on "Open WebUI".
Your default web browser should automatically open to the Open WebUI interface at http://localhost:12000.
Tip
On first run, Open WebUI downloads models. This can delay server start and cause the page to fail to load in your browser. Simply wait and refresh the page. On future launches it will open quickly.
Step 6. Create administrator account
To start using Open WebUI you must create an initial administrator account. This is a local account that you will use to access the Open WebUI interface.
In the Open WebUI interface, click the "Get Started" button at the bottom of the screen.
Fill out the administrator account creation form with your preferred credentials.
Click the registration button to create your account and access the main interface.
Step 7. Download and configure a model
Next, download a language model with Ollama and configure it for use in Open WebUI. This download happens on your DGX Spark device and may take several minutes.
Click on the "Select a model" dropdown in the top left corner of the Open WebUI interface.
Type gpt-oss:20b in the search field.
Click the Pull "gpt-oss:20b" from Ollama.com button that appears.
Wait for the model download to complete. You can monitor progress in the interface.
Once complete, select "gpt-oss:20b" from the model dropdown.
Step 8. Test the model
In the chat textarea at the bottom of the Open WebUI interface, enter:
Write me a haiku about GPUs
Press Enter to send the message and wait for the model's response.
Step 9. Stop the Open WebUI
When you are finished with your session and want to stop the Open WebUI server and reclaim resources, close the Open WebUI from NVIDIA Sync.
Click on the NVIDIA Sync icon in your system tray or taskbar to open the main application window.
Under the "Custom" section, click the x icon on the right of the "Open WebUI" entry.
This will close the tunnel and stop the Open WebUI docker container.
Step 10. Next steps
Try downloading different models from the Ollama library at https://ollama.com/library.
You can monitor GPU and memory usage through the DGX Dashboard available in NVIDIA Sync as you try different models.
If Open WebUI reports an update is available, you can pull the the container image by running this in your terminal:
docker stop open-webui
docker rm open-webui
docker pull ghcr.io/open-webui/open-webui:ollama
After the update, launch Open WebUI again from NVIDIA Sync.
Step 11. Cleanup and rollback
Steps to completely remove the Open WebUI installation and free up resources:
Warning
These commands will permanently delete all Open WebUI data and downloaded models.
Stop and remove the Open WebUI container:
docker stop open-webui
docker rm open-webui
Remove the downloaded images:
docker rmi ghcr.io/open-webui/open-webui:ollama
Remove persistent data volumes:
docker volume rm open-webui open-webui-ollama
Remove the Custom App from NVIDIA Sync by opening Settings > Custom tab and deleting the entry.
Troubleshooting
Common issues with manual setup
| Symptom | Cause | Fix |
|---|---|---|
| Permission denied on docker ps | User not in docker group | Run Step 1 completely, including logging out and logging back in or use sudo |
| Model download fails | Network connectivity issues | Check internet connection, retry download |
| GPU not detected in container | Missing --gpus=all flag |
Recreate container with correct command |
| Port 8080 already in use | Another application using port | Change port in docker command or stop conflicting service |
Common issues with setting up via NVIDIA Sync
| Symptom | Cause | Fix |
|---|---|---|
| Permission denied on docker ps | User not in docker group | Run Step 1 completely, including terminal restart |
| Browser doesn't open automatically | Auto-open setting disabled | Manually navigate to localhost:12000 |
| Model download fails | Network connectivity issues | Check internet connection, retry download |
| GPU not detected in container | Missing --gpus=all flag |
Recreate container with correct start script |
| Port 12000 already in use | Another application using port | Change port in Custom App settings or stop conflicting service |
Note
DGX Spark uses a Unified Memory Architecture (UMA), which enables dynamic memory sharing between the GPU and CPU. With many applications still updating to take advantage of UMA, you may encounter memory issues even when within the memory capacity of DGX Spark. If that happens, manually flush the buffer cache with:
sudo sh -c 'sync; echo 3 > /proc/sys/vm/drop_caches'