# Video Search and Summarization > Run the VSS Blueprint on your Spark ## Table of Contents - [Overview](#overview) - [Instructions](#instructions) --- ## Overview ## Basic idea Deploy NVIDIA's Video Search and Summarization (VSS) AI Blueprint to build intelligent video analytics systems that combine vision language models, large language models, and retrieval-augmented generation. The system transforms raw video content into real-time actionable insights with video summarization, Q&A, and real-time alerts. You'll set up either a completely local Event Reviewer deployment or a hybrid deployment using remote model endpoints. ## What you'll accomplish You will deploy NVIDIA's VSS AI Blueprint on NVIDIA Spark hardware with Blackwell architecture, choosing between two deployment scenarios: VSS Event Reviewer (completely local with VLM pipeline) or Standard VSS (hybrid deployment with remote LLM/embedding endpoints). This includes setting up Alert Bridge, VLM Pipeline, Alert Inspector UI, Video Storage Toolkit, and optional DeepStream CV pipeline for automated video analysis and event review. ## What to know before starting - Working with NVIDIA Docker containers and container registries - Setting up Docker Compose environments with shared networks - Managing environment variables and authentication tokens - Working with NVIDIA DeepStream and computer vision pipelines - Basic understanding of video processing and analysis workflows ## Prerequisites - NVIDIA Spark device with ARM64 architecture and Blackwell GPU - FastOS 1.81.38 or compatible ARM64 system - Driver version 580.82.09 installed: `nvidia-smi | grep "Driver Version"` - CUDA version 13.0 installed: `nvcc --version` - Docker installed and running: `docker --version && docker compose version` - Access to NVIDIA Container Registry with NGC API Key - [Optional] NVIDIA API Key for remote model endpoints (hybrid deployment only) - Sufficient storage space for video processing (>10GB recommended in `/tmp/`) ## Ancillary files - [VSS Blueprint GitHub Repository](https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization) - Main codebase and Docker Compose configurations - [Sample CV Detection Pipeline](https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization/tree/main/examples/cv-event-detector) - Reference CV pipeline for event reviewer workflow - [VSS Official Documentation](https://docs.nvidia.com/vss/latest/index.html) - Complete system documentation ## Time & risk **Duration:** 30-45 minutes for initial setup, additional time for video processing validation **Risks:** - Container startup can be resource-intensive and time-consuming with large model downloads - Network configuration conflicts if shared network already exists - Remote API endpoints may have rate limits or connectivity issues (hybrid deployment) **Rollback:** Stop all containers with `docker compose down`, remove shared network with `docker network rm vss-shared-network`, and clean up temporary media directories. ## Instructions ## Step 1. Verify environment requirements Check that your system meets the hardware and software prerequisites. ```bash ## Verify driver version nvidia-smi | grep "Driver Version" ## Expected output: Driver Version: 580.82.09 ## Verify CUDA version nvcc --version ## Expected output: release 13.0 ## Verify Docker is running docker --version && docker compose version ``` ## Step 2. Clone the VSS repository Clone the Video Search and Summarization repository from NVIDIA's public GitHub. ```bash ## Clone the VSS AI Blueprint repository git clone https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization.git cd video-search-and-summarization ``` ## Step 3. Run the cache cleaner script Start the system cache cleaner to optimize memory usage during container operations. ```bash ## In another terminal, start the cache cleaner script. ## Alternatively, append " &" to the end of the command to run it in the background. sudo sh deploy/scripts/sys_cache_cleaner.sh ``` ## Step 4. Set up Docker shared network Create a Docker network that will be shared between VSS services and CV pipeline containers. ```bash ## Create shared network (may require sudo depending on Docker configuration) docker network create vss-shared-network ``` > **Warning:** If the network already exists, you may see an error. Remove it first with `docker network rm vss-shared-network` if needed. ## Step 5. Authenticate with NVIDIA Container Registry Log in to NVIDIA's container registry using your [NGC API Key](https://org.ngc.nvidia.com/setup/api-keys). ```bash ## Log in to NVIDIA Container Registry docker login nvcr.io ## Username: $oauthtoken ## Password: ``` ## Step 6. Choose deployment scenario Choose between two deployment options based on your requirements: | Deployment Scenario | VLM (Cosmos-Reason1-7B) | LLM (Llama 3.1 70B) | Embedding/Reranker | CV Pipeline | |----------------------|--------------------------|---------------------|--------------------|-------------| | VSS Event Reviewer | Local | Not Used | Not Used | Local | | Standard VSS (Hybrid)| Local | Remote | Remote | Optional | Proceed with **Option A** for Event Reviewer or **Option B** for Standard VSS. ## Step 7. Option A **[VSS Event Reviewer](https://docs.nvidia.com/vss/latest/content/vss_event_reviewer.html) (Completely Local)** **7.1 Navigate to Event Reviewer directory** Change to the directory containing the Event Reviewer Docker Compose configuration. ```bash cd deploy/docker/event_reviewer/ ``` **7.2 Configure NGC API Key** Update the environment file with your NGC API Key. You can do this by editing the `.env` file directly, or by running the following command: ```bash ## Edit the .env file and update NGC_API_KEY echo "NGC_API_KEY=" >> .env ``` **7.3 Update the VSS Image path** Update `VSS_IMAGE` to `nvcr.io/nvidia/blueprint/vss-engine-sbsa:2.4.0` in `.env`. ```bash ## Edit the .env file and update VSS_IMAGE echo "VSS_IMAGE=nvcr.io/nvidia/blueprint/vss-engine-sbsa:2.4.0" >> .env ``` **7.4 Start VSS Event Reviewer services** Launch the complete VSS Event Reviewer stack including Alert Bridge, VLM Pipeline, Alert Inspector UI, and Video Storage Toolkit. ```bash ## Start VSS Event Reviewer with ARM64 and SBSA optimizations IS_SBSA=1 IS_AARCH64=1 ALERT_REVIEW_MEDIA_BASE_DIR=/tmp/alert-media-dir docker compose up ``` > **Note:** This step will take several minutes as containers are pulled and services initialize. The VSS backend requires additional startup time. **7.5 Navigate to CV Event Detector directory** In a new terminal session, navigate to the computer vision event detector configuration. ```bash cd video-search-and-summarization/examples/cv-event-detector ``` **7.6 Update the NV_CV_EVENT_DETECTOR_IMAGE Image path** Update `NV_CV_EVENT_DETECTOR_IMAGE` to `nvcr.io/nvidia/blueprint/nv-cv-event-detector-sbsa:2.4.0` in `.env`. ```bash ## Edit the .env file and update NV_CV_EVENT_DETECTOR_IMAGE echo "NV_CV_EVENT_DETECTOR_IMAGE=nvcr.io/nvidia/blueprint/nv-cv-event-detector-sbsa:2.4.0" >> .env ``` **7.7 Start DeepStream CV pipeline** Launch the DeepStream computer vision pipeline and CV UI services. ```bash ## Start CV pipeline with ARM64 and SBSA optimizations IS_SBSA=1 IS_AARCH64=1 ALERT_REVIEW_MEDIA_BASE_DIR=/tmp/alert-media-dir docker compose up ``` **7.8 Wait for service initialization** Allow time for all containers to fully initialize before accessing the user interfaces. ```bash ## Monitor container status docker ps ## Verify all containers show "Up" status and VSS backend logs show ready state ``` **7.9 Validate Event Reviewer deployment** Access the web interfaces to confirm successful deployment and functionality. ```bash ## Test CV UI accessibility (replace with your system's IP) curl -I http://:7862 ## Expected: HTTP 200 response ## Test Alert Inspector UI accessibility curl -I http://:7860 ## Expected: HTTP 200 response ``` Open these URLs in your browser: - `http://:7862` - CV UI to launch and monitor CV pipeline - `http://:7860` - Alert Inspector UI to view clips and review VLM results ## Step 8. Option B **[Standard VSS](https://docs.nvidia.com/vss/latest/content/architecture.html) (Hybrid Deployment)** In this hybrid deployment, we would use NIMs from [build.nvidia.com](https://build.nvidia.com/). Alternatively, you can configure your own hosted endpoints by following the instructions in the [VSS remote deployment guide](https://docs.nvidia.com/vss/latest/content/installation-remote-docker-compose.html). **8.1 Get NVIDIA API Key** - Log in to https://build.nvidia.com/explore/discover. - Search for **Get API Key** on the page and click on it. **8.2 Navigate to remote LLM deployment directory** ```bash cd deploy/docker/remote_llm_deployment/ ``` **8.3 Configure environment variables** Update the environment file with your API keys and deployment preferences. You can do this by editing the `.env` file directly, or by running the following commands: ```bash ## Edit .env file with required keys echo "NVIDIA_API_KEY=" >> .env echo "NGC_API_KEY=" >> .env echo "DISABLE_CV_PIPELINE=true" >> .env # Set to false to enable CV echo "INSTALL_PROPRIETARY_CODECS=false" >> .env # Set to true to enable CV ``` **8.4 Update the VSS Image path** Update `VIA_IMAGE` to `nvcr.io/nvidia/blueprint/vss-engine-sbsa:2.4.0` in `.env`. ```bash ## Edit the .env file and update VIA_IMAGE echo "VIA_IMAGE=nvcr.io/nvidia/blueprint/vss-engine-sbsa:2.4.0" >> .env ``` **8.5 Review model configuration** Verify that the config.yaml file contains the correct remote endpoints. For NIMs, it should be set to `https://integrate.api.nvidia.com/v1 `. ```bash ## Check model server endpoints in config.yaml cat config.yaml | grep -A 10 "model" ``` **8.6 Launch Standard VSS deployment** ```bash ## Start Standard VSS with hybrid deployment docker compose up ``` **8.7 Validate Standard VSS deployment** Access the VSS UI to confirm successful deployment. ```bash ## Test VSS UI accessibility (replace with your system's IP) curl -I http://:9100 ## Expected: HTTP 200 response ``` Open `http://:9100` in your browser to access the VSS interface. ## Step 9. Test video processing workflow Run a basic test to verify the video analysis pipeline is functioning based on your deployment. **For Event Reviewer deployment** Follow the steps [here](https://docs.nvidia.com/vss/latest/content/vss_event_reviewer.html#vss-alert-inspector-ui) to access and use the Event Reviewer workflow. - Access CV UI at `http://:7862` to upload and process videos - Monitor results in Alert Inspector UI at `http://:7860` **For Standard VSS deployment** Follow the steps [here](https://docs.nvidia.com/vss/latest/content/ui_app.html) to navigate VSS UI - File Summarization, Q&A, and Alerts. - Access VSS interface at `http://:9100` - Upload videos and test summarization features ## Step 10. Troubleshooting | Symptom | Cause | Fix | |---------|--------|-----| | Container fails to start with "pull access denied" | Missing or incorrect nvcr.io credentials | Re-run `docker login nvcr.io` with valid credentials | | Network creation fails | Existing network with same name | Run `docker network rm vss-shared-network` then recreate | | Services fail to communicate | Incorrect environment variables | Verify `IS_SBSA=1 IS_AARCH64=1` are set correctly | | Web interfaces not accessible | Services still starting or port conflicts | Wait 2-3 minutes, check `docker ps` for container status | ## Step 11. Cleanup and rollback To completely remove the VSS deployment and free up system resources: > **Warning:** This will destroy all processed video data and analysis results. ```bash ## For Event Reviewer deployment cd deploy/docker/event_reviewer/ IS_SBSA=1 IS_AARCH64=1 ALERT_REVIEW_MEDIA_BASE_DIR=/tmp/alert-media-dir docker compose down cd ../../examples/cv-event-detector/ IS_SBSA=1 IS_AARCH64=1 ALERT_REVIEW_MEDIA_BASE_DIR=/tmp/alert-media-dir docker compose down ## For Standard VSS deployment cd deploy/docker/remote_llm_deployment/ docker compose down ## Remove shared network (if using Event Reviewer) docker network rm vss-shared-network ## Clean up temporary media files and stop cache cleaner rm -rf /tmp/alert-media-dir sudo pkill -f sys_cache_cleaner.sh ``` ## Step 12. Next steps With VSS deployed, you can now: **Event Reviewer deployment:** - Upload video files through the CV UI at port 7862 - Monitor automated event detection and reviewing - Review analysis results in the Alert Inspector UI at port 7860 - Configure custom event detection rules and thresholds **Standard VSS deployment:** - Access full VSS capabilities at port 9100 - Test video summarization and Q&A features - Configure knowledge graphs and graph databases - Integrate with existing video processing workflows