Real-time webcam video analysis using AI

• May 13, 2025 at 08:52 IST
• 1 min (120 words)
• ai

Xuan-Son Nguyen shared a video on X where he is analyzing his webcam video feed in real-time by using local LLaMA model via ggml and Huggingface SmolVLM.

Real-time webcam demo with @huggingface SmolVLM and @ggml_org llama.cpp server.

All running locally on a Macbook M3

He also shared the GitHub repo containing the instructions on how to do it. The steps are:

1. Install llama.cpp
2. Run llama-server -hf ggml-org/SmolVLM-500M-Instruct-GGUF
   Note: you may need to add -ngl 99 to enable GPU (if you are using NVidia/AMD/Intel GPU)
   Note (2): You can also try other models here
3. Open index.html
4. Optionally change the instruction (for example, make it returns JSON)
5. Click on "Start" and enjoy

Definitely worth trying.