for self-hosted Ollama / vLLM / llama.cpp servers

Your GPU box died at 3am. Nothing watching it noticed.

GPU Watchdog watches the failure modes generic monitoring misses: models fighting each other for VRAM, cold-load storms after a reboot, and crash loops — then tells you before your users do.

Get Pro — $39 one-time → Free / open-core on GitHub
journalctl -u ollama -f

detectors

Five things it watches that nvidia-smi alone won't tell you

All five run in the free tier. A license unlocks multi-channel alerting and the keep_alive detector.

eviction thrashing
A model keeps getting reloaded because it's fighting another model for VRAM, or its keep_alive isn't sticking.
thundering herd
Multiple models cold-loading at once — usually right after a reboot — competing for disk/GPU bandwidth and making an already-healthy server look hung.
OOM crash loop
Repeated CUDA out-of-memory errors or service restarts inside a short window — the difference between a one-off hiccup and something actively on fire.
VRAM pressure
Memory pinned near the ceiling for longer than a grace period you set — the early warning before thrashing starts.
keep_alive shortfall PRO
A model evicted far sooner than your configured OLLAMA_KEEP_ALIVE predicts — usually a client request silently overriding server policy.

why this exists

It's built from a real incident, not a feature checklist

A multi-GPU Ollama box (three RTX 3060s and an old 1080) kept crash-looping under normal traffic. nvidia-smi looked fine between crashes. The dashboards looked fine. Nothing caught it until someone went digging in the logs by hand.

The real cause: MAX_LOADED_MODELS and KEEP_ALIVE were misconfigured on the server, and on top of that, three separate scripts were hardcoding keep_alive: -1 on their own requests — silently overriding the server's policy on every call. The box wasn't unstable. It was being told to do something impossible, over and over, by code nobody remembered was still running.

That's the specific pattern this tool watches for: not "is the GPU busy," but "is something forcing this server to fight itself."

— from an internal RAVEN-GPT postmortem, 2026

free vs pro

The free tier is not a trial

All five detectors run either way. A license is for teams that want more than one alert channel.

CapabilityFreePro
Core detectors (thrash / herd / crash-loop / VRAM)✓✓
keep_alive-shortfall detector—✓
Alert channels at once1Discord + Slack + email
LicenseMIT, self-hostedMIT + license key
Price$0$39 one-time

install

Running in under two minutes

$ git clone https://github.com/darkspire-dev/gpu-watchdog.git $ cd gpu-watchdog && pip install -r requirements.txt $ cp config.example.yaml config.yaml # set ollama_url + a Discord/Slack webhook $ python -m gpu_watchdog --config config.yaml

pricing

One price, no subscription

$39 one-time, lifetime license

Multi-channel alerting + the keep_alive-shortfall detector. Delivered by email within minutes of purchase, automatically.

Buy Pro →