Skip to content
NovaVMS Manual

Why NovaVMS runs one VLM stack with per-camera detection overrides

NovaVMS v5 ships a single VLM stack — Gemini 3.1 Flash-Lite (GA May 2026) via Vertex AI — plus three local CPU models in the AI Runtime sidecar. The pre-v5 per-org multi-provider selector (Gemini / OpenAI / Qwen / Ollama) was retired in the L12 cutover (docs/decisions.md D78 / D82 governance still apply, but the field of choices shrank to one).

Why one VLM, not many

A VLM choice is a cost lever, a data-residency lever, and a capability lever. The pre-v5 design exposed all three to admins. In practice:

  • Cost. Gemini 3.1 Flash-Lite is the cheapest per-event option in the May 2026 market, and the v5 evidence-first pipeline keeps Gemini calls bounded (summary + optional judge per event, not per frame).
  • Data residency. Vertex AI lets us pin processing to a regional endpoint (<region>-aiplatform.googleapis.com) and authenticate via Application Default Credentials (GOOGLE_APPLICATION_CREDENTIALS). That solves the residency lever inside one provider.
  • Capability. Gemini 3.1 Flash-Lite is multimodal, supports structured output, and supports function calling — enough to cover the v5 language surfaces (NL alert compiler, query parser, contextual judge, VLM summary). Adding a second provider would double the test surface without adding capability.

The single-provider choice means there is no per-org provider/model picker. The org-level surfaces at /admin/ai are about tuning the local pipeline, not picking a model.

Per-camera overrides in v5

Only one camera-level v5 knob is consumed by the ingest path: yolo_confidence. Set it from the camera detail page; an unset value falls through to the org-level yolo_confidence (default 0.35, range 0.05..0.95).

The pre-v5 camera columns ai_provider_override, ai_model_override, prompt_pack_id, and analysis_mode survive on the cameras table but have no consumer in the v5 ingest path. Leave them blank. Future per-camera surfaces (e.g. per-camera frame-extraction mode) will surface in /admin/ai or the camera detail page when they ship.

What the admin actually controls

The admin at /admin/ai:

  • Toggles seven per-feature AI flags org-wide (visual_search, object_detection, face_recognition, face_search, nl_alerts, nbd_suppression, vlm_understanding).
  • Tunes 11 pipeline knobs (frame extraction, runtime concurrency, snapshot quality, detection thresholds, contextual-judge thinking level).
  • Runs the 10-step AI maintenance wizard for clean migrations and backfills.
  • Inspects per-model health (loaded / checksum / RSS) for MobileCLIP2-B, YOLO26n, InsightFace, Gemini.

See Configure AI for your organization.

D82 still governs the role split

D82 records that admins govern the AI surface and operators don’t. That still applies in v5:

  • Admins: tune org-level knobs, toggle features, run maintenance, calibrate thresholds.
  • Operators: tune per-rule NBD thresholds (in alert-rule context), per-profile face match thresholds (in Face Library), per-camera yolo_confidence (in camera detail).
  • Viewers: read-only AI evidence panel scoped by user_site_access (D5-10).

What happened to the env-var contract

Pre-v5 used AI_DEFAULT_PROVIDER and AI_DEFAULT_MODEL. v5 uses:

  • VERTEX_AI_PROJECT and VERTEX_AI_LOCATION — pin Vertex AI to a region.
  • GOOGLE_APPLICATION_CREDENTIALS — path to the Vertex AI service-account JSON for ADC.
  • GEMINI_MODEL — override the default Gemini model (default gemini-3.1-flash-lite).
  • AI_RUNTIME_BASE_URL — local AI Runtime endpoint (default http://127.0.0.1:8091).
  • GEMINI_API_KEY — AI Studio API-key fallback (dev/local only; AI Studio is geo-blocked from many production VPS regions).

See AI provider fields.

Changes are forward-only

Changing an org-level knob applies to all new events from the moment the change is saved. Events analysed before the change keep their original embeddings, detections, faces, and Gemini summaries. Re-analysis of past events requires an admin to call POST /api/v1/ai/reprocess (bulk, scoped to a filter) or to re-trigger one event via POST /api/v1/events/{id}/analyze.

For a clean re-embedding (e.g. when MobileCLIP2-B is updated), use the AI maintenance wizard at /admin/ai — it covers checksum preflight, schema migration, retained-event backfill, and the eval-corpus gate before reopening.

Where to go next