Skip to content
NovaVMS Manual

Alert rule fires dozens of times per minute

You changed an alert rule’s scope, trigger, or schedule and now the notification center, your inbox, or a webhook destination is being hammered. This page applies to operators and admins who own the rule.

Symptom

Within a few minutes of saving the rule you see:

  • The notification bell badge climbing every few seconds.
  • Dozens of new rows on the Alerts page for the same rule and camera pair.
  • Recipients reporting an email flood, or a webhook endpoint returning 429s.

Likely causes

  1. Scope or compiled predicates are too broad for the camera’s activity level. A legacy event_category=motion rule on a busy entrance can fire on every matching event. A compiled rule can be similarly broad if its prompt, scope, or evidence predicates allow common activity.
  2. Compiled NBD de-duplication is disabled or too lenient. Active compiled rules use the NBD (No Big Deal) engine. If enabled=false, or the thresholds do not group the repeated events, each matching event can send.
  3. The legacy cooldown is too short. Immediate camera-event rules and supported offline system rules enforce cooldown_sec in PostgreSQL for the same rule and camera. A short value allows more alert rows. This legacy cooldown is separate from compiled NBD.
  4. Rule is scoped to “all cameras”. NBD’s de-dup key is (org_id, camera_id, rule_id, zone_hash) so each camera tracks its own group. One rule can still produce dozens of alerts per minute across the fleet if every camera independently triggers.
  5. Recipient list amplifies the noise. Compiled fanout delivers to each selected push user, email address, and enabled webhook ID. More selected destinations mean more deliveries per fire.
  6. Rule schedule mismatch. Active compiled rules enforce their recurring schedule/time windows. Legacy rules fire only with schedule_type=always; accepted legacy business_hours rows are skipped rather than restricted to business hours.

Fixes

Pause the rule first

Stop the bleeding before you diagnose.

  1. Open Alerts -> Rules.
  2. Find the rule at the top of the list (most recently fired).
  3. For a legacy rule, toggle Enabled off. The toggle takes effect immediately — no server restart.

The current Rules-page controls do not manage compiled-only rules safely. Do not use its toggle, Edit, Test, Duplicate, or Delete controls on a compiled-only row. Record the rule ID and escalate through your Admin/Owner process until compiled-aware management controls are available.

You can now investigate without new alerts piling up.

If the AOI is too broad

For a legacy rule, create a replacement with scope_type=cameras and an explicit camera_ids list. Do not switch it to legacy site: that value is accepted and persisted but skipped by the current evaluators.

For a compiled rule, recreate the rule carefully with a smaller camera selection and tighter natural-language predicates, test it against recent events, and activate the replacement. Do not reopen an existing compiled rule to edit scope or zone: the wizard does not rehydrate those fields and saving can widen the rule to all cameras and the whole frame.

If NBD suppression is disabled or too lenient

  1. For an active compiled rule, update NBD through PATCH /api/v1/alert-rules/{id}/nbd or the compatible NBD settings surface.
  2. Set enabled = true. Default window_seconds = 10800 (3h), similarity_threshold = 0.85. Raise the similarity threshold to ~0.90 to group only near-identical look-alikes; lower it to group more broadly.
  3. Save. The next event on the same camera + rule starts or updates its notification_groups row.

For a legacy rule, increase cooldown_sec within the supported 60-3600 second range. The immediate and supported system-event lanes enforce it in the alert-history insert. NBD settings apply to compiled rules, not legacy rows.

If recipients are amplifying the noise

  1. Open the rule, go to Recipients.
  2. Keep only the push users, email addresses, and webhook IDs that need each fire.
  3. For a paging webhook, use a dedicated compiled rule with tight predicates and NBD enabled. For a legacy webhook rule, use a tight camera list and database-backed cooldown.

Verify the fix

  1. Re-enable the rule.
  2. Watch the Alerts page for 10 minutes. Alert count for this rule should be single-digit.
  3. Check the rule’s Last fired timestamp — it advances at the rate you expect.
  4. Ask one recipient whether the inbox load is manageable. Volume is the real metric here, not config values.

Still stuck?

  • What drives alert response time — how the split pipeline and cooldown interact
  • Alert rule schema — every field, default, and constraint
  • Create an alert rule — the full walkthrough for rebuilding a rule from scratch
  • If the flood is coming from an event_category=motion rule on an ONVIF camera, the camera’s motion sensitivity may be set too high — see the camera’s own web UI