Incident Management

Hrida.ai includes a plugin-based incident management system that lets you connect one or more external on-call providers — Grafana OnCall, PagerDuty, and Opsgenie — through a unified admin panel. Each provider gets its own webhook URL and API token. Agents can create, acknowledge, and resolve incidents using MCP tools regardless of which provider is active.

Admin-Only Feature

The Incidents tab is only accessible to users with the admin role. Navigate to Admin Panel → Incidents.

---

Architecture

External provider (OnCall / PagerDuty / Opsgenie)
        │  Outgoing webhook (provider-specific auth)
        ▼
POST /api/v1/webhooks/incidents/{provider_id}
        │  Normalized IncidentEvent
        ├─▶ WebSocket → Admin incidents panel (real time)
        └─▶ Agent workflow trigger (optional)

Admin panel / agents
        │  REST API calls
        ▼
GET /api/v1/webhooks/incidents/{id}/incidents
        │  hrida-ai-studio proxies to provider API
        ▼
Provider REST API

The plugin layer normalizes all providers into the same event shape — the admin panel and agent tools work identically regardless of whether the alert came from OnCall, PagerDuty, or Opsgenie.

---

Supported Providers

Provider	Webhook auth	Agent tools via mcpo
Grafana OnCall	Bearer token	✅
PagerDuty	HMAC-SHA256 (X-PagerDuty-Signature)	✅
Opsgenie	Custom header (X-Hrida-Secret)	✅

---

Adding a Provider

All configuration is done through the admin panel — no env-var changes needed.

1. Navigate to Admin Panel → Incidents → Providers.
2. Click + Add Provider.
3. Select the provider type (OnCall / PagerDuty / Opsgenie).
4. Enter a name (e.g. "Production OnCall") and fill in the config fields (API token, webhook secret, etc.).
5. Click Save — the card shows the Webhook URL to copy.
6. Paste that URL into your provider's outgoing webhook settings (see each provider guide for exact steps).

You can configure multiple providers simultaneously — for example, OnCall for engineering and PagerDuty for security incidents. Each shows independently in the live feed.

Enabling / disabling providers

Each provider card has an enable toggle. Disabling a provider stops its webhook from being accepted and hides its incidents from the feed without deleting the configuration.

---

Admin Incidents Panel

Navigate to Admin Panel → Incidents to see the unified live feed across all configured providers.

Column	Description
Status	Firing (red) · Acknowledged (yellow) · Resolved (green)
Provider	Which provider the incident came from
Title	Incident / alert title
Fired	Timestamp when the alert first fired
Acknowledged	Timestamp when an engineer acknowledged
Resolved	Timestamp when the incident was resolved
Open ↗	Deep-link to the incident in the provider's UI

The table updates in real time via WebSocket — no manual refresh needed. Click Refresh to re-fetch the current list from each provider's API.

Switch to the Providers sub-tab to manage configured provider integrations.

---

Normalized Incident States

All providers map to three canonical states:

Canonical state	Grafana OnCall	PagerDuty	Opsgenie
firing	firing	triggered	action = Create
acknowledged	acknowledged	acknowledged	action = Acknowledge
resolved	resolved	resolved	action = Close

---

Agent Integration (via hrida-mcpo)

Each provider can expose its REST API as an MCP tool server through hrida-mcpo. Agents attached to the tool server can call incident operations without writing any custom code.

Grafana OnCall

Add the mcpo sidecar URL as a tool server in Admin Panel → Settings → Tool Servers:

http://hrida-mcpo-oncall:8000   (Docker internal)
http://localhost:8091            (host access)

Available tools: list_alert_groups, get_alert_group, acknowledge_alert_group, resolve_alert_group, list_schedules, list_users.

PagerDuty / Opsgenie

For cloud providers (PagerDuty, Opsgenie), run a separate hrida-mcpo instance pointed at their OpenAPI specs:

# In your docker-compose override:
hrida-mcpo-pagerduty:
  image: ghcr.io/hrida-ai/hrida-mcpo:latest
  environment:
    OPENAPI_SPEC_URL: https://api.pagerduty.com/openapi.json
    AUTH_HEADER: Authorization
    AUTH_TOKEN: Token token=<api_token>

Example: Agent that auto-creates incidents

[Agent node] Detects elevated error rate from metrics
  → calls create_alert_group (OnCall) or trigger_event (PagerDuty)
  → Provider routes to on-call engineer
  → Engineer acknowledges
  → Provider POSTs webhook to hrida-ai-studio
  → hrida-ai-studio resumes a paused workflow node

---

Workflow Trigger Integration

Any published agent workflow can fire automatically when an incident changes state. Set oncall_trigger in the workflow's Advanced → Metadata:

Value	Triggers on
firing	New incident (any provider)
acknowledged	Engineer acknowledges
resolved	Incident resolved
any	Any state change

This works for all providers — the webhook receiver normalizes the state before checking workflow triggers.

---

API Reference

Webhook Receiver (provider → hrida-ai-studio)

Each provider gets its own URL. Find it on the provider card in Admin Panel → Incidents → Providers.

POST /api/v1/webhooks/incidents/{provider_id}

Authentication is provider-specific — see the individual provider guides.

Legacy alias (Grafana OnCall only, backward-compatible):

POST /api/v1/webhooks/oncall

Automatically 307-redirects to the first enabled OnCall provider. Existing OnCall webhook configurations don't need to be updated.

Admin: List Providers

GET /api/v1/incident-providers/
Authorization: Bearer <admin-jwt>

Admin: Create Provider

POST /api/v1/incident-providers/
Authorization: Bearer <admin-jwt>
Content-Type: application/json

{
  "name": "Production OnCall",
  "type": "grafana_oncall",
  "enabled": true,
  "config": {
    "api_url": "http://oncall:8080",
    "api_token": "...",
    "webhook_secret": "..."
  }
}

Admin: Update Provider (PATCH — config is merged, not replaced)

PATCH /api/v1/incident-providers/{id}
Authorization: Bearer <admin-jwt>
Content-Type: application/json

{ "enabled": false }

Admin: Incidents Proxy

GET /api/v1/webhooks/incidents/{id}/incidents?state=firing,acknowledged
Authorization: Bearer <admin-jwt>

Returns raw incident objects from the provider API for the given states. Valid state values: firing, acknowledged, resolved, silenced.

Admin: Supported Types + Config Schemas

GET /api/v1/incident-providers/types
Authorization: Bearer <admin-jwt>

Returns each supported provider type with its required config keys — used by the admin UI to render the correct form fields.

---

Related

• Grafana OnCall Setup Guide — self-hosted Docker Compose, plugin installation, webhook configuration
• PagerDuty Setup Guide — API token, HMAC webhook signing, routing keys
• Opsgenie Setup Guide — GenieKey, custom header auth, EU region
• Analytics → Agent Workflow Analytics — track which workflows were triggered by incidents