Alerting

Overview

QODRYX's alerting system monitors your applications for errors, performance degradation, security vulnerabilities, and deployment issues. Alerts can be sent to multiple channels including Slack, Discord, email, and webhooks.

Alert Types

Notification Channels

Slack

Receive rich, interactive alerts directly in your Slack channels:

# In your project settings or via API:
{
  "channel": "slack",
  "config": {
    "webhook_url": "https://hooks.slack.com/services/...",
    "channel": "#alerts",
    "mention_users": ["@oncall"],
    "include_stacktrace": true
  }
}

Discord

Get alerts in your Discord server with rich embeds:

{
  "channel": "discord",
  "config": {
    "webhook_url": "https://discord.com/api/webhooks/...",
    "mention_role": "123456789"
  }
}

Email

Receive email notifications for critical alerts:

{
  "channel": "email",
  "config": {
    "recipients": ["team@example.com", "oncall@example.com"],
    "digest": "immediate",  // or "hourly", "daily"
    "include_details": true
  }
}

Webhooks

Send alerts to any HTTP endpoint for custom integrations:

{
  "channel": "webhook",
  "config": {
    "url": "https://your-service.com/alerts",
    "method": "POST",
    "headers": {
      "Authorization": "Bearer your-token"
    }
  }
}

Alert Rules

Create custom alert rules based on conditions:

# Example alert rules

# Alert when error rate exceeds threshold
- name: "High Error Rate"
  condition: error_rate > 5%
  window: 5m
  severity: critical
  channels: [slack, pagerduty]

# Alert on new error types
- name: "New Error Detected"
  condition: new_error_type
  severity: warning
  channels: [slack]

# Alert on performance degradation
- name: "Slow Response Time"
  condition: p95_latency > 2000ms
  window: 10m
  severity: warning
  channels: [slack, email]

# Alert on security vulnerabilities
- name: "Critical Vulnerability"
  condition: vulnerability.severity == "critical"
  severity: critical
  channels: [slack, email, pagerduty]

Alert Configuration

Severity Levels

Alert Throttling

Prevent alert fatigue with intelligent throttling:

• Rate Limiting: Maximum alerts per time window
• Deduplication: Same alert not sent within cooldown period
• Aggregation: Group similar alerts into summaries
• Quiet Hours: Suppress non-critical alerts during off-hours

# Throttling configuration
throttle:
  rate_limit: 10/hour
  cooldown: 15m
  aggregate_after: 5
  quiet_hours:
    enabled: true
    start: "22:00"
    end: "08:00"
    timezone: "Europe/Stockholm"
    allow_critical: true

Alert Actions

Define automated actions when alerts trigger:

Managing Alerts

Acknowledging Alerts

When you receive an alert, you can acknowledge it to stop further notifications:

# Via CLI
qodryx alerts ack <alert-id> --message "Looking into it"

# Via API
POST /api/alerts/{id}/acknowledge
{
  "message": "Looking into it",
  "assignee": "john@example.com"
}

Resolving Alerts

Mark alerts as resolved when the issue is fixed:

# Via CLI
qodryx alerts resolve <alert-id> --resolution "Fixed in v2.1.1"

# Via API
POST /api/alerts/{id}/resolve
{
  "resolution": "Fixed in v2.1.1",
  "commit": "abc123"
}

Best Practices

Next Steps