Auto-Triage Sentry Errors

Enable the Sentry MCP

Devin needs access to your Sentry account so it can pull stack traces, breadcrumbs, and event metadata during triage.

Go to Settings > MCP Marketplace and find Sentry
Click Enable and complete the OAuth flow with the Sentry account that has access to your projects
Test the connection — start a session and ask Devin to list recent issues in one of your projects

Once connected, Devin can query any Sentry project your account has access to: issue details, full stack traces, event breadcrumbs, release tags, and more. See the MCP Marketplace docs for setup details.

Create a Sentry Internal Integration

Sentry delivers webhooks through Internal Integrations. In your Sentry dashboard, go to Settings > Developer Settings > Custom Integrations and click Create New Integration > Internal Integration.Configure it:

Name: Devin Auto-Triage
Webhook URL: A service you’ll deploy in the next step (e.g., https://your-domain.com/sentry-webhook)
Alert Rule Action: Toggle on — this makes the integration available as an action in alert rules
Permissions: Read access to Issue & Event and Project

Save the integration, then create an alert rule. Go to Alerts > Create Alert Rule > Issue Alert for your project:

When: A new issue is created
If: The issue has more than 50 events in 1 hour (adjust to your traffic)
Then: Send a notification via Devin Auto-Triage

Wire the webhook to the Devin API

Build a small handler that receives Sentry’s alert payload and starts a Devin session. Create a service user in Settings > Service Users with ManageOrgSessions permission and store its token as DEVIN_API_KEY in your handler’s environment. Set DEVIN_ORG_ID to your organization ID — get it by calling GET https://api.devin.ai/v3/enterprise/organizations with your token.

const express = require('express');
const app = express();
app.use(express.json());

app.post('/sentry-webhook', async (req, res) => {
  const event = req.body.data?.event;
  if (!event) return res.sendStatus(200);

  const orgId = process.env.DEVIN_ORG_ID;
  const response = await fetch(
    `https://api.devin.ai/v3/organizations/${orgId}/sessions`, {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${process.env.DEVIN_API_KEY}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      prompt: [
        `A Sentry alert fired for a new issue: "${event.title}"`,
        `Culprit: ${event.culprit}`,
        `Sentry URL: ${event.web_url}`,
        ``,
        `Use the Sentry MCP to pull the full stack trace and breadcrumbs.`,
        `Identify the root cause, fix the issue, and open a PR with a`,
        `regression test.`,
      ].join('\n'),
      tags: ['sentry-auto-triage', `project:${event.project}`],
    }),
  });

  const { session_id } = await response.json();
  console.log(`Started Devin session ${session_id} for: ${event.title}`);
  res.sendStatus(200);
});

app.listen(3000);

Deploy this anywhere that can receive HTTPS traffic — a Cloudflare Worker, AWS Lambda, or a small VPS. Point your Sentry Internal Integration’s webhook URL to it.The code above tags each session with sentry-auto-triage and the Sentry project name. This lets you filter these sessions in the Devin dashboard and pull them via the API using the tags query parameter — useful for tracking how many errors Devin triages per project.To verify the pipeline works, trigger a test alert in Sentry (or lower your threshold temporarily), then check app.devin.ai for a new session tagged sentry-auto-triage.

What Devin does with each alert

When a new error crosses your threshold and the webhook fires, Devin starts a session and works through the issue:

Queries Sentry via MCP — pulls the full stack trace, breadcrumbs (user actions leading to the crash), affected browser/OS/release tags, and event frequency
Traces the root cause — reads the source file at the exact line from the stack trace, follows the data flow to understand why the value is undefined or null
Writes a targeted fix — implements error handling (optional chaining, default values, input validation) that matches your codebase’s existing patterns
Adds a regression test — creates a test case that reproduces the original crash scenario and verifies the fix prevents it
Opens a PR — links the Sentry issue URL in the PR description so reviewers can cross-reference the original error context and event volume

Example PR description:

Fix: TypeError in UserProfile when user.profile is null

Root cause: The /api/users/:id endpoint returns { profile: null }
for users who haven't completed onboarding. UserProfile.tsx:47
destructures email from user.profile without a null check.

Fix: Added optional chaining and a fallback state.
Test: Added test for users with null profile — verifies the
component renders "Complete your profile" instead of crashing.

Sentry issue: FRONTEND-1892 (340 events in the past hour)

Sharpen triage with playbooks and Knowledge

Once the pipeline is running, make the auto-triage smarter:Create a triage playbook. Start from the !triage template playbook — duplicate it and customize with your team’s error-handling conventions (error boundaries, null check patterns, logging format). Then pass the playbook ID in your webhook handler by adding "playbook_id": "your-playbook-id" to the request body. You can also use Advanced Devin to generate a playbook from a description of your triage workflow.Add Knowledge about your API contracts and known edge cases — e.g., “Responses from /api/users may return { profile: null } for users who haven’t completed onboarding. Always guard against this.” Use Advanced Devin to help you write Knowledge entries from your existing documentation. The more context Devin has about your domain, the more accurate its fixes.Scope alerts carefully. Use Sentry’s alert rule conditions to limit which errors trigger Devin — filter by project, environment (production only), or error volume. A good starting point: only fire for issues with 50+ events in the first hour to focus on high-impact errors.Set up a weekly review schedule. Create a schedule that runs once a week to review the outcomes of your auto-triage sessions and feed learnings back into your playbook and Knowledge:

Gallery

Auto-Triage Sentry Errors