Deployment Monitoring

Deployment Monitoring closes the feedback loop between Autopilot and your CI/CD pipeline. When Autopilot pushes code, DevSpec watches the resulting deployment via webhooks from Vercel, Netlify, Railway, Render, or self-hosted Coolify. If the deployment fails, DevSpec classifies the error, decides whether to retry automatically or escalate to a human, and records the full history so your team always knows what happened and why.

How It Works

The deployment monitoring pipeline follows a webhook-driven flow: your CI/CD platform sends a webhook when a deployment finishes. DevSpec receives the event, fetches build logs, classifies the failure with Claude, and decides whether to retry or escalate.

• Your CI/CD platform (Vercel, Netlify, Railway, Render, or Coolify) sends a deployment webhook to DevSpec.
• DevSpec records the event and links it to the originating action item and deployment target.
• If the deployment succeeded, DevSpec records the success and notifies relevant parties.
• If the deployment failed, DevSpec fetches the build logs via the platform API.
• Claude classifies the failure into an error category (code, config, dependency, infra, test, unknown) and provides a human-readable reason.
• The retry gate evaluates whether the error is retryable: code and dependency errors can be retried, while infra and hard-stop errors are escalated immediately.
• If retryable and under the max retry limit, the action item is re-queued for Autopilot with enriched error context.
• If not retryable or retries are exhausted, the failure is escalated to the assigned developer with a notification.

1. Setting Up Vercel

Connect Vercel to DevSpec so deployment events are forwarded automatically.

Step 1 — Create a deployment target in DevSpec

In your DevSpec project, go to Settings > Deployment Targets and click "Add Deployment Target". Choose "Vercel" as the platform and give it a name (e.g. "Production"). Save it — DevSpec will display a webhook URL. Copy the URL.

Step 2 — Open the Vercel webhook settings

Navigate to your Vercel dashboard and go to Settings > Webhooks. Click "Add Webhook".

Step 3 — Choose projects

Under "Projects", select "Specific Projects" and choose only the Vercel project you want to monitor.

Step 4 — Select events

Under "Deployment Events", check the following: "Deployment Succeeded" (deployment.succeeded), "Deployment Error" (deployment.error), and "Deployment Canceled" (deployment.canceled). Optionally check "Deployment Created" (deployment.created) for real-time in-progress tracking. Leave all other event categories unchecked.

Step 5 — Paste the endpoint URL

Paste the webhook URL you copied from DevSpec into the "Endpoint" field at the bottom. It will look like this:

https://devspec.ai/api/webhooks/deployment/{your-project-id}/{your-target-id}

Step 6 — Save and copy the signing secret

Click "Create Webhook". Vercel will show a confirmation page with a signing secret — copy it immediately, as it will not be shown again. This secret is used to verify that incoming webhooks are authentically from Vercel.

Step 7 — Save the secret in DevSpec

Go back to your deployment target in DevSpec, click Edit, and paste the Vercel signing secret into the "Webhook Secret" field. Optionally also add your Vercel API token — this lets DevSpec fetch build logs when a deployment fails.

Step 8 — Verify the connection

Trigger a test deployment in Vercel. You should see a deployment event appear in DevSpec within a few seconds.

2. Setting Up Netlify

Connect Netlify to DevSpec for deployment monitoring.

Step 1 — Open Netlify site settings

Go to your Netlify dashboard, select the site, and navigate to Site Configuration > Notifications.

Step 2 — Add an outgoing webhook

Under "Outgoing webhooks", click "Add notification" and select "Outgoing webhook".

Step 3 — Configure the webhook URL

Enter the DevSpec webhook URL:

https://devspec.ai/api/webhooks/deployment/{your-project-id}/netlify

Step 4 — Select the deploy events

Add webhooks for "Deploy succeeded" and "Deploy failed" events.

Step 5 — Add your Netlify API token in DevSpec

In the deployment target settings, paste your Netlify personal access token so DevSpec can fetch build logs on failure.

3. Setting Up Railway

Connect Railway to DevSpec for deployment monitoring.

Step 1 — Open Railway project settings

Go to your Railway dashboard, select the project, and navigate to Settings > Webhooks.

Step 2 — Add the DevSpec webhook URL

Enter the DevSpec webhook URL:

https://devspec.ai/api/webhooks/deployment/{your-project-id}/railway

Step 3 — Select deployment events

Subscribe to "Deployment Success" and "Deployment Failed" events.

Step 4 — Add your Railway API token in DevSpec

In the deployment target settings, paste your Railway API token to enable build log fetching.

4. Setting Up Render

Connect Render to DevSpec for deployment monitoring.

Step 1 — Open Render service settings

Go to your Render dashboard, select the service, and navigate to Settings > Notifications.

Step 2 — Add a webhook notification

Click "Add Notification" and choose "Webhook".

Step 3 — Configure the webhook URL

Enter the DevSpec webhook URL:

https://devspec.ai/api/webhooks/deployment/{your-project-id}/render

Step 4 — Select events

Subscribe to "Deploy succeeded" and "Deploy failed" events.

Step 5 — Add your Render API key in DevSpec

In the deployment target settings, paste your Render API key to enable log fetching for failed deployments.

5. Setting Up Coolify

Coolify is self-hosted, so it's tracked a little differently. You choose how much to connect — from a webhook with no inbound access at all, up to an API token for exact commits and build logs. The trade-offs are covered at the end of this section.

Coolify's deploy notification is team-wide: one webhook fires for every app in the team. DevSpec handles this with a single project-level URL and routes each notification to the right target by Coolify Application UUID — so you add one DevSpec target per Coolify app you want to track.

Step 1 — Create a deployment target per app

Go to Settings > Deployment Targets > Add Deployment Target. Choose "Coolify", give it a name, then set:

• Coolify Application UUID — the app this target tracks. Open the app in Coolify and copy the last segment of the dashboard URL (…/application/<uuid>) — a short random ID, not your app's public domain. (It's also the uuid= value in the app's Deploy Webhook URL.)
• Branch Filter — the branch this app deploys (e.g. main or staging); used to resolve the commit from your connected repository's data.

You don't set a webhook secret — Coolify's webhook is unsigned, so DevSpec generates a strong one for you and builds it into the URL below. (You can rotate it any time by editing the target → Regenerate webhook secret, which gives you a fresh URL to paste back into Coolify.)

Repeat for each Coolify app you want to track. Copy the webhook URL — it's the same project-level URL for all your Coolify targets.

Step 2 — Add ONE Webhook notification in Coolify

In Coolify, go to Settings → Notifications → Webhook (configured per team) and enable it. When you save the target, DevSpec shows the complete webhook URL — copy it and paste it into Coolify as-is. It already includes the generated webhook secret as a ?secret= token (the secret is encrypted at rest afterward, so grab the URL while it's shown):

https://devspec.ai/api/webhooks/deployment/{your-project-id}/coolify?secret=<your-secret>

Enable the Deployment Success and Deployment Failure events. DevSpec routes each app's deploys to the matching target automatically; deploys for apps you haven't added as targets are ignored.

That's the zero-exposure baseline — no API token, no inbound access. DevSpec records every success and failure; because Coolify's webhook doesn't include the commit, DevSpec resolves it from your connected repository (the branch's HEAD at deploy time).

Optional — connect the Coolify API. Enable Coolify's API (Settings → API), then edit the target and add:

• A read API token + an Instance Base URL (e.g. https://coolify.example.com) → DevSpec fetches the exact commit SHA instead of inferring it. A read token can read your team's resource metadata (apps, config, commits) but not secrets — low risk, and the most accurate option if your Coolify API is reachable from DevSpec.
• A read:sensitive token additionally fetches build logs, enabling AI failure classification and auto-retry. Note that read:sensitive also grants access to your environment variables, secrets, and logs — only use it if you want auto-retry and accept that trade-off.

Least privilege: a Coolify API token can read everything in the team it was created under. If you connect a token, consider putting the app(s) in a dedicated Coolify team and creating the token there, so DevSpec can only ever see that team — and set an expiration so it auto-retires.

6. Backfilling Past Deployments

Webhooks are forward-only — they tell DevSpec about deploys that happen after you connect. To pull in your history as well, use Backfill.

When you connect (or edit) a target that has everything backfill needs, DevSpec kicks off a backfill automatically in the background — so you usually don't have to do anything. You can also run it on demand: on Settings → Deployment Targets, eligible targets show a history icon (↻); click it to import recent deployments from the provider's API.

Backfill needs an API token on the target plus the provider's scoping config:

• Coolify — the Instance Base URL + Application UUID (exactly what powers exact-commit tracking).
• Vercel — the Vercel Project ID (Settings → General → Project ID on the Vercel project), so it imports only that project's deployments.

Webhook-only targets (no token) can't backfill.

What to expect:

• It imports the most recent deployments (the latest ~50), newest first, stamped with their real deploy time so they land correctly on the timeline.
• It's deduplicated — anything DevSpec already has (from a live webhook or an earlier backfill) is left untouched, so it's safe to run more than once. The toast tells you how many were imported vs. already present.
• Imported deploys are linked to action items the same way live ones are (commit tag, branch, or commit reference).
• Backfill is silent: historical deploys never fire notifications, retries, escalations, or health changes — it only fills in the record. Deploys with no VCS commit (e.g. docker-image builds) are skipped.

Understanding Results

Deployment events are displayed on the action item detail page with status badges and classification details.

Status badges

• Green "succeeded" badge: the deployment completed successfully.
• Red "deployment_failed" badge: the deployment failed and was escalated to a human after exhausting retries or hitting a hard-stop.
• Yellow "retrying" indicator: a retry has been triggered and the action item is re-queued for Autopilot.

Error categories

• code — a bug in the pushed code (syntax error, type error, runtime crash). Retryable.
• dependency — a missing or incompatible dependency. Retryable.
• config — environment variable or configuration issue. Escalated.
• infra — platform outage or infrastructure problem. Escalated.
• test — test suite failure. Retryable.
• unknown — could not be classified. Escalated.

Retry behavior

Each deployment target has a configurable max retry count (default 2). When a retryable error is detected and the attempt count is below the max, the action item is re-queued with the error context so Autopilot can fix the issue. When retries are exhausted, the failure is escalated to the assigned developer via in-app notification.

Troubleshooting

Common issues and how to resolve them.

Webhook events not arriving in DevSpec

Your CI/CD platform is sending webhooks but DevSpec does not show any deployment events.

• Verify the webhook URL contains your correct DevSpec project ID.
• Check the webhook delivery logs in your CI/CD platform for HTTP errors.
• Ensure your DevSpec instance is publicly accessible (not behind a firewall).
• Confirm you selected the correct deployment events (succeeded and failed).

Build logs showing as "unavailable"

Deployment failures are recorded but the build log excerpt says "Logs unavailable".

• Check that an API token is configured for the deployment target in DevSpec.
• Verify the token has permission to read deployment logs on the CI/CD platform.
• Some platforms rate-limit log API calls — wait a few minutes and check again.

Retries not triggering

A deployment failed with a retryable error but no retry was triggered.

• Check if the max retry count has been reached (visible in the deployment event card).
• Hard-stop keywords (e.g. "out of memory", "disk quota exceeded") bypass retry logic entirely.
• Verify the deployment target has "in_autopilot_loop" enabled — retries only work when Autopilot is managing the action item.

Classification seems incorrect

Claude classified the error in the wrong category.

The classification uses Claude Sonnet to analyze build logs. Ambiguous errors (e.g. a missing env var that looks like a code error) can be misclassified. The full build log excerpt is available in the deployment event card for manual review. If the classification is wrong, the escalation path ensures a human reviews it.

Action item stuck in "deployment_failed" status

The action item shows deployment_failed but you want to retry manually.

• Update the action item status back to "open" or "queued" to allow Autopilot to pick it up again.
• Alternatively, fix the issue manually and push a new deployment — DevSpec will pick up the new webhook event.

Multiple deployment targets for one project

You have staging and production environments and want monitoring for both.

Each deployment target is configured independently with its own webhook URL, API token, and retry settings. Add a separate deployment target in DevSpec project settings for each environment. The webhook URL includes the target ID to route events correctly.

Webhook signature verification failing

The CI/CD platform reports successful delivery but DevSpec rejects the webhook.

• Check that the webhook secret configured in DevSpec matches the one in your CI/CD platform.
• Some platforms send a test event on creation — verify these are being accepted.
• Check server logs for signature mismatch errors.

Notifications not sent on escalation

A deployment was escalated but no notification was received.

• Verify the action item has an assigned user — notifications are sent to the assigned developer.
• Check notification preferences in user settings.
• Look at the Inngest dashboard for failed notification jobs.