Repository navigation
fix(cloudflare): Capture errors that fail a workflow run outside a step callback #26316
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
JPeer264
wants to merge
1
commit into
develop
Choose a base branch
from
jp/cloudflare-workflow-capture-run-errors
base: develop
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+461
−4
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
105 changes: 105 additions & 0 deletions
105
dev-packages/cloudflare-integration-tests/suites/workflows/run-errors/index.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,105 @@ | ||
| import * as Sentry from '@sentry/cloudflare'; | ||
| import { WorkflowEntrypoint } from 'cloudflare:workers'; | ||
| import type { WorkflowEvent, WorkflowStep } from 'cloudflare:workers'; | ||
| import { NonRetryableError } from 'cloudflare:workflows'; | ||
|
|
||
| interface Env { | ||
| SENTRY_DSN: string; | ||
| RUN_ERRORS_WORKFLOW: Workflow; | ||
| } | ||
|
|
||
| interface WorkflowParams { | ||
| mode: string; | ||
| } | ||
|
|
||
| export class RunErrorsTestWorkflow extends WorkflowEntrypoint<Env, WorkflowParams> { | ||
| async run(event: WorkflowEvent<WorkflowParams>, step: WorkflowStep): Promise<void> { | ||
| const { mode } = event.payload; | ||
|
|
||
| if (mode === 'throw-before-step') { | ||
| throw new TypeError('Thrown before any step'); | ||
| } | ||
|
|
||
| if (mode === 'non-retryable-before-step') { | ||
| throw new NonRetryableError('Non-retryable before any step'); | ||
| } | ||
|
|
||
| if (mode === 'throw-after-step') { | ||
| await step.do('successful-step', async () => 'done'); | ||
| throw new Error('Thrown after a successful step'); | ||
| } | ||
|
|
||
| if (mode === 'wait-for-event-timeout') { | ||
| await step.waitForEvent('wait-for-event', { type: 'never-sent', timeout: '1 second' }); | ||
| } | ||
|
|
||
| if (mode === 'step-timeout') { | ||
| await step.do('slow-step', { timeout: '1 second', retries: { limit: 0, delay: 100 } }, async () => { | ||
| await new Promise(resolve => setTimeout(resolve, 5000)); | ||
| }); | ||
| } | ||
|
|
||
| if (mode === 'non-serializable') { | ||
| await step.do( | ||
| 'non-serializable-step', | ||
| { retries: { limit: 0, delay: 100 } }, | ||
| (async () => () => 'function') as () => Promise<never>, | ||
| ); | ||
| } | ||
|
|
||
| if (mode === 'throw-in-step') { | ||
| await step.do('failing-step', { retries: { limit: 0, delay: 100 } }, async () => { | ||
| throw new Error('Thrown inside the step callback'); | ||
| }); | ||
| } | ||
|
|
||
| if (mode === 'caught-step-error') { | ||
| try { | ||
| await step.do('failing-step', { retries: { limit: 0, delay: 100 } }, async () => { | ||
| throw new Error('Thrown inside the step callback'); | ||
| }); | ||
| } catch { | ||
| // The workflow goes on after the failed step | ||
| } | ||
| } | ||
|
|
||
| if (mode === 'rethrow-new-error') { | ||
| try { | ||
| await step.do('failing-step', { retries: { limit: 0, delay: 100 } }, async () => { | ||
| throw new Error('Thrown inside the step callback'); | ||
| }); | ||
| } catch { | ||
| throw new Error('Thrown by run() after the failed step'); | ||
| } | ||
| } | ||
| } | ||
| } | ||
|
|
||
| export default { | ||
| async fetch(request, env, _ctx) { | ||
| const url = new URL(request.url); | ||
|
|
||
| if (url.pathname === '/trigger-workflow') { | ||
| const instance = await env.RUN_ERRORS_WORKFLOW.create({ | ||
| params: { mode: url.searchParams.get('mode') || '' }, | ||
| }); | ||
|
|
||
| return Response.json({ id: instance.id }); | ||
| } | ||
|
|
||
| if (url.pathname === '/flush-marker') { | ||
| Sentry.captureMessage('flush-marker'); | ||
| return Response.json({ ok: true }); | ||
| } | ||
|
|
||
| const statusMatch = url.pathname.match(/^\/workflow-status\/(.+)$/); | ||
|
|
||
| if (statusMatch) { | ||
| const instance = await env.RUN_ERRORS_WORKFLOW.get(statusMatch[1]!); | ||
|
|
||
| return Response.json({ id: statusMatch[1], status: await instance.status() }); | ||
| } | ||
|
|
||
| return new Response('Not found', { status: 404 }); | ||
| }, | ||
| } satisfies ExportedHandler<Env>; |
5 changes: 5 additions & 0 deletions
5
dev-packages/cloudflare-integration-tests/suites/workflows/run-errors/instrument.server.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| import { defineCloudflareOptions } from '@sentry/cloudflare'; | ||
|
|
||
| export default defineCloudflareOptions((env: { SENTRY_DSN: string }) => ({ | ||
| dsn: env.SENTRY_DSN, | ||
| })); |
161 changes: 161 additions & 0 deletions
161
dev-packages/cloudflare-integration-tests/suites/workflows/run-errors/test.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,161 @@ | ||
| import type { Envelope } from '@sentry/core'; | ||
| import { expect, it } from 'vitest'; | ||
| import { createRunner } from '../../../runner'; | ||
|
|
||
| interface TriggerResponse { | ||
| id: string; | ||
| } | ||
|
|
||
| interface StatusResponse { | ||
| id: string; | ||
| status: { status: string }; | ||
| } | ||
|
|
||
| interface SentException { | ||
| type?: string; | ||
| value?: string; | ||
| mechanism?: { handled?: boolean; type?: string }; | ||
| } | ||
|
|
||
| async function waitForWorkflowStatus( | ||
| makeRequest: <T>(method: 'get' | 'post', path: string) => Promise<T | undefined>, | ||
| workflowId: string, | ||
| ): Promise<StatusResponse | undefined> { | ||
| for (let i = 0; i < 30; i++) { | ||
| const status = await makeRequest<StatusResponse>('get', `/workflow-status/${workflowId}`); | ||
| if (status?.status?.status === 'errored' || status?.status?.status === 'complete') { | ||
| return status; | ||
| } | ||
| await new Promise(r => setTimeout(r, 200)); | ||
| } | ||
| return undefined; | ||
| } | ||
|
|
||
| // The flush marker is sent after the instance has ended and must arrive directly after the expected | ||
| // error event. A second error event for the same failure would arrive in its place and fail the test. | ||
| const flushMarkerMatcher = (envelope: Envelope): void => { | ||
| const [, items] = envelope; | ||
| const [itemHeader, itemBody] = items[0] as [{ type: string }, Record<string, unknown>]; | ||
|
|
||
| expect(itemHeader.type).toBe('event'); | ||
| expect(itemBody.message).toBe('flush-marker'); | ||
| }; | ||
|
|
||
| it.for([ | ||
| { mode: 'throw-before-step', type: 'TypeError', value: 'Thrown before any step' }, | ||
| { mode: 'non-retryable-before-step', type: 'NonRetryableError', value: 'Non-retryable before any step' }, | ||
| { mode: 'throw-after-step', type: 'Error', value: 'Thrown after a successful step' }, | ||
| { mode: 'wait-for-event-timeout', type: 'Error', value: expect.stringContaining('timed out') }, | ||
| { mode: 'step-timeout', type: 'Error', value: expect.stringContaining('timed out') }, | ||
| { mode: 'non-serializable', type: 'Error', value: expect.stringContaining('not serialisable') }, | ||
| ])( | ||
| 'captures one unhandled error when run() fails outside a step callback ($mode)', | ||
| async ({ mode, type, value }, { signal }) => { | ||
| const runner = createRunner(__dirname) | ||
| .expect((envelope: Envelope): void => { | ||
| const [, items] = envelope; | ||
| const [itemHeader, itemBody] = items[0] as [{ type: string }, { exception?: { values?: SentException[] } }]; | ||
|
|
||
| expect(itemHeader.type).toBe('event'); | ||
| expect(itemBody.exception?.values?.[0]).toMatchObject({ | ||
| type, | ||
| value, | ||
| mechanism: { handled: false, type: 'auto.faas.cloudflare.workflow' }, | ||
| }); | ||
| }) | ||
| .expect(flushMarkerMatcher) | ||
| .start(signal); | ||
|
|
||
| const trigger = await runner.makeRequest<TriggerResponse>('get', `/trigger-workflow?mode=${mode}`); | ||
| expect(trigger?.id).toBeDefined(); | ||
|
|
||
| const status = await waitForWorkflowStatus(runner.makeRequest.bind(runner), trigger!.id); | ||
| expect(status?.status?.status).toBe('errored'); | ||
|
|
||
| await runner.makeRequest('get', '/flush-marker'); | ||
| await runner.completed(); | ||
| }, | ||
| ); | ||
|
|
||
| it('captures an error thrown inside a step callback only one time', async ({ signal }) => { | ||
| const runner = createRunner(__dirname) | ||
| .expect((envelope: Envelope): void => { | ||
| const [, items] = envelope; | ||
| const [itemHeader, itemBody] = items[0] as [{ type: string }, { exception?: { values?: SentException[] } }]; | ||
|
|
||
| expect(itemHeader.type).toBe('event'); | ||
| expect(itemBody.exception?.values?.[0]).toMatchObject({ | ||
| type: 'Error', | ||
| value: 'Thrown inside the step callback', | ||
| mechanism: { handled: true, type: 'auto.faas.cloudflare.workflow' }, | ||
| }); | ||
| }) | ||
| .expect(flushMarkerMatcher) | ||
| .start(signal); | ||
|
|
||
| const trigger = await runner.makeRequest<TriggerResponse>('get', '/trigger-workflow?mode=throw-in-step'); | ||
| expect(trigger?.id).toBeDefined(); | ||
|
|
||
| const status = await waitForWorkflowStatus(runner.makeRequest.bind(runner), trigger!.id); | ||
| expect(status?.status?.status).toBe('errored'); | ||
|
|
||
| await runner.makeRequest('get', '/flush-marker'); | ||
| await runner.completed(); | ||
| }); | ||
|
|
||
| it('captures no second error when run() catches a failed step', async ({ signal }) => { | ||
| const runner = createRunner(__dirname) | ||
| .expect((envelope: Envelope): void => { | ||
| const [, items] = envelope; | ||
| const [itemHeader, itemBody] = items[0] as [{ type: string }, { exception?: { values?: SentException[] } }]; | ||
|
|
||
| expect(itemHeader.type).toBe('event'); | ||
| expect(itemBody.exception?.values?.[0]?.value).toBe('Thrown inside the step callback'); | ||
| }) | ||
| .expect(flushMarkerMatcher) | ||
| .start(signal); | ||
|
|
||
| const trigger = await runner.makeRequest<TriggerResponse>('get', '/trigger-workflow?mode=caught-step-error'); | ||
| expect(trigger?.id).toBeDefined(); | ||
|
|
||
| const status = await waitForWorkflowStatus(runner.makeRequest.bind(runner), trigger!.id); | ||
| expect(status?.status?.status).toBe('complete'); | ||
|
|
||
| await runner.makeRequest('get', '/flush-marker'); | ||
| await runner.completed(); | ||
| }); | ||
|
|
||
| it('captures the step error and the new error when run() replaces a failed step error', async ({ signal }) => { | ||
| const runner = createRunner(__dirname) | ||
| .expect((envelope: Envelope): void => { | ||
| const [, items] = envelope; | ||
| const [itemHeader, itemBody] = items[0] as [{ type: string }, { exception?: { values?: SentException[] } }]; | ||
|
|
||
| expect(itemHeader.type).toBe('event'); | ||
| expect(itemBody.exception?.values?.[0]).toMatchObject({ | ||
| value: 'Thrown inside the step callback', | ||
| mechanism: { handled: true, type: 'auto.faas.cloudflare.workflow' }, | ||
| }); | ||
| }) | ||
| .expect((envelope: Envelope): void => { | ||
| const [, items] = envelope; | ||
| const [itemHeader, itemBody] = items[0] as [{ type: string }, { exception?: { values?: SentException[] } }]; | ||
|
|
||
| expect(itemHeader.type).toBe('event'); | ||
| expect(itemBody.exception?.values?.[0]).toMatchObject({ | ||
| value: 'Thrown by run() after the failed step', | ||
| mechanism: { handled: false, type: 'auto.faas.cloudflare.workflow' }, | ||
| }); | ||
| }) | ||
| .expect(flushMarkerMatcher) | ||
| .start(signal); | ||
|
|
||
| const trigger = await runner.makeRequest<TriggerResponse>('get', '/trigger-workflow?mode=rethrow-new-error'); | ||
| expect(trigger?.id).toBeDefined(); | ||
|
|
||
| const status = await waitForWorkflowStatus(runner.makeRequest.bind(runner), trigger!.id); | ||
| expect(status?.status?.status).toBe('errored'); | ||
|
|
||
| await runner.makeRequest('get', '/flush-marker'); | ||
| await runner.completed(); | ||
| }); |
7 changes: 7 additions & 0 deletions
7
dev-packages/cloudflare-integration-tests/suites/workflows/run-errors/vite.config.mts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| import { cloudflare } from '@cloudflare/vite-plugin'; | ||
| import { sentryCloudflareVitePlugin } from '@sentry/cloudflare/vite'; | ||
| import { defineConfig } from 'vite'; | ||
|
|
||
| export default defineConfig({ | ||
| plugins: [cloudflare(), sentryCloudflareVitePlugin()], | ||
| }); |
13 changes: 13 additions & 0 deletions
13
dev-packages/cloudflare-integration-tests/suites/workflows/run-errors/wrangler.jsonc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| { | ||
| "name": "workflow-run-errors-test", | ||
| "compatibility_date": "2026-05-01", | ||
| "main": "index.ts", | ||
| "compatibility_flags": ["nodejs_compat"], | ||
| "workflows": [ | ||
| { | ||
| "name": "run-errors-test-workflow", | ||
| "binding": "RUN_ERRORS_WORKFLOW", | ||
| "class_name": "RunErrorsTestWorkflow", | ||
| }, | ||
| ], | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do we need this capturedByCallback thing if we pass it through checkOrSetAlreadyCaught anyhow...?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ah wait, I think I am confused by the code. we ignore the return value here and just use this to mark this as captured, right? maybe worth it to adjust the comment a bit, e.g. "we use checkOrSetAlreadyCaught to ensure the error is marked as already sent" or something like this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll clarify