Repository navigation
[Bug][azuredevops_go] YAML Syntax Error breaks Pipeline #8838
Description
Search before asking
- I had searched in the issues and found no similar issues.
What happened
The azuredevops_go plugin fails the entire DevLake pipeline when it encounters Azure DevOps builds created by YAML syntax errors. These builds are valid Azure DevOps build records but do not have a usable Timeline, causing timeline parsing to fail and abort the pipeline.
This behavior blocks data ingestion even though the underlying build failure is expected and should be handled gracefully.
What do you expect to happen
The Azure DevOps Go plugin should gracefully handle missing or invalid timelines, for example:
- Skip builds whose timeline response cannot be parsed
- Log a warning instead of failing the pipeline
- Mark the build as failed/incomplete and continue
- Treat timeline collection as best effort, not mandatory
At minimum, timeline parsing errors should be non‑fatal for the overall pipeline.
How to reproduce
Add an Azure DevOps Project to the connection's data scope that contains a pipeline with a YAML error that has been triggered at least once within the selected timeframe. Run the data collection. The data collection job will fail at ado:<scope name>.
This an error description based on the original logs, but refined using AI to strip confidential information and increase readability.
Error running DevLake pipeline
Root cause:
subtask collectApiTimelineRecords ended unexpectedly
Stack (compressed):
- server/services.runPipeline
- core/runner.RunPipeline
- core/runner.RunPluginTask
- core/runner.RunPluginSubTasks
- plugins/azuredevops_go/tasks.CollectRecords
- helpers/pluginhelper/api.(*ApiCollector).Execute
- helpers/pluginhelper/api.(*WorkerScheduler).WaitAsync
Combined errors:
{
error parsing response from <ORG>/<PROJECT>/_apis/build/builds/<BUILD_ID_1>/Timeline?api-version=7.1
=====================
error parsing response from <ORG>/<PROJECT>/_apis/build/builds/<BUILD_ID_2>/Timeline?api-version=7.1
=====================
error parsing response from <ORG>/<PROJECT>/_apis/build/builds/<BUILD_ID_3>/Timeline?api-version=7.1
}
Notes:
- All affected builds failed due to YAML syntax errors in pipeline definitions
- Timeline API responses are empty, malformed, or non-JSON
- Failure of a single timeline causes the entire pipeline to abort
Anything else
No response
Version
v1.0.3-beta10@99376a8
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
Activity
dosubot commentedon Apr 16, 2026
Hi @techdecline! I'm Dosu and I’m helping the incubator-devlake team.
Thanks for the detailed report. I dug into the code and can confirm the root cause and suggest where a fix should go.
Root cause: The
ParseRawMessageFromRecordsresponse parser callsUnmarshalResponse, which returns a hard error on both empty responses (ErrEmptyResponse) and malformed/non-JSON bodies. These errors propagate up through theApiCollectorand abort the entire subtask.The plugin already has a pattern for gracefully skipping problematic builds —
ignoreDeletedBuildsusesapi.ErrIgnoreAndContinuein theAfterResponsehandler to skip 404s. However, when a YAML-broken build returns a 200 with an empty or non-JSON timeline body, theAfterResponsecheck passes (it's not a 404), and the error only surfaces later during response parsing where there's no ignore/skip mechanism.Suggested fix approach: There are a couple of options:
Expand
AfterResponse— Check for empty or non-JSON response bodies inignoreDeletedBuilds(or a new wrapper) and returnapi.ErrIgnoreAndContinuefor those cases too. This is the minimal change.Make
ParseRawMessageFromRecordstolerant — Return an empty[]json.RawMessageinstead of an error when the response is empty or cannot be parsed as JSON, with a warning log. This treats missing timelines as "no records" rather than a fatal error.Also worth noting: the collector query on line 52 already filters out
result != "failed"builds, but builds broken by YAML errors may still have a non-failedresult (e.g.,none) depending on how Azure DevOps categorizes them, so they slip through.Since you've indicated you're willing to submit a PR — option 1 is probably the cleanest and most consistent with the existing codebase patterns. You could also consider adding a filter for builds with
result = "none"at the query level if those are consistently the problematic ones.To reply, just mention @dosu.
Share context across your team and agents. Try Dosu.
fix(azuredevops_go): handle empty/invalid timeline responses for YAML…
techdecline commentedon Apr 16, 2026
Hey @dosu, Just created a Pull Request based on your and my own analysis.
dosubot commentedon Apr 16, 2026
Nice, thanks for jumping on that so quickly! If you link the PR here, I'd be happy to take a look at the implementation and provide feedback.
To reply, just mention @dosu.
Share context across your team and agents. Try Dosu.
techdecline commentedon Apr 16, 2026
There you go @dosu #8839
dosubot commentedon Apr 16, 2026
Thanks for sharing! I took a look at the PR — the approach looks solid. You implemented both fixes we discussed:
ignoreInvalidTimelineResponsehandler — replacesignoreDeletedBuildsand now also skips empty/non-JSON bodies viaapi.ErrIgnoreAndContinue, with the body peek-and-restore viaio.NopCloserso valid responses still flow through to the parser. Clean solution.result = 'none'builds in addition to'failed', which avoids even hitting the Timeline API for YAML-broken builds in the first place.Good to see unit tests covering the key scenarios (404, empty body, non-JSON body, valid JSON passthrough, and valid empty JSON object). The defense-in-depth approach here is nice — the query filter prevents most of the bad requests, and the
AfterResponsehandler catches anything that slips through.Looks good to me overall. Hope it gets picked up by the maintainers soon!
To reply, just mention @dosu.
Share context across your team and agents. Try Dosu.
fix(azuredevops_go): handle empty/invalid timeline responses for YAML… (
Sync upstream Apache DevLake (v1.0.3-beta9 → beta15) (#22)