Skip to content

Fix Prevent event-watcher JSON/HCL replacements from truncating files #7394 - #7409

Open
JbravoI wants to merge 9 commits into
pipe-cd:masterfrom
JbravoI:Fix_7394
Open

JbravoI wants to merge 9 commits into
pipe-cd:masterfrom
JbravoI:Fix_7394

Conversation

@JbravoI

@JbravoI JbravoI commented Sep 22, 2026

Copy link
Copy Markdown

What this PR does:

  • Prevents jsonField and HCLField event-watcher replacements from reaching a file write; they now return an explicit unsupported-format error.
  • Rejects jsonField and HCLField during event-watcher configuration validation.
  • Fixes replacement validation so any combination of multiple replacement fields is rejected.
  • Adds regression tests proving JSON/HCL replacement targets are not modified.
  • Updates the v1 configuration reference to list only the supported replacement formats: yamlField and regex.

Why we need it:

The event watcher previously accepted JSON/HCL replacement configuration but did not implement either path. When one was processed, the code wrote a nil byte slice to the target file, truncating it to zero bytes before committing the change. This change turns that destructive behavior into a clear validation or runtime error and documents the supported configuration accurately.

Which issue(s) this PR fixes:

Fixes #7394

Does this PR introduce a user-facing change?: Yes.

  • How are users affected by this change: Configurations using jsonField or HCLField now fail with an explicit unsupported-field error instead of creating a commit that empties the target file. yamlField and regex replacements are unaffected.
  • Is this breaking change: Yes, for configurations that use jsonField or HCLField. Those formats were documented but not implemented safely.
  • How to migrate (if breaking change): Replace jsonField/HCLField with a supported yamlField or regex replacement where practical. Otherwise, remove the event-watcher replacement until JSON/HCL support is implemented.

Screenshots/Videos (for documentation or website changes):

Not applicable. This documentation change removes unsupported configuration fields from a reference table and has no visual UI change.

Verification:

go test ./pkg/app/pipedv1/eventwatcher -run '^TestCommitFilesDoesNotTruncateUnsupportedReplacementFiles$' -count=1
go test ./pkg/configv1 -run '^TestEventWatcherValidate$' -count=1

Both targeted tests pass.

JbravoI and others added 4 commits September 19, 2026 17:31
Signed-off-by: Ewuji John <johnseyi51@gmail.com>
Signed-off-by: Ewuji John <johnseyi51@gmail.com>
Signed-off-by: Ewuji John <johnseyi51@gmail.com>
Copilot AI lite review requested due to automatic review settings September 22, 2026 17:54
@JbravoI
JbravoI requested review from a team as code owners September 22, 2026 17:54
@netlify

netlify Bot commented Sep 22, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for pipecd-site canceled.

Name Link
🔨 Latest commit 4c66cb2
🔍 Latest deploy log https://app.netlify.com/projects/pipecd-site/deploys/6ae489539ffd518372d1a905

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

No unresolved issues were identified, and all supplied assessments indicate approval readiness.

Review effort: Lite
Findings: None

What changed in this PR

This PR prevents unsupported JSON/HCL event-watcher replacements from truncating files by rejecting them during validation and before writes.

Changes:

  • Adds replacement validation and runtime safeguards.
  • Adds regression tests for invalid replacements and truncation.
  • Updates documentation and .gitignore.
File Summary
pkg/​configv1/​event_watcher.go Adds v1 replacement validation.
pkg/​configv1/​event_watcher_test.go Tests invalid replacement configurations.
pkg/​configv1/​application.go Validates application event watchers.
pkg/​configv1/​application_test.go Tests application-level validation.
pkg/​config/​event_watcher.go Adds legacy replacement validation.
pkg/​config/​application.go Validates legacy application event watchers.
pkg/​app/​pipedv1/​eventwatcher/​eventwatcher.go Prevents unsupported writes and reports failures.
pkg/​app/​pipedv1/​eventwatcher/​eventwatcher_test.go Tests files are not truncated.
pkg/​app/​piped/​eventwatcher/​eventwatcher.go Applies equivalent legacy safeguards.
pkg/​app/​piped/​eventwatcher/​eventwatcher_test.go Tests legacy safeguards.
docs/​content/​en/​docs-v1.0.x/​user-guide/​managing-application/​configuration-reference.md Documents supported replacement formats.
.gitignore Ignores WORKABLE.md.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@harshrajdebug harshrajdebug left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding the check to the application spec validation has a wider effect than the event watcher. I loaded the repro config from #7394 with LoadApplication on master and on this branch. On master it loads, and here it fails with "invalid event watcher handler config: replacement has an unsupported jsonField" as the error.

On that error the trigger reports the app as INVALID_CONFIG and skips it, so an app that only declares a jsonField replacement stops being deployed, even if no matching event has ever arrived. The event watcher also skips an app whose config fails to load, so for pipedv1 the new check in commitFiles is never reached and the event is never marked FAILURE.

If that's intended, it's worth stating in the user-facing section, since it's broader than the event watcher returning an error. If not, the validation at the top of commitFiles already stops the truncation on its own, before any file is written.

Separately, the .gitignore change adds WORKABLE.md, which looks like a local file.

Signed-off-by: Ewuji John <johnseyi51@gmail.com>
@JbravoI

JbravoI commented Sep 28, 2026

Copy link
Copy Markdown
Author

Hi @harshrajdebug

Kindly assist in reviewing, as I have made the necessary changes in the commit 0a3dd44

@harshrajdebug harshrajdebug left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, 0a3dd44 covers both points. The #7394 config loads again with LoadApplication on this head (on 9474ff9 it failed with the jsonField error), and when I remove the checks in commitFiles (the validation at the top and the jsonField and HCLField returns), the new truncation test fails, so those checks are what keep the file intact.

The same rejection is still in EventWatcherEvent.Validate in both config packages, which runs when LoadEventWatcher reads the deprecated event watcher files under .pipe/. I loaded a .pipe/ file with one yamlField event and one jsonField event. On master both events load. On this branch the whole file fails with "event "image-update": replacement has an unsupported jsonField" as the error.

In both pipeds a LoadEventWatcher error is logged and the watcher moves on to the next repository. For that repository, the yamlField event in the same file and the eventWatcher entries in its application configs are skipped too, on every run until the file is changed. Those .pipe/ events already reach commitFiles through updateValues, where the new check stops the truncation and skips only the event with the jsonField. Leaving the rejection out of EventWatcherEvent.Validate would keep the effect to that one event, as it now is for the application config.

JbravoI and others added 2 commits September 30, 2026 10:39
@JbravoI

JbravoI commented Sep 30, 2026

Copy link
Copy Markdown
Author

Hi @harshrajdebug

Kindly assist in reviewing, as I have made the necessary changes in the commit 188869d

@harshrajdebug harshrajdebug left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-checked at 188869d. The legacy loading point is fixed: EventWatcherEvent.Validate in both config packages matches master again, so a .pipe/ file with a jsonField event loads as before, and the tests in both event watcher packages and both config packages pass.

Two smaller things on this head.

  1. EventWatcherConfig.Validate and EventWatcherHandlerConfig.Validate have no callers since the application spec stopped calling them in 0a3dd44, so they can be removed from both config packages.

  2. The new check at the top of commitFiles counts regex as a field, so a replacement with both yamlField and regex now fails with "replacement has multiple fields". I confirmed that with a quick probe: EventWatcherEvent.Validate accepts that replacement, as it does on master, and EventWatcherReplacement.Validate rejects it. On master commitFiles applies the yamlField and ignores the regex, so a config with both goes from working to failing every event. If rejecting that pair is intended, a line in the user-facing note would help users who hit it. If not, counting regex separately from the three field types keeps the current behaviour.

@JbravoI

JbravoI commented Oct 30, 2026

Copy link
Copy Markdown
Author

Hi @harshrajdebug

The error has been fixed in commit 9987a3e

@mohammedfirdouss

Copy link
Copy Markdown
Contributor

sign off commits @JbravoI

Signed-off-by: Ewuji John <johnseyi51@gmail.com>
@JbravoI

JbravoI commented Oct 30, 2026

Copy link
Copy Markdown
Author

@harshrajdebug Commits now signed off.

@codecov

codecov Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 25.00000% with 30 lines in your changes missing coverage. Please review.
✅ Project coverage is 30.95%. Comparing base (ff572e0) to head (fd729bb).
⚠️ Report is 78 commits behind head on master.

Files with missing lines Patch % Lines
pkg/config/event_watcher.go 20.00% 4 Missing and 4 partials ⚠️
pkg/configv1/event_watcher.go 20.00% 4 Missing and 4 partials ⚠️
pkg/app/piped/eventwatcher/eventwatcher.go 30.00% 7 Missing ⚠️
pkg/app/pipedv1/eventwatcher/eventwatcher.go 30.00% 7 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7409      +/-   ##
==========================================
+ Coverage   30.35%   30.95%   +0.59%     
==========================================
  Files         601      602       +1     
  Lines       52620    52900     +280     
==========================================
+ Hits        15975    16377     +402     
+ Misses      35123    34972     -151     
- Partials     1522     1551      +29     
Flag Coverage Δ
. 24.83% <25.00%> (+0.49%) ⬆️
.-pkg-app-pipedv1-plugin-analysis 29.19% <ø> (ø)
.-pkg-app-pipedv1-plugin-ecs 34.97% <ø> (+1.21%) ⬆️
.-pkg-app-pipedv1-plugin-kubernetes 60.19% <ø> (+0.25%) ⬆️
.-pkg-app-pipedv1-plugin-kubernetes_multicluster 62.25% <ø> (+1.85%) ⬆️
.-pkg-app-pipedv1-plugin-scriptrun 58.22% <ø> (ø)
.-pkg-app-pipedv1-plugin-terraform 45.66% <ø> (+5.77%) ⬆️
.-pkg-app-pipedv1-plugin-wait 32.22% <ø> (ø)
.-pkg-app-pipedv1-plugin-waitapproval 55.14% <ø> (ø)
.-pkg-plugin-sdk 52.07% <ø> (ø)
.-tool-actions-gh-release 20.00% <ø> (ø)
.-tool-actions-plan-preview 27.91% <ø> (ø)
.-tool-codegen-protoc-gen-auth 0.00% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Signed-off-by: Ewuji John <johnseyi51@gmail.com>

@harshrajdebug harshrajdebug left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-checked at 4c66cb2. Both points from the last round are covered: the uncalled config-level Validate methods are gone (EventWatcherHandlerConfig.Validate in both config packages, plus EventWatcherConfig.Validate in configv1), and EventWatcherReplacement.Validate no longer counts regex as a field, so a replacement with both yamlField and regex passes the check at the top of commitFiles and gets the yamlField applied, as on master. The new TestEventWatcherReplacementValidate cases cover that pair, and the config and event watcher tests in both pipeds pass on this head. Looks good to me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Prevent event-watcher JSON/HCL replacements from truncating files

4 participants