Trivy Security incident 2026-03-19 conclusion

[itaysk]

This is a continuation of #10425

Dear Trivy community,
The past few weeks have been stressful for all of us as the Trivy project, and by extension its users, were targeted in a cyber-attack. First, we want to apologize. Trivy has earned the trust of many around the world, and while the security value it provides is significant, we did not meet the standard expected from us. We focused on delivering the best product but failed to adequately protect its supply chain and to respond as effectively as we should have. Since then, we have been working around the clock to investigate and remediate the incident and as a result we were brief with communications. Now that the situation is stable, we want to share our findings, the steps we have taken, and the lessons learned.

Current status

We have completed an investigation with third part incident response experts, and comprehensive review with third-party auditors of the full security posture of Trivy and its supporting infrastructure. The actions below reflect partial list of both immediate remediation and longer-term hardening measures:

1. Credential Reset
We performed a full credential reset:

• ⁠Revoked all GitHub tokens across all Trivy repositories
• Revoked all registry tokens across distribution channels
• Ensure no new tokens were issued until existing access was fully revoked
• Migrate to GitHub Apps and fine-grained tokens (in progress)

What this means: 
All previously issued credentials are no longer valid. New access is heavily scrutinized.

2. Hardened Release Pipeline
We updated the release process to reduce the risk of unintended or unauthorized changes:

• Removed all vulnerable instances of "pull_request_target" workflow trigger
• Added pipeline linters to enforce secure CI/CD configuration
• Ensured GitHub Actions are pinned to specific commit SHAs

What this means: 
New releases are produced through a more secure pipeline.

3. Artifact Immutability and Integrity Controls
We applied measures to prevent the risk of future changes:

• ⁠Enforced immutable releases where supported (Docker Hub, GitHub Releases)
  • Introduced complementary artifact monitoring for unsupported systems (detailed in Monitoring section)
• Enforced git tag protection for artifact repositories
• Added SLSA provenance attestations to Trivy releases
  • This is in addition to existing Sigstore signing

What this means: 
New releases have stronger protection against post-publication changes, and users have the tools to verify the integrity of released artifacts.

4. Access Model Hardening
We updated the access model to reduce risk and improve control:

• Enforced SSO and IP allow-listing across the organization
• Require approval before running certain workflows
• Enforce signed commits (in progress)

What this means: 
Access is now more tightly controlled.

5. Continuous Monitoring and Response
We introduced additional monitoring and response capabilities:

• Enabled audit log monitoring across GitHub and registries
• Logs are continuously monitored by a dedicated team
• Introduced an artifact tracking system that monitors both historical and new release artifacts for anomalies. This system will be made public and described in a separate post.

What this means:
We now have improved visibility into the release pipeline and faster detection and response of unexpected changes.

Next steps
These changes are now part of our baseline, and we will continue to reevaluate and challenge our security posture and practices across the board.
We are gradually resuming normal operation now, starting with vuln-list trackers and trivy-db releases.

Lessons learned

This attack could be attributed to many different factors, but we have identified few major reasons that contributed to the large impact of the attack, and made the initial response inefficient:

1. Trivy heavily relied on GitHub Actions for supply chain infrastructure, which is a very powerful system but also error-prone and risky unless properly configured. Trivy and its related projects were not uniformly hardened using all available controls, which could have reduced the impact of this attack.
2. Trivy is managed as an independent open source project within the broader aquasecurity GitHub organization. The same GitHub organization hosts different projects, teams and assets. Highly privileged service accounts were used across the org without scope or isolation and without adhering to the principle of least privilege.
3. Due to the organizational complexity, the secret rotation process was too long and complex and therefore was not effective. We later saw that while we rotated credentials of a specific compromised user, that attacker was still present in the system with another user.
4. Aqua maintains several GitHub organizations. The attack began in the public repositories under aquasecurity organization and drew attention there, but later manifested in other organizations which were previously overlooked.

These lessons are being incorporated into our security and release practices.

Timeline of events

1. February 27 - Initial access and credential theft:
   1. The attacker exploited a workflow vulnerable through a "pull_request_target" invocation in aquasecurity/trivy.
   2. Exfiltrated organization-level and repo-level secrets.
   3. With newly gained credentials, the attacker continued to steal secrets from other repositories in the organization, and in other organizations accessible with the credentials.
2. February 27 - Malicious VSCode Extension:
   1. Attacker acted on aquasecurity/trivy-vscode-extension repository.
   2. Released a malicious version of the Trivy VSCode extension into the OpenVSX Marketplace.
   3. Manipulated a security advisory in the same repository.
3. February 28 – Initial Hackerbot Claw activity:
   1. Activity involving hackerbot-claw user was observed. This appears to be an automated penetration testing bot that scans GitHub for vulnerable projects. It created several pull requests to exploit the vulnerable workflow. The user agent and behavioral patterns of hackrbot-claw are different than the other events inspected.
4. February 28 – additional secret exfiltration:
   1. Attacker initiated secret dumping workflows again.
5. March 1, VSCode extension remediation:
   1. Malicious OpenVSX extention was removed from marketplace (and maintainer's credentials were revoked)
6. March 1 - Release deletion and Repo takeover:
   1. Attacker removed previous GitHub Releases of aquasecurity/trivy.
   2. Attacker renamed the repository aquasecurity/trivy to aquasecurity/private-trivy, made it private, and created a new empty public repo in its place.
7. March 1 - Remediation effort and attacker persistence:
   1. Revoked credentials of automation accounts which were observed compromised as part of the remediation effort.
   2. Attacker initiated secret dumping workflow with another compromised users on another repository.
   3. The following activity observed on March 19 is done with the same token observed here, indicating the attacker regained persistence despite initial revocation.
8. March 19 - Malicious trivy artifacts:
   1. At 17:47 UTC Attacker used stolen credentials to build a malicious version of Trivy v0.69.4 and distribute by triggering Trivy's release workflow in GitHub Actions.
   2. At 22:42 UTC, the compromised PAT was revoked as part of the remediation effort.
9. March 22 – adjacent organization compromise:
   1. Service account in another GitHub organization which shared secrets with aquasecurity organization was invoking secret dumping workflows on more repositories.
   2. This attack demonstrated additional service accounts which had not been previously considered compromised.
   3. Later that day stolen service account began modifying 43 repositories in aquasec-com and made them public with the title name containing "tpcp-docs". This indicated a fallback mechanism of secrets exfiltration process was invoked.
10. March 22 - Malicious Docker Hub releases:
    1. At 15:43 UTC and 16:34 UTC the attacker pushed malicious 0.69.5 and 0.69.6 tags to DockerHub (aquasec/trivy repository). This didn’t occur via GitHub Workflow as before, but by directly pushing to Docker Hub with leaked credential.
    2. On March 23, At 01:40 UTC removed all malicious tags from Docker Hub.
11. Following March 23, we begun extensive investigation and remediation process which is shared above. No further suspicious events has been observed since.

[anhthii]

thank you @itaysk and AquaSecurity for being transparent and sharing the lessson for the community. It's very valuable for any open source projects to learn from and not making the same mistakes.

[HSelfient]

You potentially missed a vulnerability or lesson learned...what are you doing to ensure you know what your AI agents are doing.

[PiRomant]

Immutable tags on Docker Hub?

[nikpivkin]

Hi @PiRomant !

We have enabled tag immutability for certain repositories on Docker Hub

⁠Enforced immutable releases where supported (Docker Hub, GitHub Releases)

[mycodedoesnotcompile2]

@itaysk Thanks for all these explanations.

What is your current level of confidence that everything the attackers touched in your environment has been cleaned ? Do you have still some twilight zones ?