[ievdokdm]

converting flutter.dev/go/flutter-pluggable-ci-design into rfc

Pre-launch Checklist

• I read RFC 000.0001: Taxonomy and followed the file and path naming and metadata standards.
• I read RFC 000.0002: Process and confirmed this proposal meets the threshold for a full RFC.
• I read and agree to the Code of Conduct.
• I signed the CLA.
• I have linked an issue from flutter/flutter with the label design doc.
• All existing and new tests are passing.
• I have enabled "Allow edits from maintainers" on this PR so the bot can automatically assign an RFC number (or I will run dart run bin/assign_rfc_number.dart locally when instructed).

If you need help, consider asking for advice on the #hackers channel on Discord.

[jtmcdole]

This is mostly LGTM from me with some nits / suggestions / questions.

[eyebrowsoffire]

Thanks for writing this up, Dmitry! The security model and the delivery/idempotency semantics are well thought out. I have a few higher-level questions:

1. Consider a simpler partner-pull, report-only model

The RFC expects a few hundred deliveries a day at most (§1.2), but the design needs outbound webhooks, two Pub/Sub topics with DLQs, SSRF protections, and endpoints that partners host. Most of that comes from Cocoon pushing to partners.

Could §5 also evaluate this simpler model?

• Partners watch flutter/flutter for new commits themselves. Most CI systems can do this out of the box.
• Cocoon exposes one authenticated endpoint where partners report results for a (sha, task_name).
• Cocoon still creates pending tasks on post-submit, as it already does.

Communication is one-way, and partners don't need public endpoints or anything GCP-specific. The tradeoffs (dashboard-triggered re-runs, correlating results without a uid) would be good to spell out. If push is still preferred, could the webhook at least be optional?

Related: partner keys are fetched from a partner-hosted keys_url, which is itself a public endpoint partners have to run. With only a handful of partners, could their public keys be stored directly in config instead?

2. Onboarding mechanics

How does a partner actually get added? Is the plan for someone to hand-edit partner_config in Firestore, or should that config live in source control so changes get reviewed? Also, how are partner/iss values assigned, and how do partners learn Cocoon's JWKS URL and iss?

3. Formal API specification

External parties will build against this contract, and we'll have to maintain it. I'd like a normative spec for each endpoint rather than examples (+1 to the earlier JSON schema suggestion). It should cover:

• method, path, and query params
• field types, required vs optional, and limits
• error response bodies
• versioning: there's currently no version in the path or payload, and §7 says the webhook fields aren't settled yet

4. Test result structure

As written, details[] is a free-form list the partner can change on every run, and only task_name is known up front. Is that intentional? If test names aren't stable or unique, we can't track per-test history or identify which commit broke a given test, which seems central to the goal in §1.1. If loosely typed is fine for v1, please say so explicitly in the RFC.

[ievdokdm]

Thanks for writing this up, Dmitry! The security model and the delivery/idempotency semantics are well thought out. I have a few higher-level questions:

1. Consider a simpler partner-pull, report-only model

The RFC expects a few hundred deliveries a day at most (§1.2), but the design needs outbound webhooks, two Pub/Sub topics with DLQs, SSRF protections, and endpoints that partners host. Most of that comes from Cocoon pushing to partners.

Could §5 also evaluate this simpler model?

• Partners watch flutter/flutter for new commits themselves. Most CI systems can do this out of the box.
• Cocoon exposes one authenticated endpoint where partners report results for a (sha, task_name).
• Cocoon still creates pending tasks on post-submit, as it already does.

Communication is one-way, and partners don't need public endpoints or anything GCP-specific. The tradeoffs (dashboard-triggered re-runs, correlating results without a uid) would be good to spell out. If push is still preferred, could the webhook at least be optional?

Related: partner keys are fetched from a partner-hosted keys_url, which is itself a public endpoint partners have to run. With only a handful of partners, could their public keys be stored directly in config instead?

2. Onboarding mechanics

How does a partner actually get added? Is the plan for someone to hand-edit partner_config in Firestore, or should that config live in source control so changes get reviewed? Also, how are partner/iss values assigned, and how do partners learn Cocoon's JWKS URL and iss?

3. Formal API specification

External parties will build against this contract, and we'll have to maintain it. I'd like a normative spec for each endpoint rather than examples (+1 to the earlier JSON schema suggestion). It should cover:

• method, path, and query params
• field types, required vs optional, and limits
• error response bodies
• versioning: there's currently no version in the path or payload, and §7 says the webhook fields aren't settled yet

4. Test result structure

As written, details[] is a free-form list the partner can change on every run, and only task_name is known up front. Is that intentional? If test names aren't stable or unique, we can't track per-test history or identify which commit broke a given test, which seems central to the goal in §1.1. If loosely typed is fine for v1, please say so explicitly in the RFC.

Thanks for the feedback, bellow the answers on your suggestions and questions:

1. Added 5.3. Alternative 3: Direct GitHub Trigger with Cocoon Inbound Callback section
2. At this stage we plan to manually add partner info into partner_config collection. In inbound JWT Payload iss should be equal to partner_config.partner. We plan to make an agreement about that with partner on a integration stage.
3. Added versions into API's
4. On UI we only going to show the task_name. Free form of details is mostly for triaging the issues. to show more granular test cases/devices types test execution details/logs.

[github-actions]

Assigned RFC 610.0002.