[ROSAENG-435] osdctl cluster context should list PagerDuty incidents for HCP

[samanthajayasinghe]

Expected Behavior

When running osdctl cluster context --cluster-id for a HCP cluster, it should return relevant PagerDuty alerts.

Actual Behavior

When running it, it fails to find any incidents and returns none.

Steps to Reproduce:

Run osdctl cluster-context --cluster-id $SOME_HCP_ID

Note the lack of incidents in the output

Environment:

All

Additional Information

This is primarily due to the design of how Classic vs HCP clusters are routed to PD. For classic clusters, we create a PD service per-cluster, and the service name is inferred via the DNS name of the cluster in osdctl here. 

We should decouple this first, so it's more obvious, then figure out how to enumerate the list of service IDs in PD based on the clusters region. Once we have that, we can then list incidents for those service(s), and iterate over them, looking at the CustomDetails to map the Cluster ID.

[fullsend-ai-triage]

🤖 Finished Triage · ✅ Success · Started 5:47 AM UTC · Completed 5:51 AM UTC
Commit: 13f62d5 · View workflow run →

[fullsend-ai-triage]

Triage Summary

Severity: Medium | Category: Bug

Problem

The osdctl org context command returns zero PagerDuty incidents for HCP (Hosted Control Plane) clusters. The PD service lookup uses the cluster's DNS base domain, which only works for Classic clusters where each cluster has its own PD service. HCP clusters use region-based PD services, so the DNS query finds no matching services.

Root Cause

In cmd/org/context.go, the PD client is initialized with cluster.DNS().BaseDomain(), which GetPDServiceIDs() in pkg/provider/pagerduty/pagerduty.go uses to query PD via ListServicesWithContext(Query: baseDomain). This returns results for Classic clusters but not HCP clusters, which are organized differently in PagerDuty.

Recommended Fix

1. Detect whether the target cluster is HCP or Classic
2. For HCP clusters, enumerate PD service IDs based on the cluster's region instead of DNS base domain
3. Filter retrieved incidents by matching the Cluster ID in the incident's CustomDetails field
4. This likely requires extending the PDClient interface and NewPDClient factory to accept cluster type and region parameters

Proposed Test Case

// Test: HCP cluster PD incident lookup
// Setup: Mock PD client with region-based services containing
// incidents with CustomDetails mapping to the target HCP cluster ID
//
// Given: An HCP cluster 'test-hcp-123' in region 'us-east-1'
// And: A PD service for 'us-east-1' with mixed incidents
//
// Then: Only incidents with matching cluster_id in CustomDetails
//       should be returned for the HCP cluster
//
// Edge cases:
// - Classic clusters in the same org still use DNS-based lookup
// - HCP cluster with no matching incidents returns empty (no error)
// - Multiple HCP clusters in same region get correctly filtered incidents

---

Next steps:

• /fs-code — agent creates a PR to implement this issue
• /fs-code <your instruction> — agent implements with your specific guidance

[samanthajayasinghe]

/fs-code

[fullsend-ai-coder]

🤖 Finished Code · ✅ Success · Started 5:52 AM UTC · Completed 6:22 AM UTC
Commit: 13f62d5 · View workflow run →