Skip to content

osdctl rhobs logs: incorrect Grafana URL for HCP clusters #932

Description

@aliceh

Summary

osdctl rhobs logs -C <hcp-cluster-id> --url generates an incorrect Grafana URL for HCP (Hosted Control Plane) clusters. The URL shows no logs when opened in a browser.

Root Cause

When given an HCP cluster ID, the command:

  1. Correctly resolves the RHOBS cell by looking up the management cluster (MC)
  2. But incorrectly uses the HCP cluster's openshift_cluster_id (external UUID) as the Loki filter

HCP control-plane logs run in a namespace on the MC (e.g. ocm-production-<hcp-id>-...) and are indexed in RHOBS under the MC's openshift_cluster_id, not the HCP cluster's.

The generated query is:

{k8s_namespace_name="default"} | openshift_cluster_id = "<hcp-external-uuid>"

This returns no logs because:

  • k8s_namespace_name="default" is not where HCP control-plane logs live
  • openshift_cluster_id = "<hcp-external-uuid>" is the wrong cluster label for RHOBS HCP logs

Expected Behavior

For an HCP cluster, the URL should use:

  • Namespace: the HCP control-plane namespace on the MC, e.g. ocm-production-<hcp-id>-<suffix> (available via GetHCPNamespace)
  • openshift_cluster_id: the MC's external UUID (available via GetManagementCluster(...).ExternalID())

Working Loki query (confirmed by user investigation):

{k8s_namespace_name="ocm-production-<hcp-id>-<suffix>"} | openshift_cluster_id = "<mc-external-uuid>"

Steps to Reproduce

osdctl rhobs logs -C <hcp-cluster-id> --url

Open the generated URL — shows "No logs found".

Compare with:

osdctl rhobs logs -C <mc-cluster-id> --url

Then manually set the namespace to the HCP namespace — shows logs correctly.

Affected Code

cmd/rhobs/logs_cmd.go — the GetDefaultGrafanaLogsUrl() method and the namespace flag default value ("default").

When populateRhobsFetchers is called with an HCP cluster ID, it sets monitoredClusterId to the MC's internal ID (correct for RHOBS cell resolution) but leaves clusterExternalId as the HCP cluster's external UUID (wrong for the Loki query).

Fix

In populateRhobsFetchers (or in the logs URL generation path), when the cluster is an HCP cluster:

  1. Fetch the HCP namespace via utils.GetHCPNamespace(clusterId)
  2. Fetch the MC's external UUID via utils.GetManagementCluster(clusterId).ExternalID()
  3. Use these to build the correct Loki stream selector and filter

Note: osdctl cluster context has already been patched with this workaround separately.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions