Repository navigation
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Write the documentation asked for by CCI-MOC/MOC-issues#339 ("Document how to create alerts on either metric values or log contents"; completion criteria: documentation written covering alert creation for both metrics and logs). It is a sub-issue of the epic CCI-MOC/MOC-issues#325 (OAC: Monitoring and logging).
"starte das Doku-Dokument für #339"
It is one document in ops-docs docs/, not an ADR. All work and PRs only on the fork schwesig/ops-docs; never on CCI-MOC (no PRs, comments, issues or links there).
What Changed
docs/2026-10-20-alerting-howto/README.md, a how-to for OAC operators. It starts with a table that maps each kind of metric alert to its rule object and to where that rule is evaluated: anAlertingRulefor a platform metric on one cluster, aPrometheusRulefor a project metric on a workload cluster, and the hub'sthanos-ruler-custom-rulesConfigMap for a metric across the fleet. For each path it gives a minimal YAML example, where the rule should live as code and how to check that the alert fires.AlertingRuleobjects that use LogQL: how to enable the LokiStack ruler, how to write a rule and how to check it. A warning explains that log alerts cannot run on OAC yet, becausemaindeploys no logging stack and the open logging change has nospec.rulessection.templateEngineOverride: mdso the Prometheus{{ }}templates are not parsed by Liquid.🤖 Generated with Claude Code
Risk Assessment
✅ Low: The change adds a single documentation page. Both round-1 fixes are in place as the user instructed: the leaf-prometheus note at lines 195-198 keeps the ACM 2.17 link and adds no new quotes, and the API group is qualified at line 203. No new defects were found.
Testing
I built the real ops-docs Eleventy site from this branch into the evidence directory, served it on a local port and took a full-page screenshot of the new doc with headless Firefox. Frontmatter validation passes. The page renders the metric section (platform AlertingRule, project PrometheusRule, fleet-wide rule on the hub, how to check it fires) and the log section (enable the ruler, LokiStack AlertingRule, how to check it fires). The IMPORTANT and WARNING callouts render. The Prometheus
{{ }}template comes through intact because the page skips Liquid. The page has no dead in-page anchors. Both fixes from round 1 show up in the rendered output. Offline, all YAML examples parse into the expected kinds and rules. That is not a live check, so the YAML scenario stays untested. Promtool, oc and a cluster are not available, so the PromQL and LogQL expressions were not run against a live Prometheus or Loki.oc get alertingrule.monitoring.openshift.io -n openshift-monitoringEvidence: Rendered HTML page from the Eleventy build
Evidence: YAML examples parsed into rule objects
AlertingRule monitoring.openshift.io/v1 OACNodeRootFilesystemAlmostFull; PrometheusRule monitoring.coreos.com/v1 WebDeploymentUnavailable; ConfigMap thanos-ruler-custom-rules -> OACClusterNotSendingMetrics; LokiStack openshift-logging/logging-loki; AlertingRule loki.grafana.com/v1 WebLogsManyErrorsPipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
🔧 **Review** - 2 issues found → auto-fixed ✅
docs/2026-10-20-alerting-howto/README.md:184- The fleet section tells readers to addopenshift.io/prometheus-rule-evaluation-scope: leaf-prometheusto a projectPrometheusRuleso its alerts get forwarded to the hub. It does not say that with this label the rule is evaluated only by the user-workload Prometheus, which cannot query platform metrics. Here is how that goes wrong: a reader adds the label to the doc's own project example (kube_deployment_status_replicas_available{deployment="web"} == 0, line 126). That is a kube-state-metrics series scraped by platform Prometheus, so the expression returns nothing and the alert never fires, with no error. Fix: add one sentence saying leaf-prometheus rules only see metrics scraped by user workload monitoring in that project, so the example rule would stop working with the label.docs/2026-10-20-alerting-howto/README.md:191- The check step usesoc get alertingrule -n openshift-monitoring. Once the Loki Operator is installed (workload clusters and the dev hub under the logging change),alertingruleexists in two API groups:monitoring.openshift.ioandloki.grafana.com. The short name may then resolve to the Loki CRD and list nothing, even though the platform rule exists. The log section (line 304) already qualifies its group (alertingrule.loki.grafana.com). Usealertingrules.monitoring.openshift.iohere so the two match.🔧 Fix applied.
✅ Re-checked - no issues remain.
🔧 **Test** - 1 issue found → auto-fixed ✅
docs/2026-10-20-alerting-howto/README.md:16- The IMPORTANT callout links to #where-alerts-end-up-today and #open-points. The ops-docs site renders Markdown with markdown-it plus github-alerts only, so headings get no id attributes, and on the published page these links do nothing (they work when viewed on GitHub). There are two ways to fix it: replace the links with plain section names in the doc, or add a heading-anchor plugin such as markdown-it-anchor to eleventy.config.js. The second is a site-wide change outside this docs-only scope.{{ $labels.instance }}verbatim; screenshot shows it in the AlertingRule examplegit archive HEADinto a throwaway temp dir, thennpm ci,npm run validate-frontmatter,npm run build(Eleventy 3.1.7)Inspected the built_site/docs/2026-10-20-alerting-howto/index.html: callouts render as markdown-alert-important and markdown-alert-warning, the{{ $labels.instance }}template is kept, no[text][ref]reference links are left unresolved, and checked whether in-page anchor targets existChecked that_site/docs/index.htmllists the new docParsed every ```yaml block with js-yaml (including the nested custom_rules.yaml in the ConfigMap) and printed apiVersion, kind, name, namespace, labels and rule expressionsRead the leaf-prometheus note and theoc get alertingrule.monitoring.openshift.iocheck step to confirm the recorded review fixesServed_sitewithpython3 -m http.serverand took headless Firefox screenshots (throwaway profile) of the doc page and the docs index🔧 Fix applied.
✅ Re-checked - no issues remain.
oc get alertingrule.monitoring.openshift.io -n openshift-monitoringnpm ci(inside the worktree; node_modules removed afterwards)npm run validate-frontmatternpx eleventy --output=<evidence>/site(real site build)Checked the built page: headings, callout classes, no in-pagehref="#..."links, literal{{ $labels.instance }}in the code block, API group of the alertingrule commands, leaf-prometheus sentenceServed the built site withpython3 -m http.server 18765and confirmed the page and its entry on /docs/ return 200firefox --headless --screenshotfull-page capture of the rendered doc (chrome-devtools-axi bridge failed, so Firefox was used instead)Parsed every fenced YAML block and the nested custom_rules.yaml with the repo's js-yaml✅ **Document** - passed
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.