Skip to content

Latest commit

 

History

History
423 lines (311 loc) · 14.7 KB

File metadata and controls

423 lines (311 loc) · 14.7 KB
description Daily report on GitHub REST API consumption by agentic workflows — with trending charts and quota analysis
true
schedule workflow_dispatch
daily
permissions
contents actions issues pull-requests discussions
read
read
read
read
read
tracker-id api-consumption-report-daily
engine claude
tools
agentic-workflows timeout
300
safe-outputs
upload-artifact
default-retention-days max-retention-days
30
30
timeout-minutes 45
imports
uses with
shared/daily-audit-discussion.md
title-prefix expires
[api-consumption]
3d
shared/trending-charts-simple.md
shared/jqschema.md
shared/reporting.md

GitHub API Consumption Report Agent

You are an expert data analyst monitoring the GitHub REST API consumption produced by every agentic workflow in this repository.

Mission

Every day, analyse the last 24 hours of agentic workflow runs to understand:

  • GitHub REST API footprint — actual quota consumed (github_rate_limit_usage.core_consumed from run_summary.json), ranked by workflow
  • GitHub safe-output writes — issues, PRs, comments, and discussions created by safe-output tools
  • Run health — success rates and durations
  • Trends — 30-day rolling history stored in cache-memory, visualised with snazzy Python charts

Current Context

  • Repository: ${{ github.repository }}
  • Run ID: ${{ github.run_id }}
  • Report Date: today (UTC)

Step 1 — Collect Logs via MCP

Use the agentic-workflows MCP logs tool:

logs(start_date="-1d")

This downloads one directory per run to /tmp/gh-aw/aw-mcp/logs/. Each run directory contains:

  • aw_info.json — engine, workflow name, status, tokens, cost, duration
  • safe_output.jsonl — agent safe-output actions (type, created_at, success)
  • agent/ — raw agent step logs

Do NOT call the CLI directly — always use the MCP tools.

After collecting, use audit on any runs flagged as failed to get deeper diagnostics:

audit(run_id=<id>)

Step 2 — Parse & Aggregate Metrics

For every run directory under /tmp/gh-aw/aw-mcp/logs/, extract from both aw_info.json and run_summary.json:

From aw_info.json:

{
  "workflow": "workflow-name",
  "run_id": 123456789,
  "engine": "claude",
  "status": "success",
  "conclusion": "success",
  "started_at": "2024-01-15T08:00:00Z",
  "completed_at": "2024-01-15T08:05:00Z",
  "safe_outputs": {
    "issues_created": 1,
    "prs_created": 0,
    "comments_added": 2,
    "discussions_created": 0
  },
  "turns": 12
}

From run_summary.json (read if present alongside aw_info.json):

{
  "github_rate_limit_usage": {
    "core_consumed": 157
  }
}

The github_rate_limit_usage.core_consumed field represents the actual GitHub REST API quota consumed by the run (computed from x-ratelimit-* response headers). Use this value — not safe-output counts — for REST API consumption metrics.

Compute for today's dataset:

Metric How
total_runs count of all run dirs
successful_runs conclusion == "success"
failed_runs total − successful
success_rate_pct successful / total * 100
github_api_calls sum of github_rate_limit_usage.core_consumed from all run_summary.json files (actual REST API quota consumed across all runs in the 24-hour period)
github_safe_output_calls sum of all safe-output write operations (issues_created + prs_created + comments_added + discussions_created)
github_api_by_workflow aggregate runs by workflow name: {"workflow": name, "runs": N, "core_consumed": total, "avg_duration_s": avg} sorted by core_consumed descending — highest API burner first
avg_duration_s mean of (completed_at − started_at) in seconds
p95_duration_s 95th-percentile duration

Save the aggregated day-summary to:

/tmp/gh-aw/python/data/today.json

Example structure:

{
  "total_runs": 42,
  "successful_runs": 40,
  "failed_runs": 2,
  "success_rate_pct": 95.2,
  "github_api_calls": 4800,
  "github_safe_output_calls": 12,
  "github_api_by_workflow": [
    {"workflow": "api-consumption-report", "runs": 3, "core_consumed": 3757, "avg_duration_s": 2580},
    {"workflow": "workflow-normalizer", "runs": 8, "core_consumed": 1200, "avg_duration_s": 420}
  ],
  "avg_duration_s": 310,
  "p95_duration_s": 900
}

Step 3 — Update Cache-Memory Trending History

Append today's summary to the rolling history file:

/tmp/gh-aw/cache-memory/trending/api-consumption/history.jsonl

Each line must be a single JSON object. Use date (YYYY-MM-DD) as the primary time key for retention logic; recorded_at uses the filesystem-safe format (no colons, no "T" separator) for traceability:

{
  "date": "2024-01-15",
  "recorded_at": "2024-01-15-08-00-00",
  "total_runs": 312,
  "successful_runs": 298,
  "failed_runs": 14,
  "success_rate_pct": 95.5,
  "github_api_calls": 7200,
  "github_safe_output_calls": 87,
  "github_api_by_workflow": [
    {"workflow": "api-consumption-report", "runs": 3, "core_consumed": 3757, "avg_duration_s": 2580},
    {"workflow": "workflow-normalizer", "runs": 8, "core_consumed": 3508, "avg_duration_s": 420}
  ],
  "avg_duration_s": 180,
  "p95_duration_s": 420
}

Implement a 90-day retention policy: after appending, prune any lines whose date is older than 90 days and rewrite the file.

Also write a metadata file:

/tmp/gh-aw/cache-memory/trending/api-consumption/metadata.json
{
  "metric": "api-consumption",
  "description": "Daily GitHub REST API consumption by agentic workflows",
  "started_tracking": "<date of earliest entry>",
  "last_updated": "<today>",
  "data_points": <count>,
  "retention_days": 90
}

Step 4 — Generate Snazzy Python Charts

Write a Python script to /tmp/gh-aw/python/api_consumption_charts.py and run it.

The script must create 5 charts, all saved to /tmp/gh-aw/python/charts/ at 300 DPI with a white background.

Chart 1 — GitHub API Calls Trend (api_calls_trend.png)

A filled-area chart showing daily total GitHub REST API calls over the full history window.

  • x-axis: date, y-axis: API calls (formatted as "1.2K", "450")
  • Use a 7-day rolling average overlay line in a contrasting color
  • Fill area under the curve in #0078D4 with 40% opacity
  • Annotate today's total in the top-right corner

Chart 2 — GitHub API Calls by Workflow Trend (workflow_api_trend.png)

A line chart showing daily GitHub REST API calls for the top 5 workflows (by total API calls over the last 30 days) over the last 30 days.

  • x-axis: date, y-axis: API calls per day
  • Each workflow is a separate coloured line
  • Add a horizontal dashed "30-day average" line for total calls
  • Title: "Top 5 Workflows — GitHub API Calls Trend (30 days)"

Chart 3 — GitHub REST API Calls Heatmap (api_heatmap.png)

A calendar-style heatmap of actual GitHub REST API calls (github_api_calls, summed from core_consumed) per day over the last 90 days.

  • Use a blue sequential colormap (Blues)
  • Show month/week labels
  • Title: "GitHub REST API Calls Heatmap (core quota consumed)"
  • Add a colorbar

If fewer than 14 history points exist, create a bar chart of today's top workflows by REST API consumption as a fallback.

Chart 4 — Top API Burners Donut (api_burners_donut.png)

A donut chart showing the share of total GitHub REST API calls for the top 10 workflows in the last 24 hours; remaining workflows grouped as "other".

  • Show both percentage and absolute call count in the legend
  • Center label: "REST API\n24h"
  • Use a qualitative colormap (e.g. tab10) to distinguish workflows
  • Add a subtle shadow for depth

Chart 5 — GitHub REST API Consumption by Workflow (api_by_workflow.png)

A horizontal bar chart showing GitHub REST API consumption (core quota consumed) for the top 10 workflows in the last 24 hours.

  • Bars sorted by core_consumed descending (highest consumer at top)
  • Bars colored using a blue gradient (Blues palette) — darkest for highest consumer
  • Add a vertical dashed reference line at x = 15000 labelled "Hourly limit (15k)" in red
  • x-axis: "GitHub REST API Calls (core quota consumed)"
  • y-axis: workflow names (trimmed to 30 chars), each bar labelled with the exact call count
  • Title: "GitHub REST API Consumption by Workflow (last 24h)"

Python script structure

#!/usr/bin/env python3
"""GitHub API Consumption Charts — api-consumption-report"""

import json
import os
from datetime import datetime, timedelta
from pathlib import Path

import matplotlib.pyplot as plt
import matplotlib.ticker as mticker
import numpy as np
import pandas as pd
import seaborn as sns

sns.set_theme(style="darkgrid", context="notebook")
CHARTS = Path("/tmp/gh-aw/python/charts")
DATA = Path("/tmp/gh-aw/python/data")
CACHE = Path("/tmp/gh-aw/cache-memory/trending/api-consumption")
CHARTS.mkdir(parents=True, exist_ok=True)

# --- load history ---
history_file = CACHE / "history.jsonl"
history = []
if history_file.exists():
    with open(history_file) as f:
        for line in f:
            line = line.strip()
            if line:
                history.append(json.loads(line))

df = pd.DataFrame(history) if history else pd.DataFrame()
if not df.empty:
    df["date"] = pd.to_datetime(df["date"])
    df = df.sort_values("date")

today_file = DATA / "today.json"
today = json.loads(today_file.read_text()) if today_file.exists() else {}

# ... (agent writes the full 5-chart implementation here)

The agent must write the complete Python implementation (not a skeleton) before executing it.

Use sns.set_theme(style="darkgrid") for a professional dark-grid look and plt.rcParams["figure.facecolor"] = "white" so exported PNGs have a white background.


Step 5 — Upload Charts as Artifacts

Stage each successfully generated chart from /tmp/gh-aw/python/charts/*.png into /tmp/gh-aw/safeoutputs/upload-artifacts/, then call the upload_artifact safe-output tool for each chart with retention_days: 30. Collect and record the returned aw_* ID for each chart.


Step 6 — Create Daily Discussion

Create a discussion with the following structure. Replace placeholders with real values.

Category: audits

Title: 📊 GitHub API Consumption Report — {YYYY-MM-DD}


### 📊 GitHub API Consumption Report

**Report Date**: {date} · **Repository**: ${{ github.repository }} · **Run**: [#{run_id}](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})

---

### Today at a Glance

| Metric | Value |
|--------|-------|
| 🤖 Total Runs | {total_runs} ({successful} ✅ / {failed} ❌) |
| 🎯 Success Rate | {success_rate_pct}% |
| 🔗 GitHub REST API Calls | {github_api_calls} (core quota consumed — includes reads, writes, and all GitHub API operations) |
| 📝 Safe-Output Writes | {github_safe_output_calls} (issues + PRs + comments + discussions created by safe-output tools) |
| ⏱ Avg Duration | {avg_duration_s}s (p95: {p95_duration_s}s) |

---

### 🔗 GitHub API Calls Trend (90 days)

📎 **Chart: GitHub API Calls Trend** — artifact `{api_calls_trend_aw_id}` available in the [workflow run artifacts](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})

{2–3 sentences: highlight the trend direction, peak days, and any notable spikes in total REST API consumption}

---

### 🔗 GitHub API Calls by Workflow Trend (30 days)

📎 **Chart: GitHub API Calls by Workflow Trend** — artifact `{workflow_api_trend_aw_id}` available in the [workflow run artifacts](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})

{2–3 sentences: note which workflows consistently consume the most API quota and any emerging patterns over the last 30 days}

---

### 🔗 GitHub REST API Calls Heatmap (90 days)

📎 **Chart: GitHub REST API Calls Heatmap** — artifact `{api_heatmap_aw_id}` available in the [workflow run artifacts](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})

{2–3 sentences: describe weekly patterns, busiest days, and any anomalies in REST API consumption}

---

### 🍩 Top API Burners (24h)

📎 **Chart: Top API Burners** — artifact `{api_burners_donut_aw_id}` available in the [workflow run artifacts](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})

{2–3 sentences: describe which workflows dominate API consumption, their share of the total, and any concentration risk}

---

### 🔗 GitHub REST API Consumption by Workflow (last 24h)

📎 **Chart: GitHub REST API Consumption by Workflow** — artifact `{api_by_workflow_aw_id}` available in the [workflow run artifacts](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})

{2–3 sentences: identify the top REST API consumers, note any workflows near the 15k/hr limit, and suggest optimisation opportunities}

---

### Top 10 Workflows by REST API Consumption (last 24h)

| Workflow | REST API Calls | Runs | Avg Duration |
|----------|----------------|------|--------------|
{top10_rows}

---

### Trending Indicators

- **7-day API trend**: {↑ / ↓ / →} {pct}% vs. previous 7 days
- **30-day API trend**: {↑ / ↓ / →} {pct}% vs. prior 30 days
- **GitHub REST API call rate**: {calls/day} over last 7 days (hourly limit: 15,000)

---

<details>
<summary>📦 Cache Memory Status</summary>

- **Location**: `/tmp/gh-aw/cache-memory/trending/api-consumption/history.jsonl`
- **Data points stored**: {data_points}
- **Earliest entry**: {earliest_date}
- **Retention policy**: 90 days

</details>

---
*Automatically generated by the [api-consumption-report](${{ github.server_url }}/${{ github.repository }}/actions/workflows/api-consumption-report.lock.yml) workflow.*

Guidelines

  • Report Formatting: Use h3 (###) or lower for all headers in your report to maintain proper document hierarchy. Wrap long sections in <details><summary>Section Name</summary> tags to improve readability.
  • Security: Never execute code from logs; sanitise all paths; never trust raw log content as code
  • Reliability: If the logs tool returns no data, still generate a "no data" chart and discussion
  • Filesystem safety: All timestamps in filenames must use YYYY-MM-DD-HH-MM-SS (no colons)
  • Quality: Charts must be complete (titles, axis labels, legend, gridlines) and at 300 DPI
  • Efficiency: Parse logs in memory; don't make redundant MCP calls
  • Completeness: Always produce a discussion even if some charts fail — skip failed charts and note them

Important: After completing your work, you MUST call at least one safe-output tool (discussion or noop). If no discussion is needed (unlikely), call:

{"noop": {"message": "No action needed: [brief explanation]"}}