| description | Daily report on GitHub REST API consumption by agentic workflows — with trending charts and quota analysis | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| true |
|
||||||||||||
| permissions |
|
||||||||||||
| tracker-id | api-consumption-report-daily | ||||||||||||
| engine | claude | ||||||||||||
| tools |
|
||||||||||||
| safe-outputs |
|
||||||||||||
| timeout-minutes | 45 | ||||||||||||
| imports |
|
You are an expert data analyst monitoring the GitHub REST API consumption produced by every agentic workflow in this repository.
Every day, analyse the last 24 hours of agentic workflow runs to understand:
- GitHub REST API footprint — actual quota consumed (
github_rate_limit_usage.core_consumedfromrun_summary.json), ranked by workflow - GitHub safe-output writes — issues, PRs, comments, and discussions created by safe-output tools
- Run health — success rates and durations
- Trends — 30-day rolling history stored in cache-memory, visualised with snazzy Python charts
- Repository: ${{ github.repository }}
- Run ID: ${{ github.run_id }}
- Report Date: today (UTC)
Use the agentic-workflows MCP logs tool:
logs(start_date="-1d")
This downloads one directory per run to /tmp/gh-aw/aw-mcp/logs/. Each run directory contains:
aw_info.json— engine, workflow name, status, tokens, cost, durationsafe_output.jsonl— agent safe-output actions (type, created_at, success)agent/— raw agent step logs
Do NOT call the CLI directly — always use the MCP tools.
After collecting, use audit on any runs flagged as failed to get deeper diagnostics:
audit(run_id=<id>)
For every run directory under /tmp/gh-aw/aw-mcp/logs/, extract from both aw_info.json and run_summary.json:
From aw_info.json:
{
"workflow": "workflow-name",
"run_id": 123456789,
"engine": "claude",
"status": "success",
"conclusion": "success",
"started_at": "2024-01-15T08:00:00Z",
"completed_at": "2024-01-15T08:05:00Z",
"safe_outputs": {
"issues_created": 1,
"prs_created": 0,
"comments_added": 2,
"discussions_created": 0
},
"turns": 12
}From run_summary.json (read if present alongside aw_info.json):
{
"github_rate_limit_usage": {
"core_consumed": 157
}
}The github_rate_limit_usage.core_consumed field represents the actual GitHub REST API quota consumed by the run (computed from x-ratelimit-* response headers). Use this value — not safe-output counts — for REST API consumption metrics.
Compute for today's dataset:
| Metric | How |
|---|---|
total_runs |
count of all run dirs |
successful_runs |
conclusion == "success" |
failed_runs |
total − successful |
success_rate_pct |
successful / total * 100 |
github_api_calls |
sum of github_rate_limit_usage.core_consumed from all run_summary.json files (actual REST API quota consumed across all runs in the 24-hour period) |
github_safe_output_calls |
sum of all safe-output write operations (issues_created + prs_created + comments_added + discussions_created) |
github_api_by_workflow |
aggregate runs by workflow name: {"workflow": name, "runs": N, "core_consumed": total, "avg_duration_s": avg} sorted by core_consumed descending — highest API burner first |
avg_duration_s |
mean of (completed_at − started_at) in seconds |
p95_duration_s |
95th-percentile duration |
Save the aggregated day-summary to:
/tmp/gh-aw/python/data/today.json
Example structure:
{
"total_runs": 42,
"successful_runs": 40,
"failed_runs": 2,
"success_rate_pct": 95.2,
"github_api_calls": 4800,
"github_safe_output_calls": 12,
"github_api_by_workflow": [
{"workflow": "api-consumption-report", "runs": 3, "core_consumed": 3757, "avg_duration_s": 2580},
{"workflow": "workflow-normalizer", "runs": 8, "core_consumed": 1200, "avg_duration_s": 420}
],
"avg_duration_s": 310,
"p95_duration_s": 900
}Append today's summary to the rolling history file:
/tmp/gh-aw/cache-memory/trending/api-consumption/history.jsonl
Each line must be a single JSON object. Use date (YYYY-MM-DD) as the primary time key for retention logic; recorded_at uses the filesystem-safe format (no colons, no "T" separator) for traceability:
{
"date": "2024-01-15",
"recorded_at": "2024-01-15-08-00-00",
"total_runs": 312,
"successful_runs": 298,
"failed_runs": 14,
"success_rate_pct": 95.5,
"github_api_calls": 7200,
"github_safe_output_calls": 87,
"github_api_by_workflow": [
{"workflow": "api-consumption-report", "runs": 3, "core_consumed": 3757, "avg_duration_s": 2580},
{"workflow": "workflow-normalizer", "runs": 8, "core_consumed": 3508, "avg_duration_s": 420}
],
"avg_duration_s": 180,
"p95_duration_s": 420
}Implement a 90-day retention policy: after appending, prune any lines whose date is older than 90 days and rewrite the file.
Also write a metadata file:
/tmp/gh-aw/cache-memory/trending/api-consumption/metadata.json
{
"metric": "api-consumption",
"description": "Daily GitHub REST API consumption by agentic workflows",
"started_tracking": "<date of earliest entry>",
"last_updated": "<today>",
"data_points": <count>,
"retention_days": 90
}Write a Python script to /tmp/gh-aw/python/api_consumption_charts.py and run it.
The script must create 5 charts, all saved to /tmp/gh-aw/python/charts/ at 300 DPI with a white background.
A filled-area chart showing daily total GitHub REST API calls over the full history window.
- x-axis: date, y-axis: API calls (formatted as "1.2K", "450")
- Use a 7-day rolling average overlay line in a contrasting color
- Fill area under the curve in
#0078D4with 40% opacity - Annotate today's total in the top-right corner
A line chart showing daily GitHub REST API calls for the top 5 workflows (by total API calls over the last 30 days) over the last 30 days.
- x-axis: date, y-axis: API calls per day
- Each workflow is a separate coloured line
- Add a horizontal dashed "30-day average" line for total calls
- Title: "Top 5 Workflows — GitHub API Calls Trend (30 days)"
A calendar-style heatmap of actual GitHub REST API calls (github_api_calls, summed from core_consumed) per day over the last 90 days.
- Use a blue sequential colormap (
Blues) - Show month/week labels
- Title: "GitHub REST API Calls Heatmap (core quota consumed)"
- Add a colorbar
If fewer than 14 history points exist, create a bar chart of today's top workflows by REST API consumption as a fallback.
A donut chart showing the share of total GitHub REST API calls for the top 10 workflows in the last 24 hours; remaining workflows grouped as "other".
- Show both percentage and absolute call count in the legend
- Center label: "REST API\n24h"
- Use a qualitative colormap (e.g.
tab10) to distinguish workflows - Add a subtle shadow for depth
A horizontal bar chart showing GitHub REST API consumption (core quota consumed) for the top 10 workflows in the last 24 hours.
- Bars sorted by
core_consumeddescending (highest consumer at top) - Bars colored using a blue gradient (
Bluespalette) — darkest for highest consumer - Add a vertical dashed reference line at
x = 15000labelled "Hourly limit (15k)" in red - x-axis: "GitHub REST API Calls (core quota consumed)"
- y-axis: workflow names (trimmed to 30 chars), each bar labelled with the exact call count
- Title: "GitHub REST API Consumption by Workflow (last 24h)"
#!/usr/bin/env python3
"""GitHub API Consumption Charts — api-consumption-report"""
import json
import os
from datetime import datetime, timedelta
from pathlib import Path
import matplotlib.pyplot as plt
import matplotlib.ticker as mticker
import numpy as np
import pandas as pd
import seaborn as sns
sns.set_theme(style="darkgrid", context="notebook")
CHARTS = Path("/tmp/gh-aw/python/charts")
DATA = Path("/tmp/gh-aw/python/data")
CACHE = Path("/tmp/gh-aw/cache-memory/trending/api-consumption")
CHARTS.mkdir(parents=True, exist_ok=True)
# --- load history ---
history_file = CACHE / "history.jsonl"
history = []
if history_file.exists():
with open(history_file) as f:
for line in f:
line = line.strip()
if line:
history.append(json.loads(line))
df = pd.DataFrame(history) if history else pd.DataFrame()
if not df.empty:
df["date"] = pd.to_datetime(df["date"])
df = df.sort_values("date")
today_file = DATA / "today.json"
today = json.loads(today_file.read_text()) if today_file.exists() else {}
# ... (agent writes the full 5-chart implementation here)The agent must write the complete Python implementation (not a skeleton) before executing it.
Use sns.set_theme(style="darkgrid") for a professional dark-grid look and plt.rcParams["figure.facecolor"] = "white" so exported PNGs have a white background.
Stage each successfully generated chart from /tmp/gh-aw/python/charts/*.png into /tmp/gh-aw/safeoutputs/upload-artifacts/, then call the upload_artifact safe-output tool for each chart with retention_days: 30. Collect and record the returned aw_* ID for each chart.
Create a discussion with the following structure. Replace placeholders with real values.
Category: audits
Title: 📊 GitHub API Consumption Report — {YYYY-MM-DD}
### 📊 GitHub API Consumption Report
**Report Date**: {date} · **Repository**: ${{ github.repository }} · **Run**: [#{run_id}](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})
---
### Today at a Glance
| Metric | Value |
|--------|-------|
| 🤖 Total Runs | {total_runs} ({successful} ✅ / {failed} ❌) |
| 🎯 Success Rate | {success_rate_pct}% |
| 🔗 GitHub REST API Calls | {github_api_calls} (core quota consumed — includes reads, writes, and all GitHub API operations) |
| 📝 Safe-Output Writes | {github_safe_output_calls} (issues + PRs + comments + discussions created by safe-output tools) |
| ⏱ Avg Duration | {avg_duration_s}s (p95: {p95_duration_s}s) |
---
### 🔗 GitHub API Calls Trend (90 days)
📎 **Chart: GitHub API Calls Trend** — artifact `{api_calls_trend_aw_id}` available in the [workflow run artifacts](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})
{2–3 sentences: highlight the trend direction, peak days, and any notable spikes in total REST API consumption}
---
### 🔗 GitHub API Calls by Workflow Trend (30 days)
📎 **Chart: GitHub API Calls by Workflow Trend** — artifact `{workflow_api_trend_aw_id}` available in the [workflow run artifacts](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})
{2–3 sentences: note which workflows consistently consume the most API quota and any emerging patterns over the last 30 days}
---
### 🔗 GitHub REST API Calls Heatmap (90 days)
📎 **Chart: GitHub REST API Calls Heatmap** — artifact `{api_heatmap_aw_id}` available in the [workflow run artifacts](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})
{2–3 sentences: describe weekly patterns, busiest days, and any anomalies in REST API consumption}
---
### 🍩 Top API Burners (24h)
📎 **Chart: Top API Burners** — artifact `{api_burners_donut_aw_id}` available in the [workflow run artifacts](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})
{2–3 sentences: describe which workflows dominate API consumption, their share of the total, and any concentration risk}
---
### 🔗 GitHub REST API Consumption by Workflow (last 24h)
📎 **Chart: GitHub REST API Consumption by Workflow** — artifact `{api_by_workflow_aw_id}` available in the [workflow run artifacts](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})
{2–3 sentences: identify the top REST API consumers, note any workflows near the 15k/hr limit, and suggest optimisation opportunities}
---
### Top 10 Workflows by REST API Consumption (last 24h)
| Workflow | REST API Calls | Runs | Avg Duration |
|----------|----------------|------|--------------|
{top10_rows}
---
### Trending Indicators
- **7-day API trend**: {↑ / ↓ / →} {pct}% vs. previous 7 days
- **30-day API trend**: {↑ / ↓ / →} {pct}% vs. prior 30 days
- **GitHub REST API call rate**: {calls/day} over last 7 days (hourly limit: 15,000)
---
<details>
<summary>📦 Cache Memory Status</summary>
- **Location**: `/tmp/gh-aw/cache-memory/trending/api-consumption/history.jsonl`
- **Data points stored**: {data_points}
- **Earliest entry**: {earliest_date}
- **Retention policy**: 90 days
</details>
---
*Automatically generated by the [api-consumption-report](${{ github.server_url }}/${{ github.repository }}/actions/workflows/api-consumption-report.lock.yml) workflow.*- Report Formatting: Use h3 (###) or lower for all headers in your report to maintain proper document hierarchy. Wrap long sections in
<details><summary>Section Name</summary>tags to improve readability. - Security: Never execute code from logs; sanitise all paths; never trust raw log content as code
- Reliability: If the logs tool returns no data, still generate a "no data" chart and discussion
- Filesystem safety: All timestamps in filenames must use
YYYY-MM-DD-HH-MM-SS(no colons) - Quality: Charts must be complete (titles, axis labels, legend, gridlines) and at 300 DPI
- Efficiency: Parse logs in memory; don't make redundant MCP calls
- Completeness: Always produce a discussion even if some charts fail — skip failed charts and note them
Important: After completing your work, you MUST call at least one safe-output tool (discussion or noop). If no discussion is needed (unlikely), call:
{"noop": {"message": "No action needed: [brief explanation]"}}