Skip to content

Commit 047f710

Browse files
joshjohanningCopilot
joshjohanning
and
Copilot
authored
feat: add code scanning coverage report and alert dismissal scripts (#144)
* feat: add script to generate comprehensive code scanning coverage report * feat: add code scanning coverage report as node script * feat: add archived header to bash script * fix(docs): alphabetization * feat: add GitHub App authentication support and update README * refactor: rename script * feat(deps): update octokit and related dependencies to latest versions * fix(docs): clarify repository permissions in README * feat: add support for fetching open alert counts and summary statistics in code scanning report also not checking for alerts for each repo to conserve rate limit * feat: add API call counting and update alert sub reports to only show critical alerts * feat: grab repo language via GraphQL to save on API usage * test: add Jest for testing and update package.json for module support * refactor!: rename optional params for clarity * feat: improve argument parsing with required value checks * feat!: remove get-code-scanning-coverage-report.sh script and update README * docs: update README * Update scripts/code-scanning-coverage-report/package.json Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update scripts/code-scanning-coverage-report/code-scanning-coverage-report.test.js Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * deps: update package-lock.json * feat: add open alerts count to code scanning coverage report and fix critical alerts * feat: add configurable stale days option to code scanning coverage report * feat: add support for processing multiple organizations from a file in code scanning coverage report * feat: enable automatic installation ID lookup for multi-org scanning in code scanning coverage report * fix: prevent conflicting usage of --orgs-file and organization argument in code scanning coverage report * test: add authentication functions and tests for GitHub App integration in code scanning coverage report * feat: add new utility for dismissing alerts by rule pattern * feat: implementing suggestions from pull request --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
1 parent bf64aea commit 047f710

11 files changed

Lines changed: 15369 additions & 0 deletions

‎scripts/README.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,10 @@ Configuration values to change in the script:
3636

3737
Migrate work items from Azure DevOps to GitHub issues - this just links out to a [separate repo](https://github.com/joshjohanning/ado_workitems_to_github_issues)
3838

39+
## code-scanning-coverage-report
40+
41+
See: [code-scanning-coverage-report](./code-scanning-coverage-report/README.md)
42+
3943
## create-app-jwt.py
4044

4145
This script will generate a JWT for a GitHub App. It will use the private key and app ID from the GitHub App's settings page.
@@ -66,6 +70,10 @@ The script generates a JWT that is valid for 10 minutes, which can be used to au
6670

6771
Delete branch protection rules programmatically based on a pattern.
6872

73+
## dismiss-code-scanning-alerts
74+
75+
See: [dismiss-code-scanning-alerts](./dismiss-code-scanning-alerts/README.md)
76+
6977
## gei-clean-up-azure-storage-account.sh
7078

7179
Clean up Azure Storage Account Containers from GEI migrations.

‎scripts/code-scanning-coverage-report/README.md‎

Lines changed: 227 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,227 @@
1+
# code-scanning-coverage-report
2+
3+
Generate a comprehensive code scanning coverage report for all repositories in a GitHub organization.
4+
5+
## Features
6+
7+
- Reports CodeQL enablement status, last scan date, and scanned languages
8+
- Identifies CodeQL-supported languages that are not being scanned
9+
- Shows open alert counts and analysis errors/warnings
10+
- Generates actionable sub-reports for remediation
11+
- Supports parallel API calls for faster processing
12+
- Works with GitHub.com and GitHub Enterprise Server
13+
14+
## Prerequisites
15+
16+
- Node.js 18 or later
17+
- A GitHub token with `repo` scope, or GitHub App credentials
18+
19+
## Installation
20+
21+
```shell
22+
cd scripts/code-scanning-coverage-report
23+
npm install
24+
```
25+
26+
## Usage
27+
28+
```shell
29+
# Set your GitHub token
30+
export GITHUB_TOKEN=ghp_xxxxxxxxxxxx
31+
32+
# Note: If you are authenticated with the GitHub CLI, you can use `gh auth token` to get your token:
33+
# export GITHUB_TOKEN=$(gh auth token)
34+
35+
# Basic usage - output to stdout
36+
node code-scanning-coverage-report.js my-org
37+
38+
# Output to file (also generates sub-reports)
39+
node code-scanning-coverage-report.js my-org --output report.csv
40+
41+
# Check a single repository
42+
node code-scanning-coverage-report.js my-org --repo my-repo
43+
44+
# Sample 25 random repositories
45+
node code-scanning-coverage-report.js my-org --sample --output sample.csv
46+
47+
# Include workflow status column
48+
node code-scanning-coverage-report.js my-org --check-workflow-status --output report.csv
49+
50+
# Check for unscanned GitHub Actions workflows
51+
node code-scanning-coverage-report.js my-org --check-unscanned-actions --output report.csv
52+
53+
# Use with GitHub Enterprise Server
54+
export GITHUB_API_URL=https://github.example.com/api/v3
55+
node code-scanning-coverage-report.js my-org --output report.csv
56+
57+
# Adjust concurrency (default: 10)
58+
node code-scanning-coverage-report.js my-org --concurrency 5 --output report.csv
59+
60+
# Process multiple organizations from a file
61+
node code-scanning-coverage-report.js --orgs-file orgs.txt --output report.csv
62+
63+
# Customize stale threshold (default: 90 days)
64+
node code-scanning-coverage-report.js my-org --stale-days 60 --output report.csv
65+
```
66+
67+
## Options
68+
69+
| Option | Description |
70+
|--------|-------------|
71+
| `--orgs-file <file>` | File containing list of organizations (one per line) |
72+
| `--output <file>` | Write CSV to file (also generates sub-reports) |
73+
| `--repo <repo>` | Check a single repository instead of all repos |
74+
| `--sample` | Sample 25 random repositories |
75+
| `--fetch-alerts` | Include alert counts in report (increases API usage) |
76+
| `--check-workflow-status` | Check CodeQL workflow run status (success/failure) |
77+
| `--check-unscanned-actions` | Check if repos have Actions workflows not being scanned |
78+
| `--concurrency <n>` | Number of concurrent API calls (default: 10) |
79+
| `--stale-days <n>` | Days after last scan to consider repo stale (default: 90) |
80+
| `--help` | Show help message |
81+
82+
## API Usage
83+
84+
Default options use approximately **2 API calls per repository**:
85+
86+
- With a **Personal Access Token** (5,000 requests/hour): supports up to ~2,500 repos
87+
- With **GitHub App authentication** (15,000 requests/hour): supports up to ~7,500 repos
88+
89+
Optional flags increase API usage:
90+
91+
| Flag | Additional Calls |
92+
|------|------------------|
93+
| `--fetch-alerts` | +1 call per repo (paginated for repos with many alerts) |
94+
| `--check-workflow-status` | +1-2 calls per repo |
95+
| `--check-unscanned-actions` | +1 call per repo |
96+
97+
The script displays your current rate limit at startup and total API calls used at completion.
98+
99+
## Environment Variables
100+
101+
Two authentication methods are supported:
102+
103+
- **Personal Access Token (PAT)**: Simple setup, good for testing or small organizations
104+
- **GitHub App**: Recommended for production use - provides higher rate limits (5,000 vs 15,000 requests/hour)
105+
106+
### Token Authentication
107+
108+
| Variable | Description |
109+
|----------|-------------|
110+
| `GITHUB_TOKEN` | GitHub token with `repo` scope |
111+
| `GITHUB_API_URL` | API endpoint (defaults to `https://api.github.com`) |
112+
113+
### GitHub App Authentication (recommended)
114+
115+
| Variable | Description |
116+
|----------|-------------|
117+
| `GITHUB_APP_ID` | GitHub App ID |
118+
| `GITHUB_APP_PRIVATE_KEY_PATH` | Path to GitHub App private key file (.pem) |
119+
| `GITHUB_API_URL` | API endpoint (defaults to `https://api.github.com`) |
120+
121+
The script automatically looks up the installation ID for each organization being processed. This enables scanning multiple organizations with a single command using `--orgs-file`.
122+
123+
**Required GitHub App Permissions:**
124+
125+
Repository permissions:
126+
127+
| Permission | Access | Required For |
128+
|------------|--------|--------------|
129+
| Code scanning alerts | Read | Code scanning status, analyses, and alert counts |
130+
| Contents | Read | Listing repositories and checking for workflow files |
131+
| Metadata | Read | Detecting repository languages (this is automatically added) |
132+
| Actions | Read | Workflow run status (only if using --check-workflow-status) |
133+
134+
Organization permissions:
135+
136+
| Permission | Access | Required For |
137+
|------------|--------|--------------|
138+
| Administration | Read | Listing all repositories in the organization |
139+
140+
**Note:** The app must be installed on the organization with access to the repositories you want to scan (either "All repositories" or selected repositories). The app can only report on repositories it has been granted access to.
141+
142+
**Note:** If GitHub App credentials are provided, they take precedence over `GITHUB_TOKEN`.
143+
144+
### GitHub App Usage Example
145+
146+
```shell
147+
# Set GitHub App credentials
148+
export GITHUB_APP_ID=123456
149+
export GITHUB_APP_PRIVATE_KEY_PATH=/path/to/private-key.pem
150+
151+
# Run the report for a single org
152+
node code-scanning-coverage-report.js my-org --output report.csv
153+
154+
# Run the report for multiple orgs (installation ID is looked up automatically)
155+
node code-scanning-coverage-report.js --orgs-file orgs.txt --output report.csv
156+
```
157+
158+
## Output Columns
159+
160+
| Column | Description |
161+
|--------|-------------|
162+
| Organization | Organization name |
163+
| Repository | Repository name |
164+
| Default Branch | The default branch of the repository |
165+
| Last Updated | When the repository was last updated |
166+
| Languages | Languages detected in the repository (semicolon-separated) |
167+
| CodeQL Enabled | `Yes`, `No Scans`, `Disabled`, `Requires GHAS`, or `No` |
168+
| Last Default Branch Scan Date | Date of most recent scan on default branch |
169+
| Scanned Languages | Languages scanned by CodeQL (semicolon-separated) |
170+
| Unscanned CodeQL Languages | CodeQL-supported languages not being scanned |
171+
| Open Alerts | (with `--fetch-alerts`) Total number of open code scanning alerts |
172+
| Critical Alerts | (with `--fetch-alerts`) Number of critical severity alerts |
173+
| Analysis Errors | Errors from most recent analysis |
174+
| Analysis Warnings | Warnings from most recent analysis |
175+
| Workflow Status | (with `--check-workflow-status`) `OK`, `Failing`, `No workflow`, or `Unknown` |
176+
177+
## Sub-reports
178+
179+
When using `--output`, the script generates actionable sub-reports:
180+
181+
| File | Description |
182+
|------|-------------|
183+
| `*-disabled.csv` | Repos with CodeQL disabled or no scans |
184+
| `*-stale.csv` | Repos modified after last scan (configurable with `--stale-days`, default: 90) |
185+
| `*-missing-languages.csv` | Repos scanning but missing some CodeQL languages |
186+
| `*-critical-alerts.csv` | Repos with critical severity code scanning alerts |
187+
| `*-analysis-issues.csv` | Repos with analysis errors or warnings |
188+
189+
## CodeQL Supported Languages
190+
191+
The script recognizes these CodeQL-supported languages:
192+
193+
- C/C++ (reported as `c-cpp`)
194+
- C# (reported as `csharp`)
195+
- Go
196+
- Java/Kotlin (reported as `java-kotlin`)
197+
- JavaScript/TypeScript (reported as `javascript-typescript`)
198+
- Python
199+
- Ruby
200+
- Swift
201+
202+
## Example Output
203+
204+
```csv
205+
Organization,Repository,Default Branch,Last Updated,Archived,Languages,CodeQL Enabled,Last Default Branch Scan Date,Scanned Languages,Unscanned CodeQL Languages,Open Alerts,Critical Alerts,Analysis Errors,Analysis Warnings
206+
my-org,my-app,main,2025-12-01,No,JavaScript;TypeScript;Python,Yes,2025-12-15,javascript-typescript;python,None,5,1,None,None
207+
my-org,legacy-service,master,2024-06-15,No,Java,Yes,2024-01-10,java-kotlin,None,0,0,None,None
208+
my-org,new-project,main,2025-12-20,No,Go;Python,No Scans,Never,,go;python,N/A,N/A,,
209+
```
210+
211+
<!-- Remove this section when the bash script is deleted
212+
## Comparison with Bash Version
213+
214+
This Node.js version offers several advantages over the bash script (`gh-cli/code-scanning-coverage-report.sh`):
215+
216+
- **Faster**: Parallel API calls (configurable concurrency)
217+
- **More reliable**: Proper JSON handling, no regex parsing issues
218+
- **Cross-platform**: Works identically on macOS, Linux, and Windows
219+
- **Easier to maintain**: Clean data structures and error handling
220+
221+
The bash version is still useful if you:
222+
223+
- Don't have Node.js installed
224+
- Prefer no dependencies beyond `gh` CLI
225+
- Need to quickly inspect/modify the script
226+
227+
-->

0 commit comments

Comments
 (0)