From 362cb952b266ef43d89aed757f437e2c619f9dfb Mon Sep 17 00:00:00 2001 From: Jacob Olson Date: Mon, 9 Nov 2026 20:06:10 -0700 Subject: [PATCH] Make the Reports page honest and document it The Reports page's script overwrote Total Reports and Pending Reports with 0 every 30 seconds (the stats endpoint has neither number), the buttons led to report types that cannot be run, and the recent reports were not links. Lead with the storage audit: audit and in-progress counts, active alerts, recent audits that open the checklist. Rewrite the reports README for what works today and link it from the README and docs index. Co-Authored-By: Claude Sonnet 5.5 --- README.md | 3 + docs/README.md | 3 + krill/reports/README.md | 202 +++--------------- .../templates/reports/reports_dashboard.html | 127 ++++------- krill/reports/tests.py | 36 ++++ krill/reports/views.py | 59 ++--- 6 files changed, 134 insertions(+), 296 deletions(-) diff --git a/README.md b/README.md index 2888fa3..b9b8071 100644 --- a/README.md +++ b/README.md @@ -240,6 +240,9 @@ Contributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for gui - [Utility Scripts](scripts/README.md) - Helper scripts (whitespace cleanup, PostgreSQL migration) - [Security Notes](krill/SECURITY.md) - Security configuration and checks +**Using Krill** +- [Reports and storage audits](krill/reports/README.md) - Spot-check storage, resume and review audits, alerts + **Data** - [Data Import Guide](docs/DATA_IMPORT.md) - How to structure CSV files for importing sample data - [Database Setup Guide](docs/database.md) - SQLite and PostgreSQL configuration diff --git a/docs/README.md b/docs/README.md index a2e2420..3904d55 100644 --- a/docs/README.md +++ b/docs/README.md @@ -4,6 +4,9 @@ - [Data Import Guide](DATA_IMPORT.md) - CSV format for importing sample data - [Database Setup](database.md) - SQLite and PostgreSQL configuration +## Using Krill +- [Reports and storage audits](../krill/reports/README.md) - spot-check storage, resume and review audits, alerts + ## Deployment (`deployment/`) - [Deployment Guide](deployment/README.md) - Docker images, environment variables, security, troubleshooting - [DigitalOcean App Platform](deployment/digitalocean.md) diff --git a/krill/reports/README.md b/krill/reports/README.md index 4dce1b7..bdffcee 100644 --- a/krill/reports/README.md +++ b/krill/reports/README.md @@ -1,182 +1,50 @@ -# Reports App +# Reports app -The Reports app provides comprehensive reporting functionality for the Krill system, including report generation, alert management, issue tracking, and sample search capabilities. +Reports is where you check that what Krill says is in storage is really there (the **storage audit**), and where alerts are listed. Open it from **Reports** in the sidebar. -## Features +## Storage audit -### 1. Report Generation -- **Report Templates**: Predefined templates for common report types -- **Multiple Formats**: Support for PDF, Excel, CSV, and JSON output -- **Parameterized Reports**: Customizable report parameters -- **Scheduled Reports**: Automated report generation on schedule +A spot check of storage. Krill picks stored aliquots at random and gives you a checklist of where to find each one. You go to the freezer, look, and mark each location. -### 2. Alert System -- **System Alerts**: Monitor and manage system alerts -- **Severity Levels**: Low, Medium, High, and Critical alert levels -- **Alert Types**: Categorize alerts by type (storage, temperature, etc.) -- **Workflow Management**: Acknowledge and resolve alerts +### Run one +1. **Reports > Start Storage Audit** (or **Start Audit** on the dashboard). +2. Give it a name (it defaults to today's date) and a sample size (1 to 100, default 10). Optionally tick **Include empty slot verification** to also check a few slots that should be empty. +3. **Generate Audit Report** opens the checklist. -### 3. Issue Tracking -- **Issue Reporting**: Report bugs, feature requests, and maintenance issues -- **Priority Management**: Low, Medium, High, and Urgent priority levels -- **Assignment System**: Assign issues to team members -- **Status Tracking**: Track issue progress from open to resolved +### The checklist +Each row shows the cell line and aliquot type, the full location (freezer > cane > box > row, column), and the details that help you recognise the tube: passage, date frozen, lab member, disposition, aliquot number and the sample notes. -### 4. Sample Search -- **Advanced Search**: Search samples by name, type, and disposition -- **Filtering**: Multiple filter options for refined results -- **Export Results**: Download search results in various formats -- **Real-time Updates**: Live search with debouncing +Mark each row **Correct** (the tube is there; for an empty-slot check, it is empty), **Wrong** (missing or different), and add a note if useful. Every mark is saved as you make it, so you can walk away and carry on later. The row's left edge turns green or red, and the counts at the top (checked, correct, wrong) update. -### 5. Storage Dashboard -- **Capacity Monitoring**: Track storage usage and availability -- **Device Status**: Monitor storage device health and status -- **Statistics**: Comprehensive storage statistics and metrics +### Resume, review and clean up +- **Reports** and the **Storage Audits** list (`/reports/audit/list/`) show audits with progress ("12 of 20 checked") and whether each is in progress or complete. The dashboard lists audits still in progress. +- Open any audit to carry on or review it. The list shows everyone's audits (20 per page); **Show only mine** narrows it. +- A row records who checked it, so a review by a lab manager shows up as theirs. +- The trash button deletes an audit permanently, after a confirmation. -## Models +### Who can do what +| | View | Mark slots | Delete | +|---|---|---|---| +| Anyone signed in | Any audit | Only audits they started | Only audits they started | +| Lab managers and admins | Any audit | Any audit | Any audit | -### ReportTemplate -- Defines report structure and configuration -- Supports multiple report types (sample inventory, storage capacity, etc.) -- JSON-based template configuration +Other people see the checklist with the fields disabled. -### Report -- Generated report instances -- Tracks generation status and results -- Supports multiple output formats +### How it works +- An audit is a `Report` whose template has the type `storage_audit`. The random sample and each mark live in the report's `result_data` (`audit_slots`, and `empty_audit_slots` if requested), so there is no separate table. +- Each aliquot is one tube, so every occupied slot expects exactly one. The details on each row are read from the aliquot when the page opens, so older audits show current information. +- Saving a mark is `POST /reports/audit//save/` with the slot's kind, index, result and note. +- If building the sample fails, no report is created. -### Alert -- System alerts and notifications -- Configurable severity and status levels -- Target-specific alerts (freezer, sample, etc.) +Code: `reports/views.py` (`generate_audit_report`, `generate_storage_audit_data`, `audit_detail`, `audit_list`, `save_audit_result`, `delete_audit`), templates in `reports/templates/reports/`, tests in `reports/tests.py`. -### Issue -- Issue tracking and management -- Priority and status management -- Assignment and resolution tracking +## Alerts -### ScheduledReport -- Automated report generation -- Configurable frequency and recipients -- Parameter management for scheduled reports +**Reports > View Alerts** lists alerts with their severity (low, medium, high, critical), type (storage, sample, system, security, general) and status (active, acknowledged, resolved, dismissed). An active alert can be acknowledged and then resolved. The Reports page shows the count of active alerts and the latest five. -## API Endpoints +## Other pieces -### Dashboard Statistics -- `GET /reports/dashboard/stats/` - Enhanced dashboard statistics - -### Report Generation -- `POST /reports/generate/` - Generate new report -- `GET /reports/list/` - List user reports -- `GET /reports/detail//` - View report details -- `GET /reports/download//` - Download generated report - -### Alert Management -- `GET /reports/alerts/` - List all alerts -- `POST /reports/alerts/create/` - Create new alert -- `POST /reports/alerts//acknowledge/` - Acknowledge alert -- `POST /reports/alerts//resolve/` - Resolve alert - -### Issue Tracking -- `GET /reports/issues/` - List all issues -- `POST /reports/issues/report/` - Report new issue - -### Sample Search -- `GET /reports/samples/search/` - Search samples with filters - -### Storage Dashboard -- `GET /reports/storage/dashboard/` - Storage overview statistics - -## Usage - -### Generating Reports -1. Navigate to Reports Dashboard -2. Click "Generate Report" -3. Select template and configure parameters -4. Choose output format -5. Submit for generation - -### Managing Alerts -1. View active alerts in the Alerts section -2. Acknowledge alerts when reviewed -3. Resolve alerts when issues are fixed -4. Create new alerts for system issues - -### Reporting Issues -1. Use the Issue Tracking section -2. Fill out issue details and priority -3. Assign to team members if needed -4. Track resolution progress - -### Searching Samples -1. Use the Sample Search interface -2. Enter search criteria and filters -3. View results with pagination -4. Export results as needed - -## Configuration - -### Report Templates -Configure report templates in the Django admin: -- Define report structure and parameters -- Set output format options -- Configure access permissions - -### Alert Rules -Set up alert rules for: -- Storage capacity thresholds -- Temperature monitoring -- System performance metrics -- User activity monitoring - -### Scheduled Reports -Configure automated reports for: -- Daily/weekly/monthly summaries -- Regulatory compliance reports -- Performance monitoring -- Data archiving - -## Dependencies - -- Django 5.0+ -- Python 3.10+ -- SQLite/PostgreSQL/MySQL database -- Material Icons for UI elements - -## Installation - -1. Add `reports` to `INSTALLED_APPS` in Django settings -2. Run migrations: `python manage.py migrate` -3. Include reports URLs in main URL configuration -4. Access via `/reports/` URL path - -## Customization - -### Adding New Report Types -1. Extend the `ReportTemplate` model choices -2. Create custom report generation logic -3. Add template configuration options -4. Update admin interface - -### Custom Alert Types -1. Define new alert types in the Alert model -2. Create alert generation logic -3. Configure alert rules and thresholds -4. Add custom alert handling - -### Extended Search -1. Add new search fields to Sample model -2. Extend search form and logic -3. Update search results display -4. Add export functionality - -## Contributing - -1. Follow Django coding standards -2. Add tests for new functionality -3. Update documentation -4. Submit pull requests - -## License - -This app is part of the Krill system and follows the same licensing terms. +- `/reports/list/` lists the signed-in user's own reports, including audits. +- `/reports/storage/dashboard/` shows storage capacity statistics. +- `/reports/dashboard/stats/` returns summary numbers as JSON. +- The `generate_report` endpoint and the report types in `ReportTemplate` (sample inventory, storage capacity, user activity, aliquot tracking) are not offered anywhere in the interface. Only the storage audit can be run from the page today. diff --git a/krill/reports/templates/reports/reports_dashboard.html b/krill/reports/templates/reports/reports_dashboard.html index 5141133..837b1dc 100644 --- a/krill/reports/templates/reports/reports_dashboard.html +++ b/krill/reports/templates/reports/reports_dashboard.html @@ -5,103 +5,73 @@ {% block content %}
-
-

Reports Dashboard

-

Generate reports, manage alerts, and track system activity

+

Reports

+

Check that what Krill says is in storage is really there, and review alerts

- - -
-

Recent Reports

- +

Recent Audits

+ View All
- {% if recent_reports %} - {% for report in recent_reports %} -
- - {% if report.status == 'complete' %} - check_circle - {% elif report.status == 'pending' %} - schedule - {% elif report.status == 'failed' %} - error - {% else %} - description - {% endif %} - -
-

{{ report.name }} - {{ report.template.report_type|title }} report

- {{ report.created_at|timesince }} ago - Status: {{ report.get_status_display }} -
+ {% for item in audits %} + + {% if item.progress.complete %}check_circle{% else %}pending{% endif %} +
+

{{ item.report.name }}

+ {{ item.progress.checked }} of {{ item.progress.total }} checked + {% if item.progress.wrong %}• {{ item.progress.wrong }} wrong{% endif %} + • {{ item.report.created_by.username }} • {{ item.report.created_at|timesince }} ago
- {% endfor %} - {% else %} -
- info -
-

No recent reports found

- Generate your first report to see activity here -
+
+ {% empty %} +
+ info +
+

No audits yet

+ Start a storage audit to spot-check that tubes are where Krill says
- {% endif %} +
+ {% endfor %}
@@ -143,25 +113,4 @@

Recent Alerts

- {% endblock %} diff --git a/krill/reports/tests.py b/krill/reports/tests.py index df1700c..238c8c4 100644 --- a/krill/reports/tests.py +++ b/krill/reports/tests.py @@ -522,3 +522,39 @@ def test_checklist_shows_what_is_on_each_tube(self): for expected in ('HeLa', 'Pellet', 'Passage p8', 'Nov 1, 2019', 'By MS', f'Aliquot #{aliquot.id}', 'In Storage', 'ER+ line'): self.assertContains(page, expected) + + +class ReportsHomeTest(StorageAuditReportTest): + """The Reports page leads with the storage audit and shows only numbers that are true.""" + + def home(self): + self.client.force_login(self.user) + response = self.client.get(reverse('reports:reports_dashboard')) + self.assertEqual(response.status_code, 200) + return response + + def test_counts_audits_and_those_still_in_progress(self): + first = self.post_audit(sample_size=2).json()['report_id'] + self.post_audit(sample_size=2) + self.client.post(reverse('reports:save_audit_result', args=[first]), + json.dumps({'kind': 'audit_slots', 'index': 0, 'result': 'correct'}), content_type='application/json') + page = self.home() + self.assertEqual((page.context['audit_count'], page.context['audits_in_progress']), (2, 2)) + for index in (1,): + self.client.post(reverse('reports:save_audit_result', args=[first]), + json.dumps({'kind': 'audit_slots', 'index': index, 'result': 'wrong'}), content_type='application/json') + self.assertEqual(self.home().context['audits_in_progress'], 1) # the first is now finished + + def test_recent_audits_link_to_the_checklist(self): + report_id = self.post_audit(sample_size=2).json()['report_id'] + self.assertContains(self.home(), reverse('reports:audit_detail', args=[report_id])) + + def test_page_has_no_script_that_overwrites_the_counts_and_no_dead_report_types(self): + page = self.home() + self.assertNotContains(page, 'loadDashboardStats') # it reset Total/Pending Reports to 0 every 30s + self.assertNotContains(page, 'Pending Reports') + self.assertNotContains(page, 'All Reports') + self.assertContains(page, 'Start Storage Audit') + + def test_page_with_nothing_yet_explains_what_to_do(self): + self.assertContains(self.home(), 'No audits yet') diff --git a/krill/reports/views.py b/krill/reports/views.py index 9271158..97e9c08 100644 --- a/krill/reports/views.py +++ b/krill/reports/views.py @@ -32,48 +32,27 @@ # Main Reports Dashboard @login_required def reports_dashboard(request): - """Main reports dashboard view""" - try: - # Get basic statistics - total_reports = Report.objects.count() - pending_reports = Report.objects.filter(status='pending').count() - active_alerts = Alert.objects.filter(status='active').count() - open_issues = 0 # Issue tracking removed - - # Get recent reports - recent_reports = Report.objects.filter(created_by=request.user).order_by('-created_at')[:5] - - # Get recent alerts - recent_alerts = Alert.objects.filter(status='active').order_by('-created_at')[:5] - - # Get recent issues (removed - issue tracking disabled) - recent_issues = [] - - context = { - 'total_reports': total_reports, - 'pending_reports': pending_reports, - 'active_alerts': active_alerts, - 'open_issues': open_issues, - 'recent_reports': recent_reports, - 'recent_alerts': recent_alerts, - 'recent_issues': recent_issues, - } - - return render(request, 'reports/reports_dashboard.html', context) - except Exception as e: - # Fallback to basic template if there are any errors - return render(request, 'reports/reports_dashboard.html', { - 'total_reports': 0, - 'pending_reports': 0, - 'active_alerts': 0, - 'open_issues': 0, - 'recent_reports': [], - 'recent_alerts': [], - 'recent_issues': [], - }) + """Reports home: the storage audit, and the alerts.""" + audits = [] + in_progress = 0 + for report in Report.objects.filter( + template__report_type='storage_audit', status='complete' + ).select_related('created_by')[:200]: + progress = audit_progress(report.result_data) + if progress['total'] and not progress['complete']: + in_progress += 1 + if len(audits) < 5: + audits.append({'report': report, 'progress': progress}) + + return render(request, 'reports/reports_dashboard.html', { + 'audits': audits, + 'audit_count': Report.objects.filter(template__report_type='storage_audit', status='complete').count(), + 'audits_in_progress': in_progress, + 'active_alerts': Alert.objects.filter(status='active').count(), + 'recent_alerts': Alert.objects.filter(status='active').order_by('-created_at')[:5], + }) -# Dashboard Statistics API @login_required def dashboard_stats(request): """Enhanced dashboard statistics API with resilient error handling"""