Feature/table download - #130
Merged
Merged
Conversation
… processing, and LLM layer documentation Introduced comprehensive documentation covering the backend architecture, API surface, data models, Pydantic schemas, document processing workflows, and the LLM provider layer. This includes detailed descriptions of application entry points, runtime layers, package responsibilities, and main data flows, enhancing the overall understanding of the backend system.
… data models, document processing, LLM routing, extraction, evaluation, session sharing, and authentication. Added diagrams to improve understanding of system interactions and data flows.
…xisting documentation with additional links for better navigation. Added a new class reference appendix for detailed field-level insights on ORM models, Pydantic schemas, and service classes.
Introduced a comprehensive README file detailing the Science-GPT web application, its purpose, target users, functionality, testing methodology, and evaluation results. This documentation aims to provide a clear overview of the tool's capabilities and the benefits it offers to scientific reviewers in processing large documents.
- Introduced `14-batch-results.md` detailing the Batch Results page, including UI sections, table structure, fuzzy search, row detail modal, Excel export, state management, and API calls. - Updated `README.md` to include an overview of the Batch Results feature and its integration within the Science-GPT frontend. - Created `component-index.md` to catalog shared components used across multiple pages. - Added `hooks-contexts.md` to document custom hooks and context providers utilized in the frontend. - Defined key TypeScript interfaces in `types-interfaces.md` for better understanding of the data structures used. - Established a glossary to clarify project-specific terms and Azure services relevant to the Science-GPT project.
- TablesGallery: download individual tables as HTML or all tables bundled into a single file for Excel import - auth-service: conditional SSL for local vs Azure Postgres; run Better Auth DB migrations on startup
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Replace .includes("azure.com") substring match with URL parsing so
the azure.com check only matches the hostname (exact or subdomain),
and sslmode is checked via searchParams rather than a full-string
include. Resolves CodeQL js/incomplete-url-substring-sanitization.
Sam120204
reviewed
Jun 1, 2026
Sam120204
left a comment
Sam120204
left a comment
Contributor
There was a problem hiding this comment.
OECD GD 150 (2018) Revised Guidance Document 150 on Standardised Test Guidelines for Evaluating Chemicals for Endocrine Disruption-CURRENT.pdf I tested with this large file, it contains 266 tables, and the issue right now is that the backend is busy downloading the tables so everything is blocking and showing timeout (like displaying pdf in the frontend). im thinking about maybe implmenting batching downloading tables. but i tried other smaller files, and download is working. so i guess optimizing the download for larger files would be the priority. But overall looks good :)
Contributor
|
also remember to merge from the origin/main to this branch :) |
spencer-crook
self-requested a review
June 11, 2026 19:26
Documents with 266+ tables caused the PDF viewer to block on load and
"Download All" to time out. Three targeted fixes:
- Add GET /documents/{id}/tables/download endpoint that fetches all table
HTML files concurrently via asyncio.gather() and returns a single ZIP,
collapsing 266 HTTP round-trips into one (32x faster in benchmarks)
- Wrap tmp_file.read_bytes/write_bytes in asyncio.to_thread() so /tmp
cache I/O no longer blocks the async event loop during burst requests
- Lazy-load TablePreview cards with IntersectionObserver so only visible
cards fetch their HTML on render (eliminates 266 simultaneous requests
that were starving the PDF viewer of browser connections)
…Tstream2/SummarizationTool into feature/table-download
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Added the ability for users to download tables and extract them for use in excel and other apps