EHR queries - #103

Open

jeremyestein wants to merge 88 commits into

devfrom

ehr_queries_clean

Commits on Feb 2, 2026

first query for flow sheets
skeating
committed
820840d

Commits on Feb 4, 2026

building up scripts
skeating
committed
fb3d6da

Commits on Feb 19, 2026

adding more files
skeating
committed
8476351

Commits on Aug 5, 2026

Merge branch 'dev' into sk/queries
skeating
committed
47240ff

Commits on Sep 9, 2026

Add and refine SQL scripts for EHR data extraction

skeating
authored and
jeremyestein
committed
56ceb52
Refactor lab results query for wide format output

skeating
authored and
jeremyestein
committed
de65f8d
Started on functions to wrap snakemake around sql calls

thompson318
authored and
jeremyestein
committed
4426711
Started to implement caboodle look up code

thompson318
authored and
jeremyestein
committed
1b8545a
Setting up config and docker bits

thompson318
authored and
jeremyestein
committed
a854492
Add a ehr location to snakefile

thompson318
authored and
jeremyestein
committed
7784854
Tidied up implementation and rewrote Snakefile logic, with help from Claude Sonnet 5.0

thompson318
authored and
jeremyestein
committed
bdf0eb2
We don't need a separate container for ehr lookup

thompson318
authored and
jeremyestein
committed
f4382d2
Added ehr variables to settings and added a testing flag.

thompson318
authored and
jeremyestein
committed
de12629
added some caboodle faking logic to enable local running

thompson318
authored and
jeremyestein
committed
314a613
Caboodle setup goes into exporter.env

thompson318
authored and
jeremyestein
committed
f33621d
Format fix

thompson318
authored and
jeremyestein
committed
de875db
Beginning of test for ehr integration

thompson318
authored and
jeremyestein
committed
43e9f54
SQL formatting

thompson318
authored and
jeremyestein
committed
21e0e5b
Test with ehr lookup

thompson318
authored and
jeremyestein
committed
2b28e42
use caboodle testing

thompson318
authored and
jeremyestein
committed
d4f2121
Trying to test ehr and hash lookup with ftps

thompson318
authored and
jeremyestein
committed
a704072
Moved sql

thompson318
authored and
jeremyestein
committed
8bd10d2
refactored db to make adding ehr lookup implementation clearer

thompson318
authored and
jeremyestein
committed
76c8d9f
Started on logic to get flowsheets, first need hospital visit number

thompson318
authored and
jeremyestein
committed
b7e5ce3
The exporter needs to access uds, do duplicated some of the config from controller

thompson318
authored and
jeremyestein
committed
87a8ea1
Use enviroment to manage sql locations as it varies between containers

thompson318
authored and
jeremyestein
committed
8219da8
path into test

thompson318
authored and
jeremyestein
committed
76144af
Use format rather than parameters for schema name

thompson318
authored and
jeremyestein
committed
093563f
Default for sql path to simplify testing

thompson318
authored and
jeremyestein
committed
ece4174
Updated mocked function name in test controller

thompson318
authored and
jeremyestein
committed
6d3f470
Enable a fake star db shortcut so that snakemake integration test can run

thompson318
authored and
jeremyestein
committed
d6b51e8
Don't connect for fake star

thompson318
authored and
jeremyestein
committed
f4a629b
Tidied up getting for hospital visit number

thompson318
authored and
jeremyestein
committed
df0db43
return integer hospital visit id

thompson318
authored and
jeremyestein
committed
e84f031
(with rewritten history) Merge branch 'sk/queries' into st/queries-to-snakemake
jeremyestein
committed
1363c98
Added flowsheet query to ehr

thompson318
authored and
jeremyestein
committed
ca4789a
moved sql and implemented lab results

thompson318
authored and
jeremyestein
committed
cba9b54
Fixed column names in pseudon

thompson318
authored and
jeremyestein
committed
c0850e5
Flowsheets and labresults are star, not caboodle

thompson318
authored and
jeremyestein
committed
f497fc4
Auto stash before merge of "sk/queries" and "st/queries-to-snakemake"

skeating
authored and
jeremyestein
committed
8a08337
star flow sheet query

skeating
authored and
jeremyestein
committed
b93c17d
finished star lab queries

skeating
authored and
jeremyestein
committed
2fc2240
removed un new scripts and moved read me to current script location

skeating
authored and
jeremyestein
committed
f946149
added note about getting scripts from the waveform-private-queries repository as part of deployment

skeating
authored and
jeremyestein
committed
e249346
Merge branch 'dev' into sk/queries
jeremyestein
committed
65c74fd
Post-merge linting fix
jeremyestein
committed
1218cb5
SCHEMA_NAME is accessed regardless of testing mode, so it has to exist
jeremyestein
committed
6de2a63

Commits on Sep 10, 2026

Create a space for scripts that can't be in this repo to be installed to
jeremyestein
committed
8c4bf82
Bring in private SQL scripts
jeremyestein
committed
7968f17

Commits on Sep 11, 2026

Linting fix
jeremyestein
committed
85ea6ad
Add private repo to setup instructions
jeremyestein
committed
1e44668

Commits on Sep 14, 2026

Commits on Sep 15, 2026

First go at split of db.py into PG and SQL Server versions
jeremyestein
committed
0163626
Cursor's first go at PG->MSSQL conversion
jeremyestein
committed
edc39a7
Delete pointless test that doesn't test anything
jeremyestein
committed
3d22cba
Define all Caboodle timeouts in seconds to match API
jeremyestein
committed
39d5be6
timeout needs to be an int
jeremyestein
committed
523407f
Was getting runtime type error in pd.concat. Now always convert
jeremyestein
committed
4312789
Fix typo in SQL query
jeremyestein
committed
d2a6775
Also return DataFrame for MSSQL queries
jeremyestein
committed
3a85bc9
MSSQL column info is a bit different
jeremyestein
committed
1d5f07d
Update the list of safe EHR columns to match the queries
jeremyestein
committed
76989a9

Commits on Sep 16, 2026

First attempt at doing TZ conversion on the DB query input params. Still
jeremyestein
committed
5669337
Tidy up file structure and naming conventions
thompson318
committed
b75f8ce
Do pre- and post-query TZ adjustment for SQL Server
jeremyestein
committed
f6944e1
Merge branch 'ehr_queries_clean' into st/get_ehr
thompson318
committed
ec62ca0
Prefix EHR query result columns so we don't mix them up
jeremyestein
committed
23d683b
Fixed test_ehr
thompson318
committed
56a1ac6
Add sputum query
jeremyestein
committed
fd7214e
Attempt to switch to parquet but there are some numeric type conversion
jeremyestein
committed
cc7c454
Fix linting
jeremyestein
committed
a22267c
These come as floats from Emap so keep them as such here
jeremyestein
committed
aba26b0
Assert on some EHR parquet data and fix timezone issue in parquet (no
jeremyestein
committed
f37a639
More data assertions
jeremyestein
committed
54cd710

Commits on Sep 17, 2026

Check the entire returned dataframe
jeremyestein
committed
8209ee7
Exclude EHR lookups from the integration test as these have their own
jeremyestein
committed
6151fa3
Delete the test EHR code as it was out of date now and we have another
jeremyestein
committed
6cbbf50
Bring in file layout changes but this was assuming CSV EHR data so will
jeremyestein
committed
ea3ea7d
Merge fixup: make filenames reflect that they're parquets. Put them in
jeremyestein
committed
3072b40
Fixup to remove EHR from integration test
jeremyestein
committed
51767f3
Add exporter config flag to bypass EHR lookup with dummy output file so
jeremyestein
committed
3c8a0c3
Include EHR files in the DSH upload
jeremyestein
committed
fc2d192
Had forgotten TubeType column
jeremyestein
committed
19cd385
GROUP BY was using old names
jeremyestein
committed
73db88a
Add logging to show number of rows retrieved from EHR
jeremyestein
committed
9c51055