Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 15 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ async def main():
# Read multiple parquet files
import glob
paths = glob.glob("/cache/sinan/**/*.parquet")
df = pysus.read_parquet(paths, mode="union").df()
df = pysus.read_parquet(paths, mode="union")
```

### Using the Streamlit Web UI (experimental feature)
Expand Down Expand Up @@ -207,16 +207,16 @@ async with PySUS() as pysus:

```python
# Union mode (default) - includes all columns from any file
df = pysus.read_parquet(paths, mode="union").df()
df = pysus.read_parquet(paths, mode="union")

# Intersection mode - only common columns across all files
df = pysus.read_parquet(paths, mode="intersection").df()
df = pysus.read_parquet(paths, mode="intersection")

# Strict mode - raises error if schemas don't match
df = pysus.read_parquet(paths, mode="strict").df()
df = pysus.read_parquet(paths, mode="strict")

# With custom SQL
df = pysus.read_parquet(paths, sql="SELECT * WHERE column > 100").df()
df = pysus.read_parquet(paths, sql="SELECT * WHERE column > 100")
```

## Configuration
Expand All @@ -235,19 +235,23 @@ pysus = PySUS(db_path='/my/config.db')
### Environment Variables

- `PYSUS_CACHEPATH`: Directory for cached files
- `DADOSGOV_TOKEN`: API token for the dados.gov.br client
(required for DadosGov downloads)
- `ACCESS_KEY` / `SECRET_KEY`: S3 credentials for writing to the
PySUS bucket (catalog sync/maintenance)

## Data Sources

| Dataset | Description | Source |
|---------|-------------|--------|
| SINAN | Disease Notifications | FTP / DuckLake |
| SIM | Mortality | FTP / DuckLake |
| SINASC | Births | FTP / DuckLake |
| SINAN | Disease Notifications | FTP / DuckLake / DadosGov |
| SIM | Mortality | FTP / DuckLake / DadosGov |
| SINASC | Births | FTP / DuckLake / DadosGov |
| SIH | Hospitalizations | FTP / DuckLake |
| SIA | Ambulatory | FTP / DuckLake |
| CIHA | Hospital Admissions | FTP / DuckLake |
| CNES | Health Facilities | FTP / DuckLake |
| PNI | Immunizations | FTP / DuckLake |
| CNES | Health Facilities | FTP / DuckLake / DadosGov |
| PNI | Immunizations | FTP / DuckLake / DadosGov |
| IBGE | Geographic Data | FTP / DuckLake |


Expand Down Expand Up @@ -275,7 +279,7 @@ pre-commit run --all-files

Run tests:
```bash
pytest tests/
pytest pysus/tests/
```

Run tests inside the Docker container:
Expand Down
1 change: 1 addition & 0 deletions pysus/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,3 +24,4 @@ def get_version() -> str:
__version__: str = version

from pysus.api._impl.databases import * # noqa
from pysus.api.client import PySUS # noqa
Loading