[theobigand]

No description provided.

[Copilot]

Pull request overview

This PR focuses on adding/clarifying Python type hints across the backend ETL, DB layer, API routes, and tests to improve readability and static analysis.

Changes:

• Added return/parameter type annotations across core DB, ETL, and API modules.
• Updated test helper fakes and test function signatures with typing.
• Added typing imports (Any, Hashable, etc.) where needed for new annotations.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
backend/src/test/etl/test_extract.py	Adds typing for tmp_path fixture and return type.
backend/src/test/db/test_connect.py	Adds typing for fake Mongo client + test return types.
backend/src/etl/transform.py	Types ETL transform helpers (_to_bool, _to_int_safe).
backend/src/etl/extract.py	Types ETL extract helpers and extract() return type.
backend/src/core/logger.py	Types get_logger() return value.
backend/src/core/db/doctrine.py	Types Doctrine CRUD methods and helpers.
backend/src/core/db/connect.py	Types ConnectDB methods.
backend/src/core/classes/logs.py	Adds return types for properties/setters.
backend/src/core/classes/cars.py	Adds return types for properties/setters.
backend/src/api/v1/etl/workflow.py	Types workflow endpoint response.
backend/src/api/v1/etl/transform.py	Types transform endpoint response.
backend/src/api/v1/etl/load.py	Types load endpoint response.
backend/src/api/v1/etl/extract.py	Types extract endpoint response.
backend/src/api/v1/dataset/get_dataset.py	Types dataset endpoint return structure.
backend/src/api/v1/data/get_logs.py	Types logs endpoints responses.
backend/src/api/v1/data/get_cars.py	Types cars endpoints responses.

Comments suppressed due to low confidence (3)

backend/src/api/v1/etl/load.py:16

• This endpoint calls transform_from_file(), which will fail at runtime if extract() returns list[Cars] (since transform_cars expects a pd.DataFrame). Fix by aligning extract()/transform_from_file() types (convert to DataFrame or adjust extract() to return a DataFrame).

def run_load()-> dict[str, int | str]:
    try:
        project_root = Path(__file__).parent.parent.parent.parent.parent.parent
        csv_path = project_root / "cfg" / "dataset-cars-dirty.csv"
        df = transform_from_file(str(csv_path), logger=logger)
        data = df.to_dict(orient='records')

backend/src/api/v1/etl/extract.py:13

• This route returns data directly from extract(), which is a list[Cars] (custom class instances) and is not JSON-serializable by default. Serialize the cars (e.g. to list[dict] / Pydantic models) before returning, or change extract() to return JSON-safe structures.

def run_extract() -> dict[str, Any]:
    project_root = Path(__file__).parent.parent.parent.parent.parent.parent
    csv_path = project_root / "cfg" / "dataset-cars-dirty.csv"
    data = extract(str(csv_path))
    return {"statusCode": 200, "message": "Data extracted successfully", "data": data}

backend/src/api/v1/etl/transform.py:16

• transform_from_file() currently relies on extract() output being a pd.DataFrame, but extract() returns list[Cars] (and transform_cars expects a DataFrame). This endpoint will raise at runtime unless you convert the extracted cars into a DataFrame before calling transform_cars (or change extract() to return a DataFrame).

def run_transform() -> dict[str, Any]:
    try:
        project_root = Path(__file__).parent.parent.parent.parent.parent.parent
        csv_path = project_root / "cfg" / "dataset-cars-dirty.csv"
        df = transform_from_file(str(csv_path), logger=logger)
        return {

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The monkeypatch target cfg.db.classes.connect.MongoClient does not exist in this repo, so these tests won't patch MongoClient and may hit the real database. Patch the symbol where MongoClient is imported/used (e.g. backend.src.core.db.connect.MongoClient).

[Copilot]

Same issue as above: cfg.db.classes.connect.MongoClient is not a valid import path here, so the patch won’t apply. Patch backend.src.core.db.connect.MongoClient (or the exact module where ConnectDB resolves MongoClient).

[Copilot]

Same issue as above: the monkeypatch path cfg.db.classes.connect.MongoClient doesn't exist, so the failure-path test won't exercise the intended code. Patch the MongoClient symbol in backend.src.core.db.connect instead.

[Copilot]

extract() is annotated to return list[Cars] and currently builds/returns Cars objects, but other parts of the codebase (e.g. transform_from_file/ETL routes) treat the extract output like a pd.DataFrame. Either change extract() to return a DataFrame (and move object construction elsewhere) or update downstream callers to convert list[Cars] into a DataFrame before transforming/loading.

[Copilot]

Parameter typing is inconsistent: args is annotated as Dict[str, Any] but defaults to None. Update the annotation to Optional[...] (or broaden the type to also accept list[Dict[str, Any]] since the implementation supports insert_many).