Skip to content

Improve robustness of check_df and code refinements#3

Open
tur-ium wants to merge 7 commits into
cornerstone-data:masterfrom
tur-ium:code_refinements
Open

Improve robustness of check_df and code refinements#3
tur-ium wants to merge 7 commits into
cornerstone-data:masterfrom
tur-ium:code_refinements

Conversation

@tur-ium

@tur-ium tur-ium commented May 6, 2026

Copy link
Copy Markdown

In generate_import_factors.py Improve robustness of check_df to input data format (support Flow or Flowable as column names)

Quality-of-life and code security improvements

  • Improve dependency management by adding a pyproject.toml to allow use of uv
  • Add .gitignore file for safety and to avoid committing unnecessary/auto-generated files
  • Add support for loading API keys via dotenv

@tur-ium tur-ium changed the title Code refinements Improve robustness of check_df and code refinements May 6, 2026
@bl-young bl-young self-requested a review May 18, 2026 13:03

@bl-young bl-young left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your help here! This looks good, see my few requests/comments.

Comment thread pyproject.toml
]

[tool.uv]
exclude-newer = "1 week"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't believe this is a valid entry

Comment thread .gitignore
!import_emission_factors/output/*import_factors_exiobase_2019*
!import_emission_factors/output/aggregate*
!import_emission_factors/output/import_shares_2019*
# Standard python gitignore from https://github.com/github/gitignore/

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not opposed to moving to the more standard .gitignore but I'd like to keep the existing ignores in there as well for now

Comment thread pyproject.toml
"fedelemflowlist",
"numpy",
"openpyxl",
"pandas",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"pandas",
"pandas[excel]",

I get an ImportError as pandas requires openpyxl >= 3.1.5. I think setting this optional install should resolve it.

Comment thread pyproject.toml
"numpy",
"openpyxl",
"pandas",
"pymrio",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"pymrio",
"pymrio",
"python-dotenv",

Let's go ahead and add python-dotenv as well.

## summary sectors.
multiplier_df = df_prepare(multiplier_df, year)
check = (multiplier_df
.query('Flow == @multiplier_df["Flow"][0]')

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this chunk causing you problems? I think it was an oversight. We should probably always use Flowable instead of Flow which should work for all instances (there's no need to have both options in here).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants