Skip to content

Fix pickling contextlib-decorated functions - #620

Open
jipeng6036-del wants to merge 1 commit into
cloudpipe:masterfrom
jipeng6036-del:fix/contextlib-decorator-pickling-509
Open

jipeng6036-del wants to merge 1 commit into
cloudpipe:masterfrom
jipeng6036-del:fix/contextlib-decorator-pickling-509

Conversation

@jipeng6036-del

Copy link
Copy Markdown

Fixes #509.

Functions decorated with contextlib.contextmanager or asynccontextmanager retain an unused generator in the decorator closure, which currently prevents by-value serialization. I added reducers that recreate an unused context manager from its native generator factory and arguments, preserving custom state and shared references.

The reducers reject generators that have started or closed and factories whose calls could run construction side effects. Context managers backed by already-pickleable ordinary iterators keep their existing object reduction and iterator position. Ordinary generators remain unsupported.

Validation:

  • Full project tests on Windows CPython 3.10–3.14 and Linux CPython 3.8/3.9 and PyPy 3.9; no failures. The final test-only adjustment accepts the Python 3.8 async exit return value; the final regressions were rerun on every interpreter (48 passed on Python >=3.10; 46 passed and two stdlib capability skips on Python 3.8/3.9), and the final tree also passed the complete Python 3.8 suite. Python <3.10 skips the two async decorator cases because the stdlib lacks that capability.
  • Fresh-process standard-pickle consumers check repeated calls, return values, shared state, exception propagation, and cleanup. An independently installed wheel passes the same four consumers; wheel/sdist source bytes match Git.
  • Configured pre-commit command and changed-file hooks pass. An optional --all-files run finds the same Black formatting difference in the unmodified ipykernel test file on both baseline and candidate. Subprocess coverage setup, combination, and XML generation were exercised. macOS and the remaining OS/version combinations are left to upstream CI.

Raw verification results.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cannot pickle functions decorated through contextlib

1 participant