Skip to content

Lazy import lock contention (run in background threads) #158638

Description

@error-3317

Bug report

Description

When a lazily imported module is resolved (reified) in a background thread, the global import lock is held for the entire duration of that module's initialization. If the module has a slow initialization (e.g., heavy I/O, network requests, or computationally expensive top-level code), it completely blocks other threads from performing unrelated, otherwise fast imports.

This creates severe lock contention in multi-threaded applications that utilize lazy imports to improve startup time, as a background thread resolving a lazy import will stall the main thread (or other worker threads) if they attempt to import anything else.

Reproducible Example

Below is a minimal reproducible example demonstrating this behavior:

# slowmod.py
import time
print('-- [slowmod] Starting load...')
time.sleep(3)
print('-- [slowmod] Load complete!')
# lazy_wrapper.py
lz = __lazy_import__('slowmod')
# fastmod.py
print('-- [fastmod] Loaded instantly!')
# main.py
import threading
import time
import sys
import lazy_wrapper

def lazy_thread():
    print('[Thread-Lazy] Triggering resolution by accessing attribute...')
    # Accessing __name__ triggers the lazy import reification
    name = lazy_wrapper.lz.__name__
    print(f'[Thread-Lazy] Resolved module: {name}')

def eager_thread():
    time.sleep(0.5) 
    print('[Thread-Eager] Attempting to import fastmod...')
    start = time.time()
    import fastmod
    end = time.time()
    print(f'[Thread-Eager] fastmod imported in {end - start:.4f} seconds.')

t1 = threading.Thread(target=lazy_thread)
t2 = threading.Thread(target=eager_thread)

t1.start()
t2.start()

t1.join()
t2.join()

Actual output

[Thread-Lazy] Triggering resolution by accessing attribute...
-- [slowmod] Starting load...
[Thread-Eager] Attempting to import fastmod...
-- [slowmod] Load complete!
[Thread-Lazy] Resolved module: slowmod
-- [fastmod] Loaded instantly!
[Thread-Eager] fastmod imported in 2.5001 seconds.

Expected output

...
[Thread-Eager] fastmod imported in 0.0001 seconds.
...

CPython versions tested on:

3.15

Operating systems tested on:

Linux

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    interpreter-core(Objects, Python, Grammar, and Parser dirs)performancePerformance or resource usagetopic-lazy-importstype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions