Repository navigation
register() misses libraries that were listed before it was called #2
Description
Since joblib#249, _CDLLCache.create_controller remembers, for the lifetime of the process, every listed shared library that no controller matched (_controller_cache[original_filepath] = (None, "", "")). register() appends the new controller to _ALL_CONTROLLERS but does not clear that cache. A library that was listed before register() was called is therefore never matched by the new controller.
On Linux every loaded .so is listed. Any ThreadpoolController() or threadpool_info() call made between loading the library and registering its controller, by any package, is enough to hide the library. test_custom_controller doesn't catch this because tests/_pyMylib loads the library and registers the controller in the same import.
register() doesn't invalidate the controller returned by get_cached_controller() (joblib#257) either.
Reproducer (Linux)
import threadpoolctl
from threadpoolctl import LibController, register, threadpool_info
class FakeController(LibController):
# Matches glibc, which is always loaded on Linux.
user_api = "fake"
internal_api = "fake"
filename_prefixes = ("libc.",)
check_symbols = ("printf",)
def get_num_threads(self):
return 1
def set_num_threads(self, num_threads):
pass
def get_version(self):
return None
print(threadpoolctl.__version__)
threadpool_info() # e.g. done by any library before the controller is registered
register(FakeController)
print([lib["internal_api"] for lib in threadpool_info()])3.7.0
['fake']
3.8.0.dev0
[]
Suggested fix
Clear both caches in register():
def register(controller):
"""Register a new controller"""
_ALL_CONTROLLERS.append(controller)
_ALL_USER_APIS.append(controller.user_api)
_ALL_INTERNAL_APIS.append(controller.internal_api)
_ALL_PREFIXES.extend(controller.filename_prefixes)
# Libraries listed before this call may match the new controller.
_CDLL_CACHE._controller_cache.clear()
reset_cached_controller()Dropping only the negative entries would be enough, but register() is rare, so clearing everything costs one extra scan. The reproducer above can become the regression test.
Activity
cakedev0 commentedon Oct 29, 2026
Fixed upstream in joblib#287.