feat: Implement advanced cache management features

[frenzel-ai-dev]

This commit introduces several enhancements to the job caching mechanism:

1. Cache Size Limits & LRU Eviction:

   • The JobCache now supports limits based on the maximum number of files (max_cache_files) and the total cache size in bytes (max_cache_size_bytes).
   • An LRU (Least Recently Used) eviction policy has been implemented to automatically remove older cache entries when these limits are exceeded.
   • File access times are updated on cache hits to ensure the LRU logic is accurate.

2. Granular Cache Clearing:

   • The clear_cache method in JobCache has been enhanced to allow clearing by specific site, query, and location.
   • A new method, clear_expired_cache, has been added to remove only cache entries that have passed their defined expiration time.

3. Cache Management CLI:

   • A new CLI script, job_recommender/manage_cache_cli.py, provides command-line tools for cache administration.
   • You can now clear the cache (entirely, or filtered by site/query/location) and clear only expired entries via this CLI.
   • The CLI accepts a --cache-dir argument to specify the cache directory, which also aids in testing.

4. Unit Tests:

   • Comprehensive unit tests have been added for the new cache eviction policies (file count, size, LRU logic), granular clearing functions, and CLI commands.
   • All tests are passing.

5. Linting & Fixes:

   • Critical linting errors (e.g., undefined names) have been resolved.
   • Some stylistic linting issues (line length, unused imports) remain but do not impact functionality.

These changes provide more robust and flexible control over the caching system, preventing uncontrolled growth and allowing for more targeted cache management.

[Copilot]

Pull Request Overview

This PR adds advanced management to the job caching system, including LRU eviction based on file count and total size, granular and expired-cache clearing methods, a new CLI for cache administration, and accompanying unit tests.

• Introduce max_cache_files and max_cache_size_bytes limits with LRU eviction in JobCache.
• Enhance clear_cache to filter by site/query/location and add clear_expired_cache.
• Provide a manage_cache_cli.py script with --cache-dir, plus comprehensive tests.

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tests/test_manage_cache_cli.py	CLI tests for clear and clear-expired commands with --cache-dir.
tests/test_cache.py	Unit tests for caching, eviction, clearing, and expiration logic.
job_recommender/cache.py	Added file/size limits, LRU eviction, and refined clear methods.
job_recommender/manage_cache_cli.py	New CLI script supporting cache management commands.
job_recommender/parallel_scraper.py	Added (unused) rich.progress import.
job_recommender/job_scraper.py	Moved ParallelJobScraper import into main() to break a cycle.