fix[models]: support cache_system_prompt parameter for model configu… - #41
fix[models]: support cache_system_prompt parameter for model configu…#41rishu685 wants to merge 1 commit into
Conversation
|
I understand this commit is specifically for anthropic, what about other provider? we will do on other commits? or you think its harder? |
|
See what i think is prompt caching is handled differently across providers: --Anthropic is the only one requiring this explicit client-side flag (cache_control) here. Let me know if you'd prefer us to strictly enforce this flag only for Anthropic, or make the change for others! |
This PR addresses #31 by introducing a new
cache_system_promptfield to the model configuration (ModelConfig). This field defaults toTrueand controls whether the system prompt and tool definitions are cached on supported providers (specifically Anthropic).Scope & Implementation Details
cache_system_prompt: bool = Trueto theBaseModelConfigdataclass insrc/agent_engine/core/spec.py._validate_modelinsrc/agent_engine/parsers/yaml/parser.pyto ensurecache_system_promptis validated as a boolean._build_modeland_build_base_modelto correctly parsecache_system_promptfrom the YAML definitions, defaulting toTrue.cache_system_promptparameter tobuild_chat_modelinsrc/agent_engine/models/factory.py._build_anthropic_modelto conditionally passcache_control={"type": "ephemeral"}parameter viainit_chat_model()kwargs whencache_system_promptisTrue._model_factory_kwargsinsrc/agent_engine/engine/langgraph/engine.pyto forwardcache_system_promptfrom node configurations to the model factory.E501) and sorted imports innodes.pyandengine.py.Verification
tests/parsers/test_cache_system_prompt.py.tests/models/test_factory.py.make check) passes cleanly:ruff check src tests: Passingmypy src tests: Passingagentctl validate examples/enterprise-knowledge-assistant/agents.yaml: Passingpytest: Passing (541 tests passed)