Repository navigation
[Bug]: [RunInference] When load_model_at_runtime=True, the first batch always ignores model side-input updates. #40665
Description
What happened?
When load_model_at_runtime=True, the first batch during inference always ignores model side-input updates, since we always go into self.model = None branch. This can create inferences served with stale models.
Issue Priority
Priority: 2 (default / most bugs should be filed as P2)
Issue Components
- Component: Python SDK
- Component: Java SDK
- Component: Go SDK
- Component: Typescript SDK
- Component: IO connector
- Component: Beam YAML
- Component: Beam examples
- Component: Beam playground
- Component: Beam katas
- Component: Website
- Component: Infrastructure
- Component: Spark Runner
- Component: Flink Runner
- Component: Prism Runner
- Component: Twister2 Runner
- Component: Hazelcast Jet Runner
- Component: Google Cloud Dataflow Runner
Activity