[SERVER] Add Kubernetes engine Pod discovery latency metric

[ruanwenjun]

Why are the changes needed?

A Spark engine Pod can already exist in Kubernetes while Kyuubi has not yet received its informer event. This delay can contribute to an engine submission timeout, but the existing engine startup metric only covers engines that register successfully.

Add kyuubi.engine.kubernetes.pod.discovery.latency, a histogram in milliseconds from Pod creation to entry into the informer ADD callback. Each ADD for a Spark engine Pod records a sample, including initial listing and repeated ADD events. UPDATE and DELETE events do not record samples. Missing, malformed, or future creation timestamps are skipped without preventing Pod event handling.

How was this patch tested?

Validated with JDK 17. dev/reformat ran without the optional Python formatter (no Python files changed).

Added tests in KubernetesApplicationOperationSuite for ADD latency, repeated ADD events, first UPDATE and DELETE exclusion, invalid timestamps, continued terminal-state handling, and non-engine Pods, without a Kubernetes cluster. All 8 tests passed.

dev/reformat
build/mvn --batch-mode --no-transfer-progress test -pl kyuubi-server -am -Dtest=none -DwildcardSuites=org.apache.kyuubi.engine.KubernetesApplicationOperationSuite -Pflink-provided,hive-provided,spark-provided
git diff --check

Was this patch assisted by generative AI tooling?

Assisted-by: Codex (GPT-6)