[tkaymak]

Part of #36841. Cleans up obsolete DStream test remnants in the Structured Streaming test suite and adds streaming lifecycle coverage for timeout handling.

Context

The shared StructuredStreamingPipelineStateTest retained four tests marked with @ignore("TODO: Reactivate with streaming."). They used CreateStream, which is a legacy DStream concept not supported by Structured Streaming. Spark 4 now has dedicated streaming lifecycle tests in StreamingPipelineLifecycleTest covering RUNNING to DONE on idle, cancellation, and sibling query failure propagation.

What this changes

• runners/spark/src/test/.../StructuredStreamingPipelineStateTest.java: Removed the four obsolete @ignore streaming test cases. Inlined Create.of("one", "two") at call sites and removed the obsolete getValues() and getStreamingOptions() helpers and CreateStream imports.
• runners/spark/src/test/.../translation/streaming/SimpleSourceTest.java: Deleted this obsolete, ignored file.
• runners/spark/4/src/test/.../translation/streaming/StreamingPipelineLifecycleTest.java: Added timeoutKeepsRunningState to assert that waitUntilFinish(Duration.millis(1)) preserves RUNNING state while the query remains active, with try, finally query cancellation to prevent leaks in the shared session.

Tests

• :runners:spark:3:test passed locally (StructuredStreamingPipelineStateTest).
• :runners:spark:4:test passed locally (StreamingPipelineLifecycleTest, 4 passed).
• checkstyle and spotless verified clean on Spark 3 and Spark 4.

R: @Abacn

[Abacn]

classMethod (org.apache.beam.examples.complete.kafkatopubsub.KafkaToPubsubE2ETest) failed

known flaky test, not related to this change