Temporary mitigation for Dataflow Managed Iceberg ITs by ahmedabu98 · Pull Request #40028 · apache/beam

ahmedabu98 · 2026-09-05T00:56:10Z

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
Update CHANGES.md with noteworthy changes.
If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

ahmedabu98 · 2026-09-05T00:56:24Z

R: @chamikaramj
CC @Amar3tto

github-actions · 2026-09-05T00:57:44Z

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment assign set of reviewers

chamikaramj · 2026-09-08T18:28:37Z

LGTM.

But seems like the test is failing with this ?

Amar3tto · 2026-09-14T20:21:51Z

@ahmedabu98 could you please check

tvalentyn · 2026-09-14T21:33:02Z

Integration test triage — PR #40028

apache/beam @ da7a257 · 1 failing job(s) · triaged in 12.8s

5 failing test(s) across 1 suite(s) — 3 distinct root cause(s).

Warning

At least one failure looks like a real defect, not a flake.

Failing suite	Jobs	Failures	On `master`
IcebergIO Managed Integration Tests on Dataflow / IO_Iceberg_Managed_Integration_Tests_Dataflow (Run IcebergIO Managed Integration Tests on Dataflow)	1	5	✅ passing (4h ago)

#	Suite	Verdict	Tests	Root cause
1	IcebergIO Managed Integration Tests on Dataflow / IO_Iceberg_Managed_Integration_Tests_Dataflow (Run IcebergIO Managed Integration Tests on Dataflow)	LIKELY REAL BUG	2	AssertionError: expected:<{ str: char: modulo_5: bool_field: <BOOLE
2	IcebergIO Managed Integration Tests on Dataflow / IO_Iceberg_Managed_Integration_Tests_Dataflow (Run IcebergIO Managed Integration Tests on Dataflow)	LIKELY REAL BUG	2	AssertionError: Expected: iterable with items [<Record(value_0, a, 0, true, 0, Record(nested_str_value_0, Record(doubly_
3	IcebergIO Managed Integration Tests on Dataflow / IO_Iceberg_Managed_Integration_Tests_Dataflow (Run IcebergIO Managed Integration Tests on Dataflow)	unknown	1	ClassCastException: class java.time.Instant cannot be cast to class org.joda.time.Instant (java.time.Instant is in modul

1. LIKELY REAL BUG — BigQueryMetastoreCatalogIT (+1 more)

Suite: IcebergIO Managed Integration Tests on Dataflow / IO_Iceberg_Managed_Integration_Tests_Dataflow (Run IcebergIO Managed Integration Tests on Dataflow)

Root cause: AssertionError: expected:<{ str: <STRING NOT NULL> char: <STRING NOT NULL> modulo_5: <INT64 NOT NULL> bool_field: <BOOLEAN NOT NULL> int_field: <INT32 NOT NULL> row: <ROW NOT NULL> {

Why this verdict: a test assertion failed

Affected tests:

BigQueryMetastoreCatalogIT.testWrite (0s) — IcebergIO Managed Integration Tests on Dataflow / IO_Iceberg_Managed_Integration_Tests_Dataflow (Run IcebergIO Managed Integration Tests on Dataflow) — job log
BigQueryMetastoreCatalogIT.testStreamToPartitionedDynamicDestinations (0s) — IcebergIO Managed Integration Tests on Dataflow / IO_Iceberg_Managed_Integration_Tests_Dataflow (Run IcebergIO Managed Integration Tests on Dataflow) — job log

Log excerpt

    java.lang.AssertionError: expected:<{
      str: <STRING NOT NULL>
      char: <STRING NOT NULL>
      modulo_5: <INT64 NOT NULL>
      bool_field: <BOOLEAN NOT NULL>
      int_field: <INT32 NOT NULL>
      row: <ROW NOT NULL> {

2. LIKELY REAL BUG — BigQueryMetastoreCatalogIT (+1 more)

Suite: IcebergIO Managed Integration Tests on Dataflow / IO_Iceberg_Managed_Integration_Tests_Dataflow (Run IcebergIO Managed Integration Tests on Dataflow)

Root cause: AssertionError: Expected: iterable with items [<Record(value_0, a, 0, true, 0, Record(nested_str_value_0, Record(doubly_nested_str_value_0, 0), 0, 0.0), [], null, 0, 2025-01-01T00:02:03.456789Z, 2025-01-01T00:02:03.45678

Why this verdict: a test assertion failed

Affected tests:

BigQueryMetastoreCatalogIT.testWriteRead (0s) — IcebergIO Managed Integration Tests on Dataflow / IO_Iceberg_Managed_Integration_Tests_Dataflow (Run IcebergIO Managed Integration Tests on Dataflow) — job log
BigQueryMetastoreCatalogIT.testReadWriteStreaming (0s) — IcebergIO Managed Integration Tests on Dataflow / IO_Iceberg_Managed_Integration_Tests_Dataflow (Run IcebergIO Managed Integration Tests on Dataflow) — job log

Where (app code):

org.apache.beam.sdk.io.iceberg.catalog.IcebergCatalogBaseIT.testWriteRead(IcebergCatalogBaseIT.java:750)

Log excerpt

    java.lang.AssertionError: 
    Expected: iterable with items [<Record(value_0, a, 0, true, 0, Record(nested_str_value_0, Record(doubly_nested_str_value_0, 0), 0, 0.0), [], null, 0, 2025-01-01T00:02:03.456789Z, 2025-01-01T00:02:03.456789, 1970-01-0
         but: not matched: <Record(value_0, a, 0, true, 0, Record(nested_str_value_0, Record(doubly_nested_str_value_0, 0), 0, 0.0), [], null, 0, 2025-01-01T00:02:03.456Z, 2025-01-01T00:02:03.456789, 1970-01-01, 00:00)>
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:6)
        at org.apache.beam.sdk.io.iceberg.catalog.IcebergCatalogBaseIT.testWriteRead(IcebergCatalogBaseIT.java:750)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
        at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
        at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:288)
        at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:282)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
        at java.base/java.lang.Thread.run(Thread.java:1583)

3. unknown — BigQueryMetastoreCatalogIT

Suite: IcebergIO Managed Integration Tests on Dataflow / IO_Iceberg_Managed_Integration_Tests_Dataflow (Run IcebergIO Managed Integration Tests on Dataflow)

Root cause: ClassCastException: class java.time.Instant cannot be cast to class org.joda.time.Instant (java.time.Instant is in module java.base of loader 'bootstrap'; org.joda.time.Instant is in unnamed module of loader 'app')

Why this verdict: no rule matched — needs a human

Affected tests:

BigQueryMetastoreCatalogIT.testRead (0s) — IcebergIO Managed Integration Tests on Dataflow / IO_Iceberg_Managed_Integration_Tests_Dataflow (Run IcebergIO Managed Integration Tests on Dataflow) — job log

Where (app code):

org.apache.beam.sdk.coders.InstantCoder.encode(InstantCoder.java:32)
org.apache.beam.sdk.coders.RowCoderGenerator$EncodeInstruction.encodeDelegate(RowCoderGenerator.java:435)
org.apache.beam.sdk.coders.Coder$ByteBuddy$u1AfxOa4.encode(Unknown Source)
org.apache.beam.sdk.coders.Coder$ByteBuddy$u1AfxOa4.encode(Unknown Source)
org.apache.beam.sdk.schemas.SchemaCoder.encode(SchemaCoder.java:121)
org.apache.beam.sdk.coders.IterableLikeCoder.encode(IterableLikeCoder.java:111)

Log excerpt

    java.lang.ClassCastException: class java.time.Instant cannot be cast to class org.joda.time.Instant (java.time.Instant is in module java.base of loader 'bootstrap'; org.joda.time.Instant is in unnamed module of loade
        at org.apache.beam.sdk.coders.InstantCoder.encode(InstantCoder.java:32)
        at org.apache.beam.sdk.coders.RowCoderGenerator$EncodeInstruction.encodeDelegate(RowCoderGenerator.java:435)
        at org.apache.beam.sdk.coders.Coder$ByteBuddy$u1AfxOa4.encode(Unknown Source)
        at org.apache.beam.sdk.coders.Coder$ByteBuddy$u1AfxOa4.encode(Unknown Source)
        at org.apache.beam.sdk.schemas.SchemaCoder.encode(SchemaCoder.java:121)
        at org.apache.beam.sdk.coders.IterableLikeCoder.encode(IterableLikeCoder.java:111)
        at org.apache.beam.sdk.coders.IterableLikeCoder.encode(IterableLikeCoder.java:58)
        at org.apache.beam.sdk.coders.Coder.encode(Coder.java:140)
        at org.apache.beam.sdk.util.CoderUtils.encodeToSafeStream(CoderUtils.java:86)
        at org.apache.beam.sdk.util.CoderUtils.encodeToByteArray(CoderUtils.java:70)
        at org.apache.beam.sdk.util.CoderUtils.encodeToByteArray(CoderUtils.java:55)
        at org.apache.beam.sdk.testing.PAssert$CheckRelationAgainstExpected.<init>(PAssert.java:1290)
        at org.apache.beam.sdk.testing.PAssert$PCollectionContentsAssert.satisfies(PAssert.java:770)
        at org.apache.beam.sdk.testing.PAssert$PCollectionContentsAssert.containsInAnyOrder(PAssert.java:725)
        at org.apache.beam.sdk.testing.PAssert$PCollectionContentsAssert.containsInAnyOrder(PAssert.java:636)
        at org.apache.beam.sdk.io.iceberg.catalog.IcebergCatalogBaseIT.testRead(IcebergCatalogBaseIT.java:480)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
        at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
        at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:288)
        at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:282)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
        at java.base/java.lang.Thread.run(Thread.java:1583)

This tool can make mistakes. It is only meant to surface errors faster, and may not surface all of them accurately.

Generated by gha_triage

ahmedabu98 · 2026-09-16T14:26:22Z

No more need for this. Looks like the managed service finally updated and tests are back to green: https://github.com/apache/beam/actions/workflows/IO_Iceberg_Managed_Integration_Tests_Dataflow.yml

temp mitigation

da7a257

github-actions Bot added java io labels Sep 5, 2026

Amar3tto requested a review from chamikaramj September 8, 2026 08:54

ahmedabu98 closed this Sep 16, 2026

Navigation Menu

Repository navigation

Temporary mitigation for Dataflow Managed Iceberg ITs - #40028

Temporary mitigation for Dataflow Managed Iceberg ITs#40028
ahmedabu98 wants to merge 1 commit into
apache:masterfrom
ahmedabu98:temp-fix-managed-iceberg

ahmedabu98 commented Sep 5, 2026

Uh oh!

ahmedabu98 commented Sep 5, 2026

Uh oh!

github-actions Bot commented Sep 5, 2026

Uh oh!

chamikaramj commented Sep 8, 2026

Uh oh!

Amar3tto commented Sep 14, 2026

Uh oh!

tvalentyn commented Sep 14, 2026

Uh oh!

ahmedabu98 commented Sep 16, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

ahmedabu98 commented Sep 5, 2026

GitHub Actions Tests Status (on master branch)

Uh oh!

ahmedabu98 commented Sep 5, 2026

Uh oh!

github-actions Bot commented Sep 5, 2026

Uh oh!

chamikaramj commented Sep 8, 2026

Uh oh!

Amar3tto commented Sep 14, 2026

Uh oh!

tvalentyn commented Sep 14, 2026

Integration test triage — PR #40028

1. LIKELY REAL BUG — BigQueryMetastoreCatalogIT (+1 more)

2. LIKELY REAL BUG — BigQueryMetastoreCatalogIT (+1 more)

3. unknown — BigQueryMetastoreCatalogIT

Uh oh!

ahmedabu98 commented Sep 16, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants