ATLAS-5336: Upgrade Kafka to 3.9.1; embedded notification broker uses KRaft (no ZooKeeper) by ramackri · Pull Request #689 · apache/atlas

ramackri · 2026-07-04T16:14:11Z

Summary

Bump Apache Kafka client and test broker from 2.8.2 to 3.9.1 (Scala 2.13).
Replace ZooKeeper-based embedded notification broker with KRaft via KafkaClusterTestKit.
Update docker dev Kafka image to kafka_2.13-3.9.1.
Harden webapp integration tests after master merge (ATLAS-5337: Fix Trino Extractor Jersey client failures for AtlasEntityWithExtInfo POST/GET #690) and stabilize Kafka-related unit/integration test classpath.
Fix bridge-module IT failures exposed after Jackson 2.16 bump (HiveHookIT, SqoopHookIT, StormAtlasHookIT).
Skip atlas-trino-extractor Failsafe on default mvn verify (TestNG classpath after global Failsafe pin).

Motivation

Kafka 2.8.x clients are only partially compatible with modern 3.9.x / 4.x brokers.
Embedded dev/test broker no longer requires an in-process ZooKeeper instance.
Aligns Atlas with the 3.9.x bridge release before external cluster KRaft migration.
Master CI began failing on BasicSearchIT (3 vs 6 entity counts) and AdminExportImportTestIT teardown when ITs share embedded-server state.
Jackson 2.16.2 (required by Kafka 3.9 testkit) surfaces classpath and Jersey serialization issues in hook ITs that use Atlas REST clients.

Changes

Area	Before	After
`kafka.version`	2.8.2	3.9.1
`kafka.scala.binary.version`	2.12	2.13
`EmbeddedKafkaServer`	In-process ZK + `KafkaServer`	`KafkaClusterTestKit` (KRaft)
Docker `atlas-kafka` image	`kafka_2.12-*`	`kafka_2.13-3.9.1`
Distro embedded config	ZK-oriented settings	KRaft; `bootstrap.servers` set at runtime

`webapp/pom.xml` — why these dependencies were added

This PR bumps root jackson.version from 2.12.7 → 2.16.2 (required by Kafka 3.9 KafkaClusterTestKit test dependencies). That version flows into atlas-intg, which declares:

jackson-jaxrs-base at ${jackson.version}
jackson-jaxrs-json-provider at ${jackson.version}

Atlas server REST still runs on Jersey 1.19 (JAX-RS 1.1 / JSR-311). After the Jackson bump, the rebuilt server WAR pulled in Jackson JAX-RS 2.16 transitively while Jersey and its JSON providers expect the older 2.12-era JAX-RS integration. In manual Docker testing this produced an inconsistent classpath — Atlas failed to serve REST reliably (GET /api/atlas/admin/version did not return 200) even though the Kafka/embedded-broker changes were correct.

Fix in webapp/pom.xml:

Dependency	Version	Purpose
`jackson-jaxrs-base`	2.12.7 (pinned)	JAX-RS integration layer for Jackson — kept at 2.12.7 for Jersey 1.19 compatibility
`jackson-jaxrs-json-provider`	2.12.7 (pinned)	`JacksonJaxbJsonProvider` used by Jersey for JSON request/response serialization on Atlas REST APIs
`jsr311-api`	1.1.1	JAX-RS 1.1 API (`javax.ws.rs.*`) required by Jersey 1.x JSON providers

atlas-intg exclusions on the two jackson-jaxrs-* artifacts prevent the WAR from also loading the 2.16.2 transitive copies. Webapp now owns a single, known-good JAX-RS Jackson version for the server while the rest of the build can use Jackson 2.16.2 where Kafka requires it.

This is intentionally not a Jersey upgrade — it decouples server-side JAX-RS Jackson (2.12.7) from the core Jackson bump (2.16.2) needed for Kafka 3.9.1.

CI fixes (Kafka upgrade fallout)

Issue	Fix	File(s)
`falcon-bridge` Failsafe picked JUnit Platform after `junit-jupiter-api` leaked onto `atlas-notification` compile classpath	Remove unused compile-scoped Jupiter dep; scope Kafka test deps to `test`	`notification/pom.xml`
`KafkaNotificationTest` / `NotificationHookConsumerKafkaTest` Surefire `NoClassDefFoundError: org/junit/jupiter/api/Assertions`	Add `junit-jupiter-api` with `test` scope for unit tests	`notification/pom.xml`
`EmbeddedKafkaServer` / IT Jetty startup `NoClassDefFoundError: org/junit/jupiter/api/Assertions` at runtime	Keep `junit-jupiter-api` on `notification` compile classpath — `KafkaClusterTestKit.waitForReadyBrokers()` calls Kafka test utilities that reference Jupiter inside the WAR	`notification/pom.xml`
Failsafe reported BUILD SUCCESS with 0 tests after `junit-jupiter-api` landed on webapp test classpath	Add `surefire-testng` to `maven-failsafe-plugin` (same as Surefire) so webapp ITs stay on TestNG	`pom.xml`, `webapp/pom.xml`
`ImportTaskListenerImplTest.testStartImportConsumer_Successful` flaky on JDK 17 CI (Mockito argument mismatch from async executor leaking between tests)	Shut down executor in `@AfterMethod`; use `CountDownLatch` instead of `Thread.sleep`	`webapp/.../ImportTaskListenerImplTest.java`
`BasicSearchIT` expected 3 hive tables but got 6 when `AdminExportImportTestIT` ran first (shared `@cl1` cluster suffix)	Scope hive searches to `default.` qualified names from `hive-db-50-tables.zip`; skip scoping for fulltext (`query`) and paginated (`offset > 0`) searches	`webapp/.../BasicSearchIT.java`
`AdminExportImportTestIT.teardown` failed with `Stream Closed` / `ClientHandlerException` when embedded Jetty stopped first	Log warning and return from teardown instead of failing the suite	`webapp/.../AdminExportImportTestIT.java`
`NotificationHookConsumerIT` (6 tests) failed with `Topic ATLAS_HOOK not present in metadata after 60000 ms`	Bind embedded KRaft broker to configured `atlas.kafka.bootstrap.servers` (`localhost:19027`) so Jetty and Failsafe IT JVMs share the same broker	`notification/.../EmbeddedKafkaServer.java`
`HiveHookIT` (`testAlterTableRename`, `testTraitsPreservedOnColumnRename`) failed with `JsonMappingException: Conflicting getter definitions for property "propagate"` on `AtlasClassification`	Pre-serialize classification request bodies with `AtlasType.toJson()` before Jersey POST (same pattern as #690 entity create/update)	`client/client-v2/.../AtlasClientV2.java`
`SqoopHookIT` timed out after 80s; root cause `NoClassDefFoundError: StreamConstraintsException` in `AtlasClient.getEntity()`	Pin `jackson-databind` 2.16 + `jackson-core` 2.16 (test scope) on sqoop-bridge IT classpath — Hadoop/Hive/Sqoop transitive deps supply older `jackson-core`	`addons/sqoop-bridge/pom.xml`
`StormAtlasHookIT.testAddEntities` failed with `NoSuchFieldError: READ_UNKNOWN_ENUM_VALUES_USING_DEFAULT_VALUE` in `AtlasClient.getEntity()`	Pin `jackson-databind` 2.16 on storm-bridge IT classpath (existing `jackson-core` test dep retained) — `storm-core` supplies older `jackson-databind`	`addons/storm-bridge/pom.xml`
`atlas-trino-extractor` Failsafe `NoClassDefFoundError: org.testng.ITestListener` after ~76 min CI build	Pinning Failsafe to TestNG globally requires `testng` on the module test classpath; extractor has only a stub `TrinoExtractorIT` and no test deps — set `skipITs=true` on default verify (live ITs planned in ATLAS-5338)	`addons/trino-extractor/pom.xml`

Skipped downstream modules — Jackson fix not required

Modules skipped after storm-bridge failed in the reactor were reviewed; none need the same preemptive Jackson pin:

Module	Why no change
`trino-extractor`	Stub `TrinoExtractorIT` only; `skipITs=true` avoids Failsafe/TestNG classpath failure on default verify; full live-stack ITs in ATLAS-5338
`atlas-examples` / `sample-app`	No tests; already pins Jackson 2.16 explicitly
`classification-updater`	No tests; already has `jackson-databind` 2.16
`index-repair-tool`	No tests; Atlas stack deps only
`notification-analyzer`	No tests; `dist` profile copies Jackson 2.16 jars
`distribution` / `docs`	Packaging / docs only

Extra fix applied during testing — KRaft listener configuration

The first EmbeddedKafkaServer port-binding fix hung startup because KRaft needs both listeners:

EXTERNAL://localhost:19027 — client traffic (matches bootstrap.servers)
CONTROLLER://localhost:0 — KRaft controller (ephemeral port)

Using only PLAINTEXT://localhost:19027 removed the controller listener and caused the hang.

Updated EmbeddedKafkaServer now sets:

listeners=EXTERNAL://localhost:19027,CONTROLLER://localhost:0
advertised.listeners=EXTERNAL://localhost:19027

How listener configuration is applied (no hard-coded strings)

Nothing passes those listener strings literally. EmbeddedKafkaServer derives them at runtime from config:

Input — atlas.kafka.bootstrap.servers in atlas-application.properties (e.g. localhost:19027 for webapp ITs, localhost:9027 placeholder in distro/docker).
Read — EmbeddedKafkaServer constructor loads the atlas.kafka.* subset via ApplicationProperties.getSubsetConfiguration().
Build — buildBrokerConfig() calls toExternalListenerAddress(bootstrap.servers) → EXTERNAL://localhost:19027, then sets:
- listeners = EXTERNAL://<host>:<port>,CONTROLLER://localhost:0
- advertised.listeners = EXTERNAL://<host>:<port>
Apply — each property is passed to KafkaClusterTestKit.Builder.setConfigProp(); the testkit starts the KRaft broker with that config.
Clients — KafkaNotification (and IT producers) connect using the same bootstrap.servers value from properties.

Step	Component	Responsibility
Config	`atlas-application.properties`	Supplies `atlas.kafka.bootstrap.servers`
Broker startup	`EmbeddedKafkaServer.buildBrokerConfig()`	Derives `listeners` / `advertised.listeners`
Broker runtime	`KafkaClusterTestKit`	Binds `EXTERNAL` on the configured port; `CONTROLLER` on ephemeral port
Produce/consume	`KafkaNotification`, `NotificationHookConsumer`	Connect to `bootstrap.servers`

Why this matters for ITs: webapp integration tests run in two JVMs — Jetty (Atlas server + hook consumer) and Failsafe (test producer). Both must reach the same broker on the same port. The old ZK embedded broker bound to the configured port; the first KRaft change used an ephemeral port, so Failsafe still pointed at localhost:19027 while Jetty's broker listened elsewhere.

flowchart LR
    subgraph Jetty["Jetty JVM (port 31000)"]
        EKS[EmbeddedKafkaServer]
        Hook[NotificationHookConsumer]
    end
    subgraph Failsafe["Failsafe IT JVM"]
        IT[NotificationHookConsumerIT]
        KN[KafkaNotification producer]
    end
    EKS -->|"binds EXTERNAL on :19027"| Kafka[(Embedded KRaft broker)]
    Hook -->|consumes ATLAS_HOOK| Kafka
    KN -->|produces ATLAS_HOOK| Kafka
    IT --> KN

If bootstrap.servers is empty, no fixed listeners are set and KafkaClusterTestKit picks ephemeral ports (acceptable for same-JVM unit tests like KafkaNotificationTest, which update ApplicationProperties after startup).

Affected files

Kafka upgrade (core)

File	Change
`pom.xml`	`kafka.version` 3.9.1, Scala 2.13, Jackson 2.16.2, Failsafe `surefire-testng`
`notification/pom.xml`	Kafka 3.9 deps; Jupiter/Kafka test classpath scoping
`notification/src/main/java/org/apache/atlas/kafka/EmbeddedKafkaServer.java`	ZK → KRaft `KafkaClusterTestKit`; bind `EXTERNAL` listener to `bootstrap.servers`
`notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java`	KRaft-compatible broker startup
`distro/src/conf/atlas-application.properties`	Embedded KRaft config (no ZK)
`dev-support/atlas-docker/Dockerfile.atlas-kafka`	`kafka_2.13-3.9.1` image
`dev-support/atlas-docker/.env`	Kafka version env
`dev-support/atlas-docker/download-archives.sh`	Download Kafka 3.9.1 archive

Server / client classpath (Jackson + Jersey)

File	Change
`webapp/pom.xml`	Pin `jackson-jaxrs-*` 2.12.7 for Jersey 1.19; remove webapp-scoped Jupiter leak
`client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java`	`AtlasType.toJson()` for entity POST/PUT (#690) + classification POST/PUT (this PR)
`client/common/src/main/java/org/apache/atlas/AtlasBaseClient.java`	Client response handling (#690, merged from master)

Bridge hook IT classpath

File	Change
`addons/sqoop-bridge/pom.xml`	Pin `jackson-databind` + `jackson-core` 2.16 for `SqoopHookIT` / `AtlasClient` v1
`addons/storm-bridge/pom.xml`	Pin `jackson-databind` 2.16 for `StormAtlasHookIT` / `AtlasClient` v1
`addons/trino-extractor/pom.xml`	`skipITs=true` on default `mvn verify` (stub IT; no `testng` test dep)

Webapp integration / unit tests

File	Change
`webapp/src/test/java/org/apache/atlas/web/integration/BasicSearchIT.java`	Scope searches to `default.` entities; fulltext/pagination exceptions
`webapp/src/test/java/org/apache/atlas/web/resources/AdminExportImportTestIT.java`	Tolerant teardown when Jetty stops first
`webapp/src/test/java/org/apache/atlas/notification/ImportTaskListenerImplTest.java`	Fix async consumer test flake
`webapp/src/test/java/org/apache/atlas/examples/QuickStartIT.java`	IT adjustments (#690, merged from master)
`webapp/src/test/java/org/apache/atlas/examples/QuickStartV2IT.java`	IT adjustments (#690, merged from master)

Other (merged from master)

File	Change
`.github/workflows/ci.yml`	CI workflow update (#690)
`intg/src/main/java/org/apache/atlas/utils/AuthenticationUtil.java`	Auth util (#690)
`repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java`	Test fix (#690)
`dev-support/atlas-docker/docker-compose.atlas-hbase.yml`	Docker compose (#690)
`dev-support/atlas-docker/scripts/atlas-hbase.sh`	HBase script (#690)
`dashboard/package.json`	UI build optimizations (ATLAS-5328, merged from master)
`dashboard/src/views/Lineage/atlas-lineage/package-lock.json`	Remove nested lineage lockfile
`dashboard/src/views/Lineage/atlas-lineage/package.json`	Remove nested lineage package
`dashboard/src/views/Lineage/atlas-lineage/webpack.config.js`	Remove nested lineage webpack config
`dashboard/src/views/Statistics/StackedAreaCharts.jsx`	Dashboard chart cleanup

Testing

Unit tests

Test	Module	Command	Result
`KafkaNotificationTest`	`notification`	`mvn -pl notification -Dtest=KafkaNotificationTest test`	PASS — embedded KRaft broker starts; produce/consume on `ATLAS_HOOK`
`NotificationHookConsumerKafkaTest`	`webapp`	`mvn -pl webapp -am -DfailIfNoTests=false -Dsurefire.failIfNoSpecifiedTests=false -Dtest=NotificationHookConsumerKafkaTest -DskipEnunciate=true test`	PASS (3 tests) — embedded broker, hook consumer, import topic
`ImportTaskListenerImplTest`	`webapp`	`mvn -pl webapp -am -DfailIfNoTests=false -Dsurefire.failIfNoSpecifiedTests=false -Dtest=ImportTaskListenerImplTest -DskipEnunciate=true test`	PASS (31 tests) — async import consumer tests stabilized

NotificationHookConsumerKafkaTest and ImportTaskListenerImplTest were run with -am so atlas-server-api and sibling modules match compile-time APIs.

Integration tests (local)

Test	Command	Result
`BasicSearchIT` + `AdminExportImportTestIT`	`mvn -Pdist,embedded-solr-it -DskipEnunciate=true -DskipDocs=true -DskipUTs=true -pl notification,webapp -am -Dfailsafe.failIfNoSpecifiedTests=false verify -Dit.test=BasicSearchIT,AdminExportImportTestIT`	PASS (19 tests)
`NotificationHookConsumerIT`	`mvn -Pdist,embedded-solr-it -DskipEnunciate=true -DskipDocs=true -DskipUTs=true -pl notification,webapp -am -Dfailsafe.failIfNoSpecifiedTests=false verify -Dit.test=NotificationHookConsumerIT`	PASS (6 tests)
`HiveHookIT` (classification methods)	`AtlasClientV2` `toJson()` fix	Awaiting CI
`SqoopHookIT`	`sqoop-bridge` Jackson classpath pin	Passed CI after fix
`StormAtlasHookIT`	`storm-bridge` Jackson databind pin	Awaiting CI

Manual — embedded KRaft broker smoke test

Unit tests exercise EmbeddedKafkaServer inside a short-lived Surefire JVM. A separate manual check verified the same path in the full Atlas server process (complete WAR classpath, normal Spring startup order, real graph store and Solr).

Setup: Atlas server running in Docker with Postgres and Solr. External Kafka broker was not required — embedded mode runs the broker in-process.

Steps:

Set atlas.notification.embedded=true in atlas-application.properties (placeholder atlas.kafka.bootstrap.servers=localhost:9027).
Restart Atlas and poll GET /api/atlas/admin/version until HTTP 200.
Confirm server logs show:
- EmbeddedKafkaServer.start(isEmbedded=true)
- Starting embedded KRaft kafka (log.dir=.../data/kafka/kafka)
- Embedded KRaft kafka server started at localhost:<ephemeral-port>
Verify atlas.kafka.bootstrap.servers was rewritten at runtime to the live broker address (placeholder not used as-is).
Restore atlas.notification.embedded=false and external bootstrap servers; restart Atlas.

Result: PASS — KRaft broker started inside the real server JVM; Atlas REST returned 200; graph, Solr, Jersey, and notification stack all initialized.

Manual — external Kafka 3.9.1 (production-like path)

Verified Atlas with atlas.notification.embedded=false against an external Kafka 3.9.1 broker (kafka_2.13-3.9.1), using the rebuilt Atlas server image.

Flow	What was verified	Result
Hive metadata ingestion	`CREATE TABLE` via Hive; Atlas REST search for `hive_column` on new table	Entity present — hook consumer processed `ATLAS_HOOK`
Classification → Ranger	PII tag applied via Atlas REST; TagSync log showed `ATLAS_ENTITIES` consumption; tag mapping visible in Ranger Admin	Classification propagated
Hive tag-based access control	`SELECT` on classified column as allowed/denied users	Deny and mask enforced per policy
Hive audit	Ranger Admin access audit for Hive service	Audit rows recorded for table access
Kafka authorizer + audit	Produce/consume on Kerberos Kafka topic under Ranger policy	Authorizer allow/deny reflected in Ranger audit

Overall: PASS — full metadata path (Hive → external Kafka → Atlas → ATLAS_ENTITIES → TagSync → Ranger) and audit paths worked with Kafka 3.9.1.

Notes

Production deployments should continue using atlas.notification.embedded=false and external bootstrap.servers.
Jackson core/databind bumped to 2.16.2 for Kafka 3.9 testkit; server JAX-RS Jackson intentionally remains 2.12.7 for Jersey 1.19 compatibility.
junit-jupiter-api is intentionally not on the webapp test classpath anymore; notification owns the compile-scoped dependency required by embedded KRaft startup.
Webapp ITs use fixed localhost:19027 in test config; embedded KRaft binds EXTERNAL on that port so Jetty and Failsafe JVMs share the same broker.
Bridge hook ITs (hive-bridge, storm-bridge, sqoop-bridge) that call AtlasClient v1 via REST may need explicit Jackson 2.16 pins when Hadoop/Hive/Storm transitive deps supply older Jackson jars.
atlas-trino-extractor skips Failsafe ITs on default CI mvn verify; opt-in live-stack tests will use -Ptrino-extractor-it in ATLAS-5338.

https://issues.apache.org/jira/browse/ATLAS-5336

… KRaft Bump kafka-clients and embedded test broker from 2.8.2 to 3.9.1 (Scala 2.13), replace ZooKeeper-based EmbeddedKafkaServer with KafkaClusterTestKit, and update docker dev Kafka image packaging to kafka_2.13-3.9.1.

Point readers to ATLAS-5190 for full Jackson unification via Jersey 2.x.

It leaked to falcon-bridge via atlas-notification and caused Failsafe to select JUnit Platform, which then failed on transitive junit 3.8.1 from falcon-common during FalconHookIT discovery.

Kafka 3.9 TestUtils needs junit-jupiter-api at test runtime; keep it off the compile classpath so falcon-bridge Failsafe stays on TestNG. Align mockito and testng with other Atlas modules (test scope).

… tests. NotificationHookConsumerKafkaTest starts EmbeddedKafkaServer, which uses Kafka 3.9 TestUtils requiring JUnit Jupiter on the test classpath. Test-scoped deps from atlas-notification do not propagate to webapp.

Shut down the import executor in @AfterMethod and wait with CountDownLatch instead of Thread.sleep so Mockito state does not leak between tests.

@cl1

Scope hive searches to largedb. entities from the imported test dataset instead of the shared @cl1 cluster suffix, and tolerate Stream Closed during AdminExportImportTestIT teardown when the embedded server stops.

@cl1

Scope BasicSearchIT to default. entities from hive-db-50-tables.zip instead of the shared @cl1 suffix, skip scoping for fulltext and paginated searches, keep junit-jupiter on the notification compile classpath for embedded KRaft runtime, and pin Failsafe to TestNG after junit landed on the webapp test path.

Derive EXTERNAL and CONTROLLER listeners from atlas.kafka.bootstrap.servers so Jetty and Failsafe JVMs share the same broker port, fixing NotificationHookConsumerIT topic timeouts on CI.

… conflicts. HiveHookIT fails when Jersey serializes AtlasClassification because isPropagate() and getPropagate() both map to "propagate"; AtlasType.toJson() matches the entity fix.

SqoopHookIT fails with NoClassDefFoundError for StreamConstraintsException when jackson-databind 2.16 is paired with an older jackson-core from Hadoop/Hive.

…t v1. StormAtlasHookIT fails with NoSuchFieldError for READ_UNKNOWN_ENUM_VALUES_USING_DEFAULT_VALUE when jackson-databind 2.16 is mixed with an older databind from storm-core on the test classpath. Co-authored-by: Cursor <cursoragent@cursor.com>

Pinning Failsafe to TestNG for the Kafka upgrade requires testng on the module test classpath; the extractor only has a stub IT and no test deps yet.

ramackri force-pushed the ATLAS-5336 branch from 7392566 to ec1f39b Compare July 4, 2026 16:16

sarathsubramanian reviewed Jul 6, 2026

View reviewed changes

Comment thread webapp/pom.xml

Document jackson-jaxrs 2.12.7 pin for Jersey 1.19 compatibility.

4c766d9

Point readers to ATLAS-5190 for full Jackson unification via Jersey 2.x.

ramackri force-pushed the ATLAS-5336 branch from 9993103 to 4c766d9 Compare July 6, 2026 08:14

ramk added 10 commits July 6, 2026 15:03

Remove unused junit-jupiter-api from atlas-notification compile deps.

faa7c98

It leaked to falcon-bridge via atlas-notification and caused Failsafe to select JUnit Platform, which then failed on transitive junit 3.8.1 from falcon-common during FalconHookIT discovery.

Scope Kafka test and unit test deps in notification to test.

7ba6bff

Kafka 3.9 TestUtils needs junit-jupiter-api at test runtime; keep it off the compile classpath so falcon-bridge Failsafe stays on TestNG. Align mockito and testng with other Atlas modules (test scope).

Fix flaky ImportTaskListenerImplTest async consumer assertions on CI.

66c5a25

Shut down the import executor in @AfterMethod and wait with CountDownLatch instead of Thread.sleep so Mockito state does not leak between tests.

Merge branch 'master' into ATLAS-5336.

aeabd5d

Harden BasicSearchIT and AdminExportImportTestIT against CI flakes.

90c08fb

Scope hive searches to largedb. entities from the imported test dataset instead of the shared @cl1 cluster suffix, and tolerate Stream Closed during AdminExportImportTestIT teardown when the embedded server stops.

Bind embedded KRaft broker to configured bootstrap.servers for ITs.

09b1b2f

Derive EXTERNAL and CONTROLLER listeners from atlas.kafka.bootstrap.servers so Jetty and Failsafe JVMs share the same broker port, fixing NotificationHookConsumerIT topic timeouts on CI.

Pre-serialize classification request bodies to avoid Codehaus Jackson…

17a75eb

… conflicts. HiveHookIT fails when Jersey serializes AtlasClassification because isPropagate() and getPropagate() both map to "propagate"; AtlasType.toJson() matches the entity fix.

Pin Jackson 2.16 on sqoop-bridge IT classpath for AtlasClient v1.

e57884e

SqoopHookIT fails with NoClassDefFoundError for StreamConstraintsException when jackson-databind 2.16 is paired with an older jackson-core from Hadoop/Hive.

ramackri force-pushed the ATLAS-5336 branch from 0ec514d to e57884e Compare July 7, 2026 08:18

ramk and others added 2 commits July 7, 2026 15:22

Skip trino-extractor Failsafe ITs on default mvn verify.

42a3c0a

Pinning Failsafe to TestNG for the Kafka upgrade requires testng on the module test classpath; the extractor only has a stub IT and no test deps yet.

sarathsubramanian approved these changes Jul 8, 2026

View reviewed changes

sarathsubramanian merged commit 4bec890 into apache:master Jul 8, 2026
2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

ATLAS-5336: Upgrade Kafka to 3.9.1; embedded notification broker uses KRaft (no ZooKeeper)#689

ATLAS-5336: Upgrade Kafka to 3.9.1; embedded notification broker uses KRaft (no ZooKeeper)#689
sarathsubramanian merged 14 commits into
apache:masterfrom
ramackri:ATLAS-5336

ramackri commented Jul 4, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

ramackri commented Jul 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Motivation

Changes

webapp/pom.xml — why these dependencies were added

CI fixes (Kafka upgrade fallout)

Skipped downstream modules — Jackson fix not required

Extra fix applied during testing — KRaft listener configuration

How listener configuration is applied (no hard-coded strings)

Affected files

Kafka upgrade (core)

Server / client classpath (Jackson + Jersey)

Bridge hook IT classpath

Webapp integration / unit tests

Other (merged from master)

Testing

Unit tests

Integration tests (local)

Manual — embedded KRaft broker smoke test

Manual — external Kafka 3.9.1 (production-like path)

Notes

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ramackri commented Jul 4, 2026 •

edited

Loading

`webapp/pom.xml` — why these dependencies were added