Skip to content

spark: the dialect task declares no outputs, so DialectSuite can record a green result against a published file that fails #1233

Description

@nielspardon

tasks.register<JavaExec>("dialect") in spark/spark-4.0_2.13/build.gradle.kts writes spark/spark_dialect.yaml and declares no outputs. DialectSuite reads that same file, and all three variants' test tasks now declare it as an inputs.file. Gradle therefore sees a consumer with no producer, has no reason to order the two, and — worse — will snapshot the file at whatever state it happens to be in when the test task starts.

That is not just a theoretical ordering question. Reproduced on fix/dialect-generator-derived-model:

  1. Change name: in spark/spark_dialect.yaml to something the generator would not emit.
  2. ./gradlew --parallel :spark:spark-4.0_2.13:dialect :spark:spark-3.4_2.12:test --tests '*DialectSuite*' — passes. dialect rewrites the file back to the correct content, and the test reads the rewritten content at runtime, but Gradle had already fingerprinted the edited file as the task's input. The successful outcome is recorded under that fingerprint.
  3. Re-edit name: the same way and run ./gradlew :spark:spark-3.4_2.12:test --tests '*DialectSuite*' — reports UP-TO-DATE and passes, on a file that does not match the generator.
  4. --rerun-tasks on that same file fails three tests, which is the correct outcome.

So one interleaved invocation is enough to bank a green result keyed to content that actually fails, and the stale pass then survives across later invocations.

Declaring the output is necessary but not sufficient: adding outputs.file("../spark_dialect.yaml") to the dialect task alone makes ./gradlew :spark:spark-4.0_2.13:dialect :spark:spark-4.0_2.13:test fail with

Task ':spark:spark-4.0_2.13:test' uses this output of task ':spark:spark-4.0_2.13:dialect' without declaring an explicit or implicit dependency.

which is Gradle correctly refusing the graph rather than a regression. The second half needs an ordering rule, and a plain dependsOn is the wrong one: it would have DialectSuite's byte-for-byte assertion compare the file against a copy the same generator had just written, which passes vacuously. mustRunAfter on dialect, or keeping the two out of one graph, preserves what the test is for. Note the dialect task is registered on spark-4.0_2.13 only while all three variants' tests consume the file, so the fix spans projects.

Follow-up from review of #1133.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions