[tausbn]

When creating the target tree for some rewrite rule, we quite often want to make the construction conditional on the presence of certain captures, or on the current state of the context.

Previously we added the postfix ? operator for this purpose. This made it possible to write a tree template like

(foo 
  bar: (baz id: (identifier #{name}))?
)

wherein the entire baz node disappears (because of the postfix ?) if the name interpolation fails (e.g. if name is None).

There are a few drawbacks with this approach:

• There's no way to specify a non-None alternative.
• It only works for #{...} interpolations.
• It works on the basis of constructing the tree eagerly and then cancelling the construction if any of the interpolations failed. This could lead to orphaned nodes in the output AST if these were constructed before the interpolation was attempted.

(This also made the implementation somewhat complicated, requiring the use of labels and breaks in the lowered Rust code.)

Instead of the above, this PR implements a generalisation of this concept. Taking inspiration from the tree-sitter query language, we use square brackets to indicate a list of alternatives that should be attempted in turn. The first alternative that succeeds is the tree that is returned from this choice operator. Moreover, individual choices can have a bool-returning Rust-block as a guard.

Thus, something like

[
  tree1 if {cond1}
  tree2
  tree3 if {cond2}
]

would have the following interpretation

• If cond1 holds, and all #{...} interpolations in tree1 are valid (e.g. different from None), then return tree1
• Otherwise, if all #{...} interpolations in tree2 are valid, return that
• Otherwise, if cond3 holds and all #{...} interpolations in tree3 are valid, return that
• Otherwise return None

This fully supersedes the existing ? syntax (which we therefore remove first): (...)? can be rewritten to just [(...)].

Before constructing any trees, the implementation first checks that the relevant condition (if present) holds, and that all #{...} interpolations have usable values. Only if both of these are true is the tree construction attempted. This prevents any orphaned nodes from being constructed.

[Copilot]

🟡 Changes recommended

Two critical compilation failures remain: missing OutputId support breaks translated captures, and premature candidate construction causes moved-value errors when alternatives consume the same repeated capture.

2 open issues

• Avoid moving repeated captures before guard evaluation · New
• Preserve OutputId support for translated captures · New

What changed in this PR

Adds ordered, guarded choices to Yeast tree templates, replacing postfix ?. The new syntax lets rewrite rules express fallbacks and check interpolation dependencies before constructing template nodes.

Changes:

• Add choice parsing, validation, and lowering with guard and dependency checks.
• Update documentation and tests for fallback selection, nested choices, and orphan-node prevention.
• Migrate Swift translation rules to bracketed choices.

File	Description
unified/​extractor/​src/​languages/​swift/​swift.rs	Migrates optional nodes and conditional rewrites to choices.
shared/​yeast/​tests/​test.rs	Covers guards, fallbacks, nested choices, and construction preflight.
shared/​yeast/​src/​lib.rs	Updates interpolation diagnostics and optional-value implementations.
shared/​yeast/​doc/​yeast.md	Documents choice syntax, behavior, and examples.
shared/​yeast-macros/​src/​template_parse.rs	Parses and validates choices and guards; removes postfix ? parsing.
shared/​yeast-macros/​src/​lower.rs	Generates dependency checks and alternative-selection code.
shared/​yeast-macros/​src/​lib.rs	Updates public macro documentation for choices and return types.
shared/​yeast-macros/​src/​ast.rs	Adds choice alternatives and removes the optional-field flag.

_{Review effort: Max}

[asgerf]

Some comments after a partial review

[asgerf]

Could you add a comment explaining what these preparations are? I tried to understand it from the code and I honestly can't see it.

[tausbn]

Admittedly, the name is a bit terrible (Copilot came up with it, and I couldn't think of a better name at the time).

Basically, this is the bit that handles the "and temporarily store" bit mentioned in the commit message. Basically, when we know a choice depends on #{foo} (where foo can be any Rust expression), we evaluate foo before attempting to build the relevant tree. If it fails, we go on to the next choice (if any), but if it succeeds, then we construct the tree, which means we eventually reach a place where we want to use #{foo}. Because we don't want to evaluate foo twice, we store the result of the initial (guarding) evaluation, and reuse it when we construct the template.

Thus, the value field contains the name of an identifier in the lowered Rust code (which contains the result of evaluating foo), source_range is a different identifier that (optionally) contains the source range of the foo interpolation, and finally expression is a small bit of Rust code that evaluates foo and produces the two other values as an Option<(String,Option<Range>)> value. Thus, as part of the guard we end up evaluating roughly the following rust code for each DependencyPreparation for a particular choice branch:

match expression {
  Some((value, source_range)) => goto_next_guard_or_build_tree
  None => try_next_choice
}

I'll update the code to have some comments explaining this (and maybe try to come up with a better name for the struct).

[tausbn]

Clarification/cleanup is in 3620348.

[asgerf]

Could we add a test with nested alternatives? Both directly-nested and indirectly-nested. It seems suspicious that the context from the outer call is not passed on in lower_choice. It would make more sense to me if we only had one lower and it somehow wired up the two contexts here.

[tausbn]

If I understand your comment correctly, I think the current behaviour is actually by design.

The new choice operator always succeeds, in that it can never cause a tree to fail being built. At worst all of its alternatives fail, and None is returned by default (which means the field in which this takes place remains unset). This means that an inner choice can never affect an outer one -- whatever interpolations it inspects will be handled by the inner choice, and the outer choice is none the wiser.

Conversely, there's no reason for the inner choice to know about the outer choice's assignments. The only way this could be relevant is if the same interpolation #{foo} is present in both the inner and outer choice, but in that case we really are evaluating foo twice (or at least, that would match the semantics we have elsewhere), so the inner choice can get its own foo to evaluate. We do not try to do any clever deduplication here.

Now, we might want to also add support for choices that can fail as a whole (and not default to None). Just off the top of my head, my suggestion would be to support the form

[
  tree1 if {cond1}
  tree2
  fail
]

for the cases where we want failure if no alternatives succeed. In this case, we might want to pass some of the outer context in.

As for your test request, here's the test that inner (indirect) choices don't affect the outer ones:

codeql/shared/yeast/tests/test.rs

Lines 965 to 991 in 7027a76

	#[test]
	fn test_innermost_choice_catches_first() {
	let input = "x = 1";
	let input_ast = parse_input(input);
	let mut ast = empty_output_ast(input);
	let mut user_ctx = ();
	let mut ctx = yeast::build::BuildCtx::new(&input_ast, &mut ast, &mut user_ctx);
	// The inner choice catches, so only `child` is dropped; `left` survives.
	let absent: Option<yeast::Id> = None;
	let root = yeast::tree!(
	(assignment
	left: [
	(left_assignment_list child: [(identifier #{absent})])
	]
	)
	);
	assert_dump_eq(
	&dump_ast(ctx.output, root, input),
	r#"
	assignment
	left: left_assignment_list
	"#,
	);
	}

And here's the test that shows that direct choice nesting is disallowed by the parser:

codeql/shared/yeast-macros/src/template_parse.rs

Lines 337 to 355 in 7027a76

	#[test]
	fn rejects_choice_as_direct_alternative() {
	let result = parse_template(quote! {
	[
	[(identifier "nested")]
	(identifier "fallback")
	]
	});
	let Err(error) = result else {
	panic!("expected a direct nested choice to be rejected");
	};
	assert_eq!(
	error.to_string(),
	"a choice cannot be used directly as another choice's alternative; \
	list its alternatives directly in the enclosing choice"
	);
	}
	#[test]