yeast: Add support for output template choices

Adds a new syntactic form
```
[tree1 if {cond1}
 tree2
 tree3 if {cond3}
]
```
to the language, the intended semantics being that we output the first
tree that is usable. Two things can prevent a tree from being used:

- the guard condition (a Rust expression evaluating to a boolean) may
  fail;
- a string interpolation (e.g. `#{name}`) may fail to have a value.

If all of the alternatives fail, then the result is `None`. This makes
the syntax a generalisation of the `?` postfix operator that was removed
in the previous commit. `(foo)?` can be replaced with a single-element
alternative `[(foo)]`.

As the changes to swift.rs show, this can clean up a bunch of
constructions where we currently rely on helper functions or somewhat
complicated Rust blocks.

In terms of the implementation, we go through the alternatives in turn,
first evaluating the guard condition. If this succeeds, we evaluate (and
temporarily store) all of the `#{...}` interpolations, and if those all
return `Some(...)` values, we instantiate the template itself. A benefit
of this implementation compared to the one for `?` (which used named
labels and `break`s) is that if either requirement fails, no AST nodes
are constructed. (The `?` implementation would eagerly construct the
tree, and then abort if any of the interpolations failed. This could
leave orphaned nodes in the AST arena.)

A choice cannot contain another choice as an immediate child (as that
choice could simply have its branches inlined in the parent), but
nesting further down is allow. In that case, the inner choice takes care
of handling its internal "failures", and these do not bubble up to the
outer choice.

Author: tausbn

shared/yeast-macros/src/ast.rs

@@ -100,6 +100,15 @@ pub(crate) enum Template {
     Node(Node),
     /// An embedded Rust block such as `{args}`.
     Splice(syn::Block),
+    /// Ordered alternatives such as `[(identifier #{name}) (missing)]`.
+    Choice(Vec<Alternative>),
+}
+
+pub(crate) struct Alternative {
+    /// Template to construct when the guard and dependencies succeed.
+    pub(crate) template: Template,
+    /// Optional postfix guard such as `if { name.is_some() }`.
+    pub(crate) guard: Option<syn::Block>,
 }
 
 pub(crate) struct Node {

shared/yeast-macros/src/lib.rs

@@ -52,6 +52,8 @@ pub fn query(input: TokenStream) -> TokenStream {
 
 /// Build a single AST node from a template, returning its `OutputId`.
 ///
+/// A top-level choice returns `Option<OutputId>`.
+///
 /// # Template syntax
 ///
 /// ```text
@@ -65,8 +67,42 @@ pub fn query(input: TokenStream) -> TokenStream {
 /// field: {expr}                - extend a named field with `{expr}`'s ids
 /// location: {expr}             - override this node's source range with the
 ///                                span of one or more nodes; may be repeated
+/// field: [(kind ...)]          - set the field only if the alternative can
+///                                be constructed (see below)
+/// field: [(a) (b)]             - use the first constructible alternative
+/// field: [(a) if {guard} (b)]  - skip an alternative when its guard is false
+/// ```
+///
+/// # Template choices
+///
+/// Brackets contain one or more ordered alternatives. For each alternative,
+/// its optional `if {guard}` is evaluated first, followed by every `#{expr}`
+/// dependency beneath it. An absent dependency — an `Option` that is `None` —
+/// skips the alternative without constructing any of its template nodes. The
+/// first alternative whose guard and dependencies succeed is constructed:
+///
+/// ```text
+/// (break_expr label: {lbl.map(|l| tree!((identifier #{l})))})   // before
+/// (break_expr label: [(identifier #{lbl})])                     // after
 /// ```
 ///
+/// Dependencies may be nested arbitrarily deeply. A nested choice handles its
+/// own dependencies, so an inner absent value need not discard an outer node:
+///
+/// ```text
+/// (parameter pattern: [
+///     (name_pattern identifier: [(identifier #{name})])
+/// ] type: {ty})
+/// ```
+///
+/// Only `#{expr}` is treated as a dependency. A `{expr}` splice is unaffected,
+/// since yielding no ids already leaves its field unset. Outside a choice,
+/// interpolating an `Option` with `#{expr}` remains a compile error.
+///
+/// A choice used as a field or list item contributes zero or one node. A
+/// top-level choice passed to `tree!` returns `Option<OutputId>`; ordinary
+/// templates continue to return `OutputId`.
+///
 /// Uses the `BuildCtx` binding named `ctx` from the surrounding scope:
 ///
 /// ```text

shared/yeast-macros/src/lower.rs

@@ -1,7 +1,33 @@
 use proc_macro2::{Ident, Span, TokenStream};
 use quote::quote;
+use std::cell::RefCell;
+use std::sync::atomic::{AtomicUsize, Ordering};
 
-use crate::ast::{Cardinality, Content, Field, Node, Pattern, Replacement, Rule, Template};
+use crate::ast::{
+    Alternative, Cardinality, Content, Field, Node, Pattern, Replacement, Rule, Template,
+};
+
+#[derive(Clone, Copy, Default)]
+struct LoweringContext<'a> {
+    dependency_scope: Option<DependencyScope<'a>>,
+}
+
+#[derive(Clone, Copy)]
+struct DependencyScope<'a> {
+    preparations: &'a RefCell<Vec<DependencyPreparation>>,
+}
+
+struct DependencyPreparation {
+    value: Ident,
+    source_range: Ident,
+    expression: TokenStream,
+}
+
+fn fresh_prepared_ident(suffix: &str) -> Ident {
+    static COUNTER: AtomicUsize = AtomicUsize::new(0);
+    let n = COUNTER.fetch_add(1, Ordering::Relaxed);
+    Ident::new(&format!("__yeast_prepared_{suffix}_{n}"), Span::call_site())
+}
 
 #[derive(Clone)]
 struct CaptureBinding {
@@ -273,15 +299,19 @@ fn capture_bindings<'a>(
 
 impl Template {
     pub(crate) fn lower_root(&self) -> TokenStream {
-        self.lower()
+        match self {
+            Template::Choice(alternatives) => Self::lower_choice(alternatives),
+            _ => self.lower_single(LoweringContext::default()),
+        }
     }
 
     pub(crate) fn lower_list(templates: &[Self]) -> TokenStream {
+        let context = LoweringContext::default();
         let items = templates
             .iter()
             .map(|template| match template {
                 Template::Node(node) => {
-                    let node = node.lower();
+                    let node = node.lower(context);
                     quote! { __nodes.push(#node); }
                 }
                 Template::Splice(block) => {
@@ -292,6 +322,14 @@ impl Template {
                         ctx.extend_field_ids(__value, &mut __nodes);
                     }
                 }
+                Template::Choice(alternatives) => {
+                    let pick = Self::lower_choice(alternatives);
+                    quote! {
+                        if let ::std::option::Option::Some(__id) = #pick {
+                            __nodes.push(__id);
+                        }
+                    }
+                }
             })
             .collect::<Vec<_>>();
 
@@ -304,19 +342,92 @@ impl Template {
         }
     }
 
-    fn lower(&self) -> TokenStream {
+    fn lower_single(&self, context: LoweringContext<'_>) -> TokenStream {
         match self {
-            Template::Node(node) => node.lower(),
+            Template::Node(node) => node.lower(context),
             Template::Splice(block) => {
                 let block = rust_block(block);
                 quote! { ::std::convert::Into::<yeast::OutputId>::into(#block) }
             }
+            Template::Choice(_) => {
+                unreachable!("choices are lowered through Template::lower_choice")
+            }
         }
     }
+
+    fn lower_choice(alternatives: &[Alternative]) -> TokenStream {
+        let pick = alternatives.iter().rev().fold(
+            quote! { ::std::option::Option::None },
+            |fallback, alternative| {
+                let candidate = alternative.lower_candidate();
+                quote! {
+                    {
+                        let __yeast_candidate = #candidate;
+                        if __yeast_candidate.is_some() {
+                            __yeast_candidate
+                        } else {
+                            #fallback
+                        }
+                    }
+                }
+            },
+        );
+        quote! {{ #pick }}
+    }
+}
+
+impl Alternative {
+    fn lower_candidate(&self) -> TokenStream {
+        let preparations = RefCell::new(Vec::new());
+        let context = LoweringContext {
+            dependency_scope: Some(DependencyScope {
+                preparations: &preparations,
+            }),
+        };
+        let value = self.template.lower_single(context);
+        let candidate = quote! { ::std::option::Option::Some(#value) };
+
+        // Nest dependency matches around construction in reverse, so they are
+        // evaluated in source order and construction runs only after all have
+        // produced values.
+        let candidate = preparations.into_inner().into_iter().rev().fold(
+            candidate,
+            |candidate, preparation| {
+                let DependencyPreparation {
+                    value,
+                    source_range,
+                    expression,
+                } = preparation;
+                // Do not use `map`/`and_then` here: `candidate` may contain
+                // embedded user Rust, and putting it in a closure would change
+                // the target of `return` and the result type seen by `?`.
+                quote! {
+                    match #expression {
+                        ::std::option::Option::Some((#value, #source_range)) => #candidate,
+                        ::std::option::Option::None => ::std::option::Option::None,
+                    }
+                }
+            },
+        );
+        let candidate = if let Some(guard) = &self.guard {
+            let guard = rust_block(guard);
+            quote! {
+                if #guard {
+                    #candidate
+                } else {
+                    ::std::option::Option::None
+                }
+            }
+        } else {
+            candidate
+        };
+
+        candidate
+    }
 }
 
 impl Node {
-    fn lower(&self) -> TokenStream {
+    fn lower(&self, context: LoweringContext<'_>) -> TokenStream {
         let ctx = Ident::new("ctx", Span::call_site());
         let kind_str = self.kind.to_string();
 
@@ -325,19 +436,54 @@ impl Node {
                 Content::Static(lit) => quote! { #ctx.literal(#kind_str, #lit) },
                 Content::Computed(block) => {
                     let block = rust_block(block);
-                    quote! {
-                        {
-                            let __expr = #block;
-                            let __value =
-                                yeast::YeastDisplay::yeast_to_string(&__expr, &#ctx);
-                            let __source_range =
-                                yeast::YeastSourceRange::yeast_source_range(&__expr, &#ctx);
+                    if let Some(scope) = context.dependency_scope {
+                        let value = fresh_prepared_ident("value");
+                        let source_range = fresh_prepared_ident("source_range");
+                        scope.preparations.borrow_mut().push(DependencyPreparation {
+                            value: value.clone(),
+                            source_range: source_range.clone(),
+                            expression: quote! {
+                                {
+                                    let __expr = #block;
+                                    yeast::MaybeYeastValue::maybe_yeast_value(&__expr).map(
+                                        |__value_ref| {
+                                            (
+                                                yeast::YeastDisplay::yeast_to_string(
+                                                    __value_ref,
+                                                    &#ctx,
+                                                ),
+                                                yeast::YeastSourceRange::yeast_source_range(
+                                                    __value_ref,
+                                                    &#ctx,
+                                                ),
+                                            )
+                                        },
+                                    )
+                                }
+                            },
+                        });
+                        quote! {
                             #ctx.literal_with_source_range(
                                 #kind_str,
-                                &__value,
-                                __source_range,
+                                &#value,
+                                #source_range,
                             )
                         }
+                    } else {
+                        quote! {
+                            {
+                                let __expr = #block;
+                                let __value =
+                                    yeast::YeastDisplay::yeast_to_string(&__expr, &#ctx);
+                                let __source_range =
+                                    yeast::YeastSourceRange::yeast_source_range(&__expr, &#ctx);
+                                #ctx.literal_with_source_range(
+                                    #kind_str,
+                                    &__value,
+                                    __source_range,
+                                )
+                            }
+                        }
                     }
                 }
             };
@@ -384,8 +530,19 @@ impl Node {
                                 }
                             });
                         }
+                        Template::Choice(alternatives) => {
+                            let pick = Template::lower_choice(alternatives);
+                            stmts.push(quote! {
+                                let #temp: ::std::option::Option<yeast::OutputId> = #pick;
+                            });
+                            field_args.push(quote! {
+                                if let ::std::option::Option::Some(__id) = #temp {
+                                    __fields.push((#field_str, vec![__id]));
+                                }
+                            });
+                        }
                         Template::Node(child) => {
-                            let value = child.lower();
+                            let value = child.lower(context);
                             stmts.push(quote! { let #temp: yeast::OutputId = #value; });
                             field_args.push(quote! {
                                 __fields.push((#field_str, vec![#temp]));