Detect values of invalid types for `strategy.matrix.include` and `strategy.matrix.exclude`

[desrosj]

When include and exclude statements are specified in a job's strategy matrix, they cannot be an array type. Specifying them as such results in a run failure.

  phpunit-tests:
    name: PHP ${{ matrix.php }}
    strategy:
      matrix:
        os: [ 'ubuntu-latest', 'macos-latest' ]
        php: [ '8.3', '8.4', '8.5' ]

        include:
          # Include jobs that test with memcached.
          - os: 'ubuntu-latest'
            php: [ '8.2' ]

        exclude:
          - os: [ 'macos-latest' ]
            php: '8.3'

While it would be amazing for Actions to support non-scalar (it would allow a single include/exclude statement to include multiple combinations), it currently does not. It can be easy to accidentally specify the values as an array, especially when someone copies the line from for the same variable from the overall matrix to spell the variable name correct or not have to fully type it out.

The error message is not super descriptive and the reason may not be obvious:

Error: The template is not valid. .github/workflows/workflow-name.yml (Line: 56, Col: 33): A sequence was not expected

I believe this would be a great safety check to add to provide more context.

[muzimuzhi]

When include and exclude statements are specified in a job's strategy matrix, they cannot be an array type. Specifying them as such results in a run failure.

Both strategy.matrix.include.<entry> and stragegy.matrix.exclude.<entry> can accept array values. See my tests in muzimuzhi/hello-github-actions#53, using the following workflow:

on:
  push

jobs:
  phpunit-tests:
    name: PHP ${{ matrix.php }}
    runs-on: ubuntu-slim
    strategy:
      matrix:
        os: [ 'ubuntu-latest', 'macos-latest' ]
        php: [ '8.3', '8.4', '8.5' ]
        include:
          # Include jobs that test with memcached.
          - os: 'ubuntu-latest'
            php: [ '8.2' ]
        exclude:
          - os: [ 'macos-latest' ]
            php: '8.3'
    steps:
      - run: |
          echo os: "${{ matrix.os }}"
          echo php: "${{ matrix.php }}"

So I think it's the uses of array matrix.<entry> that caused the error.

actionlint 1.7.11 already checks (see "Matrix values") that

values in exclude: appear in matrix: or include:

To my understanding, this issue calls for checking that value types in include: match those in matrix:. But even they don't match, it's not necessarily an error.

[desrosj]

Interesting! Especially "Surprisingly, runs-on: ${{ matrix.os }} works when matrix.os is a one-element array."

To my understanding, this issue calls for checking that value types in include: match those in matrix:. But even they don't match, it's not necessarily an error.

If I am interpreting this correctly, then a check to the effect of this may be beneficial:

• array value for strategy.matrix.<entry>, array value for strategy.matrix.(include|exclude).<entry> is OK.
• array value for strategy.matrix.<entry>, scalar value for strategy.matrix.(include|exclude).<entry> is OK
• scalar value for strategy.matrix.<entry>, array value for strategy.matrix.(include|exclude).<entry> is not OK.

You can go from array to non-array, but not the other way around.

The first two cases and the ${{ matrix.os }}/one-element array nuance would be worthy of a warning if actionlint had severity levels, but probably fine to not flag since they still work and are not technically errors.

a single include/exclude statement to include multiple combinations

As an aside, when I mentioned this, I had something like the following in mind, which would allow someone to easily add multiple combinations with a single include statement. Now knowing that the values can actually be arrays, I'm wondering if this is already possible (or easy to add if not) 🤔 :

on:
  push

jobs:
  phpunit-tests:
    name: PHP ${{ matrix.php }}
    runs-on: ubuntu-slim
    strategy:
      matrix:
        os: [ 'ubuntu-latest', 'macos-latest', 'windows-latest' ]
        php: [ '8.3', '8.4', '8.5' ]
        include:
          # Include jobs that test with memcached.
          - os: 'macos-latest'
            php: [ '8.2', '8.3' ]
        exclude:
          - os: 'windows-latest'
            php: [ '8.4', '8.5']
    steps:
      - run: |
          echo os: "${{ matrix.os }}"
          echo php: "${{ matrix.php }}"

[muzimuzhi]

If I am interpreting this correctly, then a check to the effect of this may be beneficial:

• array value for strategy.matrix.<entry>, array value for strategy.matrix.(include|exclude).<entry> is OK.
• array value for strategy.matrix.<entry>, scalar value for strategy.matrix.(include|exclude).<entry> is OK
• scalar value for strategy.matrix.<entry>, array value for strategy.matrix.(include|exclude).<entry> is not OK.

Strictly speaking,

• array of type T value for strategy.matrix.<entry>, type T value for strategy.matrix.include.<entry> is OK.
• array of type T1 value for strategy.matrix.<entry>, type T2 value for strategy.matrix.include.<entry> is not OK if types T1 and T2 are different.

For the strategy.matrix.exclude.<entry> case, actionlint already checks if each of the combination to be excluded is already listed in strategy.matrix.<entry> plus ``strategy.matrix.include` entries, so no worries.

As an aside, when I mentioned this, I had something like the following in mind, which would allow someone to easily add multiple combinations with a single include statement. Now knowing that the values can actually be arrays, I'm wondering if this is already possible (or easy to add if not) 🤔 :
[...]

This is a feature request to the GitHub Actions it self, and I found a similar request posted in 2021

• Allow GitHub Actions matrix exclusions to take an array https://github.com/orgs/community/discussions/7835