[mlir][spirv] Add mlir-spirv-tests CI to run for mlir-spv target tests

[davidegrohmann]

This should execute also the MLIR SPIRV Target tests which require the SPIRV-Tools validator

[llvmbot]

@llvm/pr-subscribers-github-workflow

Author: Davide Grohmann (davidegrohmann)

Changes

---

Full diff: https://github.com/llvm/llvm-project/pull/152124.diff

1 Files Affected:

• (modified) .github/workflows/spirv-tests.yml (+2-2)

diff --git a/.github/workflows/spirv-tests.yml b/.github/workflows/spirv-tests.yml index f15ca1cb64ba5..a2bb64d400bbf 100644 --- a/.github/workflows/spirv-tests.yml +++ b/.github/workflows/spirv-tests.yml @@ -23,7 +23,7 @@ jobs: name: Test SPIR-V uses: ./.github/workflows/llvm-project-tests.yml with: - build_target: check-llvm-codegen-spirv + build_target: check-llvm-codegen-spirv check-mlir projects: - extra_cmake_args: '-DLLVM_TARGETS_TO_BUILD="SPIRV" -DLLVM_INCLUDE_SPIRV_TOOLS_TESTS=ON' + extra_cmake_args: '-DLLVM_TARGETS_TO_BUILD="X86;SPIRV" -DLLVM_INCLUDE_SPIRV_TOOLS_TESTS=ON' os_list: '["ubuntu-24.04"]' 

[kuhar]

I'm wondering if it would be better to bifurcate spirv-tests.yml and have a separate version for mlir-spirv. It's very rare that we have PRs that affect both implementations, so having them separate could save us some CI time.

[davidegrohmann]

I'm wondering if it would be better to bifurcate spirv-tests.yml and have a separate version for mlir-spirv. It's very rare that we have PRs that affect both implementations, so having them separate could save us some CI time.

Yes it is likely better. This is now done.

[kuhar]

I'm going to merge once we confirm that the new check works as intended: https://github.com/llvm/llvm-project/actions/runs/16807047739/job/47602153053?pr=152124

[IgWod-IMG]

MLIR SPIR-V tests seem to take ~1h (https://github.com/llvm/llvm-project/actions/workflows/mlir-spirv-tests.yml) compared to regular checks that finish in ~5-10min. I wonder if there is any reason for that and whether we can bring the time down?

[davidegrohmann]

MLIR SPIR-V tests seem to take ~1h (https://github.com/llvm/llvm-project/actions/workflows/mlir-spirv-tests.yml) compared to regular checks that finish in ~5-10min. I wonder if there is any reason for that and whether we can bring the time down?

I can have a look. Can you point me to the workflows scripts for the regular checks so I can compare those with the mlir spir-v tests script?

[kuhar]

Probably no build cache? When I built this configuration locally, a lot of time was spent during spirv-tools checkout. I wonder if we can optimize how this gets cloned, e.g., --depth=1 or checkout submodules with -j $(nproc).

[IgWod-IMG]

I can have a look. Can you point me to the workflows scripts for the regular checks so I can compare those with the mlir spir-v tests script?

I believe it's "CI Checks" (https://github.com/llvm/llvm-project/actions/workflows/premerge.yaml), script: https://github.com/llvm/llvm-project/blob/main/.github/workflows/premerge.yaml

[IgWod-IMG]

I have also noticed that auto-merge doesn't wait for this job: #153440 We also should address that if deemed necessary. Not sure if this PR is the best place to track those issues.

[boomanaiden154]

I believe it's "CI Checks" (https://github.com/llvm/llvm-project/actions/workflows/premerge.yaml), script: https://github.com/llvm/llvm-project/blob/main/.github/workflows/premerge.yaml

Those checks run on powerful self hosted runners (64 threads) and have good build caching. This workflow runs on the free four core Github runners and Github build caching is generally pretty poor. It is going to be a lot slower.

I have also noticed that auto-merge doesn't wait for this job: #153440 We also should address that if deemed necessary. Not sure if this PR is the best place to track those issues.

Enabling auto-merge requires making the check required. Required checks cannot have path dependencies. I also doubt the community is interested in making this check required.

[IgWod-IMG]

Those checks run on powerful self hosted runners (64 threads) and have good build caching. This workflow runs on the free four core Github runners and Github build caching is generally pretty poor. It is going to be a lot slower.

That makes sense, thanks for investigating.

Enabling auto-merge requires making the check required. Required checks cannot have path dependencies. I also doubt the community is interested in making this check required.

In the hindsight, I agree we probably don't need this check as required, it just took me by surprise, and was quite busy that day, so I wasn’t sure what to do with it :)

[davidegrohmann]

What is the plan then? Should we switch to use the more powerful runners?

[boomanaiden154]

What is the plan then? Should we switch to use the more powerful runners?

This workflow should probably stay as it is. The llvm-premerge-linux-runners group (and windows equivalent) are intended to be used for the premerge workflow defined in premerge.yaml. If the workflow latency becomes a big issue, we can look into a better solution.