[SPIR-V] Add definitions and fix implementation for atomic builtins #106107
+107 −8
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
This change adds the missing definitions for atom_{dec, inc, max, min, xchg} builtins and relevant tests. Please note that OpenCL 1.2 does not differentiate between signed and unsigned atomic_min/atomic_max. 
michalpaszkowski requested review from VyacheslavLevytskyy and iliya-diyachkov 
Member
| @llvm/pr-subscribers-backend-spir-v Author: Michal Paszkowski (michalpaszkowski) ChangesThis change adds the missing definitions for atom_{dec, inc, max, min, xchg} builtins and relevant tests. Please note that OpenCL 1.2 does not differentiate between signed and unsigned atomic_min/atomic_max. Full diff: https://github.com/llvm/llvm-project/pull/106107.diff 7 Files Affected:
diff --git a/llvm/lib/Target/SPIRV/SPIRVBuiltins.cpp b/llvm/lib/Target/SPIRV/SPIRVBuiltins.cpp index 66cf163a1a0ac2..521d015dd1924f 100644 --- a/llvm/lib/Target/SPIRV/SPIRVBuiltins.cpp +++ b/llvm/lib/Target/SPIRV/SPIRVBuiltins.cpp @@ -784,7 +784,8 @@ static bool buildAtomicRMWInst(const SPIRV::IncomingCall *Call, unsigned Opcode, Call->Arguments.size() >= 3 ? Call->Arguments[2] : Register(); MemSemanticsReg = buildMemSemanticsReg(MemSemanticsReg, PtrRegister, Semantics, MIRBuilder, GR); - Register ValueReg = Call->Arguments[1]; + Register ValueReg = + Call->Arguments.size() >= 2 ? Call->Arguments[1] : Register(); Register ValueTypeReg = GR->getSPIRVTypeID(Call->ReturnType); // support cl_ext_float_atomics if (Call->ReturnType->getOpcode() == SPIRV::OpTypeFloat) { @@ -807,13 +808,14 @@ static bool buildAtomicRMWInst(const SPIRV::IncomingCall *Call, unsigned Opcode, ValueReg = NegValueReg; } } - MIRBuilder.buildInstr(Opcode) - .addDef(Call->ReturnRegister) - .addUse(ValueTypeReg) - .addUse(PtrRegister) - .addUse(ScopeRegister) - .addUse(MemSemanticsReg) - .addUse(ValueReg); + auto MIB = MIRBuilder.buildInstr(Opcode) + .addDef(Call->ReturnRegister) + .addUse(ValueTypeReg) + .addUse(PtrRegister) + .addUse(ScopeRegister) + .addUse(MemSemanticsReg); + if (ValueReg) + MIB.addUse(ValueReg); return true; } @@ -1458,6 +1460,10 @@ static bool generateAtomicInst(const SPIRV::IncomingCall *Call, case SPIRV::OpAtomicXor: case SPIRV::OpAtomicAnd: case SPIRV::OpAtomicExchange: + case SPIRV::OpAtomicUMin: + case SPIRV::OpAtomicUMax: + case SPIRV::OpAtomicIIncrement: + case SPIRV::OpAtomicIDecrement: return buildAtomicRMWInst(Call, Opcode, MIRBuilder, GR); case SPIRV::OpMemoryBarrier: return buildBarrierInst(Call, SPIRV::OpMemoryBarrier, MIRBuilder, GR); diff --git a/llvm/lib/Target/SPIRV/SPIRVBuiltins.td b/llvm/lib/Target/SPIRV/SPIRVBuiltins.td index 5c057a79afa0c3..bffbe511598a7a 100644 --- a/llvm/lib/Target/SPIRV/SPIRVBuiltins.td +++ b/llvm/lib/Target/SPIRV/SPIRVBuiltins.td @@ -596,6 +596,8 @@ defm : DemangledNativeBuiltin<"__spirv_AtomicXor", OpenCL_std, Atomic, 4, 4, OpA defm : DemangledNativeBuiltin<"atom_and", OpenCL_std, Atomic, 2, 4, OpAtomicAnd>; defm : DemangledNativeBuiltin<"atomic_and", OpenCL_std, Atomic, 2, 4, OpAtomicAnd>; defm : DemangledNativeBuiltin<"__spirv_AtomicAnd", OpenCL_std, Atomic, 4, 4, OpAtomicAnd>; +defm : DemangledNativeBuiltin<"atom_xchg", OpenCL_std, Atomic, 2, 2, OpAtomicExchange>; +defm : DemangledNativeBuiltin<"atomic_xchg", OpenCL_std, Atomic, 2, 2, OpAtomicExchange>; defm : DemangledNativeBuiltin<"atomic_exchange", OpenCL_std, Atomic, 2, 4, OpAtomicExchange>; defm : DemangledNativeBuiltin<"atomic_exchange_explicit", OpenCL_std, Atomic, 2, 4, OpAtomicExchange>; defm : DemangledNativeBuiltin<"AtomicEx__spirv_change", OpenCL_std, Atomic, 2, 4, OpAtomicExchange>; @@ -620,8 +622,16 @@ defm : DemangledNativeBuiltin<"__spirv_AtomicFlagClear", OpenCL_std, Atomic, 3, defm : DemangledNativeBuiltin<"atomic_flag_clear_explicit", OpenCL_std, Atomic, 2, 3, OpAtomicFlagClear>; defm : DemangledNativeBuiltin<"__spirv_AtomicSMin", OpenCL_std, Atomic, 4, 4, OpAtomicSMin>; defm : DemangledNativeBuiltin<"__spirv_AtomicSMax", OpenCL_std, Atomic, 4, 4, OpAtomicSMax>; +defm : DemangledNativeBuiltin<"atom_min", OpenCL_std, Atomic, 2, 2, OpAtomicUMin>; +defm : DemangledNativeBuiltin<"atomic_min", OpenCL_std, Atomic, 2, 2, OpAtomicUMin>; defm : DemangledNativeBuiltin<"__spirv_AtomicUMin", OpenCL_std, Atomic, 4, 4, OpAtomicUMin>; +defm : DemangledNativeBuiltin<"atom_max", OpenCL_std, Atomic, 2, 2, OpAtomicUMax>; +defm : DemangledNativeBuiltin<"atomic_max", OpenCL_std, Atomic, 2, 2, OpAtomicUMax>; defm : DemangledNativeBuiltin<"__spirv_AtomicUMax", OpenCL_std, Atomic, 4, 4, OpAtomicUMax>; +defm : DemangledNativeBuiltin<"atom_inc", OpenCL_std, Atomic, 1, 1, OpAtomicIIncrement>; +defm : DemangledNativeBuiltin<"atomic_inc", OpenCL_std, Atomic, 1, 1, OpAtomicIIncrement>; +defm : DemangledNativeBuiltin<"atom_dec", OpenCL_std, Atomic, 1, 1, OpAtomicIDecrement>; +defm : DemangledNativeBuiltin<"atomic_dec", OpenCL_std, Atomic, 1, 1, OpAtomicIDecrement>; // Barrier builtin records: defm : DemangledNativeBuiltin<"barrier", OpenCL_std, Barrier, 1, 3, OpControlBarrier>; diff --git a/llvm/test/CodeGen/SPIRV/transcoding/atomic_dec.ll b/llvm/test/CodeGen/SPIRV/transcoding/atomic_dec.ll new file mode 100644 index 00000000000000..b7625d43e3f85d --- /dev/null +++ b/llvm/test/CodeGen/SPIRV/transcoding/atomic_dec.ll @@ -0,0 +1,16 @@ +; RUN: llc -O0 -mtriple=spirv64-unknown-unknown %s -o - | FileCheck %s +; RUN: %if spirv-tools %{ llc -O0 -mtriple=spirv64-unknown-unknown %s -o - -filetype=obj | spirv-val %} + +; CHECK-DAG: %[[#INT:]] = OpTypeInt 32 0 +; CHECK-DAG: %[[#LONG:]] = OpTypeInt 64 0 +; CHECK-DAG: %[[#PTR:]] = OpTypePointer CrossWorkgroup %[[#LONG]] +; CHECK-DAG: %[[#PARAM:]] = OpFunctionParameter %[[#PTR]] +; CHECK-DAG: %[[#CONST0:]] = OpConstant %[[#INT]] 0 +; CHECK-DAG: %[[#CONST2:]] = OpConstant %[[#INT]] 2 +; CHECK: %[[#]] = OpAtomicIDecrement %[[#LONG]] %[[#PARAM]] %[[#CONST2]] %[[#CONST0]] +define void @test_atomic_dec(ptr addrspace(1) %p, i64 %val) { + %call1 = call i64 @_Z8atom_decPU3AS3Vl(ptr addrspace(1) %p) + ret void +} + +declare i64 @_Z8atom_decPU3AS3Vl(ptr addrspace(1)) diff --git a/llvm/test/CodeGen/SPIRV/transcoding/atomic_inc.ll b/llvm/test/CodeGen/SPIRV/transcoding/atomic_inc.ll new file mode 100644 index 00000000000000..47622fe9ec0af7 --- /dev/null +++ b/llvm/test/CodeGen/SPIRV/transcoding/atomic_inc.ll @@ -0,0 +1,16 @@ +; RUN: llc -O0 -mtriple=spirv64-unknown-unknown %s -o - | FileCheck %s +; RUN: %if spirv-tools %{ llc -O0 -mtriple=spirv64-unknown-unknown %s -o - -filetype=obj | spirv-val %} + +; CHECK-DAG: %[[#INT:]] = OpTypeInt 32 0 +; CHECK-DAG: %[[#LONG:]] = OpTypeInt 64 0 +; CHECK-DAG: %[[#PTR:]] = OpTypePointer CrossWorkgroup %[[#LONG]] +; CHECK-DAG: %[[#PARAM:]] = OpFunctionParameter %[[#PTR]] +; CHECK-DAG: %[[#CONST0:]] = OpConstant %[[#INT]] 0 +; CHECK-DAG: %[[#CONST2:]] = OpConstant %[[#INT]] 2 +; CHECK: %[[#]] = OpAtomicIIncrement %[[#LONG]] %[[#PARAM]] %[[#CONST2]] %[[#CONST0]] +define void @test_atomic_inc(ptr addrspace(1) %p, i64 %val) { + %call1 = call i64 @_Z8atom_incPU3AS3Vl(ptr addrspace(1) %p) + ret void +} + +declare i64 @_Z8atom_incPU3AS3Vl(ptr addrspace(1)) diff --git a/llvm/test/CodeGen/SPIRV/transcoding/atomic_max.ll b/llvm/test/CodeGen/SPIRV/transcoding/atomic_max.ll new file mode 100644 index 00000000000000..a7f29c613770e9 --- /dev/null +++ b/llvm/test/CodeGen/SPIRV/transcoding/atomic_max.ll @@ -0,0 +1,17 @@ +; RUN: llc -O0 -mtriple=spirv64-unknown-unknown %s -o - | FileCheck %s +; RUN: %if spirv-tools %{ llc -O0 -mtriple=spirv64-unknown-unknown %s -o - -filetype=obj | spirv-val %} + +; CHECK-DAG: %[[#INT:]] = OpTypeInt 32 0 +; CHECK-DAG: %[[#LONG:]] = OpTypeInt 64 0 +; CHECK-DAG: %[[#PTR:]] = OpTypePointer CrossWorkgroup %[[#LONG]] +; CHECK-DAG: %[[#PARAM:]] = OpFunctionParameter %[[#PTR]] +; CHECK-DAG: %[[#VALUE:]] = OpFunctionParameter %[[#LONG]] +; CHECK-DAG: %[[#CONST0:]] = OpConstant %[[#INT]] 0 +; CHECK-DAG: %[[#CONST2:]] = OpConstant %[[#INT]] 2 +; CHECK: %[[#]] = OpAtomicUMax %[[#LONG]] %[[#PARAM]] %[[#CONST2]] %[[#CONST0]] %[[#VALUE]] +define void @test_atomic_max(ptr addrspace(1) %p, i64 %val) { + %call1 = call i64 @_Z8atom_maxPU3AS3Vmm(ptr addrspace(1) %p, i64 %val) + ret void +} + +declare i64 @_Z8atom_maxPU3AS3Vmm(ptr addrspace(1), i64) diff --git a/llvm/test/CodeGen/SPIRV/transcoding/atomic_min.ll b/llvm/test/CodeGen/SPIRV/transcoding/atomic_min.ll new file mode 100644 index 00000000000000..d07147f51fb81d --- /dev/null +++ b/llvm/test/CodeGen/SPIRV/transcoding/atomic_min.ll @@ -0,0 +1,17 @@ +; RUN: llc -O0 -mtriple=spirv64-unknown-unknown %s -o - | FileCheck %s +; RUN: %if spirv-tools %{ llc -O0 -mtriple=spirv64-unknown-unknown %s -o - -filetype=obj | spirv-val %} + +; CHECK-DAG: %[[#INT:]] = OpTypeInt 32 0 +; CHECK-DAG: %[[#LONG:]] = OpTypeInt 64 0 +; CHECK-DAG: %[[#PTR:]] = OpTypePointer CrossWorkgroup %[[#LONG]] +; CHECK-DAG: %[[#PARAM:]] = OpFunctionParameter %[[#PTR]] +; CHECK-DAG: %[[#VALUE:]] = OpFunctionParameter %[[#LONG]] +; CHECK-DAG: %[[#CONST0:]] = OpConstant %[[#INT]] 0 +; CHECK-DAG: %[[#CONST2:]] = OpConstant %[[#INT]] 2 +; CHECK: %[[#]] = OpAtomicUMin %[[#LONG]] %[[#PARAM]] %[[#CONST2]] %[[#CONST0]] %[[#VALUE]] +define void @test_atomic_min(ptr addrspace(1) %p, i64 %val) { + %call1 = call i64 @_Z8atom_minPU3AS3Vmm(ptr addrspace(1) %p, i64 %val) + ret void +} + +declare i64 @_Z8atom_minPU3AS3Vmm(ptr addrspace(1), i64) diff --git a/llvm/test/CodeGen/SPIRV/transcoding/atomic_xchg.ll b/llvm/test/CodeGen/SPIRV/transcoding/atomic_xchg.ll new file mode 100644 index 00000000000000..5ccf8eaa136bda --- /dev/null +++ b/llvm/test/CodeGen/SPIRV/transcoding/atomic_xchg.ll @@ -0,0 +1,17 @@ +; RUN: llc -O0 -mtriple=spirv64-unknown-unknown %s -o - | FileCheck %s +; RUN: %if spirv-tools %{ llc -O0 -mtriple=spirv64-unknown-unknown %s -o - -filetype=obj | spirv-val %} + +; CHECK-DAG: %[[#INT:]] = OpTypeInt 32 0 +; CHECK-DAG: %[[#LONG:]] = OpTypeInt 64 0 +; CHECK-DAG: %[[#PTR:]] = OpTypePointer CrossWorkgroup %[[#LONG]] +; CHECK-DAG: %[[#PARAM:]] = OpFunctionParameter %[[#PTR]] +; CHECK-DAG: %[[#VALUE:]] = OpFunctionParameter %[[#LONG]] +; CHECK-DAG: %[[#CONST0:]] = OpConstant %[[#INT]] 0 +; CHECK-DAG: %[[#CONST2:]] = OpConstant %[[#INT]] 2 +; CHECK: %[[#]] = OpAtomicExchange %[[#LONG]] %[[#PARAM]] %[[#CONST2]] %[[#CONST0]] %[[#VALUE]] +define void @test_atomic_xchg(ptr addrspace(1) %p, i64 %val) { + %call1 = call i64 @_Z9atom_xchgPU3AS1Vll(ptr addrspace(1) %p, i64 %val) + ret void +} + +declare i64 @_Z9atom_xchgPU3AS1Vll(ptr addrspace(1), i64) |
VyacheslavLevytskyy approved these changes Sep 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
This change adds the missing definitions for atom_{dec, inc, max, min, xchg} builtins and relevant tests.
Please note that OpenCL 1.2 does not differentiate between signed and unsigned atomic_min/atomic_max.