[libc][libm][GPU] Added missing vendor entry points to `libm` for GPUs #66031

AntonRydahl · 2023-09-11T23:45:01Z

There are a number of mathematical functions where no target-agnostic implementations exist, and the compiler built-ins are not correctly lowered on all GPU targets. This patch adds inlined wrappers for those functions to the GPU version of libm for AMDGPU and NVPTX targets.

There are a number of mathematical functions where no target agnostic implementations exist and the compiler builtins are not correctly lowered. This patch adds inlined wrappers for those functions to the GPU version of `libm` for AMDGPU and NVPTX targets.

This doesn't change the current behavior of the function, but the explicit declaration looks cleaner.

…m#65674) Add a DAG combine to form a masked.load from a masked_strided_load intrinsic with stride equal to element size. This covers a couple of extra test cases, and allows us to simplify and common some existing code on the concat_vector(load, ...) to strided load transform. This is the first in a mini-patch series to try and generalize our strided load and gather matching to handle more cases, and common up different approaches to the same problems in different places.

COMPILER_RT_DEBUG was just added to sanitizer-ppc64le-linux, and this test is already broken there.

Instead of eagerly creating a diagnostic that will be discarded in the normal case, switch to lazy initialization on error.

…ool (llvm#65991) To match other internal symbolizer functions. This makes harder to distighush small buffer from a different failure, but we has the same problem for the rest of the lib. Still we use 16k buffer so it should be enough most of the time. We can fix all function togerher if future, if needed.

Deprecate the `gpu-to-cubin` & `gpu-to-hsaco` passes in favor of the `TargetAttr` workflow. This patch removes remaining upstream uses of the aforementioned passes, including the option to use them in `mlir-opt`. A future patch will remove these passes entirely. The passes can be re-enabled in `mlir-opt` by adding the CMake flag: `-DMLIR_ENABLE_DEPRECATED_GPU_SERIALIZATION=1`.

…cc.loop (llvm#65521) The `cache` directive may appear at the top of (inside of) a loop. It specifies array elements or subarrays that should be fetched into the highest level of the cache for the body of the loop. The `cache` directive is modeled as a data entry operands attached to the acc.loop operation.

The cache directive is attached directly to the acc.loop operation when the directive appears in the loop. When it appears before a loop, the OpenACCCacheConstruct is saved and attached when the acc.loop is created. Directive that cannot be attached to a loop are silently discarded. Depends on llvm#65521

…` canonicalization patterns. (llvm#66002) This pattern fits better with the other canonicalization patterns that exist for `linalg.fill`.

Otherwise they are dangling if lldMain is called more than once.

llvm#65776) Since the OpenACC atomics specification is a subset of OpenMP atomics, the same lowering implementation can be used. This change extracts out the necessary pieces from the OpenMP lowering and puts them in a shared spot. The shared spot is a header file so that each implementation can template specialize directly. After putting the OpenMP implementation in a common spot, the following changes were needed to make it work for OpenACC: * Ensure parsing works correctly by avoiding hardcoded offsets. * Templatize based on atomic type. * The checking whether it is OpenMP or OpenACC is done by checking for OmpAtomicClauseList (OpenACC does not implement this so we just templatize with void). It was preferable to check this instead of atomic type because in some cases, like atomic capture, the read/write/update implementations are called - and we want compile time evaluation of these conditional parts. * The memory order and hint are used only for OpenMP. * Generate acc dialect operations instead of omp dialect operations.

… configurable (llvm#65687) "descriptive summaries" should only be used for small to medium binaries because of the performance penalty the cause when completing types. I'm defaulting it to false. Besides that, the "raw child" for synthetics should be optional as well. I'm defaulting it to false. Both options can be set via a launch or attach config, following the pattern of most settings. javascript extension wrappers can set these settings on their own as well.

Fixes: llvm#65806 Currently clang put extern shared var ODR-used by host device functions in global var __clang_gpu_used_external. This behavior was due to https://reviews.llvm.org/D123441. However, clang should not do that for extern shared vars since their addresses are per warp, therefore cannot be accessed by host code.

…/macos triples" This reverts commit 9f77fac. The change unintentionally changed lots of codegen, see llvm#47698 (comment) Also revert a follow-up: This reverts commit b40a5be.

…yped memory. (llvm#66009) Exposes the existing `get(ShapedType, StringRef, AsmResourceBlob)` builder publicly (was protected) and adds a CAPI `mlirUnmanagedDenseBlobResourceElementsAttrGet`. While such a generic construction interface is a big help when it comes to interop, it is also necessary for creating resources that don't have a standard C type (i.e. f16, the f8s, etc). Previously reviewed/approved as part of https://reviews.llvm.org/D157064

This should fix the warning seen in https://lab.llvm.org/buildbot/#/builders/13/builds/39980/steps/6/logs/stdio

Make sure every conditional branch constructed by `LoopUnrollRuntime` code sets branch weights. - Add new 1:127 weights for the conditional jumps checking whether the whole (unrolled) loop should be skipped in the generated prolog or epilog code. - Remove `updateLatchBranchWeightsForRemainderLoop` function and just add weights immediately when constructing the relevant branches. This leads to simpler code and makes the code more obvious as every call to `CreateCondBr` now has a `BranchWeights` parameter. - Rework formula for epilogue latch weights, to assume equal distribution of remainders and remove `assert` (as I was able to reach this code when forcing small unroll factors on the commandline). Differential Revision: https://reviews.llvm.org/D158642

With this, check-llvm passes on an arm mac if x86 isn't in LLVM_TARGETS_TO_BUILD. This pattern to skip the tests if x86 isn't enabled is used in every other test in this file.

According to 7.5.6.3 point 3, finalization occurs when > A nonpointer, nonallocatable object that is not a dummy argument or function result is finalized immediately before it would become undefined due to execution of a RETURN or END statement (19.6.6, item (3)). We were not calling the finalization on empty derived-type. There is no such restriction so this patch updates the code so the finalization is called for empty type as well.

The test is flaky after Kernel upgrade from 6.0 to 6.5.

The issue is uncovered by llvm#47698: for IR files without a target triple, -mtriple= specifies the full target triple while -march= merely sets the architecture part of the default target triple, leaving a target triple which may not make sense, e.g. riscv64-apple-darwin. Therefore, -march= is error-prone and not recommended for tests without a target triple. The issue has been benign as we recognize $unknown-apple-darwin as ELF instead of rejecting it outrightly.

The issue is uncovered by llvm#47698: for assembly files, -triple= specifies the full target triple while -arch= merely sets the architecture part of the default target triple, leaving a target triple which may not make sense, e.g. riscv64-apple-darwin. Therefore, -arch= is error-prone and not recommended for tests. The issue has been benign as we recognize $unknown-apple-darwin as ELF instead of rejecting it outrightly. Due to the nature of the issue, we don't see the issue in tests using architectures that any of Mach-O/COFF/XCOFF supports.

Similar to 806761a

The test only applies to ELF. On Linux, when a default target triple is, say, Mach-O, the test should be excluded as well.

Reviewed By: #bolt, maksfb Differential Revision: https://reviews.llvm.org/D154120

Reduce YAML profile processing times: - preprocessProfile: speed up buildNameMaps by replacing ProfileNameToProfile mapping with ProfileFunctionNames set and ProfileBFs vector. Pre-look up YamlBF->BF correspondence, memoize in ProfileBFs. - readProfile: replace iteration over all functions in the binary by iteration over profile functions (strict match and LTO name match). On a large binary (1.9M functions) and large YAML profile (121MB, 30k functions) reduces profile steps runtime: pre-process profile data: 12.4953s -> 10.7123s process profile data: 9.8195s -> 5.6639s Compared to fdata profile reading: pre-process profile data: 8.0268s process profile data: 1.0265s process profile data pre-CFG: 0.1644s Reviewed By: #bolt, maksfb Differential Revision: https://reviews.llvm.org/D159460

Currently, dimlvlmap with identity affine map will be treated as empty affine map. But the new syntax would treat it as an actual identity affine map such as {d0} -> {d0}. This mismatch could raise an error when we are comparing sparse encodings.

This is information that the compiler already has, and should be exposed so that the library doesn't need to reimplement the exact same functionality. Differential Revision: https://reviews.llvm.org/D135341

Fixed formatting of the section violating 80-char line limit.

For some reasons enable_aliases is not set when we LLVM_ENABLE_RUNTIMES=compiler-rt instead of LLVM_ENABLE_PROJECTS.

llvmbot · 2023-09-11T23:45:32Z

@llvm/pr-subscribers-libc

Changes

There are a number of mathematical functions where no target-agnostic implementations exist, and the compiler built-ins are not correctly lowered on all GPU targets. This patch adds inlined wrappers for those functions to the GPU version of `libm` for AMDGPU and NVPTX targets.

Patch is 51.98 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/66031.diff

50 Files Affected:

(modified) libc/config/gpu/entrypoints.txt (+34-3)
(modified) libc/src/math/CMakeLists.txt (+15)
(added) libc/src/math/acos.h (+18)
(added) libc/src/math/acosh.h (+18)
(added) libc/src/math/asin.h (+18)
(added) libc/src/math/asinh.h (+18)
(added) libc/src/math/atan.h (+18)
(added) libc/src/math/atan2.h (+18)
(added) libc/src/math/atan2f.h (+18)
(added) libc/src/math/atanh.h (+18)
(added) libc/src/math/erf.h (+18)
(added) libc/src/math/expm1.h (+18)
(modified) libc/src/math/gpu/vendor/CMakeLists.txt (+143)
(added) libc/src/math/gpu/vendor/acos.cpp (+18)
(added) libc/src/math/gpu/vendor/acosh.cpp (+18)
(modified) libc/src/math/gpu/vendor/amdgpu/amdgpu.h (+25)
(modified) libc/src/math/gpu/vendor/amdgpu/declarations.h (+31-3)
(added) libc/src/math/gpu/vendor/asin.cpp (+18)
(added) libc/src/math/gpu/vendor/asinh.cpp (+18)
(added) libc/src/math/gpu/vendor/atan.cpp (+18)
(added) libc/src/math/gpu/vendor/atan2.cpp (+20)
(added) libc/src/math/gpu/vendor/atan2f.cpp (+21)
(added) libc/src/math/gpu/vendor/atanh.cpp (+18)
(added) libc/src/math/gpu/vendor/erf.cpp (+18)
(added) libc/src/math/gpu/vendor/erff.cpp (+18)
(added) libc/src/math/gpu/vendor/exp.cpp (+18)
(added) libc/src/math/gpu/vendor/exp10.cpp (+18)
(added) libc/src/math/gpu/vendor/exp2.cpp (+18)
(added) libc/src/math/gpu/vendor/expm1.cpp (+18)
(added) libc/src/math/gpu/vendor/log.cpp (+18)
(added) libc/src/math/gpu/vendor/log10.cpp (+18)
(added) libc/src/math/gpu/vendor/log10f.cpp (+19)
(added) libc/src/math/gpu/vendor/log1p.cpp (+18)
(added) libc/src/math/gpu/vendor/log1pf.cpp (+18)
(added) libc/src/math/gpu/vendor/log2.cpp (+18)
(added) libc/src/math/gpu/vendor/log2f.cpp (+18)
(added) libc/src/math/gpu/vendor/logb.cpp (+18)
(added) libc/src/math/gpu/vendor/logbf.cpp (+19)
(added) libc/src/math/gpu/vendor/logf.cpp (+18)
(added) libc/src/math/gpu/vendor/lrint.cpp (+18)
(added) libc/src/math/gpu/vendor/lrintf.cpp (+18)
(added) libc/src/math/gpu/vendor/lround.cpp (+18)
(added) libc/src/math/gpu/vendor/lroundf.cpp (+20)
(modified) libc/src/math/gpu/vendor/nvptx/declarations.h (+25)
(modified) libc/src/math/gpu/vendor/nvptx/nvptx.h (+25)
(added) libc/src/math/gpu/vendor/tgamma.cpp (+18)
(added) libc/src/math/gpu/vendor/tgammaf.cpp (+18)
(added) libc/src/math/sincos.h (+18)
(added) libc/src/math/tgamma.h (+18)
(added) libc/src/math/tgammaf.h (+18)

diff --git a/libc/config/gpu/entrypoints.txt b/libc/config/gpu/entrypoints.txt index 0e314c60870c6a..730f76cbd6fbb9 100644 --- a/libc/config/gpu/entrypoints.txt +++ b/libc/config/gpu/entrypoints.txt @@ -113,11 +113,19 @@ set(TARGET_LIBC_ENTRYPOINTS set(TARGET_LIBM_ENTRYPOINTS # math.h entrypoints + libc.src.math.acos libc.src.math.acosf + libc.src.math.acosh libc.src.math.acoshf + libc.src.math.asin libc.src.math.asinf + libc.src.math.asinh libc.src.math.asinhf + libc.src.math.atan libc.src.math.atanf + libc.src.math.atan2 + libc.src.math.atan2f + libc.src.math.atanh libc.src.math.atanhf libc.src.math.ceil libc.src.math.ceilf @@ -127,9 +135,15 @@ set(TARGET_LIBM_ENTRYPOINTS libc.src.math.cosf libc.src.math.cosh libc.src.math.coshf + libc.src.math.erf + libc.src.math.erff + libc.src.math.exp10 libc.src.math.exp10f + libc.src.math.exp2 libc.src.math.exp2f + libc.src.math.exp libc.src.math.expf + libc.src.math.expm1 libc.src.math.expm1f libc.src.math.fabs libc.src.math.fabsf @@ -157,15 +171,26 @@ set(TARGET_LIBM_ENTRYPOINTS libc.src.math.llrintf libc.src.math.llround libc.src.math.llroundf - libc.src.math.pow - libc.src.math.powf - libc.src.math.sin + libc.src.math.log10 + libc.src.math.log10f + libc.src.math.log1p + libc.src.math.log1pf + libc.src.math.log2 + libc.src.math.log2f + libc.src.math.log + libc.src.math.logf + libc.src.math.lrint + libc.src.math.lrintf + libc.src.math.lround + libc.src.math.lroundf libc.src.math.modf libc.src.math.modff libc.src.math.nearbyint libc.src.math.nearbyintf libc.src.math.nextafter libc.src.math.nextafterf + libc.src.math.pow + libc.src.math.powf libc.src.math.remainder libc.src.math.remainderf libc.src.math.remquo @@ -176,6 +201,10 @@ set(TARGET_LIBM_ENTRYPOINTS libc.src.math.roundf libc.src.math.scalbn libc.src.math.scalbnf + libc.src.math.sin + libc.src.math.sinf + libc.src.math.sincos + libc.src.math.sincosf libc.src.math.sinh libc.src.math.sinhf libc.src.math.sqrt @@ -184,6 +213,8 @@ set(TARGET_LIBM_ENTRYPOINTS libc.src.math.tanf libc.src.math.tanh libc.src.math.tanhf + libc.src.math.tgamma + libc.src.math.tgammaf libc.src.math.trunc libc.src.math.truncf ) diff --git a/libc/src/math/CMakeLists.txt b/libc/src/math/CMakeLists.txt index 8b2021cac8239f..f1f72714981a9e 100644 --- a/libc/src/math/CMakeLists.txt +++ b/libc/src/math/CMakeLists.txt @@ -54,14 +54,23 @@ function(add_math_entrypoint_object name) ) endfunction() +add_math_entrypoint_object(acos) add_math_entrypoint_object(acosf) +add_math_entrypoint_object(acosh) add_math_entrypoint_object(acoshf) +add_math_entrypoint_object(asin) add_math_entrypoint_object(asinf) +add_math_entrypoint_object(asinh) add_math_entrypoint_object(asinhf) +add_math_entrypoint_object(atan) add_math_entrypoint_object(atanf) +add_math_entrypoint_object(atan2) +add_math_entrypoint_object(atan2f) + +add_math_entrypoint_object(atanh) add_math_entrypoint_object(atanhf) add_math_entrypoint_object(ceil) @@ -77,6 +86,7 @@ add_math_entrypoint_object(cosf) add_math_entrypoint_object(cosh) add_math_entrypoint_object(coshf) +add_math_entrypoint_object(erf) add_math_entrypoint_object(erff) add_math_entrypoint_object(exp) @@ -88,6 +98,7 @@ add_math_entrypoint_object(exp2f) add_math_entrypoint_object(exp10) add_math_entrypoint_object(exp10f) +add_math_entrypoint_object(expm1) add_math_entrypoint_object(expm1f) add_math_entrypoint_object(fabs) @@ -198,6 +209,7 @@ add_math_entrypoint_object(scalbn) add_math_entrypoint_object(scalbnf) add_math_entrypoint_object(scalbnl) +add_math_entrypoint_object(sincos) add_math_entrypoint_object(sincosf) add_math_entrypoint_object(sin) @@ -216,6 +228,9 @@ add_math_entrypoint_object(tanf) add_math_entrypoint_object(tanh) add_math_entrypoint_object(tanhf) +add_math_entrypoint_object(tgamma) +add_math_entrypoint_object(tgammaf) + add_math_entrypoint_object(trunc) add_math_entrypoint_object(truncf) add_math_entrypoint_object(truncl) diff --git a/libc/src/math/acos.h b/libc/src/math/acos.h new file mode 100644 index 00000000000000..53b299f1da8902 --- /dev/null +++ b/libc/src/math/acos.h @@ -0,0 +1,18 @@ +//===-- Implementation header for acos --------------------------*- C++ -*-===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#ifndef LLVM_LIBC_SRC_MATH_ACOS_H +#define LLVM_LIBC_SRC_MATH_ACOS_H + +namespace __llvm_libc { + +double acos(double x); + +} // namespace __llvm_libc + +#endif // LLVM_LIBC_SRC_MATH_ACOS_H diff --git a/libc/src/math/acosh.h b/libc/src/math/acosh.h new file mode 100644 index 00000000000000..851c619fdb9f79 --- /dev/null +++ b/libc/src/math/acosh.h @@ -0,0 +1,18 @@ +//===-- Implementation header for acosh -------------------------*- C++ -*-===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#ifndef LLVM_LIBC_SRC_MATH_ACOSH_H +#define LLVM_LIBC_SRC_MATH_ACOSH_H + +namespace __llvm_libc { + +double acosh(double x); + +} // namespace __llvm_libc + +#endif // LLVM_LIBC_SRC_MATH_ACOSH_H diff --git a/libc/src/math/asin.h b/libc/src/math/asin.h new file mode 100644 index 00000000000000..d8f3f191a35785 --- /dev/null +++ b/libc/src/math/asin.h @@ -0,0 +1,18 @@ +//===-- Implementation header for asin --------------------------*- C++ -*-===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#ifndef LLVM_LIBC_SRC_MATH_ASIN_H +#define LLVM_LIBC_SRC_MATH_ASIN_H + +namespace __llvm_libc { + +double asin(double x); + +} // namespace __llvm_libc + +#endif // LLVM_LIBC_SRC_MATH_ASIN_H diff --git a/libc/src/math/asinh.h b/libc/src/math/asinh.h new file mode 100644 index 00000000000000..098880f26ecc80 --- /dev/null +++ b/libc/src/math/asinh.h @@ -0,0 +1,18 @@ +//===-- Implementation header for asinh -------------------------*- C++ -*-===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#ifndef LLVM_LIBC_SRC_MATH_ASINH_H +#define LLVM_LIBC_SRC_MATH_ASINH_H + +namespace __llvm_libc { + +double asinh(double x); + +} // namespace __llvm_libc + +#endif // LLVM_LIBC_SRC_MATH_ASINH_H diff --git a/libc/src/math/atan.h b/libc/src/math/atan.h new file mode 100644 index 00000000000000..73d67e526d7d5a --- /dev/null +++ b/libc/src/math/atan.h @@ -0,0 +1,18 @@ +//===-- Implementation header for atan --------------------------*- C++ -*-===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#ifndef LLVM_LIBC_SRC_MATH_ATAN_H +#define LLVM_LIBC_SRC_MATH_ATAN_H + +namespace __llvm_libc { + +double atan(double x); + +} // namespace __llvm_libc + +#endif // LLVM_LIBC_SRC_MATH_ATAN_H diff --git a/libc/src/math/atan2.h b/libc/src/math/atan2.h new file mode 100644 index 00000000000000..cac5b7ddfbb62c --- /dev/null +++ b/libc/src/math/atan2.h @@ -0,0 +1,18 @@ +//===-- Implementation header for atan2 -------------------------*- C++ -*-===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#ifndef LLVM_LIBC_SRC_MATH_ATAN2_H +#define LLVM_LIBC_SRC_MATH_ATAN2_H + +namespace __llvm_libc { + +double atan2(double x, double y); + +} // namespace __llvm_libc + +#endif // LLVM_LIBC_SRC_MATH_ATAN2_H diff --git a/libc/src/math/atan2f.h b/libc/src/math/atan2f.h new file mode 100644 index 00000000000000..d3e81a2b8d5f2d --- /dev/null +++ b/libc/src/math/atan2f.h @@ -0,0 +1,18 @@ +//===-- Implementation header for atan2f ------------------------*- C++ -*-===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#ifndef LLVM_LIBC_SRC_MATH_ATAN2F_H +#define LLVM_LIBC_SRC_MATH_ATAN2F_H + +namespace __llvm_libc { + +float atan2f(float x, float y); + +} // namespace __llvm_libc + +#endif // LLVM_LIBC_SRC_MATH_ATAN2F_H diff --git a/libc/src/math/atanh.h b/libc/src/math/atanh.h new file mode 100644 index 00000000000000..de854451e7a281 --- /dev/null +++ b/libc/src/math/atanh.h @@ -0,0 +1,18 @@ +//===-- Implementation header for atanh -------------------------*- C++ -*-===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#ifndef LLVM_LIBC_SRC_MATH_ATANH_H +#define LLVM_LIBC_SRC_MATH_ATANH_H + +namespace __llvm_libc { + +double atanh(double x); + +} // namespace __llvm_libc + +#endif // LLVM_LIBC_SRC_MATH_ATANH_H diff --git a/libc/src/math/erf.h b/libc/src/math/erf.h new file mode 100644 index 00000000000000..86a146182b4f72 --- /dev/null +++ b/libc/src/math/erf.h @@ -0,0 +1,18 @@ +//===-- Implementation header for erf ---------------------------*- C++ -*-===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#ifndef LLVM_LIBC_SRC_MATH_ERF_H +#define LLVM_LIBC_SRC_MATH_ERF_H + +namespace __llvm_libc { + +double erf(double x); + +} // namespace __llvm_libc + +#endif // LLVM_LIBC_SRC_MATH_ERF_H diff --git a/libc/src/math/expm1.h b/libc/src/math/expm1.h new file mode 100644 index 00000000000000..7ae354e459feb1 --- /dev/null +++ b/libc/src/math/expm1.h @@ -0,0 +1,18 @@ +//===-- Implementation header for expm1 -------------------------*- C++ -*-===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#ifndef LLVM_LIBC_SRC_MATH_EXPM1_H +#define LLVM_LIBC_SRC_MATH_EXPM1_H + +namespace __llvm_libc { + +double expm1(double x); + +} // namespace __llvm_libc + +#endif // LLVM_LIBC_SRC_MATH_EXPM1_H diff --git a/libc/src/math/gpu/vendor/CMakeLists.txt b/libc/src/math/gpu/vendor/CMakeLists.txt index 2ee74a06a02d46..58910ac517bdc9 100644 --- a/libc/src/math/gpu/vendor/CMakeLists.txt +++ b/libc/src/math/gpu/vendor/CMakeLists.txt @@ -29,6 +29,17 @@ endif() # will link in identity metadata from both libraries. This silences the warning. list(APPEND bitcode_link_flags "-Wno-linker-warnings") +add_entrypoint_object( + acos + SRCS + acos.cpp + HDRS + ../../acos.h + COMPILE_OPTIONS + ${bitcode_link_flags} + -O2 +) + add_entrypoint_object( acosf SRCS @@ -40,6 +51,17 @@ add_entrypoint_object( -O2 ) +add_entrypoint_object( + acosh + SRCS + acosh.cpp + HDRS + ../../acosh.h + COMPILE_OPTIONS + ${bitcode_link_flags} + -O2 +) + add_entrypoint_object( acoshf SRCS @@ -51,6 +73,17 @@ add_entrypoint_object( -O2 ) +add_entrypoint_object( + asin + SRCS + asin.cpp + HDRS + ../../asin.h + COMPILE_OPTIONS + ${bitcode_link_flags} + -O2 +) + add_entrypoint_object( asinf SRCS @@ -62,6 +95,17 @@ add_entrypoint_object( -O2 ) +add_entrypoint_object( + asinh + SRCS + asinh.cpp + HDRS + ../../asinh.h + COMPILE_OPTIONS + ${bitcode_link_flags} + -O2 +) + add_entrypoint_object( asinhf SRCS @@ -73,6 +117,17 @@ add_entrypoint_object( -O2 ) +add_entrypoint_object( + atan + SRCS + atan.cpp + HDRS + ../../atan.h + COMPILE_OPTIONS + ${bitcode_link_flags} + -O2 +) + add_entrypoint_object( atanf SRCS @@ -84,6 +139,39 @@ add_entrypoint_object( -O2 ) +add_entrypoint_object( + atan2 + SRCS + atan2.cpp + HDRS + ../../atan2.h + COMPILE_OPTIONS + ${bitcode_link_flags} + -O2 +) + +add_entrypoint_object( + atan2f + SRCS + atan2f.cpp + HDRS + ../../atan2f.h + COMPILE_OPTIONS + ${bitcode_link_flags} + -O2 +) + +add_entrypoint_object( + atanh + SRCS + atanh.cpp + HDRS + ../../atanh.h + COMPILE_OPTIONS + ${bitcode_link_flags} + -O2 +) + add_entrypoint_object( atanhf SRCS @@ -139,6 +227,28 @@ add_entrypoint_object( -O2 ) +add_entrypoint_object( + erf + SRCS + erf.cpp + HDRS + ../../erf.h + COMPILE_OPTIONS + ${bitcode_link_flags} + -O2 +) + +add_entrypoint_object( + erff + SRCS + erff.cpp + HDRS + ../../erff.h + COMPILE_OPTIONS + ${bitcode_link_flags} + -O2 +) + add_entrypoint_object( exp10f SRCS @@ -172,6 +282,17 @@ add_entrypoint_object( -O2 ) +add_entrypoint_object( + expm1 + SRCS + expm1.cpp + HDRS + ../../expm1.h + COMPILE_OPTIONS + ${bitcode_link_flags} + -O2 +) + add_entrypoint_object( expm1f SRCS @@ -515,6 +636,28 @@ add_entrypoint_object( -O2 ) +add_entrypoint_object( + tgamma + SRCS + tgamma.cpp + HDRS + ../../tgamma.h + COMPILE_OPTIONS + ${bitcode_link_flags} + -O2 +) + +add_entrypoint_object( + tgammaf + SRCS + tgammaf.cpp + HDRS + ../../tgammaf.h + COMPILE_OPTIONS + ${bitcode_link_flags} + -O2 +) + add_entrypoint_object( frexp SRCS diff --git a/libc/src/math/gpu/vendor/acos.cpp b/libc/src/math/gpu/vendor/acos.cpp new file mode 100644 index 00000000000000..7e6d00be3c3e58 --- /dev/null +++ b/libc/src/math/gpu/vendor/acos.cpp @@ -0,0 +1,18 @@ +//===-- Implementation of the GPU acos function ---------------------------===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#include "src/math/acos.h" +#include "src/__support/common.h" + +#include "common.h" + +namespace __llvm_libc { + +LLVM_LIBC_FUNCTION(double, acos, (double x)) { return internal::acos(x); } + +} // namespace __llvm_libc diff --git a/libc/src/math/gpu/vendor/acosh.cpp b/libc/src/math/gpu/vendor/acosh.cpp new file mode 100644 index 00000000000000..2ea150dcf78e03 --- /dev/null +++ b/libc/src/math/gpu/vendor/acosh.cpp @@ -0,0 +1,18 @@ +//===-- Implementation of the GPU acosh function --------------------------===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#include "src/math/acosh.h" +#include "src/__support/common.h" + +#include "common.h" + +namespace __llvm_libc { + +LLVM_LIBC_FUNCTION(double, acosh, (double x)) { return internal::acosh(x); } + +} // namespace __llvm_libc diff --git a/libc/src/math/gpu/vendor/amdgpu/amdgpu.h b/libc/src/math/gpu/vendor/amdgpu/amdgpu.h index 7755174e445b22..a763f2e3d0f53f 100644 --- a/libc/src/math/gpu/vendor/amdgpu/amdgpu.h +++ b/libc/src/math/gpu/vendor/amdgpu/amdgpu.h @@ -16,19 +16,30 @@ namespace __llvm_libc { namespace internal { +LIBC_INLINE double acos(double x) { return __ocml_acos_f64(x); } LIBC_INLINE float acosf(float x) { return __ocml_acos_f32(x); } +LIBC_INLINE double acosh(double x) { return __ocml_acosh_f64(x); } LIBC_INLINE float acoshf(float x) { return __ocml_acosh_f32(x); } +LIBC_INLINE double asin(double x) { return __ocml_asin_f64(x); } LIBC_INLINE float asinf(float x) { return __ocml_asin_f32(x); } +LIBC_INLINE double asinh(double x) { return __ocml_asinh_f64(x); } LIBC_INLINE float asinhf(float x) { return __ocml_asinh_f32(x); } +LIBC_INLINE double atan(double x) { return __ocml_atan_f64(x); } LIBC_INLINE float atanf(float x) { return __ocml_atan_f32(x); } +LIBC_INLINE double atan2(double x, double y) { return __ocml_atan2_f64(x, y); } +LIBC_INLINE float atan2f(float x, float y) { return __ocml_atan2_f32(x, y); } +LIBC_INLINE double atanh(double x) { return __ocml_atanh_f64(x); } LIBC_INLINE float atanhf(float x) { return __ocml_atanh_f32(x); } LIBC_INLINE double cos(double x) { return __ocml_cos_f64(x); } LIBC_INLINE float cosf(float x) { return __ocml_cos_f32(x); } LIBC_INLINE double cosh(double x) { return __ocml_cosh_f64(x); } LIBC_INLINE float coshf(float x) { return __ocml_cosh_f...

llvmbot · 2023-09-11T23:46:00Z

@llvm/pr-subscribers-backend-amdgpu

Changes

There are a number of mathematical functions where no target-agnostic implementations exist, and the compiler built-ins are not correctly lowered on all GPU targets. This patch adds inlined wrappers for those functions to the GPU version of `libm` for AMDGPU and NVPTX targets.

Patch is 52.08 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/66031.diff

50 Files Affected:

(modified) libc/config/gpu/entrypoints.txt (+34-3)
(modified) libc/src/math/CMakeLists.txt (+15)
(added) libc/src/math/acos.h (+18)
(added) libc/src/math/acosh.h (+18)
(added) libc/src/math/asin.h (+18)
(added) libc/src/math/asinh.h (+18)
(added) libc/src/math/atan.h (+18)
(added) libc/src/math/atan2.h (+18)
(added) libc/src/math/atan2f.h (+18)
(added) libc/src/math/atanh.h (+18)
(added) libc/src/math/erf.h (+18)
(added) libc/src/math/expm1.h (+18)
(modified) libc/src/math/gpu/vendor/CMakeLists.txt (+143)
(added) libc/src/math/gpu/vendor/acos.cpp (+18)
(added) libc/src/math/gpu/vendor/acosh.cpp (+18)
(modified) libc/src/math/gpu/vendor/amdgpu/amdgpu.h (+25)
(modified) libc/src/math/gpu/vendor/amdgpu/declarations.h (+31-3)
(added) libc/src/math/gpu/vendor/asin.cpp (+18)
(added) libc/src/math/gpu/vendor/asinh.cpp (+18)
(added) libc/src/math/gpu/vendor/atan.cpp (+18)
(added) libc/src/math/gpu/vendor/atan2.cpp (+20)
(added) libc/src/math/gpu/vendor/atan2f.cpp (+21)
(added) libc/src/math/gpu/vendor/atanh.cpp (+18)
(added) libc/src/math/gpu/vendor/erf.cpp (+18)
(added) libc/src/math/gpu/vendor/erff.cpp (+18)
(added) libc/src/math/gpu/vendor/exp.cpp (+18)
(added) libc/src/math/gpu/vendor/exp10.cpp (+18)
(added) libc/src/math/gpu/vendor/exp2.cpp (+18)
(added) libc/src/math/gpu/vendor/expm1.cpp (+18)
(added) libc/src/math/gpu/vendor/log.cpp (+18)
(added) libc/src/math/gpu/vendor/log10.cpp (+18)
(added) libc/src/math/gpu/vendor/log10f.cpp (+19)
(added) libc/src/math/gpu/vendor/log1p.cpp (+18)
(added) libc/src/math/gpu/vendor/log1pf.cpp (+18)
(added) libc/src/math/gpu/vendor/log2.cpp (+18)
(added) libc/src/math/gpu/vendor/log2f.cpp (+18)
(added) libc/src/math/gpu/vendor/logb.cpp (+18)
(added) libc/src/math/gpu/vendor/logbf.cpp (+19)
(added) libc/src/math/gpu/vendor/logf.cpp (+18)
(added) libc/src/math/gpu/vendor/lrint.cpp (+18)
(added) libc/src/math/gpu/vendor/lrintf.cpp (+18)
(added) libc/src/math/gpu/vendor/lround.cpp (+18)
(added) libc/src/math/gpu/vendor/lroundf.cpp (+20)
(modified) libc/src/math/gpu/vendor/nvptx/declarations.h (+25)
(modified) libc/src/math/gpu/vendor/nvptx/nvptx.h (+25)
(added) libc/src/math/gpu/vendor/tgamma.cpp (+18)
(added) libc/src/math/gpu/vendor/tgammaf.cpp (+18)
(added) libc/src/math/sincos.h (+18)
(added) libc/src/math/tgamma.h (+18)
(added) libc/src/math/tgammaf.h (+18)

diff --git a/libc/config/gpu/entrypoints.txt b/libc/config/gpu/entrypoints.txt index 0e314c60870c6ae..730f76cbd6fbb9c 100644 --- a/libc/config/gpu/entrypoints.txt +++ b/libc/config/gpu/entrypoints.txt @@ -113,11 +113,19 @@ set(TARGET_LIBC_ENTRYPOINTS set(TARGET_LIBM_ENTRYPOINTS # math.h entrypoints + libc.src.math.acos libc.src.math.acosf + libc.src.math.acosh libc.src.math.acoshf + libc.src.math.asin libc.src.math.asinf + libc.src.math.asinh libc.src.math.asinhf + libc.src.math.atan libc.src.math.atanf + libc.src.math.atan2 + libc.src.math.atan2f + libc.src.math.atanh libc.src.math.atanhf libc.src.math.ceil libc.src.math.ceilf @@ -127,9 +135,15 @@ set(TARGET_LIBM_ENTRYPOINTS libc.src.math.cosf libc.src.math.cosh libc.src.math.coshf + libc.src.math.erf + libc.src.math.erff + libc.src.math.exp10 libc.src.math.exp10f + libc.src.math.exp2 libc.src.math.exp2f + libc.src.math.exp libc.src.math.expf + libc.src.math.expm1 libc.src.math.expm1f libc.src.math.fabs libc.src.math.fabsf @@ -157,15 +171,26 @@ set(TARGET_LIBM_ENTRYPOINTS libc.src.math.llrintf libc.src.math.llround libc.src.math.llroundf - libc.src.math.pow - libc.src.math.powf - libc.src.math.sin + libc.src.math.log10 + libc.src.math.log10f + libc.src.math.log1p + libc.src.math.log1pf + libc.src.math.log2 + libc.src.math.log2f + libc.src.math.log + libc.src.math.logf + libc.src.math.lrint + libc.src.math.lrintf + libc.src.math.lround + libc.src.math.lroundf libc.src.math.modf libc.src.math.modff libc.src.math.nearbyint libc.src.math.nearbyintf libc.src.math.nextafter libc.src.math.nextafterf + libc.src.math.pow + libc.src.math.powf libc.src.math.remainder libc.src.math.remainderf libc.src.math.remquo @@ -176,6 +201,10 @@ set(TARGET_LIBM_ENTRYPOINTS libc.src.math.roundf libc.src.math.scalbn libc.src.math.scalbnf + libc.src.math.sin + libc.src.math.sinf + libc.src.math.sincos + libc.src.math.sincosf libc.src.math.sinh libc.src.math.sinhf libc.src.math.sqrt @@ -184,6 +213,8 @@ set(TARGET_LIBM_ENTRYPOINTS libc.src.math.tanf libc.src.math.tanh libc.src.math.tanhf + libc.src.math.tgamma + libc.src.math.tgammaf libc.src.math.trunc libc.src.math.truncf ) diff --git a/libc/src/math/CMakeLists.txt b/libc/src/math/CMakeLists.txt index 8b2021cac8239fe..f1f72714981a9e5 100644 --- a/libc/src/math/CMakeLists.txt +++ b/libc/src/math/CMakeLists.txt @@ -54,14 +54,23 @@ function(add_math_entrypoint_object name) ) endfunction() +add_math_entrypoint_object(acos) add_math_entrypoint_object(acosf) +add_math_entrypoint_object(acosh) add_math_entrypoint_object(acoshf) +add_math_entrypoint_object(asin) add_math_entrypoint_object(asinf) +add_math_entrypoint_object(asinh) add_math_entrypoint_object(asinhf) +add_math_entrypoint_object(atan) add_math_entrypoint_object(atanf) +add_math_entrypoint_object(atan2) +add_math_entrypoint_object(atan2f) + +add_math_entrypoint_object(atanh) add_math_entrypoint_object(atanhf) add_math_entrypoint_object(ceil) @@ -77,6 +86,7 @@ add_math_entrypoint_object(cosf) add_math_entrypoint_object(cosh) add_math_entrypoint_object(coshf) +add_math_entrypoint_object(erf) add_math_entrypoint_object(erff) add_math_entrypoint_object(exp) @@ -88,6 +98,7 @@ add_math_entrypoint_object(exp2f) add_math_entrypoint_object(exp10) add_math_entrypoint_object(exp10f) +add_math_entrypoint_object(expm1) add_math_entrypoint_object(expm1f) add_math_entrypoint_object(fabs) @@ -198,6 +209,7 @@ add_math_entrypoint_object(scalbn) add_math_entrypoint_object(scalbnf) add_math_entrypoint_object(scalbnl) +add_math_entrypoint_object(sincos) add_math_entrypoint_object(sincosf) add_math_entrypoint_object(sin) @@ -216,6 +228,9 @@ add_math_entrypoint_object(tanf) add_math_entrypoint_object(tanh) add_math_entrypoint_object(tanhf) +add_math_entrypoint_object(tgamma) +add_math_entrypoint_object(tgammaf) + add_math_entrypoint_object(trunc) add_math_entrypoint_object(truncf) add_math_entrypoint_object(truncl) diff --git a/libc/src/math/acos.h b/libc/src/math/acos.h new file mode 100644 index 000000000000000..53b299f1da8902e --- /dev/null +++ b/libc/src/math/acos.h @@ -0,0 +1,18 @@ +//===-- Implementation header for acos --------------------------*- C++ -*-===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#ifndef LLVM_LIBC_SRC_MATH_ACOS_H +#define LLVM_LIBC_SRC_MATH_ACOS_H + +namespace __llvm_libc { + +double acos(double x); + +} // namespace __llvm_libc + +#endif // LLVM_LIBC_SRC_MATH_ACOS_H diff --git a/libc/src/math/acosh.h b/libc/src/math/acosh.h new file mode 100644 index 000000000000000..851c619fdb9f799 --- /dev/null +++ b/libc/src/math/acosh.h @@ -0,0 +1,18 @@ +//===-- Implementation header for acosh -------------------------*- C++ -*-===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#ifndef LLVM_LIBC_SRC_MATH_ACOSH_H +#define LLVM_LIBC_SRC_MATH_ACOSH_H + +namespace __llvm_libc { + +double acosh(double x); + +} // namespace __llvm_libc + +#endif // LLVM_LIBC_SRC_MATH_ACOSH_H diff --git a/libc/src/math/asin.h b/libc/src/math/asin.h new file mode 100644 index 000000000000000..d8f3f191a357856 --- /dev/null +++ b/libc/src/math/asin.h @@ -0,0 +1,18 @@ +//===-- Implementation header for asin --------------------------*- C++ -*-===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#ifndef LLVM_LIBC_SRC_MATH_ASIN_H +#define LLVM_LIBC_SRC_MATH_ASIN_H + +namespace __llvm_libc { + +double asin(double x); + +} // namespace __llvm_libc + +#endif // LLVM_LIBC_SRC_MATH_ASIN_H diff --git a/libc/src/math/asinh.h b/libc/src/math/asinh.h new file mode 100644 index 000000000000000..098880f26ecc80c --- /dev/null +++ b/libc/src/math/asinh.h @@ -0,0 +1,18 @@ +//===-- Implementation header for asinh -------------------------*- C++ -*-===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#ifndef LLVM_LIBC_SRC_MATH_ASINH_H +#define LLVM_LIBC_SRC_MATH_ASINH_H + +namespace __llvm_libc { + +double asinh(double x); + +} // namespace __llvm_libc + +#endif // LLVM_LIBC_SRC_MATH_ASINH_H diff --git a/libc/src/math/atan.h b/libc/src/math/atan.h new file mode 100644 index 000000000000000..73d67e526d7d5a5 --- /dev/null +++ b/libc/src/math/atan.h @@ -0,0 +1,18 @@ +//===-- Implementation header for atan --------------------------*- C++ -*-===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#ifndef LLVM_LIBC_SRC_MATH_ATAN_H +#define LLVM_LIBC_SRC_MATH_ATAN_H + +namespace __llvm_libc { + +double atan(double x); + +} // namespace __llvm_libc + +#endif // LLVM_LIBC_SRC_MATH_ATAN_H diff --git a/libc/src/math/atan2.h b/libc/src/math/atan2.h new file mode 100644 index 000000000000000..cac5b7ddfbb62c1 --- /dev/null +++ b/libc/src/math/atan2.h @@ -0,0 +1,18 @@ +//===-- Implementation header for atan2 -------------------------*- C++ -*-===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#ifndef LLVM_LIBC_SRC_MATH_ATAN2_H +#define LLVM_LIBC_SRC_MATH_ATAN2_H + +namespace __llvm_libc { + +double atan2(double x, double y); + +} // namespace __llvm_libc + +#endif // LLVM_LIBC_SRC_MATH_ATAN2_H diff --git a/libc/src/math/atan2f.h b/libc/src/math/atan2f.h new file mode 100644 index 000000000000000..d3e81a2b8d5f2d4 --- /dev/null +++ b/libc/src/math/atan2f.h @@ -0,0 +1,18 @@ +//===-- Implementation header for atan2f ------------------------*- C++ -*-===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#ifndef LLVM_LIBC_SRC_MATH_ATAN2F_H +#define LLVM_LIBC_SRC_MATH_ATAN2F_H + +namespace __llvm_libc { + +float atan2f(float x, float y); + +} // namespace __llvm_libc + +#endif // LLVM_LIBC_SRC_MATH_ATAN2F_H diff --git a/libc/src/math/atanh.h b/libc/src/math/atanh.h new file mode 100644 index 000000000000000..de854451e7a281f --- /dev/null +++ b/libc/src/math/atanh.h @@ -0,0 +1,18 @@ +//===-- Implementation header for atanh -------------------------*- C++ -*-===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#ifndef LLVM_LIBC_SRC_MATH_ATANH_H +#define LLVM_LIBC_SRC_MATH_ATANH_H + +namespace __llvm_libc { + +double atanh(double x); + +} // namespace __llvm_libc + +#endif // LLVM_LIBC_SRC_MATH_ATANH_H diff --git a/libc/src/math/erf.h b/libc/src/math/erf.h new file mode 100644 index 000000000000000..86a146182b4f729 --- /dev/null +++ b/libc/src/math/erf.h @@ -0,0 +1,18 @@ +//===-- Implementation header for erf ---------------------------*- C++ -*-===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#ifndef LLVM_LIBC_SRC_MATH_ERF_H +#define LLVM_LIBC_SRC_MATH_ERF_H + +namespace __llvm_libc { + +double erf(double x); + +} // namespace __llvm_libc + +#endif // LLVM_LIBC_SRC_MATH_ERF_H diff --git a/libc/src/math/expm1.h b/libc/src/math/expm1.h new file mode 100644 index 000000000000000..7ae354e459feb1f --- /dev/null +++ b/libc/src/math/expm1.h @@ -0,0 +1,18 @@ +//===-- Implementation header for expm1 -------------------------*- C++ -*-===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#ifndef LLVM_LIBC_SRC_MATH_EXPM1_H +#define LLVM_LIBC_SRC_MATH_EXPM1_H + +namespace __llvm_libc { + +double expm1(double x); + +} // namespace __llvm_libc + +#endif // LLVM_LIBC_SRC_MATH_EXPM1_H diff --git a/libc/src/math/gpu/vendor/CMakeLists.txt b/libc/src/math/gpu/vendor/CMakeLists.txt index 2ee74a06a02d461..58910ac517bdc97 100644 --- a/libc/src/math/gpu/vendor/CMakeLists.txt +++ b/libc/src/math/gpu/vendor/CMakeLists.txt @@ -29,6 +29,17 @@ endif() # will link in identity metadata from both libraries. This silences the warning. list(APPEND bitcode_link_flags "-Wno-linker-warnings") +add_entrypoint_object( + acos + SRCS + acos.cpp + HDRS + ../../acos.h + COMPILE_OPTIONS + ${bitcode_link_flags} + -O2 +) + add_entrypoint_object( acosf SRCS @@ -40,6 +51,17 @@ add_entrypoint_object( -O2 ) +add_entrypoint_object( + acosh + SRCS + acosh.cpp + HDRS + ../../acosh.h + COMPILE_OPTIONS + ${bitcode_link_flags} + -O2 +) + add_entrypoint_object( acoshf SRCS @@ -51,6 +73,17 @@ add_entrypoint_object( -O2 ) +add_entrypoint_object( + asin + SRCS + asin.cpp + HDRS + ../../asin.h + COMPILE_OPTIONS + ${bitcode_link_flags} + -O2 +) + add_entrypoint_object( asinf SRCS @@ -62,6 +95,17 @@ add_entrypoint_object( -O2 ) +add_entrypoint_object( + asinh + SRCS + asinh.cpp + HDRS + ../../asinh.h + COMPILE_OPTIONS + ${bitcode_link_flags} + -O2 +) + add_entrypoint_object( asinhf SRCS @@ -73,6 +117,17 @@ add_entrypoint_object( -O2 ) +add_entrypoint_object( + atan + SRCS + atan.cpp + HDRS + ../../atan.h + COMPILE_OPTIONS + ${bitcode_link_flags} + -O2 +) + add_entrypoint_object( atanf SRCS @@ -84,6 +139,39 @@ add_entrypoint_object( -O2 ) +add_entrypoint_object( + atan2 + SRCS + atan2.cpp + HDRS + ../../atan2.h + COMPILE_OPTIONS + ${bitcode_link_flags} + -O2 +) + +add_entrypoint_object( + atan2f + SRCS + atan2f.cpp + HDRS + ../../atan2f.h + COMPILE_OPTIONS + ${bitcode_link_flags} + -O2 +) + +add_entrypoint_object( + atanh + SRCS + atanh.cpp + HDRS + ../../atanh.h + COMPILE_OPTIONS + ${bitcode_link_flags} + -O2 +) + add_entrypoint_object( atanhf SRCS @@ -139,6 +227,28 @@ add_entrypoint_object( -O2 ) +add_entrypoint_object( + erf + SRCS + erf.cpp + HDRS + ../../erf.h + COMPILE_OPTIONS + ${bitcode_link_flags} + -O2 +) + +add_entrypoint_object( + erff + SRCS + erff.cpp + HDRS + ../../erff.h + COMPILE_OPTIONS + ${bitcode_link_flags} + -O2 +) + add_entrypoint_object( exp10f SRCS @@ -172,6 +282,17 @@ add_entrypoint_object( -O2 ) +add_entrypoint_object( + expm1 + SRCS + expm1.cpp + HDRS + ../../expm1.h + COMPILE_OPTIONS + ${bitcode_link_flags} + -O2 +) + add_entrypoint_object( expm1f SRCS @@ -515,6 +636,28 @@ add_entrypoint_object( -O2 ) +add_entrypoint_object( + tgamma + SRCS + tgamma.cpp + HDRS + ../../tgamma.h + COMPILE_OPTIONS + ${bitcode_link_flags} + -O2 +) + +add_entrypoint_object( + tgammaf + SRCS + tgammaf.cpp + HDRS + ../../tgammaf.h + COMPILE_OPTIONS + ${bitcode_link_flags} + -O2 +) + add_entrypoint_object( frexp SRCS diff --git a/libc/src/math/gpu/vendor/acos.cpp b/libc/src/math/gpu/vendor/acos.cpp new file mode 100644 index 000000000000000..7e6d00be3c3e589 --- /dev/null +++ b/libc/src/math/gpu/vendor/acos.cpp @@ -0,0 +1,18 @@ +//===-- Implementation of the GPU acos function ---------------------------===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#include "src/math/acos.h" +#include "src/__support/common.h" + +#include "common.h" + +namespace __llvm_libc { + +LLVM_LIBC_FUNCTION(double, acos, (double x)) { return internal::acos(x); } + +} // namespace __llvm_libc diff --git a/libc/src/math/gpu/vendor/acosh.cpp b/libc/src/math/gpu/vendor/acosh.cpp new file mode 100644 index 000000000000000..2ea150dcf78e03a --- /dev/null +++ b/libc/src/math/gpu/vendor/acosh.cpp @@ -0,0 +1,18 @@ +//===-- Implementation of the GPU acosh function --------------------------===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#include "src/math/acosh.h" +#include "src/__support/common.h" + +#include "common.h" + +namespace __llvm_libc { + +LLVM_LIBC_FUNCTION(double, acosh, (double x)) { return internal::acosh(x); } + +} // namespace __llvm_libc diff --git a/libc/src/math/gpu/vendor/amdgpu/amdgpu.h b/libc/src/math/gpu/vendor/amdgpu/amdgpu.h index 7755174e445b222..a763f2e3d0f53f1 100644 --- a/libc/src/math/gpu/vendor/amdgpu/amdgpu.h +++ b/libc/src/math/gpu/vendor/amdgpu/amdgpu.h @@ -16,19 +16,30 @@ namespace __llvm_libc { namespace internal { +LIBC_INLINE double acos(double x) { return __ocml_acos_f64(x); } LIBC_INLINE float acosf(float x) { return __ocml_acos_f32(x); } +LIBC_INLINE double acosh(double x) { return __ocml_acosh_f64(x); } LIBC_INLINE float acoshf(float x) { return __ocml_acosh_f32(x); } +LIBC_INLINE double asin(double x) { return __ocml_asin_f64(x); } LIBC_INLINE float asinf(float x) { return __ocml_asin_f32(x); } +LIBC_INLINE double asinh(double x) { return __ocml_asinh_f64(x); } LIBC_INLINE float asinhf(float x) { return __ocml_asinh_f32(x); } +LIBC_INLINE double atan(double x) { return __ocml_atan_f64(x); } LIBC_INLINE float atanf(float x) { return __ocml_atan_f32(x); } +LIBC_INLINE double atan2(double x, double y) { return __ocml_atan2_f64(x, y); } +LIBC_INLINE float atan2f(float x, float y) { return __ocml_atan2_f32(x, y); } +LIBC_INLINE double atanh(double x) { return __ocml_atanh_f64(x); } LIBC_INLINE float atanhf(float x) { return __ocml_atanh_f32(x); } LIBC_INLINE double cos(double x) { return __ocml_cos_f64(x); } LIBC_INLINE float cosf(float x) { return __ocml_cos_f32(x); } LIBC_INLINE double cosh(double x) { return __ocml_cosh_f64(x); } LIBC_INLINE float coshf...

AntonRydahl and others added 30 commits September 11, 2023 16:26

[LLDB][NFC] Add the mojo language to Language::GetPrimaryLanguage

6e4e47f

This doesn't change the current behavior of the function, but the explicit declaration looks cleaner.

[test][tsan] Disable flaky test on PPC

9969f95

COMPILER_RT_DEBUG was just added to sanitizer-ppc64le-linux, and this test is already broken there.

Lazy initialize diagnostic when handling MLIR properties (llvm#65868)

82f0005

Instead of eagerly creating a diagnostic that will be discarded in the normal case, switch to lazy initialization on error.

[mlir][Linalg] Move linalg.fill -> linalg.pack pattern into `fill…

66f1645

…` canonicalization patterns. (llvm#66002) This pattern fits better with the other canonicalization patterns that exist for `linalg.fill`.

[ELF] Reset two member variables in Ctx::reset

3ed7d3e

Otherwise they are dangling if lldMain is called more than once.

Revert "[Driver] Properly report error for unsupported powerpc darwin…

7206f48

…/macos triples" This reverts commit 9f77fac. The change unintentionally changed lots of codegen, see llvm#47698 (comment) Also revert a follow-up: This reverts commit b40a5be.

[ORC] Fix implicit conversion warning due to 5293109.

0b4ef29

This should fix the warning seen in https://lab.llvm.org/buildbot/#/builders/13/builds/39980/steps/6/logs/stdio

[test] Add x86-registered-target to amdgpu_throw_trap.cpp

730f597

[cfi-verify tests]: Skip two x86-only tests if x86 is not enabled

c82b897

With this, check-llvm passes on an arm mac if x86 isn't in LLVM_TARGETS_TO_BUILD. This pattern to skip the tests if x86 isn't enabled is used in every other test in this file.

[test][hwasan] Disable test failing on x86_64 with no -lstdc++

cd4428b

[test][hwasan] Relax test condition

45d89b5

The test is flaky after Kernel upgrade from 6.0 to 6.5.

[test] Change llc -march= to -mtriple= & llvm-mc -arch= to -triple=

b3f442c

Similar to 806761a

[test] debug-info-correlate.ll requires an ELF target triple

21051b4

The test only applies to ELF. On Linux, when a default target triple is, say, Mach-O, the test should be excluded as well.

Fix a few messed up links in the ReleaseNotes

dba3333

[BOLT][NFC] Use formatv in DataAggregator/DataReader prints

d48a0e8

Reviewed By: #bolt, maksfb Differential Revision: https://reviews.llvm.org/D154120

yinying-lisa-li and others added 6 commits September 11, 2023 16:37

adds __reference_constructs_from_temporary

c38cc54

This is information that the compiler already has, and should be exposed so that the library doesn't need to reimplement the exact same functionality. Differential Revision: https://reviews.llvm.org/D135341

[libc++][ranges] Fix a split_view test accidentally using lazy_split

ed23c52

[Clang][Docs] Fix typo in clang-offload-packager documentation

ea98683

Fixed formatting of the section violating 80-char line limit.

[test][hwsasan] Invert enable_aliases check

5a851d8

For some reasons enable_aliases is not set when we LLVM_ENABLE_RUNTIMES=compiler-rt instead of LLVM_ENABLE_PROJECTS.

Merge branch 'llvm:main' into libc_vendor

59e4cef

AntonRydahl added the libc label Sep 11, 2023

AntonRydahl requested review from EthanLuisMcDonough, jhuber6 and sivachandra September 11, 2023 23:45

AntonRydahl requested review from a team as code owners September 11, 2023 23:45

llvmbot added the backend:AMDGPU label Sep 11, 2023

Merge branch 'llvm:main' into libc_vendor

b5fca7b

AntonRydahl closed this Sep 12, 2023

AntonRydahl deleted the libc_vendor branch September 12, 2023 00:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[libc][libm][GPU] Added missing vendor entry points to `libm` for GPUs #66031

[libc][libm][GPU] Added missing vendor entry points to `libm` for GPUs #66031

Uh oh!

AntonRydahl commented Sep 11, 2023

llvmbot commented Sep 11, 2023

There are a number of mathematical functions where no target-agnostic implementations exist, and the compiler built-ins are not correctly lowered on all GPU targets. This patch adds inlined wrappers for those functions to the GPU version of `libm` for AMDGPU and NVPTX targets.

llvmbot commented Sep 11, 2023

There are a number of mathematical functions where no target-agnostic implementations exist, and the compiler built-ins are not correctly lowered on all GPU targets. This patch adds inlined wrappers for those functions to the GPU version of `libm` for AMDGPU and NVPTX targets.

Labels

23 participants

[libc][libm][GPU] Added missing vendor entry points to libm for GPUs #66031

[libc][libm][GPU] Added missing vendor entry points to libm for GPUs #66031

Uh oh!

Conversation

AntonRydahl commented Sep 11, 2023

llvmbot commented Sep 11, 2023

There are a number of mathematical functions where no target-agnostic implementations exist, and the compiler built-ins are not correctly lowered on all GPU targets. This patch adds inlined wrappers for those functions to the GPU version of libm for AMDGPU and NVPTX targets.

llvmbot commented Sep 11, 2023

There are a number of mathematical functions where no target-agnostic implementations exist, and the compiler built-ins are not correctly lowered on all GPU targets. This patch adds inlined wrappers for those functions to the GPU version of libm for AMDGPU and NVPTX targets.

Labels

23 participants

[libc][libm][GPU] Added missing vendor entry points to `libm` for GPUs #66031

[libc][libm][GPU] Added missing vendor entry points to `libm` for GPUs #66031

There are a number of mathematical functions where no target-agnostic implementations exist, and the compiler built-ins are not correctly lowered on all GPU targets. This patch adds inlined wrappers for those functions to the GPU version of `libm` for AMDGPU and NVPTX targets.

There are a number of mathematical functions where no target-agnostic implementations exist, and the compiler built-ins are not correctly lowered on all GPU targets. This patch adds inlined wrappers for those functions to the GPU version of `libm` for AMDGPU and NVPTX targets.