3
$\begingroup$

I have checked documentation about CUDA usage in Mathematica and it seems that it is quite few applications of CUDA.

Are the any additional packages/functions/tutorials/documentations for CUDA in Mathematica?

I mean some "everyday" calculations:

  1. Working with large lists and calculation of Map, Table, for instance finding list of f[x] for a given list {x}. I know about listability and parallel programming but may be these functions (as Map, Table and similar) can be computed with CUDA. I mean quite "arbitrary" function, not functions from CUDAMap documentation.
  2. In documentation there is no info about zero-frequency component in CUDAFourier. Is this the first element of list as in Fourier?
  3. Can be CUDA used for numerical integration? Do any packages with relates to CUDA and numerical integration exist?
Improve this question
$\endgroup$
3
  • 4
    $\begingroup$ It's all here: reference.wolfram.com/language/CUDALink/guide/CUDALink.html For Map you have CUDAMap and likewise CUDASort, CUDAFold. $\endgroup$ Commented Jul 2, 2020 at 18:18
  • 2
    $\begingroup$ For CUDAFourier - just read the documentation - it says "The result agrees with the Wolfram Language:" right here so yes it's the same (though consider precision issues as GPU precision may be different from normal Fourier's precision ) $\endgroup$ Commented Jul 2, 2020 at 21:58
  • $\begingroup$ For 3), that's something I'd like to see Wolfram do more of. CUDA numerical root finding, minimization, integration and monte-carlo, gradients etc... would all add greatly to Mathematica but at the moment we have to write our own kernels. $\endgroup$ Commented Jul 2, 2020 at 22:20

1 Answer 1

Reset to default
2
$\begingroup$

CUDA (GPU computing) works well as long as you have many independent calculations on lists, matrices,... . Shifting data back and forth from memory CPU to GPU takes time. This spoils the calculation time vs. multi-threaded approaches. In addition, operations that generate e.g. one number out of many (summing up, determinant of matrix,...) take time as many threads wait for the others and a single thread is rather slow. For me this is the reason why GPU applications are still limited.

Memory sharing can overcome this, which is available in the CUDA language. Mathematica has built-in functions to access GPU computations, moreover you can make up your own functions in Mathematica using CUDA libraries. Once you have the framework to link Mathematica functions with CUDA functions, you can do a lot more (e.g. graph theory, ...).

References for CUDA: https://developer.nvidia.com/gpu-accelerated-libraries#deep-learning https://docs.nvidia.com/cuda/#cuda-api-references

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ I find the answer is a bit limited. I would start of with explaining the difference between CUDA and GPU computing. Then, you pointed to the difficulties that arise due to the limited speed of communication between different memories. But can you support this with small benchmark study? I would also mention cuBLAS as the case where GPU computing is really strong. community.wolfram.com/groups/-/m/t/1128397 $\endgroup$ Commented Jul 6, 2020 at 7:43

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.