Understanding the thread, block, and grid concepts

In Chapter 4 we finally executed truly parallel CUDA programs, and we saw that a configuration is needed for launching the kernel. More than that, this launch configuration forms part of the program itself. Taking a closer look at the vector addition program, we saw that the CPU version has a for loop while the GPU version has a configuration in place of this for loop. Thus we need to discuss the concepts that determine how execution takes place. Threads, blocks, and grids work together, but let’s get to know them individually first.

Threads

A thread is the basic unit of execution in CUDA, with each thread having its own unique ID that allows it to independently access a specific portion of the data. For a given problem, we define a set of threads that can handle the requisite amount of data. Thread IDs can be one-dimensional, two-dimensional, or three-dimensional (threadIdx.x, threadIdx.y, threadIdx.z), depending on...