GPU Programming with CUDA Training Course

Introduction

• Fundamentals of CUDA.
• Comparative analysis: CUDA vs. OpenCL vs. SYCL.
• Key features and architectural overview of CUDA.
• Configuring the development environment.

Getting Started

• Initiating a new CUDA project in Visual Studio Code.
• Navigating the project structure and file organization.
• Compiling and executing the program.
• Outputting results using printf and fprintf.

CUDA API

• The role of the CUDA API within host applications.
• Querying device information and capabilities via the CUDA API.
• Managing device memory allocation and deallocation.
• Transferring data between host and device memory.
• Launching kernels and synchronizing threads using the API.
• Error handling and exception management strategies.

CUDA C/C++

• The function of CUDA C/C++ in device-side code.
• Writing GPU-executable kernels for data manipulation.
• Data types, qualifiers, operators, and expressions in CUDA C/C++.
• Built-in functions for math, atomic operations, and warp-level tasks.
• Built-in variables like threadIdx, blockIdx, and blockDim.
• Leveraging libraries such as cuBLAS, cuFFT, and cuRAND.

CUDA Memory Model

• Distinguishing between host and device memory architectures.
• Utilizing global, shared, constant, and local memory spaces.
• Working with memory objects including pointers, arrays, textures, and surfaces.
• Managing memory access modes: read-only, write-only, and read-write.
• Understanding the memory consistency model and synchronization mechanisms.

CUDA Execution Model

• Differences between host and device execution flows.
• Defining parallelism using threads, blocks, and grids.
• Thread-level functions and variables (e.g., threadIdx, blockIdx).
• Block-level synchronization (e.g., __syncthreads, __threadfence_block).
• Grid-level functions and cooperative groups (e.g., gridDim, gridSync).

Debugging

• Identifying common errors and bugs in CUDA applications.
• Using the Visual Studio Code debugger for variables, breakpoints, and call stacks.
• Debugging on Linux with CUDA-GDB.
• Detecting memory errors and leaks using CUDA-MEMCHECK.
• Debugging and analyzing Windows-based CUDA programs with NVIDIA Nsight.

Optimization

• Key factors influencing CUDA program performance.
• Improving memory throughput with coalescing techniques.
• Reducing latency through caching and prefetching.
• Optimizing access patterns using shared and local memory.
• Measuring and improving efficiency with profiling tools.