HAL system provide GPU profile functionality via NVIDIA Nsight System CLI and NVIDIA Nsight Compute CLI.

NVIDIA Nsight Systems CLI

NVIDIA Nsight Systems is a low overhead performance analysis tool designed to provide insights developers need to optimize their software. Unbiased activity data is visualized within the tool to help users investigate bottlenecks, avoid inferring false-positives, and pursue optimizations with higher probability of performance gains. Users will be able to identify issues, such as GPU starvation, unnecessary GPU synchronization, insufficient CPU parallelizing, and even unexpectedly expensive algorithms across the CPUs and GPUs of their target platform.

nsys [global-options] profile [options] <application> [application-arguments]

NVIDIA Nsight Compute CLI

NVIDIA Nsight Compute CLI (nv-nsight-cu-cli) provides a non-interactive way to profile applications from the command line. It can print the results directly on the command line or store them in a report file. It can also be used to simply launch the target application and later attach with NVIDIA Nsight Compute or another nv-nsight-cu-cli instance.

nv-nsight-cu-cli -o profile <application> [application-arguments]