Bo-Yuan: you may need to read through NVIDIA documentation and confirm what capabilities your GPUs have. NVIDIA’s datacenter GPUs (e.g., Quadro, Tesla) have more capabilities than the consumer-focused GPUs. I understand some GPUs start in P2 mode by default. But for any GPU, you cannot manually force it to a higher Performance State. Instead, you must ensure it is properly powered and cooled. Then, you can manually set the Graphics and Memory Clocks to max speed and the GPU will maintain those speeds as much as possible.

To start, you’ll want to query the status of your GPUs to determine if they might be throttled for power and/or thermal reasons. Start by running:
nvidia-smi --query --display=TEMPERATURE,POWER,PERFORMANCE

You might see that a Thermal Slowdown is Active, which means the GPUs are not being fully cooled. Several other slowdown reasons could also be shown.

If no Clock Throttle Reasons are Active, you can double-check that the GPU clocks are set to max. Query possible clock speeds with:
nvidia-smi --query-gpu=clocks.max.memory,clocks.max.graphics --format=csv,nounits

Take the two max clock speed values from the above command and manually set to these speeds (and note that these settings reset at reboot) with:
nvidia-smi --applications-clocks=[mem clock],[graphics clock]

Hi Daniel,
We used the E5-2690v4, in a dual socket configuration. Each E5-2690v4 has fourteen Xeon cores.