Commit 58747e35 authored by Joel Hestness's avatar Joel Hestness Committed by Jonathan Hseu
Browse files

PhiloxRandom: Fix race in GPU fill function (#10298)

* PhiloxRandom: Fix race in GPU fill function

The PhiloxRandom fill kernel for the GPU had race conditions that caused the
outputs to be non-deterministic. In particular, the code previously executed
with N GPU threads (# thread contexts per GPU), but it would only advance the
fill addresses by N-1 stride in each step. This incorrect stride caused the
0th and N-1st threads to write to the same memory locations, racing for which
was last to write their common locations. Make the stride equal to the number
of threads to eliminate the race.

BONUS: By fixing this race, PhiloxRandom constant-sized GPU initializers now
match CPU initializers.

* Update random_ops_test.py to find race conditions

Increasing the size of arrays in the random_ops_test.py test to manifest
the race conditions to be resolved.
parent 2cbcda08
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please to comment