GPUReplay (GR) is a novel way for deploying GPU-accelerated computation on
mobile and embedded devices. It addresses high complexity of a modern GPU stack
for deployment ease and security. The idea is to record GPU executions on the
full GPU stack ahead of time and replay the executions on new input at run
time. We address key challenges towards making GR feasible, sound, and
practical to use. The resultant replayer is a drop-in replacement of the
original GPU stack. It is tiny (50 KB of executable), robust (replaying long
executions without divergence), portable (running in a commodity OS, in TEE,
and baremetal), and quick to launch (speeding up startup by up to two orders of
magnitude). We show that GPUReplay works with a variety of integrated GPU
hardware, GPU APIs, ML frameworks, and 33 neural network (NN) implementations
for inference or training. The code is available at
https://github.com/bakhi/GPUReplay.

By admin