Even though your computer may have an AMD video card, there are cases where Windows may have installed INTEL video drivers that can be interfering with recording with your AMD video card's H.264 encoder. It happens because Plays needs access to AMD's OpenCL runtime and on some computers that have outdated or corrupted INTEL video drivers installed (but not in use), Plays may fail with error 40001 when attempting to record a game.


To uninstall INTEL video drivers, please follow instructions on INTEL web site:


https://www.intel.com/content/www/us/en/support/articles/000005598/graphics-drivers.html 


After uninstalling INTEL video driver, reboot the computer and try to record again. If the issue persists, please try a clean installation of AMD video drivers as described here.