

ROCm works just fine on consumer cards for inferencing and is competetive or superior in $/Token/s and beats NVIDIA power consumption. ROCm 7.0 seems to be giving >2x uplift on consumer cards over 6.9, so that’s lovely. Haven’t tried 7 myself yet, waiting for the dust to settle, but I have no issues with image gen, text gen, image tagging, video scanning etc using containers and distroboxes on Bazzite with a 7800XT.
Bleeding edge and research tends to be CUDA, but mainstream use cases are getting ported reasonably quickly. TLDR unless you’re training or researching (unlikely on consumer cards) AMD is fine and performant, plus you get stable linux and great gaming.
Shit without a human in the loop needs a Geneva Convention clause yesterday, this will not end well.