Some other folks and I have been discussing issues related to the experimental audio driver here but I figured it would be worth it to make a HomePod-specific topic about both audio drivers in the appropriate sub-forum.
I'm currently running the latest TestFlight app on the 2021 Apple TV 4K with a HomePod mini stereo pair set to default audio systemwide and ARC turned on, but historically I've seen the same issues in the App Store version. I submitted some diagnostics from my Apple TV earlier today.
Default video and Default audio drivers: The audio slowly starts to get out of sync with the video as the video micro-stutters. The video delay is very noticeable after 2-3 minutes. No app freezing.
Experimental video and Default audio drivers: The video delay doesn't seem quite as bad but there are constant micro-stutters and the entire picture is visibly jittery. This is most evident on 60fps feeds with tickers on the bottom of the screen like CNBC. No app freezing.
Either video driver and Experimental audio driver: No video stuttering or delay, but app performance takes a nosedive. As the audio cache increases with time spent on a single channel, the app will freeze when you exit the video feed or change a channel. Audio from the channel being exited will also continue playing. Either action can take anywhere from 5 seconds to 30 seconds.
In all of my testing, the stats show zero dropped frames so the stuttering doesn't seem to be from the feed. I'm sticking to the experimental drivers for now but the app freezing is very frustrating. If the devs want to improve performance with HomePod/AirPlay 2 output, it seems like either the default audio driver needs to be tweaked to eliminate the video stuttering or the experimental audio driver needs to have its freezing issues sorted out. The experimental audio driver works quite well if you are switching between channels frequently, but sticking on the same feed for 5+ minutes and then switching to anything else isn't a good experience.