I've been noticing this frequently. I have the latest FireTV Stick. When watching something on Channels and hitting the Home button to exit, the audio from a random video in the same channel will begin playing over the FireTV home screen.
Just to clarify, it's not that the audio from what I was watching is continuing to play after exiting. It's also not the audio from either the current, previous, or next video on the channel. It's audio from completely random file in the group of files from the last channel I was on, and it starts from the beginning. I'm not sure if it ends on its own after a bit; I've waited as long as about 10 seconds to see. But I usually just turn the TV off, figuring it'll end on its own at some point. I've never turned the TV on the next day and found it continuing to play audio.
If I re-open the Channels app, the audio stops and everything works as normal. Doesn't seem to do it again if I close it.
FireTV is up-to-date, as is the Channels DVR container and FireTV app, in case anyone wants to know.