Doing some more testing I enabled standard definition channel in the HDHomeRun.
The standard definition channel works without stuttering while the High Definition channel stutters on the non 4K Apple TV.
The 4K Apple TV still works without stuttering.