Well, transcoding isn't transcoding at the same frame rate as direct. Usually a little bit lower.
I just pushed a couple of commits to try and address this. I'm going to test watching in the browser now and see what that looks like.
Edit: still not quite right.
I'm testing something, and it seems like there might be a better back end for this than the one I've been using.
I might revert it, but I'm testing right now, and I'll let you guys know how it pans out. I'll be sure to test in both the browser and as well as my Apple TV.
Okay, I'm doing a bit of a refactor now to try to get this right for everyone. Fingers crossed I'm not up until 3 a.m. today. 
So...whats the best model to use on an Intel system? I'm guessing Cohere?
Hang on, I'm refactoring. Just let me cook because cohere was a little too slow.
1 Like
I spent way too much time at a refactor that ended up being crap.
I added Parakeet Unified, which is quite good actually. It's very accurate, although it's English only. It's faster than Cohere transcribe with just about the same accuracy. I really don't know what's going on with the web interface. I just have a feeling different programs display closed captions differently. Is experiencing it as well. So it's not like it's, you know, like a processor or something.
I did tweak some of the settings so maybe it got better? Hopefully you guys can test and let me know, but I stayed up way too late working on this. 
Edit
Okay, I think I got it right. I tested in a WebKit based browser called Orion because I don't have Firefox on this computer and the web player is not working with Vivaldi. Regardless, it seemed to be running at about the same speed as my Apple TV. The only thing I noticed was it's three lines sometimes instead of two, which I just chalk up to the player. I can't think of another reason for that in the source code.
1 Like
I let fable loose at this problem, and it may have found something. I will update this comment once I have something available.
Latest changes have been built as :beta4. Thanks for working so hard on this @mackid1993! It's already very cool -- well on its way to fabulous. 
1 Like
@bnhf did you get the push I just literally pushed like 5 minutes ago?
I just made a couple of changes that hopefully were intended to address the issues you rose.
It's on my fork right now if you want to try it before accepting it into the beta branch. I'm not sure if it helps the issue you were having with the captions being too fast? It was intended to.
It does now. New :beta4 pushed.
1 Like
Let me know if it helps with the issue you were having. Hopefully, it solves it. Also, I have to say I really like Parakeet Unified. it's pretty good. Quite fast as well and handles my insane amount of tuners!! I'm surprised I didn't find it sooner.
I just plopped up beta4. So far Im really diggin the Parakeet Unified, with default settings. Nice..
Only "quirk" I noticed in the ah4c CC UI, when downloading the model there was no progress bar, so the download button looked unresponsive. However after the download, the Green downloaded button appeared.
will keep testing, but lookin pretty cool!!
Initial impressions are positive! I'm watching Perry Mason with some fast talkers, and it's keeping up nicely -- while still being readable. I'm still using the Nemotron model, but otherwise I have everything set to recommended or default. This is using the Android client on an Android tablet.
I'd suggest at this point, we not worry too much about the WebUI. It works, but I believe the transcoding leads to some captions disappearing too quickly. Watching the same thing in an Android client, with no transcoding, looks great and has good pacing even at 160 wpm.
I'd imagine remote users will have the best experience without transcoding as well.
EDIT: Having said the above, I do believe things are looking better generally in the WebUI. The two lines (using the two line box), are closer together -- and it's only occasionally that a caption comes-and-goes too quickly. I did also switch to Parakeet Unified, so that may be having an impact on my impressions.
Hopefully we can get a few more testers (@Jean0987654321, @uspino and maybe one or two more?), as I'd really like to merge this into main. A little more testing, a code base that isn't quite as kinetic, and we'll be there.
It'd be good to have more testers also. I'm trying something out right now.
It's not a major deviation, don't worry. I'm nearly certain Parakeet Unified supports a streaming mode where it has much lower latency. So the caption should show up on screen closer in time to when people actually speak. This is similar to Nemotron and useful for people who have powerful hardware or low tuner count. My goal would be to just have a dropdown menu when you select that model, and you can choose if you want real time or not. With a note saying that real time uses more processing power or something like that.
I'm working through it right now. It shouldn't be too hard of a lift. I'll let you all know if it ends up working. I'm just going to update this comment here.
Update: I can confirm beyond a shadow of a doubt, Parakeet Unified works in real time, and Nemotron works sentence by sentence as well. So, both of those options are available for everyone. I will be pushing a few commits soon.
1 Like
All I have is a USB encoder so probably not much use there.
@mackid1993 Can you please change the currently hardcoded playbackStaticFor that's used for PLAYBACK_DETECTION=true into an env var? The current 2s value is way too short for the DTV app, perhaps using PLAYBACK_STATIC_TIMEOUT? With that value changeable, I believe I can make playback detection work for the app as well as for Ospreys.
For now I'd prefer if you could roll this into the current PR#20, rather than creating a separate PR. After we pull PR#20 into main, we'll move to a more sane, smaller PR approach, where we're both making changes to the same branch.
Actually the typical USB device is capture only, and does not re-encode. Maybe someday, but we're a ways from supporting one of those. That would mean the encoding and caption generating would all need to be done on your CPU and/or GPU.
Will do. I should have a roll-up for you in a little bit.
3 Likes