I agree a toggle would be nice. We like captions for some content, not others. Live sports is an example that I don’t want captions on. But almost anything else we do. So I’d guess I turn them off and back on a few times a week on average.
One way to do this elegantly might be to add another section at the bottom of the pull down menu that is labeled as captions on/off. As soon as it is selected the captions toggle and the menu vanishes. That would mean three clicks or swipes down would immediately toggle captions and go back to the content.