How to change offset of EIA-608 subtitles in an MP4?

I have an MP4 file containing H.264 Video, AAC Audio, and EIA-608 Closed Captions:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'file.mp4': Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2avc1mp41 encoder : Lavf57.83.100 Duration: 00:11:26.02, start: 0.000000, bitrate: 4952 kb/s Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], Closed Captions, 4818 kb/s, 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc (default) Metadata: handler_name : VideoHandler Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default) Metadata: handler_name : SoundHandler

However, the subtitles are displaying about 2 seconds too early.

The catch is, as you can see, ffmpeg doesn't recognize eia_608 as a "stream" that can be manipulated with the -map commands (as in this question). So inputting the file multiple times to remux the subtitles back in with a delay gives the following error:

Stream map '1:s' matches no streams.
To ignore this, add a trailing '?' to the map.

And obviously -itsoffset affects the video exactly as much as the subtitles in every case, anyway.

So how do I add a slight delay to eia_608 subtitles in an MP4 file without recoding?

Know someone who can answer? Share a link to this question via email, Twitter, or Facebook.