How to continously load synchronised videos from multiple streams?

  api, c++, mpv, python, video

I have 3 independent video streams with the following structure:

├── dash
│   ├── dash-1.mp4
│   ├── dash-2.mp4
│   ├── dash-3.mp4
│   └── dash-4.mp4
├── gopro
│   ├── gopro-1.mp4
│   ├── gopro-2.mp4
└── gps
    └── gps-1.mp4

Each stream is a list of video files that are supposed to be played one after another, i.e. continuously until the end of the list. Streams are of equal length total and files in each are in order, i.e. gopro-1.mp4 and gopro-2.mp4 played one after another have the same total length as gps-1.mp4, etc. I want to synchronise and simultaneously play all 3 streams in an overlay. The use case is to simultaneously play videos from a bike dashcam, a helmet mounted GoPro and a GPS (that last stream has an alpha channel but it doesn’t matter). The culprit is individual files in each stream are of different length. MPV API that I (up to this point) utilised in a shell script can easily overlay file gps-1.mp4 on top of gopro-1.mp4 and play them together but can’t handle loading gopro-2.mp4 while gps-1.mp4 is still playing so I need a different solution.

For reference, a shellscript (runs on both Windows and Linux providing MPV is installed) that works OK with 2 streams where both streams have the same number of videos and videos have the same length:


cd ~/Videos/Bike/gopro


for gopro in $(ls -tr); do
    gps=../gps/${gopro/gopro-/gps-} # for gopro-1 look for ../gps/gps-1 and so on
    Yres=$(ffprobe -v error -select_streams v:0 -show_entries stream=height -of csv=p=0 $gopro) # gopro video height
    Yres=$(($Yres/3)) # map will be 3rd of the gopro video height
    overlays+=" --{ $gopro --external-file=$gps --lavfi-complex=[vid2]scale=-1:$Yres[small];[vid1][small]overlay[vo] --}"

exec mpv $overlays "[email protected]"

I thought about real-time trimming of gps-1.mp4 into gps-1-1 and gps-1-2 so that I could overlay gps-1-1 on gopro-1 and gps-1-2 on gopro-2 but it seems to be a limitation of MPV API that grouping videos (an equivalent of --{ and --} in CLI) and trimming the video length (--start=$S --length=$L in CLI) can’t be used together. Therefore I’m up to switching to something like Python or C++ but I have no idea how this task can be handled in either.

P.s. There are several dedicated dashcam players that do this already but none of them works on Linux – ideally the solution would be platform independent, e.g. I used MPV because it works on many platforms, but it doesn’t have to be MPV or shellscript/C/Python. Literally anything will do. My current solution is pasted above only to help visualise the problem. A Windows-specific C++ or Python solution is fine, I’ll gladly make it work on different platforms. I just hope it won’t be in Java because personal reasons.

Source: Windows Questions C++