I want to create a new video (finalVideo.mp4) with:

1. 3 image files image1.jpg, image2.jpg, image3.jpg, each image to be seen for 4 seconds
2. 1 video file (video1.mp4), lasts for about 30 seconds with a frame rate of 30fps
3. 1 audio file (audio1.mp3)

In the final video (finalVideo.mp4) I want the audio (audio1.mp3) to play only when the images are present and the video's (video1.mp4) audio when the video is present.

For example: The final video (finalVideo.mp4) contains image1.jpg, video1.mp4, image2.jpg and image3.jpg (in order). The audio (audio1.mp3) should play for the first 4 seconds, then the video's (video1.mp4) audio for the next 30 seconds and then the audio (audio1.mp3) for the next 8 (4*2) seconds.

Please let me know how to do this programmatically. I was hoping to figure this out using Java (JavaCV) or Python (OpenCV). But there is no programming language restriction as such, the answer can be in any language.

EDIT 1

Here is my attempt at making this work:

String ffmpeg = Loader.load(org.bytedeco.ffmpeg.ffmpeg.class);
        ProcessBuilder processBuilder = new ProcessBuilder(
                ffmpeg,
                "-loop", "1",
                "-framerate", "30",
                "-t", "4",
                "-i", "image1.jpg",
                "-i", "video1.mp4",
                "-loop", "1",
                "-framerate", "30",
                "-t", "4",
                "-i", "image2.jpg",
                "-loop", "1",
                "-framerate", "30",
                "-t", "4",
                "-i", "image3.jpg",
                "-filter_complex", "[0][1][2][3]concat=n=4:v=1:a=0",
                "finalVideo.mp4"
        );
        processBuilder.inheritIO().start().waitFor();

The above code only solves the video creation part without audio. Please let me know how to add audio to this video as stated above.

CodePudding user response：

You can do the same with the audio, use -ss and -t input options to control where to clip it.

String ffmpeg = Loader.load(org.bytedeco.ffmpeg.ffmpeg.class);
        ProcessBuilder processBuilder = new ProcessBuilder(
                ffmpeg,
                "-loop", "1",
                "-framerate", "30",
                "-t", "4",
                "-i", "image1.jpg",
                "-i", "video1.mp4",
                "-loop", "1",
                "-framerate", "30",
                "-t", "4",
                "-i", "image2.jpg",
                "-loop", "1",
                "-framerate", "30",
                "-t", "4",
                "-i", "image3.jpg",
                "-t" 4, "-i", "audio1.mp3",
                "-ss" 4, "-t", 8, "audio1.mp3",
                "-filter_complex", 
                "[0][1:v][2][[3]concat=n=4:v=1:a=0;[4][1:a][5]concat=n=3:v=0:a=1;",
                "finalVideo.mp4"
        );
        processBuilder.inheritIO().start().waitFor();

(not 100% if you can use the same input file twice, but this should work.)