Swift AVAssetTrack not merging properly-CodePudding

Attempting to merge multiple videos with AVMutableComposition, the tracks are gathering correctly in the avassettrack. However, they simply overlap and only the second video is displayed. Also the length of the output is whatever the second video length is.

here is the merge func:

    public func mergeCapturedVideos(completion: @escaping (_ completedMovieURL: URL) -> Void) {
        let mixComposition = AVMutableComposition()
        let movieAssets: [AVAsset] = self.capturedMovieURLs.map({ AVAsset(url: $0) })
        var insertTime: CMTime = CMTime.zero
        if movieAssets.count == self.capturedMovieURLs.count {
            for movieAsset in movieAssets {
                do {
                    if let compositionVideoTrack: AVMutableCompositionTrack = mixComposition.addMutableTrack(withMediaType: .video, preferredTrackID: Int32(kCMPersistentTrackID_Invalid)) {
                        let tracks: [AVAssetTrack] = movieAsset.tracks(withMediaType: .video)
                        let assetTrack: AVAssetTrack = tracks[0] as AVAssetTrack
                        try compositionVideoTrack.insertTimeRange(CMTimeRange(start: .zero, duration: movieAsset.duration), of: assetTrack, at: insertTime)
                        print("1 \(insertTime)")
                        insertTime = CMTimeAdd(insertTime, movieAsset.duration)
                        print("2 \(insertTime)")
                    }
                } catch let error as NSError {
                    print("Error merging movies: \(error)")
                }
                print("MIX: \(mixComposition)")
            }
            let completeMovieURL: URL = self.capturedMovieURLs[0]
            if let exporter = AVAssetExportSession(asset: mixComposition, presetName: AVAssetExportPresetHEVC1920x1080WithAlpha) {
                exporter.outputURL = completeMovieURL
                exporter.outputFileType = .mp4
                exporter.exportAsynchronously {
                    if let url = exporter.outputURL {
                        completion(url)
                    } else if let error = exporter.error {
                        print("Merge exporter error: \(error)")
                    }
                }
            }
        }
    }

This is what the "MIX: " print statement is outputting:

MIX: <AVMutableComposition: 0x2815670a0 tracks = (
    "<AVMutableCompositionTrack: 0x281567ea0 trackID = 1, mediaType = vide, editCount = 1>",
    "<AVMutableCompositionTrack: 0x281567da0 trackID = 2, mediaType = vide, editCount = 2>"
)>

As you can see, the tracks are being added properly, so it must be a issue with the insertTime var. But that is also printing the expected output:

1 CMTime(value: 0, timescale: 1, flags: __C.CMTimeFlags(rawValue: 1), epoch: 0)
2 CMTime(value: 2121, timescale: 600, flags: __C.CMTimeFlags(rawValue: 1), epoch: 0)
1 CMTime(value: 2121, timescale: 600, flags: __C.CMTimeFlags(rawValue: 1), epoch: 0)
2 CMTime(value: 3560, timescale: 600, flags: __C.CMTimeFlags(rawValue: 1), epoch: 0)

As shown, the insertTime var is properly adding the duration of each track on every loop.

This is leaving me very puzzled, why is the output not a singular movie with each video?

CodePudding user response：

Are you sure it is the second movie displaying only and not the first ?

I believe what is happening is this:

You create a valid output url let completeMovieURL: URL = self.capturedMovieURLs[0]
You then try to do an export, but the export is failing so no new video is written to the output url
You check if the if let url = exporter.outputURL this will always be valid as you provided it the url of a valid video
To check if there were errors, you should do a AVAssetExportSession status and only on completed you should call your completion handler
Because there was an error, no new movie was written, your completion handler was called with the video self.capturedMovieURLs[0] and this is what you see

That explains the issue of why you see 1 video, but why is the merge failing, I think this is due to 2 reasons and it is not because of your time ranges:

You are creating a new AVMutableCompositionTrack for each video when actually you should reuse it and keep adding new videos to it. Initialize this outside the loop
The AVAssetExportPreset used is AVAssetExportPresetHEVC1920x1080WithAlpha - I recommend not using this unless you want to configure some instructions to support this format, better to go with AVAssetExportPresetHighestQuality if you want to keep things simple

Here are some updates I made to your existing code with my comments above and hopefully you should get the output you want

public func mergeCapturedVideos(completion: @escaping (_ completedMovieURL: URL) -> Void)
{
    // No change to your code
    let mixComposition = AVMutableComposition()
    let movieAssets: [AVAsset] = self.capturedMovieURLs.map({ AVAsset(url: $0) })
    var insertTime = CMTime.zero
    
    // Initialize a AVMutableCompositionTrack outside the loop
    guard let compositionVideoTrack
            = mixComposition.addMutableTrack(withMediaType: .video,
                                             preferredTrackID: Int32(kCMPersistentTrackID_Invalid))
    else
    {
        // error initializing video track
        return
    }
    
    
    if movieAssets.count == self.capturedMovieURLs.count
    {
        // Use the existing compositionVideoTrack in the loop, don't create a new
        // one each time
        for movieAsset in movieAssets
        {
            do {
                let tracks: [AVAssetTrack] = movieAsset.tracks(withMediaType: .video)
                let assetTrack: AVAssetTrack = tracks[0] as AVAssetTrack
                
                // Insert a new track into the existing
                try compositionVideoTrack.insertTimeRange(CMTimeRange(start: .zero,
                                                                      duration: movieAsset.duration),
                                                          of: assetTrack,
                                                          at: insertTime)
                
                print("1 \(insertTime)")
                insertTime = CMTimeAdd(insertTime, movieAsset.duration)
                print("2 \(insertTime)")
            }
            catch let error as NSError {
                print("Error merging movies: \(error)")
            }
            
            print("MIX: \(mixComposition)")
        }
        
        // Configure where the exported file will be stored
        let documentsURL = FileManager.default.urls(for: .documentDirectory,
                                                    in: .userDomainMask)[0]
        
        // Generate a file name or reuse your existing url
        // Not sure if AVAssetExportSession can overwrite
        let fileName = "\(UUID().uuidString).mp4"
        let dirPath = documentsURL.appendingPathComponent(fileName)
        let outputFileURL = dirPath
        
        // Use the preset `AVAssetExportPresetHighestQuality` if you don't want
        // to mess with additional configuration
        if let exporter = AVAssetExportSession(asset: mixComposition,
                                               presetName: AVAssetExportPresetHighestQuality)
        {
            exporter.outputURL = outputFileURL
            exporter.outputFileType = .mp4
            
            // Check if export has succeeded
            exporter.exportAsynchronously
            {
                switch exporter.status
                {
                    case .completed:
                        if let url = exporter.outputURL
                        {
                            DispatchQueue.main.async
                            {
                                completion(url)
                            }
                        }
                    default:
                        if let error = exporter.error
                        {
                            print("Merge exporter error: \(error)")
                        }
                }
            }
        }
    }
}