45

Joining multiple files using ffmpeg concat seems to result in a mismatch of the timestamps or offsets for the audio. I've tried with several videos and noticed the same problem for h.264 / MP4.

Using concat and encoding the video seems to work fine. The audio stays in sync as ffmpeg does the full conversion calculations and seems to get everything right.

However, simply concatenating the videos without any transformation or encoding results in a slowly increasing sync issue. Obviously, encoding the videos rather than simply joining them will result in a loss of information/quality so I would rather find a way around this problem.

I've tried several flags to sort out this problem that appears to be based on the timestamps. None of these seem to correct the problem though.

ffmpeg -f concat -fflags +genpts -async 1 -i segments.txt test.mov ffmpeg -auto_convert 1 -f concat -fflags +genpts -async 1 -i segments.txt -c copy test2.mov ffmpeg -f concat -i segments.txt -c copy -fflags +genpts test3.mp4 ffmpeg -f concat -fflags +genpts -async 1 -i segments.txt -copyts test4.mov ffmpeg -f concat -i segments.txt -copyts test5.mov ffmpeg -f concat -i segments.txt -copyts -c copy test6.mov ffmpeg -f concat -fflags +genpts -i segments.txt -copyts -c copy test7.mov

Note: all other questions that I could find on SO seem to "fix" the problem by simply encoding the videos over again. Not a good solution.

Update

I realized the concat wasn't the problem. The original set of clips had mis-matched timestamps. Somehow concat + encoding fixed the issue, but I don't want to re-encode the videos and loose quality each time.

ffmpeg -y -ss 00:00:02.750 -i input.MOV -c copy -t 00:00:05.880 output.MOV

Which resulted in the following data

ffprobe -v quiet -show_entries stream=start_time,duration output.MOV start_time=-0.247500 duration=6.131125 start_time=-0.257333 duration=6.155333

Since then I've tried to use -tom and -t in different places along with -af apad -c:v copy and I've still failed to get the duration to be the same.

Here is the full ffprobe output

Here is the original (red) vs the segment (green)

Detailed Sample Files

I recorded a sample video, added the commands to chop it up, then concat it. http://davidpennington.me/share/audio_sync_test_video.zip

Improve this question
4
  • 1
    Audio may have to be re-encoded but not video. You can use -video_track_timescale to change video timebase of MOV/MP4s without re-encoding. If you paste details of input files, that will be helpful. Commented Feb 15, 2016 at 19:27
  • I think it might be related to this ffmpeg ticket for mp4/aac Commented Feb 28, 2016 at 21:51
  • This comment about keyframes might be part of the problem with the audio sync issues. Commented Feb 28, 2016 at 23:05
  • sboisse suggests using ffprobe to find the nearest keyframes, then clip at that point. Commented Feb 29, 2016 at 0:10

5 Answers 5

Reset to default
29
+250

This two step process should work

Step 1 Pad out the audio in each segment

ffmpeg -i segment1.mov -af apad -c:v copy <audio encoding params> -shortest -avoid_negative_ts make_zero -fflags +genpts padded1.mov

Or

Generate segments with synced streams

ffmpeg -y -ss 00:00:02.750 -i input.MOV -c copy -t 00:00:05.880 -avoid_negative_ts make_zero -fflags +genpts segment.MOV

Step 2 Concat

ffmpeg -f concat -i segments.txt -c copy test.mov

where segments.txt consists of the names of the padded files.

Improve this answer
Sign up to request clarification or add additional context in comments.

26 Comments

What does "padding the audio" do? How does it fix the timestamps?
The audio in some or all of the segments is not equal to the video length. So the audio joints aren't at the same time as the video joints hence the async. The first step pads the audio i.e. adds indefinite period of silence at the end of each segment, but the shortest stops the operation when the video stream ends thus rendering both audio and video to be the same length (as much as possible).
I tried -af apad -c:v copy <audio encoding params> -shortest but it still didn't fix the video duration.
I don't know the codecs of the streams in the file but generally the durations will not match, since both streams are quantized i.e. for a 25 fps video, duration will be multiples of 0.04s, and for AAC audio @ 48 kHz, multiples of 0.0213s. I doubt that's the problem here. Post a ffprobe readout for the whole input and one of the segments you made (before my apad suggestion)
Your segments have negative PTS because ffmpeg is cutting segments at the keyframe before your split point but is assigning PTS 0 to your split point so frames before have negative PTS. So my edited command remedies that. However, there's a hitch. The amount of audio before the split point is not equal to the video before, so there will still be some silence at the joints. sboisse's method may be the safest.
|
17

I encountered a similar problem and found a solution that worked, at least for me. In my case, I was also concatenating files, and found audio/video sync problems with iOs, but not with Windows (e.g., VLC media player showed no synchronization problems using the same mp4 file). The symptom for iOs playing this concatenated mp4 was initially good synchronization followed by an increasing loss of synchronization as the movie played, with audio going faster than video. Interestingly, the sync could be restored temporarily by advancing the movie progress slider to any point in the movie, but then the sync would be lost again as the movie continued to play in iOs. By playing the same movie simultaneously in both iOs and Windows VLC, and initially synchronized with each other as well as I could, by observing the evolution of the "echo" between them, I concluded that the iOs audio was going too fast (assuming the Windows player is correct).

For me, the solution was to add the audio filter option -af aresample=async=1000 to the ffmpeg command, which I found as an example in the ffmpeg online documentation and used verbatim. I don't know if this setting is optimal, but the result was a mp4 with audio and video remaining synchronized when played by both iOs and VLC. This ffmpeg option yielded proper iOs synchronization both during concatenation and afterwards when re-encoding the already concatenated file.

Improve this answer

3 Comments

It's the only solution that worked for me. It requires to reencode the audio tho (error: Filtergraph 'aresample=async=1000' was defined for audio output stream 0:1 but codec copy was selected. Filtering and streamcopy cannot be used together) so I had to change the flag -c copy to -c:v copy.
I use this argument with the concat filter and it works. And I was like But why?
This solves the issue of async indeed, besides is a way to sync video on top of audio? it may require re-encoding of video track but thats okay with me.
3

I have been struggling with this one for quite some time as well. Particularly when working with Panasonic AVCHD-generated MTS files. My current solution is to concatenate them on the OS level not ffmpeg. I do this on windows and it looks something like this:

COPY /b input_1.mts + input_2.mts + input_3.mts output.mts

On linux it should be something like:

$ cat input_1.mts input_2.mts input_3.mts > output.mts

You can look up documentation for the windows and linux binary concatenation.

This method of concatenation as apposed to transcoding is the way to go if the original format will work for you. This method practically uses no CPU processing and preserves the original quality. A win-win when dealing with bulk media of high quality.

Improve this answer

3 Comments

I would think this would fail with formats that have meta-data as the first X bytes of the file. Maybe not with all the safeguards built into media handling (like reading until the end of file regardless of what the stream data says?)
This is a valid concern and should be taken into consideration when concatenating files on a binary level.
TS is a streaming container. This won't work with MP4 or MOVs..etc
3

you can use filter_complex to concat different options in one go

ffmpeg -i input1.mp4 -i input2.webm \ -filter_complex "[0:v:0] [0:a:0] [1:v:0] [1:a:0] concat=n=2:v=1:a=1 [v] [a]" \ -map "[v]" -map "[a]" <encoding options> output.mkv
Improve this answer

1 Comment

Your command users a filter, and therefore will re-encode, but Xeoncross wants to avoid that.
1

If the input videos have the same video format, audio format, dimensions, etc., you can use mkvmerge from mkvtoolnix to concatenate the videos without re-encoding:

mkvmerge -o output.mkv file1.mkv + file2.mkv + file3.mkv

mkvmerge also accepts input files with an MP4 container, but the output file will have an MKV container even if you try to specify the filename extension of the output file as .mp4. You can change the container with ffmpeg:

mkvmerge -o output.mkv file1.mp4 + file2.mp4 + file3.mp4 ffmpeg -i output.mkv -c copy output.mp4

I needed to concatenate videos from different sources that had been encoded with different settings, so I first used a command like this to resize and re-encode the input videos:

for f in *.mp4;do w=1280;h=720;ffmpeg -i $f -filter:v "scale=iw*min($w/iw\,$h/ih):ih*min($w/iw\,$h/ih),pad=$w:$h:($w-iw*min($w/iw\,$h/ih))/2:($h-ih*min($w/iw\,$h/ih))/2" -c:v libx264 -crf 22 -preset slow -pix_fmt yuv420p -c:a aac -q:a 1 -ac 2 -ar 44100 ${f%mp4}mkv;done

Some of my input videos didn't have an audio channel, so I used a command like this to add a silent audio channel to the videos:

for f in *.mkv;do ffprobe $f|&grep -q 1:\ Audio||{ ffmpeg -i $f -f lavfi -i anullsrc -c:a aac -shortest -c:v copy temp-$f;mv temp-$f $f;};done

I then concatenated the videos using mkvmerge:

mkvmerge -o output.mkv `printf %s\\n *.mkv|sed '1!s/^/+ /'`
Improve this answer

4 Comments

OP asks about ffmpeg. What's the point to offer other tool? Like you ask something about C# and someone answers about qBasic...
After spending hours trying every FFmpeg suggestion/fix i could fine i gave up and tired this. It worked first time. Thanks a lot i wish i had tired it first.
@AlexSham They have basically the same licensing, so it doesn't matter at all which one you're using. There is no particular advantage to using one tool for two separate purposes, in fact the entire Unix philosophy is about doing the opposite of that. Anyway, ffmpeg has seen fit to make this complicated enough that using two tools is a really, really good idea. I love ffmpeg but this has actually gotten harder over time -- it used to be a lot easier to do with ffmpeg.
When ffmpeg over and over started failing me I immediately reached for mkvmerge, but trying to process the file further afterwards yielded the same issues. MP4Box instead of mkvmerge & ffmpeg was what worked for me.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.