32

I've looked at a number of questions but still can't quite figure this out. I'm using PyQt, and am hoping to run ffmpeg -i file.mp4 file.avi and get the output as it streams so I can create a progress bar.

I've looked at these questions: Can ffmpeg show a progress bar? catching stdout in realtime from subprocess

I'm able to see the output of a rsync command, using this code:

import subprocess, time, os, sys cmd = "rsync -vaz -P source/ dest/" p, line = True, 'start' p = subprocess.Popen(cmd, shell=True, bufsize=64, stdin=subprocess.PIPE, stderr=subprocess.PIPE, stdout=subprocess.PIPE) for line in p.stdout: print("OUTPUT>>> " + str(line.rstrip())) p.stdout.flush()

But when I change the command to ffmpeg -i file.mp4 file.avi I receive no output. I'm guessing this has something to do with stdout / output buffering, but I'm stuck as to how to read the line that looks like 

frame= 51 fps= 27 q=31.0 Lsize= 769kB time=2.04 bitrate=3092.8kbits/s

Which I could use to figure out progress.

Can someone show me an example of how to get this info from ffmpeg into python, with or without the use of PyQt (if possible)

EDIT: I ended up going with jlp's solution, my code looked like this:

#!/usr/bin/python import pexpect cmd = 'ffmpeg -i file.MTS file.avi' thread = pexpect.spawn(cmd) print "started %s" % cmd cpl = thread.compile_pattern_list([ pexpect.EOF, "frame= *\d+", '(.+)' ]) while True: i = thread.expect_list(cpl, timeout=None) if i == 0: # EOF print "the sub process exited" break elif i == 1: frame_number = thread.match.group(0) print frame_number thread.close elif i == 2: #unknown_line = thread.match.group(0) #print unknown_line pass

Which gives this output:

started ffmpeg -i file.MTS file.avi frame= 13 frame= 31 frame= 48 frame= 64 frame= 80 frame= 97 frame= 115 frame= 133 frame= 152 frame= 170 frame= 188 frame= 205 frame= 220 frame= 226 the sub process exited

Perfect!

Improve this question
4
  • 1
    your code in the edit doesn't look right (and doesn't work for me)... I don't think you want to catch a wildcard pattern and do nothing (you only need to catch patterns you care about) and more importantly - you want the thread.close to be outside the while loop rather than called the first time you catch your pattern of interest. @jlp 's code seems more correct and works for me once adapted to ffmpeg output. Commented Jul 26, 2012 at 8:59
  • In case of Python3, it should be: frame_number = thread.match.group(0).decode('utf-8') Commented Apr 19, 2019 at 3:16
  • For an error catching you should put after while: thread.close() if thread.exitstatus: print(thread.before) else: print('Ok') Commented May 21, 2020 at 16:58
  • What does this part '(.+)' do in the code? Also, the program I am working with, I need to detect failures in output, is there a way to do multiple patterns? Thanks. Commented Dec 6, 2020 at 18:24

9 Answers 9

Reset to default
23

In this specific case for capturing ffmpeg's status output (which goes to STDERR), this SO question solved it for me: FFMPEG and Pythons subprocess

The trick is to add universal_newlines=True to the subprocess.Popen() call, because ffmpeg's output is in fact unbuffered but comes with newline-characters.

cmd = "ffmpeg -i in.mp4 -y out.avi" process = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT,universal_newlines=True) for line in process.stdout: print(line)

Also note that in this code sample the STDERR status output is directly redirected to subprocess.STDOUT

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

Does not work for me, at least on python 3.9.0b5, ffmpeg 4.3-2, WSL2 Ubuntu
18

The only way I've found to get dynamic feedback/output from a child process is to use something like pexpect:

#! /usr/bin/python import pexpect cmd = "foo.sh" thread = pexpect.spawn(cmd) print "started %s" % cmd cpl = thread.compile_pattern_list([pexpect.EOF, 'waited (\d+)']) while True: i = thread.expect_list(cpl, timeout=None) if i == 0: # EOF print "the sub process exited" break elif i == 1: waited_time = thread.match.group(1) print "the sub process waited %d seconds" % int(waited_time) thread.close()

the called sub process foo.sh just waits a random amount of time between 10 and 20 seconds, here's the code for it:

#! /bin/sh n=5 while [ $n -gt 0 ]; do ns=`date +%N` p=`expr $ns % 10 + 10` sleep $p echo waited $p n=`expr $n - 1` done

You'll want to use some regular expression that matches the output you're getting from ffmpeg and does some kind of calculation on it to show the progress bar, but this will at least get you the unbuffered output from ffmpeg.

Improve this answer

5 Comments

Exactly what I wanted, thanks. I had seen pexpect but hadn't figured out how to use it, your example demonstrated it very clearly. Sorry I'm a newbie and have no points to vote up your answer!
No sweat. I'm a newbie, too. Someone else will vote it up at some point. Glad it helped.
I believe you are right, @Anentropic, edited my post to reflect the correct calling convention.
@jlp cool thanks your answer helped me get started with pexpect
In my case the python script, following line was not working. thread = pexpect.spawn(cmd) I altered the previous line cmd = "foo.sh" to cmd = "./foo.sh" hope it will help somebody :)
3
  1. Calling from the shell is generally not required.
  2. I know from experince that part of the ffmpeg output comes on stderr, not stdout.

If all you want to do is print the output line, like in your example above, then simply this will do:

import subprocess cmd = 'ffmpeg -i file.mp4 file.avi' args = cmd.split() p = subprocess.Popen(args)

Note that the line of ffmpeg chat is terminated with \r, so it will overwrite in the same line! I think this means you can't iterate over the lines in p.stderr, as you do with your rsync example. To build your own progress bar, then, you may need to handle the reading yourself, this should get you started:

p = subprocess.Popen(args, stderr=subprocess.PIPE) while True: chatter = p.stderr.read(1024) print("OUTPUT>>> " + chatter.rstrip())
Improve this answer

Comments

3

I wrote a dedicated package that gives you a generator function for ffmpeg progress in Python: ffmpeg-progress-yield.

Simply run:

pip3 install ffmpeg-progress-yield

Then, simply do:

from ffmpeg_progress_yield import FfmpegProgress cmd = [ "ffmpeg", "-i", "test/test.mp4", "-c:v", "libx264", "-vf", "scale=1920x1080", "-preset", "fast", "-f", "null", "/dev/null", ] ff = FfmpegProgress(cmd) for progress in ff.run_command_with_progress(): print(f"{progress}/100")

Note that this only works for input files where the duration is known in advance.

Improve this answer

1 Comment

Thanks, and now I can make a pyside6 progressbar work with a ffmpeg process...
2

This answers didn't worked for me :/ Here is the way I did it.

Its from my project KoalaBeatzHunter.

Enjoy!

def convertMp4ToMp3(mp4f, mp3f, odir, kbps, callback=None, efsize=None): """ mp4f: mp4 file mp3f: mp3 file odir: output directory kbps: quality in kbps, ex: 320000 callback: callback() to recieve progress efsize: estimated file size, if there is will callback() with % Important: communicate() blocks until the child process returns, so the rest of the lines in your loop will only get executed after the child process has finished running. Reading from stderr will block too, unless you read character by character like here. """ cmdf = "ffmpeg -i "+ odir+mp4f +" -f mp3 -ab "+ str(kbps) +" -vn "+ odir+mp3f lineAfterCarriage = '' print deleteFile(odir + mp3f) child = subprocess.Popen(cmdf, shell=True, stderr=subprocess.PIPE) while True: char = child.stderr.read(1) if char == '' and child.poll() != None: break if char != '': # simple print to console # sys.stdout.write(char) # sys.stdout.flush() lineAfterCarriage += char if char == '\r': if callback: size = int(extractFFmpegFileSize(lineAfterCarriage)[0]) # kb to bytes size *= 1024 if efsize: callback(size, efsize) lineAfterCarriage = ''

Next, you need 3 more functions to implement it.

def executeShellCommand(cmd): p = Popen(cmd , shell=True, stdout=PIPE, stderr=PIPE) out, err = p.communicate() return out.rstrip(), err.rstrip(), p.returncode def getFFmpegFileDurationInSeconds(filename): cmd = "ffmpeg -i "+ filename +" 2>&1 | grep 'Duration' | cut -d ' ' -f 4 | sed s/,//" time = executeShellCommand(cmd)[0] h = int(time[0:2]) m = int(time[3:5]) s = int(time[6:8]) ms = int(time[9:11]) ts = (h * 60 * 60) + (m * 60) + s + (ms/60) return ts def estimateFFmpegMp4toMp3NewFileSizeInBytes(duration, kbps): """ * Very close but not exact. duration: current file duration in seconds kbps: quality in kbps, ex: 320000 Ex: estim.: 12,200,000 real: 12,215,118 """ return ((kbps * duration) / 8)

And finally you do:

# get new mp3 estimated size secs = utls.getFFmpegFileDurationInSeconds(filename) efsize = utls.estimateFFmpegMp4toMp3NewFileSizeInBytes(secs, 320000) print efsize utls.convertMp4ToMp3("AwesomeKoalaBeat.mp4", "AwesomeKoalaBeat.mp3", "../../tmp/", 320000, utls.callbackPrint, efsize)

Hope this will help!

Improve this answer

Comments

1

You can also do it pretty clearly with PyQt4's QProcess (as asked in the original question) by connecting a slot from the QProcess to a QTextEdit or whatever. I'm still pretty new to python and pyqt but here's how I just managed to do it:

import sys from PyQt4 import QtCore, QtGui class ffmpegBatch(QtGui.QWidget): def __init__(self): super(ffmpegBatch, self).__init__() self.initUI() def initUI(self): layout = QtGui.QVBoxLayout() self.edit = QtGui.QTextEdit() self.edit.setGeometry(300, 300, 300, 300) run = QtGui.QPushButton("Run process") layout.addWidget(self.edit) layout.addWidget(run) self.setLayout(layout) run.clicked.connect(self.run) def run(self): # your commandline whatnot here, I just used this for demonstration cmd = "systeminfo" proc = QtCore.QProcess(self) proc.setProcessChannelMode(proc.MergedChannels) proc.start(cmd) proc.readyReadStandardOutput.connect(lambda: self.readStdOutput(proc)) def readStdOutput(self, proc): self.edit.append(QtCore.QString(proc.readAllStandardOutput())) def main(): app = QtGui.QApplication(sys.argv) ex = ffmpegBatch() ex.show() sys.exit(app.exec_()) if __name__ == '__main__': main()
Improve this answer

Comments

1

There's an example on the ffmpeg-python github -- doesn't work out of the box for windows machines machines (uses unix sockets)... https://github.com/kkroening/ffmpeg-python/blob/master/examples/show_progress.py

A simple, (IMO, better... :)) video converter script w/ progress (ffmpeg-python and tqdm_rich) here: https://gist.github.com/pbouill/fddf767221b47f83c97d7813c03569c4

example video encoding/conversion with progress

No need to overwrite sys.stdout/stderr then revert like some of the tqdm documentation suggests for grabbing subprocess outputs/intercepting console outputs (https://github.com/tqdm/tqdm/blob/master/examples/redirect_print.py).

Just create a video converter instance by supplying an input file path:

vc = VideoConverter(input_path=pathlib.Path('./path/to/my/file')) vc.convert()

If no output path is supplied, output video will be placed in the same dir as the input file with "-converted.mp4" appended. You can adjust the output/conversion kwargs as well... by default it's using:

class VideoConverter: FF_OUTPUT_KWARGS = { 'vcodec': 'libx265', 'crf': 35 } ...

e.g.

vc = VideoConverter( input_path=pathlib.Path('./path/to/my/input_file.mp4'), output_path=pathlib.Path('./path/to/my/output_file.mp4')), output_kwargs={ 'crf': 30, 'vcodec': 'libx264' } ) vc.convert()

You can video-time based progress report, or frame-based progress report by supplying a progress metric choice when calling "convert":

class VideoConverter: ... class ProgressMetric(Enum): FRAMES = auto() TIME = auto() ... vc.convert(self, progress_metric = VideoConverter.ProgressMetric.TIME)

You can convert everything in the script directory by simply calling:

VideoConverter.convert_all()

(can also specify input dir, output dir, output_kwargs as keyword arguments to this function)

Hope this helps!

Improve this answer

Comments

0

If you have the duration (Which you can also get from the FFMPEG output) you can calculate the progress by reading the elapsed time (time) output when encoding.

A simple example: 

 pipe = subprocess.Popen( cmd, stderr=subprocess.PIPE, close_fds=True ) fcntl.fcntl( pipe.stderr.fileno(), fcntl.F_SETFL, fcntl.fcntl(pipe.stderr.fileno(), fcntl.F_GETFL) | os.O_NONBLOCK, ) while True: readx = select.select([pipe.stderr.fileno()], [], [])[0] if readx: chunk = pipe.stderr.read() if not chunk: break result = re.search(r'\stime=(?P<time>\S+) ', chunk) elapsed_time = float(result.groupdict()['time']) # Assuming you have the duration in seconds progress = (elapsed_time / duration) * 100 # Do something with progress here callback(progress) time.sleep(10)
Improve this answer

1 Comment

Select will not work on files not created from WinSock on windows unfortunately :(
0

Looks to me that the solution is actually simple by using the -progress url (global) option of ffmpeg:

ffmpeg -progress progress.log -i file.mp4 file.avi

The file progress.log will be appended with progress information in real time (The update period is set using -stats_period option)

The information will look like this:

frame=584 fps=52.40 stream_0_0_q=0.0 bitrate=N/A total_size=762 out_time_us=0 out_time_ms=0 out_time=00:00:00.000000 dup_frames=0 drop_frames=0 speed= 0x progress=continue

From this information it is possible to calculate the progress.

A more detailed example in Programster's Blog

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.