8

I've got something fairly simple I want to do. I want to use montage on a directory that contains thousands of images, with very few options, namely:

me@home$ montage -size 256x256 DSC01*.JPG.svg output.png

...but that's just not good enough, as it only grabs about 100 images at a time; neither is

me@home$ montage -size 256x256 *.svg output.png

...which grabs all of the images at the same time, as the resulting file is too big to parse.

What I want to do is to iterate over something like 100-200 files at a time. I guess this could be implemented using a for loop (?), but I'm just a bit confused about how to do that. I guess there's probably a clever way to use find -exec or xargs that I'm not thinking of. I'm using bash, but I use zsh occasionally.

So, in conclusion, I'm looking for a one liner that, given 2600 image files, calls montage about 13 or 26 times (once for each 100-200 files), and given n files, can be called a multiple of n times.

Improve this question
1
  • 1
    Are your files all named DSC0100.JPG.svg ... DSC2600.JPG.svg? Commented Jul 12, 2012 at 15:46

4 Answers 4

Reset to default
6

A bash method, using special array features; probably translatable to zsh with some modification:

image_files=(*.svg) # use your own glob expression n=200 # number of files per command line; adjust to taste for ((i=0; i < ${#image_files[@]}; i+=n)); do montage -size 256x256 "${image_files[@]:i:n}" output-"$i".png done
Improve this answer
1
  • 1
    i found this bit of bash scripting is also very extensible. i just used it to move some files around (16 files per directory) and it worked on the first try, which was a bit of a surprise. thank you. Commented Aug 13, 2012 at 0:28
5

You can use xargs for that; unfortunately, it is not possibly to combine -I (for instering into the middle of a command line) and -L (for limiting the number of files for a single call to the executable). Therefore, I created this command line as an example (but beware of special characters in file names, they are not supported):

 ls . | \ xargs -n 100 echo | \ (a=1; while read args; do echo montage -size 256x256 $args output-$a.png; a=$((a+1)); done )

Remove the echo if you want to really execute the command.

Caveats:

  • filenames may not contain spaces or other special characters
  • the last montage line might have less than 100 files in it

Update:

This is the corresponding for loop, which (I hope) solves the problem with spaces in file names:

a=0 b=0 lst= for f in *; do a=$((a+1)) lst="$lst '$f'" if test $a -ge 100; then eval echo montage --args $lst target-$b.png b=$((b+1)) a=0 lst= fi done

Update 2: A python solution, which should be immune to special characters in file names

#!/usr/bin/env python # iterate.py """Usage: %prog <number per call> <file pattern> <command prefix> -- <command postfix> e.g. %prog 100 "DSC01*.jpg.svg" montage -size 256x256 -- output-%i.png """ import sys,subprocess,glob,os if len(sys.argv) < 5: print __doc__.replace("%prog", os.path.basename(sys.argv[0])) sys.exit(1) def chunks(l, n): for i in xrange(0, len(l), n): yield l[i:i+n] num, pattern, args = int(sys.argv[1]), sys.argv[2], sys.argv[3:] files, idx = glob.glob(pattern), args.index("--") before, after = args[0:idx], args[idx+1:] for idx,chunk in enumerate(chunks(files,num)): subprocess.call( before + chunk + [s.replace("%i",str(idx)) for s in after] )
Improve this answer
4
  • 2
    If you are going to recommend using ls in a pipe to parse its output, you should also warn about the many dangers of doing so prominently, and at the beginning to make sure people see it. Commented Jul 12, 2012 at 15:49
  • @jw013 +1 Yes, that's definitely a concern. However, his posting let me assume that he was using photos directly imported from a digital camera, which do not contain any special characters. How would you suggest to tackle that problem? Commented Jul 12, 2012 at 18:00
  • Yes, it looks like the file names are relatively benign (hence no downvote). However the OP hasn't really specified what they look like beyond *.svg (which is why I posted a comment on the question asking). In the most general case where you need to handle all file names, you'd have to resort to shell globbing and arrays or find -print0 | xargs -0 constructs. See my answer for an example of the former. Commented Jul 12, 2012 at 19:35
  • @jw013 Your answer is really nice! I never took the effort to learn how arrays work in bash. Maybe I should. Commented Jul 13, 2012 at 7:58
3

With GNU Parallel you can do:

parallel -N200 montage -size 256x256 {} output{#}.png ::: *.svg

It is of course safe for files with special characters (as you can normally expect of GNU Parallel).

Minimal installation

If you just need parallel and do not have 'make' installed (maybe the system is old or Microsoft Windows):

wget http://git.savannah.gnu.org/cgit/parallel.git/plain/src/parallel chmod 755 parallel cp parallel sem mv parallel sem dir-in-your-$PATH/bin/

Watch the intro video for a quick introduction: https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1 or at http://tinyogg.com/watch/TORaR/ and http://tinyogg.com/watch/hfxKj/

Improve this answer
2

Here's a version using xargs that's safe for any filename, but requires a temporary file to store the count. Adjust the '-n 100' to adjust how many files per montage. You can also swap the "printf" with a "find -print0", but make sure it doesn't find "count.temp".

echo 1 >count.temp printf "%s\0" *.svg | xargs -0 -n 100 sh -c ' a=`cat count.temp` montage --blah "$@" output-"$a".png let a=a+1 echo "$a" >count.temp ' rm count.temp
Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.