1

I have roughly 1 million images on a directory. The files were numbered from 1 to n. I am using for loop to iterate over each image. Since each iteration is checked by individuals, only certain number of iterations could be done in a day. When I begin the loop again the subsequent day, the loop obviously begins from the first file again.

I saved the files iterated through the loop in a text file and read the last line of the text file before the loop starts every time. I am trying to use the last read file as a beginning for the for loop.

The following is the code done so far:

query=/ImageFolder/*.jpg fil=$( tail -n 1 readfiles.txt ) for f in $query do python ~/runprog.py --query $f done

I am not sure how to use the $fil as my starting point in the for loop and the start iterating subsequent files from thereon.

Improve this question
3
  • You could, once you processed an image, move this image to a different directory, say to_be_reviewed. Commented Dec 14, 2017 at 11:04
  • @user1934428 Thank you...I tried this with a small set of images...and it worked...but since the size of images were big and also were in large numbers....I wanted to keep all the images in just one file. Commented Dec 14, 2017 at 11:18
  • You all pictures in the same file? For sure you don't. You have them in the same folder. BTW, moving the file to a different folder doesn't take longer for large pictures than for small ones, as long as both folders are in the same file system. Commented Dec 15, 2017 at 6:04

1 Answer 1

Reset to default
3

If your readfiles.txt contains all already processed files, you can use grep to look up if a certain was done or not.

After running the python script, update that file with the processed file.

for f in /ImageFolder/*.jpg; do if ! grep -q "$f" readfiles.txt; then python ~/runprog.py --query "$f" echo "$f" >> readfiles.txt fi done
Improve this answer
1
  • excellent...thank you. Works like a charm. I was struggling and stuck with using the last file. Commented Dec 14, 2017 at 9:30

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.