2

I've got a parent folder with a series of 'histogram_0000_0000' folders inside it. I'm trying to make a bash script that searches for the file 'out.txt' in each folder, and returns for each time it finds the file in a folder (to check that the file exists in all folders). The script I've got is;

#!/bin/bash joblist='job_list.txt' njobs=`wc ${joblist} | awk '{print $1}'` cwd=`pwd` for ((i=1 ; i <= ${njobs} ; i++ )); do folder=`awk '(NR=='${i}'){print}' ${joblist}` echo $folder cd ${folder} if [ find -name "out.txt" ] then echo out.txt found in $folder fi cd ${cwd} done

But every time it runs I get an error;

./checkrun.sh: line 10: [: -name: binary operator expected

I've had a look around, tried using '[[' and ']]', but still don't know why I'm having any luck! Any help would be great. Thanks, -Jake

Improve this question

3 Answers 3

Reset to default
2

[ ... ] tests string or numeric expressions. If you want to see if the find command matches you need to test to see if it has provided any output (a non-empty string, tested with -n).

if [[ -n "$(find -name 'out.txt')" ]] then echo "out.txt found in $folder" fi

Notice I'm using the modern $(...) syntax to capture the output of a command instead of the deprecated backticks `...`.

Other comments,

  • You can simplify njobs=wc ${joblist} | awk '{print $1}'`` with njobs=$(wc -l <"$joblist") to get just the number of lines.

  • Pass $i into awk as a variable, folder=$(awk -v i="$i" 'NR==i {print}' "$joblist}")

  • Don't cd to a subfolder and then rely on using cd to get back to where you were. Either use relative paths to reference the subdirectory or use a subshell to keep the cd to a limited context. Here I've use the first approach

     echo "$folder" if [[ -n "$(find "$folder" -name 'out.txt')" ]] then echo "out.txt found in $folder" fi

  • If $folder is always a relative path then reference it everywhere it's used as a path as "./$folder". This protects against it beginning with a dash (-), which can confuse commands that might try to interpret it as a series of options.

  • The bash shell already has a variable for the current working directory, so you can simplify cwd=`pwd` to cwd="$PWD"

  • Use https://shellcheck.net/ to check your code. (Or install it locally if you're concerned about sharing private code.)

Improve this answer
2
  • Immaculate, thank you so much for your help roaima! Works perfectly :) Is cd considered bad form inside 'for' loops? Or is it just more efficient to use subshells etc.? Commented May 19, 2021 at 15:46
  • It's more about symlinks confusing the shell, or handling the situation where the original directory gets renamed while you're in a subdirectory. The cd "$originalDir" command at the end of a loop can fail in these situations. Commented May 19, 2021 at 15:58
1 
#!/bin/bash joblist='job_list.txt' njobs=`wc ${joblist} | awk '{print $1}'` cwd=`pwd` while IFS= read -r line do folder="$line" #`awk '(NR=='${i}'){print}' ${joblist}` echo $folder cd ${folder} if [ -f out.txt ] then echo out.txt found in $folder fi cd ${cwd} done < job_list.txt

I hope this helped. A bit easier... line=current-line-from'job_list.txt' It reads line-per-line, so the file looks like that:

folder1 folder2 folder234

my output:

france1@macubuntu:/tmp/tmdf$ bash script.sh folder1 folder2 folder234 out.txt found in folder234 france1@macubuntu:/tmp/tmdf$

hope I helped...

Improve this answer
1
  • 1
    Thanks for your reply! The for loop doesn't seem to be the issue as I've been using it to submit job scripts from the folders. The issue seems to only be with the 'if' statement! Commented May 19, 2021 at 15:38
0

I'd use zsh for that instead of bash:

#! /bin/zsh - joblist='job_list.txt' dirs=( ${(f)"$(<$joblist)"} ) for dir in $dirs; do first_out=( $dir/**/out.txt(NDY1) ) if (( $#first_out )); then print -r out.txt found in $dir fi done

With bash and other GNU find, the equivalent would be:

#! /bin/bash - joblist='job_list.txt' dirs=(); readarray -t dirs < "$joblist" for dir in "${dirs[@]}"; do [[ -n $dir ]] || continue [[ $dir = [/.]* ]] || dir=./$dir first_out=$(find "$dir" -name out.txt -print -quit) if [[ -n $first_out ]]; then printf '%s\n' "out.txt found in $dir" fi done

A few notes:

  • readarray -t lines < file can store the lines of a file into an array, but note that if file can't be opened, readarray won't even be run so $dirs won't be set, so it's important to initialise the variable beforehand.
  • bash eventually added support for zsh-style recursive globbing (**/) with the globstar option. However, it's still somewhat broken when it comes to symlink handling (with the behaviour varying with the version) and also more importantly in this instance, it still hasn't added support for glob qualifiers like the N for nullglob, D for dotglob, or Y1 to stop at the first match without even bothering sorting the list used above.
  • find would break in find "$dir" if $dir started with -. While find does support -- to mark the end of options, that doesn't help here as arguments starting with - would still be considered as predicates if not options (as would (, ) or !). That's why we prepend ./ to $dirs that are not absolute and don't already start with . (so -my-annoying-dir- becomes ./-my-annoying-dir-). BSD find supports find -f "$dir" to avoid the problem, but that's not supported by GNU find yet.
  • If you're only wanting to check for the existence of at least one out.txt, there's no point finding them all. You can stop at the first match. Hence the (GNU specific) -quit (some BSDs have -exit for that).
  • zsh's ${(f)"$(<file)"} splits on newline characters, but when unquoted, removes empty elements (and if it didn't the next unquoted $dirs below would remove them). Empty paths usually are not valid paths so should generate errors, but when concatenated with /something, they annoyingly become an irrelevant possibly valid absolute path. And when prepended with ./ become the current directory. So best is to skip them here, hence the [[ -n $dir ]] || continue in the bash variant.
Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.