1

1) I have directories from 0 to 3120203 (note directories are not in sequence they are random)

-bash-4.1$ ls 0 1261826 211205 2398339 267475 295482 339902 395546 457254 503959 538784 583032 643106 78093 906653 1000791 126359 211250 2398362 267592 295488 340070 39565 457378 504052 538846 583103 643168 78143 91594 1001022 126944 2114355 2398373 267667 29583 341088 395652 457471 504160 540470 583316 64447 781579 91777 1002557 127163 2129010 2398380 267771 2959608 341300 395786 457628 504219 540632 583875 645373 782220 921760 1004183 127316 2129165 2398388 268076 296331 341452 396207 457758 504278 541300 583998 645437 78227 924976 1004399 127416 2132965 2398396 2681923 296456 341512 39720 457820 504337 541754 584219 645816 782272 925382 1005369 130416 2137199 2398482 268333 2964801 341688 39779 457879 504404 541994 584278 645876 782704 928134

2) I want to sort and group all those directories and move it under specific range of 50000.

eg: directories from 0 to 50000 should move under 50000, 50001 to 100000 move under 100000 and so on. iterations should be of 50000

I am not expert on shell scripting:

here is my shell script:

#!/bin/bash #set -x #Please pass document library path SOURCE=/opt/dms/ DEST=/opt/nes_dms cd $SOURCE ls | sort -n >> /opt/ncm/list.txt for dir in `cat /opt/ncm/list.txt` do #echo $dir if [ "$dir" -le "50000" ] then echo $dir is less then $i mv $dir $DEST fi done

I need the script to be incremented automatically by 50000+50000=100000 and move all directories between 50001 to 100000 in 100000 dir till we read all dir in a file.

Improve this question
2
  • 1
    Do not use shreenshots; paste the output as code instead. Not all of it, just enough to get the picture. Commented Apr 8, 2018 at 17:21
  • @HaukeLaging removed screenshot and updated with the output of ls command. Commented Apr 8, 2018 at 17:33

3 Answers 3

Reset to default
1 
#!/bin/bash #set -x #Please pass document library path SOURCE='/crypto/home/hl/tmp/stackexchange/dir_groups/source' DEST='/crypto/home/hl/tmp/stackexchange/dir_groups/target' declare -a target_dir_indexes cd "$SOURCE" || exit 1 for dir in *; do test "$DEBUG" = 'yes' && echo "$dir" test -d "$dir" && ! test -L "$dir" || continue if ! [[ "$dir" =~ ^(0|[1-9][0-9]*)$ ]]; then echo "error: dir name '${dir}'; skipping" continue fi target_index=$((dir/50000)) if [ "$dir" -gt 0 ]; then if [ $((dir%50000)) -eq 0 ]; then ((target_index--)) fi fi target_dir_name=$(((target_index+1)*50000)) target_dir_path="${DEST}/${target_dir_name}" # avoid unneccessary calls to mkdir or the VFS if [ -z "${target_dir_indexes[target_index]}" ]; then mkdir -p "$target_dir_path" target_dir_indexes[target_index]=1 fi echo "${dir} is moved to ${target_dir_name}" mv -i "$dir" "$target_dir_path" done
Improve this answer
5
  • 0 dir in $SOURCE is getting skipped to be moved in $target_dir_path Commented Apr 8, 2018 at 18:38
  • @NitinMestry Oh, there actually is a 0 entry, I should have had a closer look at the list... I have adapted the regex to include 0. Commented Apr 8, 2018 at 20:06
  • @isaac I already noticed that when reading your answer and fixed it. Commented Apr 8, 2018 at 20:16
  • @NitinMestry I have made another modification because the border numbers (n*50000) were put in the wrong group. Commented Apr 8, 2018 at 20:17
  • need your help on modifying the script: With this script mail gloal is achieved to move given range of directories, now 2nd thing i want is if we rerun the script it should only move modified contents in source under destination directory. which should take less time as we are just moving only modified directories. Commented Jun 6, 2018 at 11:21
1

The math is somewhat tricky. A simple integer division (file/50000+1)*50000 will fail. If the file number is 50000 the result is 100000. Not what you want. We need a shift of origin:

$(( ((file-1)/50000+1)*50000 ))

Also, a simpler code scheme is ok:

#!/bin/bash fsource=/opt/dms; fdest=/opt/nes_dms; istep=50000 cd "$fsource" for f In *; do i=$(( 1+(f-1)/istep )) # Which bucket? fd="$fdest/$(( istep*i ))" echo \ mv -t "$fd" "$f" done

Adding a couple of checks [(is a directory?) And (only digits)]:

#!/bin/bash fsource=/opt/dms/; fdest=/opt/nes_dms; istep=50000 for f in "$fsource"/*; do [[ -d $f ]] || continue # Is a dir? i=${f##*/} # Remove path. [[ $i =~ ^[0-9]+$ ]] || continue # Only digits? i=$((1+(i-1)/istep)) # Which bucket? fd="$fdest/$(( istep*i ))" echo \ mv -t "$fd" "$f" done

Comment out the echo when you are happy with the values and you want to execute the moves.

Improve this answer
1

WARN: script bellow doesn't handle errors when directory is not numeric named.

Posible solution:

#!/bin/bash #set -x SOURCE=/opt/dms/ DEST=/opt/nes_dms for dir in `ls $SOURCE | sort -n` do # define destination subdir dst_dir=$(( (($dir/50000) + 1) * 50000 )) # check if destination subdir exists if [ ! -d $DEST/$dst_dir ] then mkdir $DEST/$dst_dir fi mv $SOURCE/$dir $DEST/$dst_dir done
Improve this answer
6
  • one correction here, to move $dir we should be inside $SOURCE dir or need to have absolute path else will get "no such file or directory" Commented Apr 8, 2018 at 21:35
  • it should be mv $SOURCE/$dir :) Commented Apr 8, 2018 at 21:43
  • Sorry, I hurried :) fixed again. Commented Apr 8, 2018 at 21:44
  • @NitinMestry May I interested why you change right answer? Answer from Hauke Laging looks very good. Commented Apr 8, 2018 at 21:47
  • to be honest its not about good looks, for me execution time and same level accuracy matters. your script take 7.05 mins which loops and perform operation on each dir. In Hauke Laging - execution time is just 4.32 mins due to use of array. I am from java and have achieved the same in java code before coming to this stack. Thanks you :) Commented Apr 8, 2018 at 22:43

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.