I have a correlation matrix of 22000 genes and for some analysis, I need to split each row of the matrix into a new file. Which means I need to create 22000 individual files.
I don't want to use the split command (because I want to get the output file as the gene_name.txt) Eg Input file
IGHD2-15 IGHD3-22 IGHD3-16 IGHD3-10 IGHD2-15 1 0.696084 0.799736 0.818788 IGHD3-22 0.696084 1 0.691419 0.67505 IGHD3-16 0.799736 0.691419 1 0.810656 IGHD3-10 0.818788 0.67505 0.810656 1 Assuming your gene names are in the first column, all you need is:
awk '{print >> $1".txt"; close(n".txt")}' matrix.txt That will print each line into a file whose name is the 1st field of that line plus a (completely optional) .txt extension. If you don't want the gene name in the file, use:
awk '{n=$1; $1="";print >> n".txt"; close(n".txt")}' matrix.txt And, if your first line is a header, use:
awk 'NR>1{print >> $1".txt"; close($1".txt")}' matrix.txt Finally, in the unlikely case where your file can contain lines whose first field isn't a simple gene name but can contain NULL or a valid path, so you need to sanitize your input, you can use:
awk 'NR > 1 && ($1 ~ /^[A-Z0-9-]+$/) { print >> $1; close($1) }' close () each file after printing... >>. What benefit would there be if I added close(n".txt") so the file would be closed each time? Actually, isn't that what awk will do anyway? How would an explicit close help? awk will crash if you exceed the OPEN_MAX system configuration value, or its internal maximum (if different). Like: awk: cannot open "/tmp/1022" for output (Too many open files) and getconf OPEN_MAX 1024 Since you didn't give and example of what you wanted each file to have in it, or what the files should be named im guessing.
This one will take the file "DATA" from your current directory, create a new file (in the same directory) named after the first column of each row, then fill that file with the data from the rest of the columns.
Meaning
IGHD2-15 1 0.696084 0.799736 0.818788 Creates a file called IGHD2-15 and puts this in it
1 0.696084 0.799736 0.818788 Script:
#!/bin/bash while read -r line; do newFileName="$(echo "$line" | awk '{print $1}')" newFileData="$(echo "$line" | awk '{$1 = ""; print $0}')" echo $newFileData > $newFileName done < DATA I tried by below method as checked it worked fine too
Here each individual line is copied to new file. file name will be first column of each line
cat data_file.txt IGHD2-15 1 0.696084 0.799736 0.818788 IGHD3-22 0.696084 1 0.691419 0.67505 IGHD3-16 0.799736 0.691419 1 0.810656 IGHD3-10 0.818788 0.67505 0.810656 1 root@praveen_linux_example dev]# j=`cat data_file.txt| wc -l` [root@praveen_linux_example dev]# for ((z=1;z<=$j;z++)); do filename=`awk -v line="$z" 'NR==line{print $1}' data_file.txt`; sed -n ''$z'p' data_file.txt >$filename.txt;done [root@praveen_linux_example dev]# 