1

I have a file with two columns separated by one space. The second column will only have either T or F in it. I want to read the letter in the first column, eg: A has 3 lines with the same letter T in the second column, then I will keep A, however, when has mixed letter in the second column as for B and D, will be removed.

A T A T A T B T B T B F C F C F D F D T D F

I want to print if for the same 1st field there's a single value of 2nd field on all lines. Expected output:

A T A T A T C F C F

I think awk may able to do this? Thank you ahead for your help!

Improve this question
3
  • Not alway 3 rows, it is random for rows in each value, and yes they were sorted in consecutive Commented Feb 28, 2017 at 21:02
  • Why 5th and 6th lines would not printed? There is 2 B with T? Commented Feb 28, 2017 at 21:33
  • Are we allowed to use GNU datamash? Commented Feb 28, 2017 at 21:36

5 Answers 5

Reset to default
2

Here's one way with awk:

awk 'NR==FNR{if (x[$1]++){if ($2!=t){z[$1]++}} else {t=$2}; next}!($1 in z)' infile infile

This process the file twice - 1st pass it checks if there are different values of 2nd field when 1st field is the same value - if so it uses $1 as an array index then on 2nd pass it prints only if 1st field is not an index of the array.
Or, if you don't mind using sort with awk:

sort -u infile | awk 'NR==FNR{seen[$1]++;next}seen[$1]==1' - infile

sort -u removes the duplicate lines from the file and pipes the result to awk which counts the 1st field occurrences and then processes the whole file again printing the lines if the count is 1.

Improve this answer
2 
sed -e ' # this is a do-while loop which collects lines till the time the first # field remains the same. We break out of the loop when we see # a line whose 1st field != prev lines 1st field **OR** we hit the eof. :a $bb N /^\(\S\+\) .\(\n\1 .\)*$/ba :b # all equal # **Action:** Print and quit /^\(\S\+ .\)\(\n\1\)*$/q # all same 1st fld, but lines unequal, otw would have matched above # **Action:** Drop the whole block as its uninteresting /^\(\S\+\) .\(\n\1 .\)*$/d # all equal, and trailing line part of next line # **Action:** Display upto the last newline and restart # with the trailing portion /^\(\(\S\+ .\)\(\n\2\)*\)\n[^\n]*$/{ h s//\1/p g } # of same 1st fld but some lines unequal, and trailing portion has # next line # **Action:** strip till the last newline, and restart over with the # trailing part s/.*\(\n\)/\1/ D ' yourfile

This is a very interesting problem for "Sed" to solve. However, what I find lacking is a somewhat better or shall I say larger set of inputs provided by OT on SE. What I propose is that the testcases of realistic magnitude and variety can be placed on the htttp:/pastebin site which is very useful for such things.

Improve this answer
1

If you have access to GNU datamash, then you can collapse the data as follows:

datamash -W groupby 1 countunique 2 collapse 2 < file A 1 T,T,T B 2 T,T,F C 1 F,F D 2 F,T,F

That makes it simple to post-process with awk for example:

datamash -W groupby 1 countunique 2 collapse 2 < file | awk '$2==1 {n = split($3,a,","); for (i=1;i<=n;i++) print $1, a[i]}' A T A T A T C F C F
Improve this answer
1
  • I'd do it like datamash -t' ' -g1 unique 2 <infile | grep -v , | grep -Ff- infile (assuming there's no comma in the input values) Commented Feb 28, 2017 at 21:41
1

Just for the record, one more alternative with AWK without the need to double read the file . Seems to work even with unsorted file or even with entries found only once in the file. 

$ awk '( a[$1] && (($2!=a[$1]) || ($1 in bad)) ) {bad[$1]++}; \ {a[$1]=$2;f1[FNR]=$1;f2[FNR]=$2}; \ END {for (i in f1) {if (!(f1[i] in bad)) print i,f1[i],f2[i]}}' ./tmp/file12 1 A T 2 A T 3 A T 12 C F 13 C F 21 E F

In where file12 is like:

$ cat -n ./tmp/file12 1 A T 2 A T 3 A T 4 B T 5 B T 6 B F 7 B F 8 B T 9 B F 10 B F 11 B F 12 C F 13 C F 14 D F 15 D T 16 D F 17 D F 18 D F 19 D F 20 D F 21 E F

Some explanation:

( a[$1] && (($2!=a[$1]) || ($1 in bad)) ) {bad[$1]++}; # The first time value $1 is checked,a[$1] will return null/0 since a[$1] # has never been assigned. Thus a[$1] will be evaluated as false by # condition check and further condition check and record bad marking will be skipped. # The second time (second line) a[$1] will have a value and will be evaluated as true . # Then we check if $2 is equal to previous a[$1]=$2. If it is equal all ok. # Otherwise if current $2 <> previous a[$1] $2value, mark this record as bad. # Finally there is a last check if record has been already been marked as bad. {a[$1]=$2;f1[FNR]=$1;f2[FNR]=$2}; #some array delcaration END {for (i in f1) {if (!(f1[i] in bad)) print i,f1[i],f2[i]}}' ./tmp/file12 # Printing the lines of file that does not belong in bad boys.
Improve this answer
0 
sed ' /\n/D :1 $! { N /^\(\S\+\s\).*\n\1[^\n]\+$/ b1 } /^\([^\n]\+\n\)\(\1\)\+[^\n]\+$/! D h s/\n[^\n]\+$//p g s/.*\n/\n/ D ' file
Improve this answer
1
  • This fails when a value occurs only once, try adding the line G F to OP's input... Commented Mar 1, 2017 at 11:43

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.