Return to Answer

Rollback to Revision 5 - Edit approval overridden by post owner or moderator

edited May 29, 2017 at 8:27

59.7k
53
224
298

Here is an AWK one liner.

$ PATH=$(printf %s "$PATH" \ | awk -vRS=: -vORS= '!a[$0]++ {if (NR>1) printf(":"); printf("%s", $0) }' )

where:

printf %s "$PATH" prints the content of $PATH without a trailing newline
RS=: changes the input record delimiter character (default is newline)
ORS= changes the output record delimiter to the empty string
a the name of an implicitly created array
$0 references the current record
a[$0] is a associative array dereference
++ is the post-increment operator
!a[$0]++ guards the right hand side, i.e. it makes sure that the current record is only printed, if it wasn't printed before
NR the current record number, starting with 1

That means that AWK is used to split the PATH content along the : delimiter characters and to filter out duplicate entries without modifying the order.

Since AWK associative arrays are implemented as hash tables the runtime is linear (i.e. in O(n)).

Note that we don't need look for quoted : characters because shells don't provide quoting to support directories with : in its name in the PATH variable.

awkAwk + paste

The above can be simplified with pastepaste:

$ PATH=$(printf %s "$PATH" | awk -v RS=vRS=: '!a[$0]++' | paste -s -d: -)

The paste command is used to intersperse the awk output with colons. This simplifies the awk action to printing (which is the default action).

Python

The same as Python two-liner:

$ PATH=$(python3 -c 'import os; from collections import OrderedDict; \ l=os.environ["PATH"].split(":"); print(":".join(OrderedDict.fromkeys(l)))' )

Here is an AWK one liner.

$ PATH=$(printf %s "$PATH" \ | awk -vRS=: -vORS= '!a[$0]++ {if (NR>1) printf(":"); printf("%s", $0) }' )

where:

printf %s "$PATH" prints the content of $PATH without a trailing newline
RS=: changes the input record delimiter character (default is newline)
ORS= changes the output record delimiter to the empty string
a the name of an implicitly created array
$0 references the current record
a[$0] is a associative array dereference
++ is the post-increment operator
!a[$0]++ guards the right hand side, i.e. it makes sure that the current record is only printed, if it wasn't printed before
NR the current record number, starting with 1

That means that AWK is used to split the PATH content along the : delimiter characters and to filter out duplicate entries without modifying the order.

Since AWK associative arrays are implemented as hash tables the runtime is linear (i.e. in O(n)).

Note that we don't need look for quoted : characters because shells don't provide quoting to support directories with : in its name in the PATH variable.

awk + paste

The above can be simplified with paste:

$ PATH=$(printf %s "$PATH" | awk -v RS=: '!a[$0]++' | paste -s -d: -)

The paste command is used to intersperse the awk output with colons. This simplifies the awk action to printing (which is the default action).

Python

The same as Python two-liner:

$ PATH=$(python3 -c 'import os; from collections import OrderedDict; \ l=os.environ["PATH"].split(":"); print(":".join(OrderedDict.fromkeys(l)))' )

Here is an AWK one liner.

$ PATH=$(printf %s "$PATH" \ | awk -vRS=: -vORS= '!a[$0]++ {if (NR>1) printf(":"); printf("%s", $0) }' )

where:

printf %s "$PATH" prints the content of $PATH without a trailing newline
RS=: changes the input record delimiter character (default is newline)
ORS= changes the output record delimiter to the empty string
a the name of an implicitly created array
$0 references the current record
a[$0] is a associative array dereference
++ is the post-increment operator
!a[$0]++ guards the right hand side, i.e. it makes sure that the current record is only printed, if it wasn't printed before
NR the current record number, starting with 1

That means that AWK is used to split the PATH content along the : delimiter characters and to filter out duplicate entries without modifying the order.

Since AWK associative arrays are implemented as hash tables the runtime is linear (i.e. in O(n)).

Note that we don't need look for quoted : characters because shells don't provide quoting to support directories with : in its name in the PATH variable.

Awk + paste

The above can be simplified with paste:

$ PATH=$(printf %s "$PATH" | awk -vRS=: '!a[$0]++' | paste -s -d:)

The paste command is used to intersperse the awk output with colons. This simplifies the awk action to printing (which is the default action).

Python

The same as Python two-liner:

$ PATH=$(python3 -c 'import os; from collections import OrderedDict; \ l=os.environ["PATH"].split(":"); print(":".join(OrderedDict.fromkeys(l)))' )

Corrected the `awk + paste` solution.

Source Link

edit approved May 28, 2017 at 22:20

codeforester

Here is an AWK one liner.

$ PATH=$(printf %s "$PATH" \ | awk -vRS=: -vORS= '!a[$0]++ {if (NR>1) printf(":"); printf("%s", $0) }' )

where:

printf %s "$PATH" prints the content of $PATH without a trailing newline
RS=: changes the input record delimiter character (default is newline)
ORS= changes the output record delimiter to the empty string
a the name of an implicitly created array
$0 references the current record
a[$0] is a associative array dereference
++ is the post-increment operator
!a[$0]++ guards the right hand side, i.e. it makes sure that the current record is only printed, if it wasn't printed before
NR the current record number, starting with 1

That means that AWK is used to split the PATH content along the : delimiter characters and to filter out duplicate entries without modifying the order.

Since AWK associative arrays are implemented as hash tables the runtime is linear (i.e. in O(n)).

Note that we don't need look for quoted : characters because shells don't provide quoting to support directories with : in its name in the PATH variable.

Awkawk + paste

The above can be simplified with pastepaste:

$ PATH=$(printf %s "$PATH" | awk -vRS=v RS=: '!a[$0]++' | paste -s -d: -)

The paste command is used to intersperse the awk output with colons. This simplifies the awk action to printing (which is the default action).

Python

The same as Python two-liner:

$ PATH=$(python3 -c 'import os; from collections import OrderedDict; \ l=os.environ["PATH"].split(":"); print(":".join(OrderedDict.fromkeys(l)))' )

Here is an AWK one liner.

$ PATH=$(printf %s "$PATH" \ | awk -vRS=: -vORS= '!a[$0]++ {if (NR>1) printf(":"); printf("%s", $0) }' )

where:

printf %s "$PATH" prints the content of $PATH without a trailing newline
RS=: changes the input record delimiter character (default is newline)
ORS= changes the output record delimiter to the empty string
a the name of an implicitly created array
$0 references the current record
a[$0] is a associative array dereference
++ is the post-increment operator
!a[$0]++ guards the right hand side, i.e. it makes sure that the current record is only printed, if it wasn't printed before
NR the current record number, starting with 1

That means that AWK is used to split the PATH content along the : delimiter characters and to filter out duplicate entries without modifying the order.

Since AWK associative arrays are implemented as hash tables the runtime is linear (i.e. in O(n)).

Note that we don't need look for quoted : characters because shells don't provide quoting to support directories with : in its name in the PATH variable.

Awk + paste

The above can be simplified with paste:

$ PATH=$(printf %s "$PATH" | awk -vRS=: '!a[$0]++' | paste -s -d:)

The paste command is used to intersperse the awk output with colons. This simplifies the awk action to printing (which is the default action).

Python

The same as Python two-liner:

$ PATH=$(python3 -c 'import os; from collections import OrderedDict; \ l=os.environ["PATH"].split(":"); print(":".join(OrderedDict.fromkeys(l)))' )

Here is an AWK one liner.

$ PATH=$(printf %s "$PATH" \ | awk -vRS=: -vORS= '!a[$0]++ {if (NR>1) printf(":"); printf("%s", $0) }' )

where:

printf %s "$PATH" prints the content of $PATH without a trailing newline
RS=: changes the input record delimiter character (default is newline)
ORS= changes the output record delimiter to the empty string
a the name of an implicitly created array
$0 references the current record
a[$0] is a associative array dereference
++ is the post-increment operator
!a[$0]++ guards the right hand side, i.e. it makes sure that the current record is only printed, if it wasn't printed before
NR the current record number, starting with 1

That means that AWK is used to split the PATH content along the : delimiter characters and to filter out duplicate entries without modifying the order.

Since AWK associative arrays are implemented as hash tables the runtime is linear (i.e. in O(n)).

Note that we don't need look for quoted : characters because shells don't provide quoting to support directories with : in its name in the PATH variable.

awk + paste

The above can be simplified with paste:

$ PATH=$(printf %s "$PATH" | awk -v RS=: '!a[$0]++' | paste -s -d: -)

The paste command is used to intersperse the awk output with colons. This simplifies the awk action to printing (which is the default action).

Python

The same as Python two-liner:

$ PATH=$(python3 -c 'import os; from collections import OrderedDict; \ l=os.environ["PATH"].split(":"); print(":".join(OrderedDict.fromkeys(l)))' )

replaced http://stackoverflow.com/ with https://stackoverflow.com/

Source Link

edited May 23, 2017 at 12:39

Community Bot

Here is an AWK one liner.

$ PATH=$(printf %s "$PATH" \ | awk -vRS=: -vORS= '!a[$0]++ {if (NR>1) printf(":"); printf("%s", $0) }' )

where:

printf %s "$PATH" prints the content of $PATH without a trailing newline
RS=: changes the input record delimiter character (default is newline)
ORS= changes the output record delimiter to the empty string
a the name of an implicitly created array
$0 references the current record
a[$0] is a associative array dereference
++ is the post-increment operator
!a[$0]++ guards the right hand side, i.e. it makes sure that the current record is only printed, if it wasn't printed before
NR the current record number, starting with 1

That means that AWK is used to split the PATH content along the : delimiter characters and to filter out duplicate entries without modifying the order.

Since AWK associative arrays are implemented as hash tables the runtime is linear (i.e. in O(n)).

Note that we don't need look for quoted : characters because shells don't provide quoting don't provide quoting to support directories with : in its name in the PATH variable.

Awk + paste

The above can be simplified with paste:

$ PATH=$(printf %s "$PATH" | awk -vRS=: '!a[$0]++' | paste -s -d:)

The paste command is used to intersperse the awk output with colons. This simplifies the awk action to printing (which is the default action).

Python

The same as Python two-liner:

$ PATH=$(python3 -c 'import os; from collections import OrderedDict; \ l=os.environ["PATH"].split(":"); print(":".join(OrderedDict.fromkeys(l)))' )

Here is an AWK one liner.

$ PATH=$(printf %s "$PATH" \ | awk -vRS=: -vORS= '!a[$0]++ {if (NR>1) printf(":"); printf("%s", $0) }' )

where:

printf %s "$PATH" prints the content of $PATH without a trailing newline
RS=: changes the input record delimiter character (default is newline)
ORS= changes the output record delimiter to the empty string
a the name of an implicitly created array
$0 references the current record
a[$0] is a associative array dereference
++ is the post-increment operator
!a[$0]++ guards the right hand side, i.e. it makes sure that the current record is only printed, if it wasn't printed before
NR the current record number, starting with 1

That means that AWK is used to split the PATH content along the : delimiter characters and to filter out duplicate entries without modifying the order.

Since AWK associative arrays are implemented as hash tables the runtime is linear (i.e. in O(n)).

Note that we don't need look for quoted : characters because shells don't provide quoting to support directories with : in its name in the PATH variable.

Awk + paste

The above can be simplified with paste:

$ PATH=$(printf %s "$PATH" | awk -vRS=: '!a[$0]++' | paste -s -d:)

The paste command is used to intersperse the awk output with colons. This simplifies the awk action to printing (which is the default action).

Python

The same as Python two-liner:

$ PATH=$(python3 -c 'import os; from collections import OrderedDict; \ l=os.environ["PATH"].split(":"); print(":".join(OrderedDict.fromkeys(l)))' )

Here is an AWK one liner.

$ PATH=$(printf %s "$PATH" \ | awk -vRS=: -vORS= '!a[$0]++ {if (NR>1) printf(":"); printf("%s", $0) }' )

where:

printf %s "$PATH" prints the content of $PATH without a trailing newline
RS=: changes the input record delimiter character (default is newline)
ORS= changes the output record delimiter to the empty string
a the name of an implicitly created array
$0 references the current record
a[$0] is a associative array dereference
++ is the post-increment operator
!a[$0]++ guards the right hand side, i.e. it makes sure that the current record is only printed, if it wasn't printed before
NR the current record number, starting with 1

That means that AWK is used to split the PATH content along the : delimiter characters and to filter out duplicate entries without modifying the order.

Since AWK associative arrays are implemented as hash tables the runtime is linear (i.e. in O(n)).

Note that we don't need look for quoted : characters because shells don't provide quoting to support directories with : in its name in the PATH variable.

Awk + paste

The above can be simplified with paste:

$ PATH=$(printf %s "$PATH" | awk -vRS=: '!a[$0]++' | paste -s -d:)

The paste command is used to intersperse the awk output with colons. This simplifies the awk action to printing (which is the default action).

Python

The same as Python two-liner:

$ PATH=$(python3 -c 'import os; from collections import OrderedDict; \ l=os.environ["PATH"].split(":"); print(":".join(OrderedDict.fromkeys(l)))' )

add awk/paste and Python version

Source Link

edited Apr 21, 2017 at 10:00

maxschlepzig

59.7k
53
224
298

finish Gilles' edit.

Source Link

edited Sep 5, 2016 at 11:38

Stéphane Chazelas

586.8k
96
1.1k
1.7k

don't break on spaces and backslashes

Source Link

edited Sep 3, 2016 at 10:53

Gilles 'SO- stop being evil'

866.1k
205
1.8k
2.3k

Source Link

answered Apr 13, 2014 at 9:27

maxschlepzig

59.7k
53
224
298