Timeline for How to remove duplicates from pipe delimited file using awk with Pipe in Values?
Current License: CC BY-SA 4.0
15 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Apr 8, 2022 at 14:49 | vote | accept | vks009 | ||
| Apr 8, 2022 at 6:19 | comment | added | Fravadona | The devil is in the details: You should edit your question for adding an input example that illustrates your quoting rules. From what I understood (inside double-quotes you have \" for " and \\ for \ ), the current answers do not take them into account. | |
| Apr 7, 2022 at 0:11 | answer | added | Ed Morton | timeline score: 1 | |
| Apr 6, 2022 at 21:35 | answer | added | steeldriver | timeline score: 1 | |
| Apr 6, 2022 at 21:32 | comment | added | vks009 | Yes, I went through that but not able to replicate it for pipe | |
| Apr 6, 2022 at 21:15 | comment | added | steeldriver | You can do it with GNU awk, using FPAT to define what comes between separators - see for example awk when both delimiter and quotes are used for a field | |
| Apr 6, 2022 at 21:06 | comment | added | vks009 | Delimiter here is pipe and awk is taking into account the pipe in value that are in double quotes as shown in line 2. awk should ignore the pipes within "2025035|6|15|0|0|15|39" | |
| Apr 6, 2022 at 20:53 | comment | added | Chris Davies | @Fravadona it handles proper CSV escaping rules, so two comma separated fields of "item, item",another" "item would be properly parsed as item, item and another" "item, but I can't understand what vks009 is trying to show for the escaping rules. It would be much easier if they provided a (well formatted) example in their question | |
| Apr 6, 2022 at 20:19 | history | edited | Jeff Schaller♦ | edited tags | |
| Apr 6, 2022 at 20:10 | comment | added | Fravadona | @roaima does csvkit handle that kind of escaping rules? | |
| Apr 6, 2022 at 20:09 | comment | added | Chris Davies | Use one of the csvkit tools | |
| Apr 6, 2022 at 20:06 | comment | added | vks009 | It will be like this for "\"\"" for \ | |
| Apr 6, 2022 at 19:59 | comment | added | Fravadona | What are the escaping rules of your format? for ex when you have a |, ", \n as part of the data? | |
| S Apr 6, 2022 at 19:51 | review | First questions | |||
| Apr 7, 2022 at 0:11 | |||||
| S Apr 6, 2022 at 19:51 | history | asked | vks009 | CC BY-SA 4.0 |