How to read csv on python with newline separator @

Question

I need to read a csv on Python, and the text file that I have has this structure:

"114555","CM13","0004","0","C/U"@"99172","CM13","0001","0","C/U"@"178672","CM13","0001","0","C/U"

delimeter: ,

newline: @

My code so far:

import csv data = [] with open('stock.csv') as csvfile: reader = csv.reader(csvfile, delimiter=',', lineterminator='@') for row in reader: data.append({'MATERIAL': row[0],'CENTRO': row[1], 'ALMACEN': row[2], 'STOCK_VALORIZADO' : row[3], 'STOCK_UMB':row[4]}) print(data) #this print just one row

This code only print one row, because it's not recognize @ as a newline, and prints it with quotes:

[{'MATERIAL': '114555', 'CENTRO': 'CM13', 'ALMACEN': '0004', 'STOCK_VALORIZADO': '0', 'STOCK_UMB': 'C/U@"99172"'}]

quoting=csv.QUOTE_NONE "Instructs reader to perform no special processing of quote characters." If you want it without the quotes don't use csv.QUOTE_NONE. — Steven Rumbalski
– Steven Rumbalski, Commented Apr 27, 2015 at 18:26
I just tried that, but still the same results. Could be a problem with the quotes? — Marcelo
– Marcelo, Commented Apr 27, 2015 at 18:26
lineterminator won't work. The docs say "The reader is hard-coded to recognise either '\r' or '\n' as end-of-line, and ignores lineterminator. This behavior may change in the future." This means you need to fix your data before the csv.reader gets to it. — Steven Rumbalski
– Steven Rumbalski, Commented Apr 27, 2015 at 18:30
Out of curiosity, who on earth writes CSVs with '@' as lineterminator? — smci
– smci, Commented Apr 27, 2015 at 19:13

Gio · Accepted Answer · 2015-05-24 01:19:38Z

According to https://docs.python.org/2/library/csv.html : "The reader is hard-coded to recognise either '\r' or '\n' as end-of-line, and ignores lineterminator. This behavior may change in the future." Hence for now, providing the argument lineterminator='@' will not work.

I think the best option is to read your entire file into a variable, and replace all '@' characters, you can do this as follows:

with open("stock.csv", "r") as myfile: data = myfile.read().replace('@', '\n')

Now you need to adjust your algorithm in such a way that you can pass the variable data to csv.reader (instead of the file stock.csv), according to the python doc:

"The "iterable" argument can be any object that returns a line of input for each iteration, such as a file object or a list. [...]"

Hence you can pass data.splitlines() to csv.reader.

Hopefully, none of the fields contain '@', otherwise the problem gets thornier.
I agree, however I think we may asume that this is not the case here.
Might need data.splitlines(keepends=True). On normal file iteration the newlines are preserved. The csv.reader may count on that. (I'm too lazy to test whether or not that's the case.)
I'm not sure about that, in the example provided by the OP, it looks like the '@' character is the actual newline character. I know this sounds weird, but without more information of the OP we cannot make any other assumptions.
What I mean is that when you iterate over a file the newlines are part of each line. When you do data.splitlines() you get lines that have no line ending. So keeping the newlines might be needed. Otherwise your answer could be shortened to data = myfile.read().split('@'). I think data = myfile.read().replace('@', '\n').splitlines(keepends=True) may be more correct.

Steven Kaspar · Accepted Answer · 2019-10-30 18:52:17Z

I was struggling with CRLF ('\r\n') line endings using csv.reader. I was able to get it working using the newline parameter in open

with open(local_file, 'r', newline='\r\n') as f: reader = csv.reader(f)

Collectives™ on Stack Overflow

How to read csv on python with newline separator @

2 Answers 2

6 Comments

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

6 Comments

Comments

Linked

Related