pandas read_csv can't handle additional commas in double quotes

Question

Seems like this has been addressed so many times, yet I'm not able to resolve it. Here's a minimal example of my CSV:

Issue, Content Test, "A, B" Test, "A, B, C"

Here's the read_csv code (tried all sorts of combinations regarding parameters):

df = pd.read_csv('data.csv', delimiter=",", quotechar='"', encoding="utf-8")

Here's the error: ParserError:

Error tokenizing data. C error: Expected 3 fields in line 3, saw 4

I created the CSV file with a plain text editor. Wondering also why the interpreter expects 3 fields..

change your syntax to : df = pd.read_csv('data.csv', delimiter=",", quotechar='"', quoting=csv.QUOTE_MINIMAL, encoding="utf-8") — Achraf Ben Salah
– Achraf Ben Salah, Commented Sep 5, 2023 at 8:35
Your CSV is invalid, there shouldn't be a space between the delimiter and the quotes, use skipinitialspace=True — mozway
– mozway, Commented Sep 5, 2023 at 8:44

Mario Locatelli · Accepted Answer · 2023-09-05 08:43:08Z

2

Try

df = pd.read_csv('data.csv', delimiter=",", quotechar='"', skipinitialspace = True, encoding="utf-8")

answered Sep 5, 2023 at 8:43

613 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

you might want to explain why, see my comment ;)

PV8 · Accepted Answer · 2023-09-05 08:46:35Z

For what you need the delimiter in this case?

 df = pd.read_csv('data.csv', quotechar="'", encoding="utf-8", sep=",")