Python: UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position 0: ordinal not in range(128)

Question

I'm currently have an issue with my python 3 code.

 replace_line('Products.txt', line, tenminus_str)

Is the line I'm trying to turn into utf-8, however when I try to do this like I would with others, I get errors such as no attribute ones and when I try to add, for example...

.decode("utf8")

...to the end of it, I still get errors that it is using ascii. I also tried other methods that worked with other lines such as adding io. infront and adding a comma with

encoding = 'utf8'

The function that I am using for replace_line is:

def replace_line(file_name, line_num, text): lines = open(file_name, 'r').readlines() lines[line_num] = text out = open(file_name, 'w') out.writelines(lines) out.close()

How would I fix this issue? Please note that I'm very new to Python and not advanced enough to do debugging well.

EDIT: Different fix to this question than 'duplicate'

EDIT 2:I have another error with the function now.

File "FILELOCATION", line 45, in refill replace_line('Products.txt', str(line), tenminus_str) File "FILELOCATION", line 6, in replace_line lines[line_num] = text TypeError: list indices must be integers, not str

What does this mean and how do I fix it?

use utf_8_sig, instead of utf8, your file might start with bom — YOU
– YOU, Commented Nov 16, 2016 at 0:02

Mark Tolonen · Accepted Answer · 2016-11-16 08:20:30Z

Change your function to:

def replace_line(file_name, line_num, text): with open(file_name, 'r', encoding='utf8') as f: lines = f.readlines() lines[line_num] = text with open(file_name, 'w', encoding='utf8') as out: out.writelines(lines)

encoding='utf8' will decode your UTF-8 file correctly.

with automatically closes the file when its block is exited.

Since your file started with \xef it likely has a UTF-8-encoding byte order mark (BOM) character at the beginning. The above code will maintain that on output, but if you don't want it use utf-8-sig for the input encoding. Then it will be automatically removed.

Enix · Accepted Answer · 2016-11-16 01:59:59Z

4

codecs module is just what you need. detail here

import codecs def replace_line(file_name, line_num, text): f = codecs.open(file_name, 'r', encoding='utf-8') lines = f.readlines() lines[line_num] = text f.close() w = codecs.open(file_name, 'w', encoding='utf-8') w.writelines(lines) w.close()

answered Nov 16, 2016 at 1:59

Enix

4,5991 gold badge26 silver badges37 bronze badges

5 Comments

Mark Ransom Over a year ago

No need for codecs in Python 3, open also supports the encoding parameter.

Enix Over a year ago

@MarkRansom thank you for pointing out. actually, i am a developer of python2... :)

Name not Found Over a year ago

I have another error with the function now. File "LOCATION", line 45, in refill replace_line('Products.txt', str(line), tenminus_str) File "LOCATION", line 6, in replace_line lines[line_num] = text TypeError: list indices must be integers, not str What does this mean and how do I fix it?

Enix Over a year ago

@NamenotFound the second parameter line_num should be an integer represent which line you need to replace. you have passed a str to the function, so it will be failed with that error. You should call the function like replace_line("Products.txt", 1, tenminus_str), it means, you want to replace the second line with string tenminus_str.

Mark Tolonen Over a year ago

No need for codecs in Python 2, either. io.open is present in both Python 2 and Python 3 and works like Python 3's built-in open.

luyishisi · Accepted Answer · 2016-11-16 01:44:54Z

Handling coding problems You can try adding the following settings to your head

import sys reload(sys) sys.setdefaultencoding('utf-8') Type = sys.getfilesystemencoding()

tausif · Accepted Answer · 2019-04-23 10:34:14Z

Try adding encoding='utf8' if you are reading a file

with open("../file_path", encoding='utf8'): # your code

Collectives™ on Stack Overflow

Python: UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position 0: ordinal not in range(128)

4 Answers 4

Comments

5 Comments

1 Comment

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

5 Comments

1 Comment

Comments

Linked

Related