39

I have this code:

 printinfo = title + "\t" + old_vendor_id + "\t" + apple_id + '\n' # Write file f.write (printinfo + '\n')

But I get this error when running it:

 f.write(printinfo + '\n') UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 7: ordinal not in range(128)

It's having toruble writing out this:

Identité secrète (Abduction) [VF]

Any ideas please, not sure how to fix.

Cheers.

UPDATE: This is the bulk of my code, so you can see what I am doing:

def runLookupEdit(self, event): newpath1 = pathindir + "/" errorFileOut = newpath1 + "REPORT.csv" f = open(errorFileOut, 'w') global old_vendor_id for old_vendor_id in vendorIdsIn.splitlines(): writeErrorFile = 0 from lxml import etree parser = etree.XMLParser(remove_blank_text=True) # makes pretty print work path1 = os.path.join(pathindir, old_vendor_id) path2 = path1 + ".itmsp" path3 = os.path.join(path2, 'metadata.xml') # Open and parse the xml file cantFindError = 0 try: with open(path3): pass except IOError: cantFindError = 1 errorMessage = old_vendor_id self.Error(errorMessage) break tree = etree.parse(path3, parser) root = tree.getroot() for element in tree.xpath('//video/title'): title = element.text while '\n' in title: title= title.replace('\n', ' ') while '\t' in title: title = title.replace('\t', ' ') while ' ' in title: title = title.replace(' ', ' ') title = title.strip() element.text = title print title ######################################### ######## REMOVE UNWANTED TAGS ######## ######################################### # Remove the comment tags comments = tree.xpath('//comment()') q = 1 for c in comments: p = c.getparent() if q == 3: apple_id = c.text p.remove(c) q = q+1 apple_id = apple_id.split(':',1)[1] apple_id = apple_id.strip() printinfo = title + "\t" + old_vendor_id + "\t" + apple_id # Write file # f.write (printinfo + '\n') f.write(printinfo.encode('utf8') + '\n') f.close()
Improve this question
2
  • 6
    If you look at the right side of the question, you will notice a column of "Related" questions. I suggest you start by looking at them. You would also have gotten a list of possibly duplicates when writing your question title. Commented Nov 7, 2013 at 10:26
  • @MartijnPieters: you are right, as usual. Comment erased. Commented Nov 7, 2013 at 10:37

1 Answer 1

Reset to default
73

You need to encode Unicode explicitly before writing to a file, otherwise Python does it for you with the default ASCII codec.

Pick an encoding and stick with it:

f.write(printinfo.encode('utf8') + '\n')

or use io.open() to create a file object that'll encode for you as you write to the file:

import io f = io.open(filename, 'w', encoding='utf8')

You may want to read:

before continuing.

Improve this answer
Sign up to request clarification or add additional context in comments.

15 Comments

Using f.write(printinfo.encode('utf8') + '\n') works but creates odd characters Identit√© secr√®te (Abduction) [VF] which should be accented Identité secrète (Abduction) [VF]
@speedyrazor: please do read the links I provided. You are opening a UTF-8 file with something that displays the bytes as a different encoding instead. Pick the right encoding for your application.
@Martin Pieters: I have had a read through, but don't really understand. If I have "Identité secrète" in my XML file I am reading, I pick lines out and write them to a file, but that line comes out as "Identit√© secr√®te". Sorry to ask, but what code would sort this out please?
@speedyrazor: Your XML file uses a codec too. It either uses UTF-8 or has a different codec specified on the first line of the XML file. The XML parser then decodes that data to a Unicode value. When writing out the values to a file, you need to pick a codec again to write bytes. I picked UTF-8 for you because that codec can encode all of unicode, but whatever you used to view the resulting file used a different codec to interpret the bytes. The é character is unicode codepoint U+00E9. UTF-8 encodes that to two bytes, hex C3 and A9. Misinterpreting those two bytes gives you √©.
@speedyrazor: without knowing how you are reading the produces file again, I cannot help you further.
|

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.