Python: write unicode value in file

Question

I'm writing a script that writes files in multiple languages including none ascii text therefore I'm writing the content in unicode.

Here is a print of the data:

[['LATEST', u'\u0928\u0935\u0940\u0928\u0924\u092e'], ['RECOMMENDED', u'\u0938\u093f\u092b\u093e\u0930\u093f\u0936 \u0915\u093f\u092f\u093e \u0917\u092f\u093e']]

here is the code that I use to write :

f = codecs.open(file,encoding='utf-8', mode='w') f.write(el)

This works fine to have to text appear in Hindi in an text editor but because of the file format expected by the server, I need to directly write out

\u0928\u0935\u0940\u0928\u0924\u092e

I'm currently running

os.system("native2ascii -encoding utf-8 ./output/nls_hi.properties ./output/nls_hi.properties")

but this takes too much time and I can't help but think that there must be a way to directly write it the right way.

Ideas ?

Thanks

Jason

user711413 · Accepted Answer · 2012-09-05 21:55:17Z

4

You probably want to use something like my_string.encode('raw_unicode_escape')

Well, f = codecs.open('bla.txt', encoding='raw_unicode_escape', mode='w')

Then the file will contain the escaped string: \u0928\u0935\u0940\u0928\u0924\u092e

edited Sep 5, 2012 at 21:55

answered Sep 5, 2012 at 21:49

user711413

7975 silver badges12 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Jason Rogers Over a year ago

that worked, I had looked up the table of encodings but looked at unicode... and didn't think of looking at raw...; still got a long way to go before mastering the documentation !

AechoLiu Over a year ago

You save my lift !!

Adam Rosenfield · Accepted Answer · 2012-09-05 21:51:49Z

What file format does the server expect? Does it need a byte-order mark (BOM)? Whatever the answer, it's easiest to just directly use str.encode:

data = u'text with Unicode chars etc.' with open(filename, 'w') as f: # For UTF-8, no BOM: f.write(data.encode('utf-8')

For UTF-16, use data.encode('utf-16'), which will come with a BOM. If you don't want a BOM, explicitly use either utf-16le (little-endian) or utf-16be (big-endian).

I had already tried that but I then get "ascii' codec can't decode byte 0xe0 in position 7: ordinal not in range(128)" however the solution of user711413 worked so problem solved thanks

Collectives™ on Stack Overflow

Python: write unicode value in file

2 Answers 2

2 Comments

1 Comment

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

1 Comment

Related