I am trying to learn python and couldn't figure out how to translate the following perl script to python:
#!/usr/bin/perl -w use open qw(:std :utf8); while(<>) { s/\x{00E4}/ae/; s/\x{00F6}/oe/; s/\x{00FC}/ue/; print; } The script just changes unicode umlauts to alternative ascii output. (So the complete output is in ascii.) I would be grateful for any hints. Thanks!
For converting to ASCII you might want to try ASCII, Dammit or this recipe, which boils down to:
>>> title = u"Klüft skräms inför på fédéral électoral große" >>> import unicodedata >>> unicodedata.normalize('NFKD', title).encode('ascii','ignore') 'Kluft skrams infor pa federal electoral groe' 
fileinput module to loop over standard input or a list of files,translate methodtranslit.py would look like this:
#!/usr/bin/env python2.6 # -*- coding: utf-8 -*- import fileinput table = { 0xe4: u'ae', ord(u'ö'): u'oe', ord(u'ü'): u'ue', ord(u'ß'): None, } for line in fileinput.input(): s = line.decode('utf8') print s.translate(table), And you could use it like this:
$ cat utf8.txt sömé täßt sömé täßt sömé täßt $ ./translit.py utf8.txt soemé taet soemé taet soemé taet In case you are using python 3 strings are by default unicode and you dont' need to encode it if it contains non-ASCII characters or even a non-Latin characters. So the solution will look as follow:
line = 'Verhältnismäßigkeit, Möglichkeit' table = { ord('ä'): 'ae', ord('ö'): 'oe', ord('ü'): 'ue', ord('ß'): 'ss', } line.translate(table) >>> 'Verhaeltnismaessigkeit, Moeglichkeit' 
print s.translate(table).encode('ascii', 'ignore'), I guess.
ord(u'ß'): None in this code is to delete the ß ("eszett") character. It should be ord(u'ß'): u'ss'. Upvotes?? Accepted answer???You could try unidecode to convert Unicode into ascii instead of writing manual regular expressions. It is a Python port of Text::Unidecode Perl module:
#!/usr/bin/env python import fileinput import locale from contextlib import closing from unidecode import unidecode # $ pip install unidecode def toascii(files=None, encoding=None, bufsize=-1): if encoding is None: encoding = locale.getpreferredencoding(False) with closing(fileinput.FileInput(files=files, bufsize=bufsize)) as file: for line in file: print unidecode(line.decode(encoding)), if __name__ == "__main__": import sys toascii(encoding=sys.argv.pop(1) if len(sys.argv) > 1 else None) It uses FileInput class to avoid global state.
Example:
$ echo 'äöüß' | python toascii.py utf-8 aouss I use translitcodec
>>> import translitcodec >>> print '\xe4'.decode('latin-1') ä >>> print '\xe4'.decode('latin-1').encode('translit/long').encode('ascii') ae >>> print '\xe4'.decode('latin-1').encode('translit/short').encode('ascii') a You can change the decode language to whatever you need. You may want a simple function to reduce length of a single implementation.
def fancy2ascii(s): return s.decode('latin-1').encode('translit/long').encode('ascii') 
Quick and dirty (python2):
def make_ascii(string): return string.decode('utf-8').replace(u'ü','ue').replace(u'ö','oe').replace(u'ä','ae').replace(u'ß','ss').encode('ascii','ignore'); 