How can I write to files using Python (on Windows) and use the Unix end of line character?
e.g. When doing:
f = open('file.txt', 'w') f.write('hello\n') f.close() Python automatically replaces \n with \r\n.
Add a comment |
4 Answers
The modern way: use newline=''
Use the newline= keyword parameter to io.open() to use Unix-style LF end-of-line terminators:
import io f = io.open('file.txt', 'w', newline='\n') This works in Python 2.6+. In Python 3 you could also use the builtin open() function's newline= parameter instead of io.open().
The old way: binary mode
The old way to prevent newline conversion, which does not work in Python 3, is to open the file in binary mode to prevent the translation of end-of-line characters:
f = open('file.txt', 'wb') # note the 'b' meaning binary but in Python 3, binary mode will read bytes and not characters so it won't do what you want. You'll probably get exceptions when you try to do string I/O on the stream. (e.g. "TypeError: 'str' does not support the buffer interface").
4 Comments
Osify
This help me by using this: # newline='' means don't convert \n plus.google.com/+PongsametreySOK/posts/2Bg8AdNJCxV
12431234123412341234123
You can set newline to
'\n' direct, this is more explicit than '', and i think it is easier to read it.
Colin D Bennett
@12431234123412341234123 I agree, using
'\n' is clearer than ''. Then I won't need a comment to explain what the code is doing :) But be aware that this is only true for writing files. When reading files, newline='\n' is not exactly the same as newline='', since readlines() will split only on \n in the former case but with newline='' it will do universal newline handling but still still return the actual newlines from the file in the data. Thanks for the suggestion! Python test program
bballdave025
Python 3.6+ (maybe previous Python3 versions, as well). If you worry a little bit about seeing the same
'\n' that had been switched previously, you could use the Unicode codepoint (make sure the encoding is utf-8 or something else compatible). with open('file.txt', 'w', encoding='utf-8', newline='\u000A') ; P.S. the result isn't any different from the newline='\n' version, this is just if someone wants some more comfort.For Python 2 & 3
See: The modern way: use newline='' answer on this very page.
For Python 2 only (original answer)
Open the file as binary to prevent the translation of end-of-line characters:
f = open('file.txt', 'wb') Quoting the Python manual:
On Windows, 'b' appended to the mode opens the file in binary mode, so there are also modes like 'rb', 'wb', and 'r+b'. Python on Windows makes a distinction between text and binary files; the end-of-line characters in text files are automatically altered slightly when data is read or written. This behind-the-scenes modification to file data is fine for ASCII text files, but it’ll corrupt binary data like that in JPEG or EXE files. Be very careful to use binary mode when reading and writing such files. On Unix, it doesn’t hurt to append a 'b' to the mode, so you can use it platform-independently for all binary files.
Comments
You'll need to use the binary pseudo-mode when opening the file.
f = open('file.txt', 'wb') 
Comments
def dos2unix(inp_file, out_file=None): if out_file: out_file_tmp = out_file else: out_file_tmp = inp_file + '_tmp' if os.path.isfile(out_file_tmp): os.remove(out_file_tmp) with open(out_file_tmp, "w", newline='\n') as fout: with open(inp_file, "r") as fin: lines = fin.readlines() lines = map(lambda line: line.strip() + '\n', lines) fout.writelines(lines) if not out_file: shutil.move(out_file_tmp, inp_file) print(f'dos2unix() {inp_file} is overwritten with converted data !') else: print(f'dos2unix() {out_file} is created with converted data !') 
1 Comment
Community
Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.