1

Suppose I have a file (say file1.txt) with data around 3mb or more. If I want to write this data to a second file (say file2.txt), which one of the following approaches will be better?

Language used: Python 2.7.3

Approach 1:

file1_handler = file("file1.txt", 'r') for lines in file1_handler: line = lines.strip() # Perform some operation file2_handler = file("file2.txt", 'a') file2_handler.write(line) file2_handler.write('\r\n') file2_handler.close() file1_handler.close()

Approach 2:

file1_handler = file("file1.txt", 'r') file2_handler = file("file2.txt", 'a') for lines in file1_handler: line = lines.strip() # Perform some operation file2_handler.write(line) file2_handler.write('\r\n') file2_handler.close() file1_handler.close()

I think approach two will be better because you just have to open and close file2.txt once. What do you say?

Improve this question
1
  • 2
    Open a file with open, not with file. Commented Mar 20, 2013 at 13:23

3 Answers 3

Reset to default
6

Use with, it will close the files automatically for you:

with open("file1.txt", 'r') as in_file, open("file2.txt", 'a') as out_file: for lines in in_file: line = lines.strip() # Perform some operation out_file.write(line) out_file.write('\r\n')

Use open instead of file, file is deprecated.

Of course it's unreasonable to open file2 on every line of file1.

Improve this answer
Sign up to request clarification or add additional context in comments.

14 Comments

I was writing the same think :) @Hemant, look at: docs.python.org/2/whatsnew/2.5.html#pep-343-the-with-statement
Regarding f2.write('\r\n'): in order to do this you need to open f2 as binary file (appending "b" to the flag).
oops! I thought open has been deprecated :p ( i din't read the documents properly) so does the speed of writing increase? Because approach one was taking almost 2 hours for copying 1 MB of data.
Binary mode has no effect on Unix. It writes what it's told to write.
Text mode on Windows converts \r and \n to \r\n. \r\n is left unchanged.
|
0

I was recently doing something similar (if I understood you well). How about:

file = open('file1.txt', 'r') file2 = open('file2.txt', 'wt') for line in file: newLine = line.strip() # You can do your operation here on newLine file2.write(newLine) file2.write('\r\n') file.close() file2.close()

This approach works like a charm!

Improve this answer

Comments

0

My solution (derived from Pavel Anossov + buffering):

dim = 1000 buffer = [] with open("file1.txt", 'r') as in_file, open("file2.txt", 'a') as out_file: for i, lines in enumerate(in_file): line = lines.strip() # Perform some operation buffer.append(line) if i%dim == dim-1: for bline in buffer: out_file.write(bline) out_file.write('\r\n') buffer = []

Pavel Anossov gave the right solution first: this is just a suggestion ;) Probably it exists a more elegant way to implement this function. If anyone knows it, please tell us.

Improve this answer

3 Comments

@ Francesco: Hey thanks for this answer :) but I am not much familiar with enumerate method. Can please explain me benefits of using enumerate?
@Hemant: enumerate is useful :) Look here: docs.python.org/2/library/functions.html#enumerate
@ Francesco: thanks for the doc. now i understand your example more clearly. :)

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.