Insert line at middle of file with Python? [duplicate]

If you want to search a file for a substring and add a new text to the next line, one of the elegant ways to do it is the following:

import os, fileinput old = "A" new = "B" for line in fileinput.FileInput(file_path, inplace=True): if old in line : line += new + os.linesep print(line, end="") 

There is a combination of techniques which I found useful in solving this issue:

with open(file, 'r+') as fd: contents = fd.readlines() contents.insert(index, new_string) # new_string should end in a newline fd.seek(0) # readlines consumes the iterator, so we need to start over fd.writelines(contents) # No need to truncate as we are increasing filesize 

In our particular application, we wanted to add it after a certain string:

with open(file, 'r+') as fd: contents = fd.readlines() if match_string in contents[-1]: # Handle last line to prevent IndexError contents.append(insert_string) else: for index, line in enumerate(contents): if match_string in line and insert_string not in contents[index + 1]: contents.insert(index + 1, insert_string) break fd.seek(0) fd.writelines(contents) 

If you want it to insert the string after every instance of the match, instead of just the first, remove the else: (and properly unindent) and the break.

Note also that the and insert_string not in contents[index + 1]: prevents it from adding more than one copy after the match_string, so it's safe to run repeatedly.

The accepted answer has to load the whole file into memory, which doesn't work nicely for large files. The following solution writes the file contents with the new data inserted into the right line to a temporary file in the same directory (so on the same file system), only reading small chunks from the source file at a time. It then overwrites the source file with the contents of the temporary file in an efficient way (Python 3.8+).

from pathlib import Path from shutil import copyfile from tempfile import NamedTemporaryFile sourcefile = Path("/path/to/source").resolve() insert_lineno = 152 # The line to insert the new data into. insert_data = "..." # Some string to insert. with sourcefile.open(mode="r") as source: destination = NamedTemporaryFile(mode="w", dir=str(sourcefile.parent)) lineno = 1 while lineno < insert_lineno: destination.file.write(source.readline()) lineno += 1 # Insert the new data. destination.file.write(insert_data) # Write the rest in chunks. while True: data = source.read(1024) if not data: break destination.file.write(data) # Finish writing data. destination.flush() # Overwrite the original file's contents with that of the temporary file. # This uses a memory-optimised copy operation starting from Python 3.8. copyfile(destination.name, str(sourcefile)) # Delete the temporary file. destination.close() 

EDIT 2020-09-08: I just found an answer on Code Review that does something similar to above with more explanation - it might be useful to some.

You can just read the data into a list and insert the new record where you want.

names = [] with open('names.txt', 'r+') as fd: for line in fd: names.append(line.split(' ')[-1].strip()) names.insert(2, "Charlie") # element 2 will be 3. in your list fd.seek(0) fd.truncate() for i in xrange(len(names)): fd.write("%d. %s\n" %(i + 1, names[i])) 

You don't show us what the output should look like, so one possible interpretation is that you want this as the output:

1. Alfred
2. Bill
3. Charlie
4. Donald

(Insert Charlie, then add 1 to all subsequent lines.) Here's one possible solution:

def insert_line(input_stream, pos, new_name, output_stream): inserted = False for line in input_stream: number, name = parse_line(line) if number == pos: print >> output_stream, format_line(number, new_name) inserted = True print >> output_stream, format_line(number if not inserted else (number + 1), name) def parse_line(line): number_str, name = line.strip().split() return (get_number(number_str), name) def get_number(number_str): return int(number_str.split('.')[0]) def format_line(number, name): return add_dot(number) + ' ' + name def add_dot(number): return str(number) + '.' input_stream = open('input.txt', 'r') output_stream = open('output.txt', 'w') insert_line(input_stream, 3, 'Charlie', output_stream) input_stream.close() output_stream.close() 

1. Parse the file into a python list using file.readlines() or file.read().split('\n')
2. Identify the position where you have to insert a new line, according to your criteria.
3. Insert a new list element there using list.insert().
4. Write the result to the file.

location_of_line = 0 with open(filename, 'r') as file_you_want_to_read: #readlines in file and put in a list contents = file_you_want_to_read.readlines() #find location of what line you want to insert after for index, line in enumerate(contents): if line.startswith('whatever you are looking for') location_of_line = index #now you have a list of every line in that file context.insert(location_of_line, "whatever you want to append to middle of file") with open(filename, 'w') as file_to_write_to: file_to_write_to.writelines(contents) 

That is how I ended up getting whatever data I want to insert to the middle of the file.

this is just pseudo code, as I was having a hard time finding clear understanding of what is going on.

essentially you read in the file to its entirety and add it into a list, then you insert your lines that you want to that list, and then re-write to the same file.

i am sure there are better ways to do this, may not be efficient, but it makes more sense to me at least, I hope it makes sense to someone else.

A simple but not efficient way is to read the whole content, change it and then rewrite it:

line_index = 3 lines = None with open('file.txt', 'r') as file_handler: lines = file_handler.readlines() lines.insert(line_index, 'Charlie') with open('file.txt', 'w') as file_handler: file_handler.writelines(lines) 

I write this in order to reutilize/correct martincho's answer (accepted one)

! IMPORTANT: This code loads all the file into ram and rewrites content to the file

Variables index, value may be what you desire, but pay attention to making value string and end with '\n' if you don't want it to mess with existing data.

with open("path_to_file", "r+") as f: # Read the content into a variable contents = f.readlines() contents.insert(index, value) # Reset the reader's location (in bytes) f.seek(0) # Rewrite the content to the file f.writelines(contents) 

See the python docs about file.seek method: Python docs

Below is a slightly awkward solution for the special case in which you are creating the original file yourself and happen to know the insertion location (e.g. you know ahead of time that you will need to insert a line with an additional name before the third line, but won't know the name until after you've fetched and written the rest of the names). Reading, storing and then re-writing the entire contents of the file as described in other answers is, I think, more elegant than this option, but may be undesirable for large files.

You can leave a buffer of invisible null characters ('\0') at the insertion location to be overwritten later:

num_names = 1_000_000 # Enough data to make storing in a list unideal max_len = 20 # The maximum allowed length of the inserted line line_to_insert = 2 # The third line is at index 2 (0-based indexing) with open(filename, 'w+') as file: for i in range(line_to_insert): name = get_name(i) # Returns 'Alfred' for i = 0, etc. file.write(F'{i + 1}. {name}\n') insert_position = file.tell() # Position to jump back to for insertion file.write('\0' * max_len + '\n') # Buffer will show up as a blank line for i in range(line_to_insert, num_names): name = get_name(i) file.write(F'{i + 2}. {name}\n') # Line numbering now bumped up by 1. # Later, once you have the name to insert... with open(filename, 'r+') as file: # Must use 'r+' to write to middle of file file.seek(insert_position) # Move stream to the insertion line name = get_bonus_name() # This lucky winner jumps up to 3rd place new_line = F'{line_to_insert + 1}. {name}' file.write(new_line[:max_len]) # Slice so you don't overwrite next line 

Unfortunately there is no way to delete-without-replacement any excess null characters that did not get overwritten (or in general any characters anywhere in the middle of a file), unless you then re-write everything that follows. But the null characters will not affect how your file looks to a human (they have zero width).