0

I am planning an application in python3 which makes use of either three text files or three database tables,am not sure what to use.These files consits of just strings or a single column in a database table with a string in each row.

The app has three programs program_A ,program_B ,program_C which manipulate these three files,file_A and file_B and file_C.Program A reads/writes file_A reads file_C and writes into file_B,program_B reads/writes file_B,program_C writes file_A and file_C.file_A and file_B are nothing but imaginary queue so a program reads a line from the file and then removes that line from the file,(hence i mentioned reads/writes) and a program that writes to a file writes a line to the end of the file.

I am not sure how i can get these programs to work since these programs are simultaneously accessing the files.I have looked into sqlite file locking states and i don't know how i can use it to solve my scenario.I am also not sure if i should be using sqlite databases or simple text files.

I am also considering using some kind of shared memory,or inmemory database where i can load the contents of these files when the app starts and writing back to the files when the app exits,but am worried if some software hardware failure would mean i would lose the work done on that run of the app since the files were not updated because of the failure.

Improve this question

1 Answer 1

Reset to default
1

Reading and writing in a same file? No. This will only bring pain and hardship. You can try threading's or multiprocessing's mechanisms for sharing data and have a queue implementation that will fit your needs.

The sqlite approach seems fine too as you can add the necessary data to set the order of these "queues" and so on, but I am no fan of processes communicating through a database.

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

Thank you.I think multiprocessing's mechanism can fit my needs,especially the queue.now am going through the mechanism to implement shared memory,in multiprocessing's documentation.I have seen lot of replies for similar questions discouraging the shared memory,so i will look into the topic more.
If I understand correctly your situation, you don't need the shared memory mechanism and queues will suffice. However, don't trust qsize and empty. Always stop a process after it explicitly receives confirmation that the queue it waits on is empty (for example put None in the queue and when the process reads it, it stops).
Thanks for the tip about 'None' in the queue.Queues will suffice file_A,file_B part , but the file_C part is not a queue.The program_C does write to the end of the file,but program_A needs to go through all the strings present in this file.
Oops, I must've missed that. Well shared memory ain't that bad. You just need to be careful.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.