Can pip handle concurrency?

Question

I want to allow multiple users to manipulate the same virtual environment. What happens if multiple processes try to install/update/delete the same package using pip? Should I use file locking?

Here is the situation. There is a web app which can have multiple admins. Admin A and B login and see an update is available. They both click on the update button. A request is sent to the server in order to update the app's package. Now what happens?

In what kind of environment? Directly where all users have access to the same file system, indirectly where you have a server doing this as a result of a user-initiated action, or something else? — dskrypa
– dskrypa, Commented Jan 12, 2023 at 22:54
Please edit your question and provide us with more details as to exactly what the situation is, who/what will be running pip and how, and what the desired end results are. Are the file system manipulations such as running pip install or pip uninstall being performed by a process, like a Flask or Django server, or through the shell, such as multiple users logged into the same server using the command line? This question is too vague to answer as-is. Also, please indicate why having multiple users control pip in the same environment is necessary. — MattDMo
– MattDMo, Commented Jan 12, 2023 at 23:23
Not an answer nor necessarily a defect of the question, but consider that even if pip does not break during concurrent operations: Would the result be desirable? If user A updates package X while user B deletes package X, at least one of them will find the packages in an undesirable state. — MisterMiyagi
– MisterMiyagi, Commented Jan 13, 2023 at 8:37
Ugh. The recent edit has completely changed the focus of the question. "They both click on the update button. A request is sent to the server in order to update the app's package." This completely depends on what the server actually does, not pip! Does the server queue, deduplicate, normalise, ... such requests? Do you actually care about "the server" or would you adjust it based on what pip does? Retrofitting such details can really mess with questions... — MisterMiyagi
– MisterMiyagi, Commented Jan 13, 2023 at 11:00

MisterMiyagi · Accepted Answer · 2023-01-13 09:00:25Z

No. Pip does not perform any locking or synchronisation across multiple pip instances. This means that multiple pip instances acting on the same environment may lead to undefined behaviour.

There is an open feature request since 2015:
pypa/pip#2361: Handle multiple concurrent pip processes on the same environment

In principle, acting on separate packages in the same environment is fine. However, any operations targeting the same package may leave it in incomplete or inconsistent state.

Note that even if you protect against packages being corrupted via file locking for example, the environment is still likely to be broken. If one user installs a package at version A, another installs it at version B, and a third deletes it, the resulting environment will be inappropriate for two of these users.

Collectives™ on Stack Overflow

Can pip handle concurrency?

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Related