Trying to understand Django select_for_update() with MySQL and Processes

The good news is that it's mostly there, I did some reading around and based on an answer I found here

I am assuming that you are running on Linux as that seems to be the behaviour on the platform.

It looks like under Linux the default Process start strategy is the fork strategy, which is usually what you want, however in this exact circumstance it appears that resources (such as DB connections) are being shared, resulting in the db operations being treated as the same transaction and thus are not blocked. To get the behaviour you want, each process would appear to need its own resources and to not share resouces with its parent process (and subsequently any other children of the parent).

It is possible to get the behaviour you want using the following code, however be aware that I had to split the code into two files.

fn.py

from time import sleep from django.db import transaction import django django.setup() from myapp.main.models import CompoundBase def setup() -> int: cb = CompoundBase.objects.first() cb.corporate_id = 'foo' cb.save() return cb.pk def select_and_sleep(pk: int) -> None: try: with transaction.atomic(): cb = CompoundBase.objects.select_for_update().get(pk=pk) print('Locking') sleep(5) cb.corporate_id = 'baz' cb.save() print('Updated after sleep') except Exception: logger.exception('select_and_sleep') def sleep_and_update(pk: int) -> None: try: sleep(2) print('Updating') with transaction.atomic(): cb = CompoundBase.objects.select_for_update().get(pk=pk) cb.corporate_id = 'bar' cb.save() print('Updated without sleep') except Exception: logger.exception('sleep_and_update') 

proc_test.py

from concurrent import futures from multiprocessing import get_context from time import sleep import logging import fn logger = logging.getLogger() executor = futures.ProcessPoolExecutor(mp_context=get_context("forkserver")) # executor = futures.ThreadPoolExecutor() def test() -> None: pk = fn.setup() f1 = executor.submit(fn.select_and_sleep, pk) f2 = executor.submit(fn.sleep_and_update, pk) futures.wait([f1, f2]) test() 

There are three strategies in starting a process, fork, spawn, and forkserver, using either spawn or forkserver appears to get you the behaviour that you are looking for.

References:

• Multiprocessing Locks

• Connections