Nodejs, MongoDB concurrent request creates duplicate record

Question

Let me be real simple. I am running node.js server. When I receive data from patch request, first I need to check in database. If the row exists I will update it, otherwise I will create that record. Here is my code. This is what I am calling in the request handler callback.

let dbCollection = db$master.collection('users'); createUser(req.body).then(user => { dbCollection.updateMany( { rmisId: req.body.rmisId }, { $set: { ...user } }, { upsert: true } ).then((result) => { log.debug("RESULTS", user); return result; }) .catch((err => { log.debug(err); return err; })); })

This is working fine in sequential requests. But its creating duplicate record when I receive 10 concurrent request. I am running on my local machine and replicating concurrent request using Apache JMeter. Please help me if you have experienced this kind of problem.

Thank you !

UPDATE

I have tested another approach that reads the database like dbCollection.find({rmisId: req.body.rmisId}) the database for determine its existing or no. But it has no difference at all.

Just reading about this morning :P docs.mongodb.com/manual/core/read-isolation-consistency-recency/… docs.mongodb.com/manual/core/transactions — dm03514
– dm03514, Commented Dec 18, 2019 at 20:27
I got that atomicity thing. For my case, I don't have replica set in my database. In a production we are running only two instance of the node server. Problem is we don't have a load balancer or message queue between our microservices. — jad
– jad, Commented Dec 18, 2019 at 20:48
Also, starting with MongoDB 4.0 you can use sessions with transaction support. — barrypicker
– barrypicker, Commented Dec 18, 2019 at 21:23
Can I say every request will be separated transaction in mongo? — jad
– jad, Commented Dec 18, 2019 at 21:25

Burak Serdar · Accepted Answer · 2019-12-18 20:38:10Z

You cannot check-and-update. Mongodb operations are atomic at the document level. After you check and see that the record does not exist, another request may create the document you just checked, and after that you can recreate the same record if you don't have unique indexes or if you're generating the IDs.

Instead, you can use upsert, as you're already doing, but without the create. It looks like you're getting the ID from the request, so simply search using that ID, and upsert the user record. That way if some other thread inserts it before you do, you'll update what the previous thread inserted. If this is not something you prefer, add a unique index for that user ID field.

I have tried that "search first then upsert" approach. But it was no difference. So thank you so much.. anyway, I will put the index in that field is my last chance.
No, not search-then-upsert, only upsert. Upsert is atomic. search-then-upsert is not.

Collectives™ on Stack Overflow

Nodejs, MongoDB concurrent request creates duplicate record

1 Answer 1

2 Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Related