1

When I am inserting/updating a document in a collection, is the lock applied on the database or the collection. Suppose I have two collections and they are independant of each other in the same database and wants to do write operations on them concurrently. Is this possible?

Here is the code I am using to test this: 

var assert = require('assert'), MongoClient = require('mongodb').MongoClient, async = require('async'); var station_list = require('./station_list.json'), trains_list = require('./trains_list.json'); var stationList = [], trainsList = []; var MONGO_URL = 'mongodb://localhost:27017/test'; for(var i=0; i<station_list.stations.length; i++) stationList.push(station_list.stations[i].station_code); for(var i=0; i<trains_list.trains.length; i++) trainsList.push(trains_list.trains[i].code); console.log('trains : ' + trainsList.length + ' stations : ' + stationList.length); populateTrains(); populateStations(); function populateTrains() { async.eachSeries(trainsList, populateTrainDb, function (err) { assert.equal(null, err); }); } function populateTrainDb(code, callback) { MongoClient.connect(MONGO_URL, function (err, db) { assert.equal(null, err); var jsonData = {}; jsonData.code = code; db.collection('trainsCon').replaceOne( {'code' : code}, jsonData, {upsert: true, w:1}, function (err, res) { assert.equal(null, err); db.close(); callback(); }); }); } function populateStations() { async.eachSeries(stationList, populateStationDb, function (err) { assert.equal(null, err); }); } function populateStationDb(code, callback) { MongoClient.connect(MONGO_URL, function (err, db) { assert.equal(null, err); var jsonData = {}; jsonData.code = code; db.collection('stationsCon').replaceOne( {'code' : code}, jsonData, {upsert:true, w:1}, function (err, res) { assert.equal(null, err); db.close(); callback(); }); }); }

The two json files : station_list.json and trains_list.json have around 5000 entries. So after running the given program I get this error after a while : 

C:\Users\Adnaan\Desktop\hopSmart\node_modules\mongodb\lib\server.js:242 process.nextTick(function() { throw err; }) ^ AssertionError: null == { [MongoError: connect EADDRINUSE 127.0.0.1:27017] name: 'MongoError', message: 'connect EADDRINUSE 127.0.0.1:27017' } at C:\Users\Adnaan\Desktop\hopSmart\testing.js:52:10 at C:\Users\Adnaan\Desktop\hopSmart\node_modules\mongodb\lib\mongo_client.js:276:20 at C:\Users\Adnaan\Desktop\hopSmart\node_modules\mongodb\lib\db.js:224:14 at null.<anonymous> (C:\Users\Adnaan\Desktop\hopSmart\node_modules\mongodb\lib\server.js:240:9) at g (events.js:273:16) at emitTwo (events.js:100:13) at emit (events.js:185:7) at null.<anonymous> (C:\Users\Adnaan\Desktop\hopSmart\node_modules\mongodb-core\lib\topologies\server.js:301:68) at emitTwo (events.js:100:13) at emit (events.js:185:7)

When I check the number of entries entered the database, around 4000 entries had already been entered in both the collections. So what I get from the above experiment was that an error might have occured when one write was being attempted while inside other collection a document must have been getting written.

So how should I proceed to have this concurrency without conflicting locks.

Improve this question
4
  • Have you tried this yourself yet? What is result? Show us your effort. Commented Mar 6, 2016 at 12:11
  • @Saleem I have added the code example along with the error. Commented Mar 7, 2016 at 14:35
  • This may be an issue of driver. Which driver version are you using? Please also mention MongoDB version too. Commented Mar 7, 2016 at 15:44
  • Also include sample input documents from both collections. Commented Mar 7, 2016 at 15:46

3 Answers 3

Reset to default
6

The answer to this question can be quite long and depends on various factors (MongoDB version, storage engine, type of operations you are doing, sharding, etc.). I can only recommend you to read carefully the Concurrency section of the MongoDB documentation, and in particular the lock granularity part.

Make sure to choose the right version of MongoDB first as the behaviour varies greatly from one version to another (e.g. database locking in pre-3.0 vs. collection locking for most operations in post-3.0 using NMAPv1).

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

0

I don't think it's concurrency issue with MongoDB, but I could be driver or even with test itself. I have created a sample application couple of weeks ago to stress test MongoDB while working on a nasty bug. I used C# and MongoDB 3.0 on Windows 10. I have inserted million of documents in multithreaded environment but couldn't crash MongoDB. 

Parallel.For(0, 10000, (x => { var lstDocs = new List<BsonDocument>(); for (var i = 0; i < 100; i++) { lstDocs.Add(new BsonDocument(doc)); } collection.InsertMany(lstDocs); lstDocs.Clear(); }));

You can find code in gist here.

Improve this answer

Comments

0

You should not be calling MongoClient.connect every time. That's causing a ton of connections to open and close all the time which is overloading mongo. You should let the MongoClient manage the connection pool. Change it so that you store the db object from MongoClient.connect. Something like this:

var db MongoClient.connect(url, function(err, database){ db = database; }
Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.