E11000 duplicate key error index in mongodb mongoose

Question

Following is my user schema in user.js model -

var userSchema = new mongoose.Schema({ local: { name: { type: String }, email : { type: String, require: true, unique: true }, password: { type: String, require:true }, }, facebook: { id : { type: String }, token : { type: String }, email : { type: String }, name : { type: String } } }); var User = mongoose.model('User',userSchema); module.exports = User;

This is how I am using it in my controller -

var user = require('./../models/user.js');

This is how I am saving it in the db -

user({'local.email' : req.body.email, 'local.password' : req.body.password}).save(function(err, result){ if(err) res.send(err); else { console.log(result); req.session.user = result; res.send({"code":200,"message":"Record inserted successfully"}); } });

Error -

{"name":"MongoError","code":11000,"err":"insertDocument :: caused by :: 11000 E11000 duplicate key error index: mydb.users.$email_1 dup key: { : null }"}

I checked the db collection and no such duplicate entry exists, let me know what I am doing wrong ?

FYI - req.body.email and req.body.password are fetching values.

I also checked this post but no help STACK LINK

If I removed completely then it inserts the document, otherwise it throws error "Duplicate" error even I have an entry in the local.email

Had the same error! During development we disabled auto indexing and were using lowercase schema names/keys. We later decided to use TitleCase instead but failed to update our index names (ie: titleCase instead of TitleCase). So when we enabled indexing, we got this error. Took a little while to figure it out. You'll want to make sure all names/keys are named exactly everywhere. — Jeach
– Jeach, Commented May 5, 2019 at 0:38
I had the same error and setting mongoose model to unique: false was not having any impact. I realised that I had first to drop the table and then it would work. You can do seomthing like db.whateverthecollection.drop({}). Be careful, it deletes the collection. — aixecador
– aixecador, Commented Jan 10, 2020 at 13:02
_id: mongoose.Types.ObjectId add because unique id is required — Mayank Pandav
– Mayank Pandav, Commented Mar 20, 2020 at 9:42

Community · Accepted Answer · 2020-06-20 09:12:55Z

The error message is saying that there's already a record with null as the email. In other words, you already have a user without an email address.

The relevant documentation for this:

If a document does not have a value for the indexed field in a unique index, the index will store a null value for this document. Because of the unique constraint, MongoDB will only permit one document that lacks the indexed field. If there is more than one document without a value for the indexed field or is missing the indexed field, the index build will fail with a duplicate key error.

You can combine the unique constraint with the sparse index to filter these null values from the unique index and avoid the error.

unique indexes

Sparse indexes only contain entries for documents that have the indexed field, even if the index field contains a null value.

In other words, a sparse index is ok with multiple documents all having null values.

sparse indexes

From comments:

Your error says that the key is named mydb.users.$email_1 which makes me suspect that you have an index on both users.email and users.local.email (The former being old and unused at the moment). Removing a field from a Mongoose model doesn't affect the database. Check with mydb.users.getIndexes() if this is the case and manually remove the unwanted index with mydb.users.dropIndex(<name>).

If I removed completely then it inserts the document, otherwise it throws error "Duplicate" error even I have an entry in the local.email
Your error says that the key is named mydb.users.$email_1 which makes me suspect that you have an index on both users.email and users.local.email (The former being old and unused at the moment). Removing a field from a Mongoose model doesn't affect the database. Check with mydb.users.getIndexes() if this is the case and manually remove the unwanted index with mydb.users.dropIndex(<name>).
Or use db.users.dropIndexes() if you're making multiple index changes
This was an issue for me when I was using mongoose plugin passport-local-mongoose without setting a username for new users.
It's a matter or perspective or maybe even opinion, @titoih - is null or the absence of a value a value just like any other or is it a special case? "[email protected]" and "[email protected]" are the same, are "nothing" and "nothing" the same as well? With a non-sparse index the answer is "yes" in MongoDB. Other databases (such as MySQL) would say "no".

Rick · Accepted Answer · 2015-09-13 14:56:12Z

101

If you are still in your development environment, I would drop the entire db and start over with your new schema.

From the command line

➜ mongo use dbName; db.dropDatabase(); exit

answered Sep 13, 2015 at 14:56

Rick

13.6k3 gold badges46 silver badges42 bronze badges

7 Comments

JorgeGarza Over a year ago

I think this solution is very aggressive, I think is enough with db.collection.dropIndexes() as cs_stackX said

Sandip Subedi Over a year ago

@JorgeGarza I had the same issue. I just dropped the collection and everything started working fine after that.

Meir Snyder Over a year ago

I accidentally initialized the collection with a unique Id for two of my fields. Later on when I wanted to add multiple documents with the same Id It wouldn't let me until I deleted the collection and reaccessed it.

Janac Meena Over a year ago

If you are new in your development, this is a good chance to experiment and figure out how to solve this...rather than run into this issue once you have gigabytes of user data already stored

retr0 Over a year ago

@JorgeGarza 's comment makes sense. Dropped indexes will be rebuilt once again by the Schema file when the server is restarted. Also, it worked.

|

Daggie Blanqx - Douglas Mwangi · Accepted Answer · 2018-08-18 10:24:28Z

I want to explain the answer/solution to this like I am explaining to a 5-year-old , so everyone can understand .

I have an app.I want people to register with their email,password and phone number . In my MongoDB database , I want to identify people uniquely based on both their phone numbers and email - so this means that both the phone number and the email must be unique for every person.

However , there is a problem : I have realized that everyone has a phonenumber but not everyone has an email address .

Those that don`t have an email address have promised me that they will have an email address by next week. But I want them registered anyway - so I tell them to proceed registering their phonenumbers as they leave the email-input-field empty .

They do so .

My database NEEDS an unique email address field - but I have a lot of people with 'null' as their email address . So I go to my code and tell my database schema to allow empty/null email address fields which I will later fill in with email unique addresses when the people who promised to add their emails to their profiles next week .

So its now a win-win for everyone (but you ;-] ): the people register, I am happy to have their data ...and my database is happy because it is being used nicely ...but what about you ? I am yet to give you the code that made the schema .

Here is the code : NOTE : The sparse property in email , is what tells my database to allow null values which will later be filled with unique values .

var userSchema = new mongoose.Schema({ local: { name: { type: String }, email : { type: String, require: true, index:true, unique:true,sparse:true}, password: { type: String, require:true }, }, facebook: { id : { type: String }, token : { type: String }, email : { type: String }, name : { type: String } } }); var User = mongoose.model('User',userSchema); module.exports = User;

I hope I have explained it nicely . Happy NodeJS coding / hacking!

if you want a sparse index and unique validation, and have other fields that are not required you'll need to use the partialFilterExpression option. See this answer stackoverflow.com/a/34600171/728287

bumsoverboard · Accepted Answer · 2020-03-20 09:12:53Z

49

In this situation, log in to Mongo find the index that you are not using anymore (in OP's case 'email'). Then select Drop Index

answered Mar 20, 2020 at 9:12

bumsoverboard

8918 silver badges9 bronze badges

3 Comments

xv8 Over a year ago

Thank you for this graphical solution to the problem! Worked perfectly.

Roshan Wickramaarachchi Over a year ago

thanks, it works perfectly. this is the only solution I have found

Dulal Sandip Over a year ago

Thank you this worked me.I just did drop index for email and phone in my case.It worked for me

Yegor · Accepted Answer · 2015-06-24 02:35:55Z

Check collection indexes.

I had that issue due to outdated indexes in collection for fields, which should be stored by different new path.

Mongoose adds index, when you specify field as unique.

Humble Cucumber · Accepted Answer · 2023-02-07 22:54:44Z

Edit: This solution still works in 2023 and you don't need to drop your collection or lose any data.

Here's how I solved same issue in September 2020. There is a super-fast and easy way from the mongodb atlas (cloud and desktop). Probably it was not that easy before? That is why I feel like I should write this answer in 2020.

First of all, I read above some suggestions of changing the field "unique" on the mongoose schema. If you came up with this error I assume you already changed your schema, but despite of that you got a 500 as your response, and notice this: specifying duplicated KEY!. If the problem was caused by schema configuration and assuming you have configurated a decent middleware to log mongo errors the response would be a 400.

Why this happens (at least the main reason)

Why is that? In my case was simple, that field on the schema it used to accept only unique values but I just changed it to accept repeated values. Mongodb creates indexes for fields with unique values in order to retrieve the data faster, so on the past mongo created that index for that field, and so even after setting "unique" property as "false" on schema, mongodb was still using that index, and treating it as it had to be unique.

How to solve it

Dropping that index. You can do it in 2 seconds from Mongo Atlas or executing it as a command on mongo shell. For the sack of simplicity I will show the first one for users that are not using mongo shell.

Go to your collection. By default you are on "Find" tab. Just select the next one on the right: "Indexes". You will see how there is still an index given to the same field is causing you trouble. Just click the button "Drop Index". Done.

So don't drop your database everytime this happens

I believe this is a better option than just dropping your entire database or even collection. Basically because this is why it works after dropping the entire collection. Because mongo is not going to set an index for that field if your first entry is using your new schema with "unique: false".

My problem was, i changed my schema and removed that field that was required, and every time i tried to insert anything it kept showing up. Thanks for the info.

Daksh M. · Accepted Answer · 2019-02-08 13:16:19Z

Well basically this error is saying, that you had a unique index on a particular field for example: "email_address", so mongodb expects unique email address value for each document in the collection.

So let's say, earlier in your schema the unique index was not defined, and then you signed up 2 users with the same email address or with no email address (null value).

Later, you saw that there was a mistake. so you try to correct it by adding a unique index to the schema. But your collection already has duplicates, so the error message says that you can't insert a duplicate value again.

You essentially have three options:

Drop the collection

db.users.drop();
Find the document which has that value and delete it. Let's say the value was null, you can delete it using:

db.users.remove({ email_address: null });
Drop the Unique index:

db.users.dropIndex(indexName)

I Hope this helped :)

mad Man · Accepted Answer · 2016-11-07 14:23:33Z

I faced similar issues , I Just clear the Indexes of particular fields then its works for me . https://docs.mongodb.com/v3.2/reference/method/db.collection.dropIndexes/

Kobe Luo · Accepted Answer · 2015-05-21 09:38:36Z

This is my relavant experience:

In 'User' schema, I set 'name' as unique key and then ran some execution, which I think had set up the database structure.

Then I changed the unique key as 'username', and no longer passed 'name' value when I saved data to database. So the mongodb may automatically set the 'name' value of new record as null which is duplicate key. I tried the set 'name' key as not unique key {name: {unique: false, type: String}} in 'User' schema in order to override original setting. However, it did not work.

At last, I made my own solution:

Just set a random key value that will not likely be duplicate to 'name' key when you save your data record. Simply Math method '' + Math.random() + Math.random() makes a random string.

I don't think this is a valid solution, your just masking the problem.

Sandip Subedi · Accepted Answer · 2016-12-28 05:02:10Z

I had the same issue. Tried debugging different ways couldn't figure out. I tried dropping the collection and it worked fine after that. Although this is not a good solution if your collection has many documents. But if you are in the early state of development try dropping the collection.

db.users.drop();

Ignatius Andrew · Accepted Answer · 2016-08-16 19:18:33Z

This is because there is already a collection with the same name with configuration..Just remove the collection from your mongodb through mongo shell and try again.

db.collectionName.remove()

now run your application it should work

Alamin · Accepted Answer · 2021-04-08 07:20:29Z

I have solved my problem by this way.

Just go in your mongoDB account -> Atlast collection then drop your database column. Or go mongoDB compass then drop your database,

It happed sometimes when you have save something null inside database.

FYI - you don't need to drop the db. you can go to Compass, then to the collection, then to the indexes tab and you'll be able to just remove that index

halfer · Accepted Answer · 2016-10-02 17:15:48Z

I got this same issue when I had the following configuration in my config/models.js

module.exports.models = { connection: 'mongodb', migrate: 'alter' }

Changing migrate from 'alter' to 'safe' fixed it for me.

module.exports.models = { connection: 'mongodb', migrate: 'safe' }

Christian Hartmann · Accepted Answer · 2016-10-06 13:29:41Z

same issue after removing properties from a schema after first building some indexes on saving. removing property from schema leads to an null value for a non existing property, that still had an index. dropping index or starting with a new collection from scratch helps here.

note: the error message will lead you in that case. it has a path, that does not exist anymore. im my case the old path was ...$uuid_1 (this is an index!), but the new one is ....*priv.uuid_1

Embedded_Mugs · Accepted Answer · 2018-10-29 14:02:39Z

I had a similar problem and I realized that by default mongo only supports one schema per collection. Either store your new schema in a different collection or delete the existing documents with the incompatible schema within the your current collection. Or find a way to have more than one schema per collection.

Inamur Rahman · Accepted Answer · 2019-04-11 05:35:02Z

I have also faced this issue and I solved it. This error shows that email is already present here. So you just need to remove this line from your Model for email attribute.

unique: true

This might be possible that even if it won't work. So just need to delete the collection from your MongoDB and restart your server.

As Md Habibullah · Accepted Answer · 2020-07-02 10:01:46Z

It's not a big issue but beginner level developers as like me, we things what kind of error is this and finally we weast huge time for solve it.

Actually if you delete the db and create the db once again and after try to create the collection then it's will be work properly.

➜ mongo use dbName; db.dropDatabase(); exit

Dharman · Accepted Answer · 2020-09-01 13:59:38Z

Drop you database, then it will work.

You can perform the following steps to drop your database

step 1 : Go to mongodb installation directory, default dir is "C:\Program Files\MongoDB\Server\4.2\bin"

step 2 : Start mongod.exe directly or using command prompt and minimize it.

step 3 : Start mongo.exe directly or using command prompt and run the following command

i) use yourDatabaseName (use show databases if you don't remember database name)

ii) db.dropDatabase()

This will remove your database. Now you can insert your data, it won't show error, it will automatically add database and collection.

Isaacs Katongole · Accepted Answer · 2023-03-16 15:51:23Z

I have the same issue however I discovered having set the email to be unique i.e email : { type: String, require: true, unique: true }, means I have to have a different email on every entry.

Mykola Berezhniuk · Accepted Answer · 2024-08-02 10:08:38Z

This is related with indexes in Mobgo DB, which you created previously. If you don't want to drop your collection you can check and remove some indexes withe the help of MongoDB Compass

Rahul P · Accepted Answer · 2017-02-17 13:26:53Z

I had the same issue when i tried to modify the schema defined using mangoose. I think the issue is due to the reason that there are some underlying process done when creating a collection like describing the indices which are hidden from the user(at least in my case).So the best solution i found was to drop the entire collection and start again.

John Conde · Accepted Answer · 2021-03-03 01:19:28Z

If you are in the early stages of development: Eliminate the collection. Otherwise: add this to each attribute that gives you error (Note: my English is not good, but I try to explain it)

index:true, unique:true, sparse:true

sparse is the true winner here. this should be upvoted much more than anything else.

tariel · Accepted Answer · 2022-07-12 09:14:26Z

in my case, i just forgot to return res.status(400) after finding that user with req.email already exists

Naveen Kamath · Accepted Answer · 2022-08-24 09:34:38Z

0

Go to your database and click on that particular collection and delete all the indexes except id.

answered Aug 24, 2022 at 9:34

Naveen Kamath

671 silver badge6 bronze badges

1 Comment

Humble Cucumber Over a year ago

So you have a problem with one duplicated field and you drop ALL indexes? Plus you are potentially making your queries slower now.

CoderC · Accepted Answer · 2023-11-29 11:47:28Z

Problem is that you have same values null in mongodb database for different Documents. If you can give different values to different documents then this problem will not come.

I was facing this error when i was using passport.js and want to add google auth in my project and I solved it !

How ???

See here everyone in above answer is saying just drop that collection from mongodb, But it is not the final solution of this problem. After Droping collection you will get that error again after some time.

Try to remove that field from the document of mogodb, I mean remove from schema of your collection.

If that field is important then try to write your document in this way. Let's take a example :

const userSchema = mongoose.Schema({ email:{ Type:String, unique:true, //Dont write this line Or Remove if this exists in code }, //Write Other Fields which you want to add .... });

Read More About It: Source MongoDB Official

And Other way is Just give some values to this email field in document so that it can store in mongodb and it will not generate problem for you. Before Storing Data in mongodb Try to give some unique value or just some value in email field. Be Ensure that you are not using unique:true option in Schema.

Collectives™ on Stack Overflow

E11000 duplicate key error index in mongodb mongoose

25 Answers 25

8 Comments

7 Comments

1 Comment

3 Comments

Comments

Edit: This solution still works in 2023 and you don't need to drop your collection or lose any data.

Why this happens (at least the main reason)

How to solve it

So don't drop your database everytime this happens

2 Comments

Comments

Comments

2 Comments

Comments

Comments

1 Comment

Comments

Comments

Comments

Comments

Comments

Comments

Comments

Comments

Comments

1 Comment

Comments

1 Comment

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

25 Answers 25

8 Comments

7 Comments

1 Comment

3 Comments

Comments

Edit: This solution still works in 2023 and you don't need to drop your collection or lose any data.

Why this happens (at least the main reason)

How to solve it

So don't drop your database everytime this happens

2 Comments

Comments

Comments

2 Comments

Comments

Comments

1 Comment

Comments

Comments

Comments

Comments

Comments

Comments

Comments

Comments

Comments

1 Comment

Comments

1 Comment

Comments

Linked

Related