Info

tl;dr: There is no consistent way to upsert an identifier-only document between 2.4.x and 2.5.x. Allowing modifiers in a newObj to be an empty BSON object would help. I noticed the following while testing Doctrine ODM against 2.5.x. In 2.4.x and earlier versions, I believe there was only a single way to upsert a document that only contained an identifier field: > db.foo.update({_id:1}, {$set: {}}, true); db.getLastErrorObj(); { "updatedExisting" : false, "n" : 1, "connectionId" : 1, "err" : null, "ok" : 1 } > db.foo.update({_id:2}, {}, true); db.getLastErrorObj(); { "updatedExisting" : false, "upserted" : ObjectId("52cb12553ad84dc22b80c72f"), "n" : 1, "connectionId" : 1, "err" : null, "ok" : 1 } > db.foo.update({_id:3}, {$set: {_id:3}}, true); db.getLastErrorObj(); { "err" : "Mod on _id not allowed", "code" : 10148, "n" : 0, "connectionId" : 1, "ok" : 1 } > db.foo.find() { "_id" : 1 } { "_id" : ObjectId("52cb12553ad84dc22b80c72f") } Using an empty object for the newObj argument results in the upsert ignoring the client-provided _id. $set cannot be used on _id, even if that would technically be OK for an upsert. $setOnInsert would make more sense, but it also doesn't work – it's also 2.4+ only, so I wouldn't rely on it for essential ODM logic. In 2.5.x, the one working method from 2.4.x no longer works. The two methods that didn't work in 2.4.x do work in 2.5.x: > db.foo.update({_id:1}, {$set: {}}, true); db.getLastErrorObj(); { "err" : "'$set' is empty. You must specify a field like so: {$mod: {: ...}}", "code" : 16840, "n" : 0, "connectionId" : 2, "ok" : 1 } > db.foo.update({_id:2}, {}, true); db.getLastErrorObj(); { "updatedExisting" : false, "upserted" : 2, "n" : 1, "connectionId" : 2, "syncMillis" : 0, "writtenTo" : null, "err" : null, "ok" : 1 } > db.foo.update({_id:3}, {$set: {_id:3}}, true); db.getLastErrorObj(); { "updatedExisting" : false, "upserted" : 3, "n" : 1, "connectionId" : 2, "syncMillis" : 0, "writtenTo" : null, "err" : null, "ok" : 1 } > db.foo.find() { "_id" : 2 } { "_id" : 3 } The strict validation that makes the 2.4.x solution no longer work looks to have been introduced in this commit for SERVER-7175.

Top User Comments

jacobgh111 commented on Fri, 18 Jul 2014 14:03:21 +0000: I would also like to chime in about the harm that this change has made. +1 to Glenn Maynard's explanation above, which explains a code flow that appears all over my code. Furthermore, with the default write concern the way it is (which I think is new), update exceptions are not explicitly reported back to the system. As a result, data can be mysteriously lost for no apparent reason, which is what I've discovered using the PyMongo drivers. This is dangerous. Please revert this change. glenn commented on Fri, 20 Jun 2014 17:34:10 +0000: Rejecting empty modifiers is inconsistent, introduces bugs, and creates busywork for users to work around this strange behavior, as evidenced above as people struggle to find workarounds. > I spoke to Scott Hernandez about this today, and he explained the new strictness around empty modifiers is intended to alert users that were inadvertently sending over empty updates. This is a reason to add a warning that you can turn off, not for making it an error. Making it an error is forcing premature optimization on users. This has broken a common pattern that I use all the time in my code: sets = {} unset = {} if x_update_required(): sets["x"] = "y" if y_needs_removal(): unset["y"] = true ... db.update(query, {$set: sets, $unset: unset}) This regularly sends empty modifiers, which is intentional and harmless. Sometimes it sends nothing but empty modifiers, which is also intentional, because the update is often a findAndModify where the result is wanted even if nothing has changed. This change is harmful and narrow-sighted. Please revert it. valeri.karpov commented on Fri, 6 Jun 2014 21:07:48 +0000: Great suggestion Nathan, I think I'll use that. Guess I can work around this by including $unset with a reserved keyword. boutell commented on Mon, 19 May 2014 20:45:00 +0000: This is so MySQL (: I used to appreciate Mongo's willingness not to give me a hard time for asking to fetch any of zero things, or to update nothing. Mongo can optimize that away in one place, why make a thousand query builders do it? nathan@expectedbehavior.com commented on Wed, 14 May 2014 18:36:46 +0000: We have also encountered this issue, and our 'workaround' was to include $unset: _this_field_does_not_exist_ever_ if the $set command is empty. Very annoying behavior change, especially coupled with the changes in the way $set works with _id. valeri.karpov commented on Wed, 7 May 2014 19:25:05 +0000: My conclusions from having wrangled with this for the last couple days in the context of findAndModify: Scott's proposed solution would clobber any existing document, so wouldn't work. The idea of copying the query parameters into `$set` is great, except for 2.2 and 2.4 will throw an error if `$set` attempts to set `_id`. This puts you between a rock and a hard place if you want to support 2.2, 2.4, and 2.6. What mongoose will probably do for now is copy query parameters minus `_id` into `$set`, which will work as expected on 2.2 and 2.4, but leave an unfortunate corner case on 2.6 where if the query parameter to findAndModify only has `_id` and there is nothing to update the findAndModify will fail. If this particular server behavior isn't going to change in the future (although IMO it absolutely should) I can work it into mongoose's API in the next major release. jmikola@gmail.com commented on Wed, 22 Jan 2014 18:52:17 +0000: I spoke to scotthernandez about this today, and he explained the new strictness around empty modifiers is intended to alert users that were inadvertently sending over empty updates. We discussed the server possibly having some sort of merge functionality (like save, but setting fields instead of overwriting the entire document) down the line, but for now, I will have to attempt a pre-2.6 or 2.6+ update and then fall back on error (unless I can detect the server version up front). For 2.6+, "{$set: {_id:3}}" will be the preferred method. jmikola@gmail.com commented on Fri, 10 Jan 2014 14:44:06 +0000: This is the conditional in question: https://github.com/mongodb/mongo/blob/8b57f45b149b9df8ed2e9d7f74bf5d33a650cd5c/src/mongo/db/ops/update_driver.cpp#L115 I also spoke with justin.lee@10gen.com yesterday about Morphia and he said they use the second example ("{}" as the newObj argument), which actually disregards any custom identifier before 2.5.x. But since Morphia also tends to store the class name in another field by default, most users would never have such empty updates (with only _id). jmikola@gmail.com commented on Thu, 9 Jan 2014 06:31:47 +0000: I looked into that, but it opens the possibility of data loss. If a developer creates a document object, assigns their own identifier, and persists/flushes, our ODM opts to do an upsert. If the developer didn't set a specific identifier, the ODM will generate one upon persisting and opt to perform an insert upon flush. When a document's changeset contains only the identifier, we basically want "insert if not exists". That's essentially what the NOOP newObj argument accomplished in the issue description. Other discussions (here and here) suggest performing an insert and ignoring a possible unique index exception. Some additional context: https://github.com/doctrine/mongodb-odm/pull/750/files scotthernandez commented on Wed, 8 Jan 2014 23:03:32 +0000: You can use save/update with the doc as the query + update (w/upsert true). var idDoc = {_id:1}; db.save(idDoc); db.update(idDoc, idDoc, true);

Steps to Reproduce