Launching soon: The world's first vendor agnostic bug scrubLearn more & join waitlist
OPERATIONAL DEFECT DATABASE
...
...
It appears that if a 2.4 DB which was backed up using MongoDump is then restored via MongoRestore into 2.6 and that dump contains an index which is too large to index, then the restore stops and leaves the DB incompleted. Notice how the last line from MongoRestore is Error creating index LearningLine.Conversation: 17282 err: "Btree::insert: key too large to index, failing LearningLine.Conversation.$find_messages_by_activity 1426 { : { Attachments: [], LastAttachmentId: 0, C..." This is NOT the final collection to restore.
dan@10gen.com commented on Wed, 20 Jan 2016 22:11:52 +0000: Pretty sure you've found a bug in the upgradeChecker in 3.2.1. Can you use the 2.6 mongo shell and try again. Also, upgradeCheckAllDBs() is just a convenience method to do the iteration for you. dblock commented on Wed, 20 Jan 2016 21:47:19 +0000: Running 3.2.1 shell against a 2.4 MongoDB. $ mongo ... MongoDB shell version: 3.2.1 connecting to: ... > db.upgradeCheck() database 'gravity-staging' for 2.6 upgrade compatibility ... Everything in 'gravity-staging' is ready for the upgrade! ^^^ lies I cannot run upgradeCheckAllDBs() because this is a Compose deployment, I don't have admin access. I assume that it just iterates over and does upgradeCheck? dan@10gen.com commented on Wed, 20 Jan 2016 21:33:08 +0000: What exactly are you running for upgradeCheck? What version of the shell and mongodb are you using? Are you following this documentation: https://docs.mongodb.org/manual/reference/method/db.upgradeCheckAllDBs? dblock commented on Wed, 20 Jan 2016 21:14:40 +0000: Finding these has been a painful trial/error process, mongodump, mongorestore, restart once the next one fails. db.upgradeCheck() returned nothing The error from the index creation from mongoimport just leaves you guessing, hunting for the document ID that causes the problem. It would be nice it it at least returned the ID of the document - the one below for example doesn't get found with a regex because it's not in the index (partial index)? Failed: gravity-staging.events: error creating indexes for gravity-staging.events: createIndex error: Btree::insert: key too large to index, failing gravity-staging.events.$name_1 6590 { : "Activities New York / Description Soonja Han's Paris Studio gets reconstructed within the de Buck gallery in Chelsea, NY: An interrogation of the ar..." } dan@10gen.com commented on Wed, 20 Jan 2016 20:52:05 +0000: Hi DB, Sorry you hit this, but as you can see this repairs a major correctness issue. Starting with 2.6, if the index key is too long, the document is rejected. The change is described in this ticket: SERVER-5290. If the run parameter failIndexKeyTooLong is set to false, the document will be inserted, but the index entry will not be created. This is documented here: https://docs.mongodb.org/manual/reference/parameters/#param.failIndexKeyTooLong dblock commented on Wed, 20 Jan 2016 05:13:46 +0000: FWIW I spent the majority of the afternoon working around this error, collection after collection. We query data by the problematic indexe(s) so this restriction and the inability to ignore and continue came as a surprise on a mongorestore. Being able to run the server in an old(er) mode is fine, but we don't run our own, I would have liked to get (or be able to get) a warning instead of a hard error here. Related, I couldn't find in the documentation anything about the semantics of inserting a record when the index key produced would be too long - does the record get inserted or not? mkennedy66996693 commented on Mon, 3 Mar 2014 15:58:29 +0000: Hi Dan, Yes, that is what my understanding was. I filed the bug because I think this is wrong. It's fine for runtime. But I want a flag or something for *mongorestore* which says something like 'continue on index error' or 'drop indexes that are incompatible with 2.6' or something. I do not want to run the server is a downgraded mode. I just want to import the data, acknowledging the problems and changes. dan@10gen.com commented on Mon, 3 Mar 2014 05:19:39 +0000: With the new restrictions on index key length, you have a couple options to restore databases with such documents. 1. restore with a 2.4 server, then restart with 2.6 binary. 2. run with server with the failIndexKeyTooLong set to false to override the new behavior. In either case, it is highly recommended that the documents are cleaned up to avoid condition if possible as results from indexes will not be usable for docs with this keys that are too long. mkennedy66996693 commented on Sun, 2 Mar 2014 04:43:03 +0000: Thanks for looking at this Dan. I'm not sure I would classify it as duplicate. I think there is a difference between attempting a DB restore and attempting to create an improper index and runtime or doing an insert which is invalid. I don't want to run the server in a degraded mode, I just want to restore data and acknowledge any index errors (but continue) dan@10gen.com commented on Sun, 2 Mar 2014 00:02:02 +0000: duplicate of SERVER-12982.
Use this command and see screenshot. mongorestore --drop --db DB ".\DB" echo done