Info

During an initial sync of a capped collection, the collection scan reads until it hits EOF. If records are inserted on the sync source during the collection scan, the scan may read more documents than the collection cap allows, which will result in records being deleted during the collection scan. The collection cloner for initial sync uses a MultiIndexBlock builder, which does not expect records to be deleted while it builds indexes. If records are deleted before the MultiIndexBlock commits at the end of the collection scan, it can leave dangling index records pointing to deleted records. On WiredTiger, this simply results in an error when the index records are scanned. On MMAPv1, the index records point to invalid record data, which may or may not produce a BSON parsing error that looks like this: 2017-05-15T09:45:48.697-0400 I - [conn3] Assertion: 10334:BSONObj size: -286331154 (0xEEEEEEEE) is invalid. Size must be between 0 and 16793600(16MB) First element: _id: ObjectId('5919b0b5f337b48af7a0be4d') src/mongo/bson/bsonobj.cpp 58 The collections in question will fail an index validation by the validate command. To repair this issue, run the reindex command.

Top User Comments

xgen-internal-githook commented on Wed, 14 Jun 2017 13:07:20 +0000: Author: {u'username': u'milkie', u'name': u'Eric Milkie', u'email': u'milkie@10gen.com'} Message: SERVER-29197 initial sync now builds capped indexes correctly In initial sync, capped indexes were built along with regular indexes, as documents were inserted, using the MultiIndexBlock. This structure requires that no records are deleted before committing the index builds. But the collection cap might delete records prior to commit. The new way to build capped indexes will be to create the indexes on a capped collection prior to inserting any records, for initial sync. (cherry picked from commit 3a3a6def395bab81e7545b15a93bee7799d9efb1) Branch: v3.4 https://github.com/mongodb/mongo/commit/34d292df70c97fadbac07296dea3cb04b6021a4d xgen-internal-githook commented on Wed, 31 May 2017 19:05:13 +0000: Author: {u'username': u'milkie', u'name': u'Eric Milkie', u'email': u'milkie@10gen.com'} Message: SERVER-29197 initial sync now builds capped indexes correctly In initial sync, capped indexes were built along with regular indexes, as documents were inserted, using the MultiIndexBlock. This structure requires that no records are deleted before committing the index builds. But the collection cap might delete records prior to commit. The new way to build capped indexes will be to create the indexes on a capped collection prior to inserting any records, for initial sync. Branch: master https://github.com/mongodb/mongo/commit/3a3a6def395bab81e7545b15a93bee7799d9efb1

Steps to Reproduce