Info

KVStorageEngine::reconcileCatalogAndIdents returns a list of (collection name, index name) pairs that need to be rebuilt. If a collection requires rebuilding multiple indexes, they will be separate entries in this list. This API suggests that rebuilding indexes one by one would work, but that's unfortunately not true.[1] There are two obvious ways to correct catalog::openCatalog rebuilding indexes one by one. Complete the TODO to enable building an index without the collection object. Have catalog::openCatalog group all the indexes to rebuild per collection before passing them to rebuildIndexesOnCollection. [1] More completely: creating the collection object will cascade and create all of the in-memory index objects for that collection. The WiredTigerIndex constructor will fail when searching for the underlying table. rebuildIndexesOnCollection avoids this situation by first dropping all indexes that are to be rebuilt from the storage engine's implementation of the collection catalog. Thus when the Collection object is constructed, it does not observe the indexes to be rebuilt. Thus this bug only manifests when there is more than one missing index to rebuild on a collection.

Top User Comments

siyuan.zhou@10gen.com commented on Wed, 14 Feb 2018 19:56:09 +0000: Since the commit got reverted, no BF was filed according to the BF procedure. Tracking the issue in SERVER-33322 is sufficient. Thanks for letting me know! CC the current Build Baron, adam.martin. kyle.suarez commented on Wed, 14 Feb 2018 14:51:52 +0000: Filed SERVER-33322 for the index building fix. kyle.suarez commented on Wed, 14 Feb 2018 14:13:18 +0000: daniel.gottlieb, looks like reconcileCatalogAndIdents() shouldn't include unready indexes in the list of indexes to rebuild when the --noIndexBuildRetry option is set: [js_test:index_no_retry] 2018-02-13T22:06:10.241+0000 2018-02-13T22:06:10.240+0000 E QUERY [thread1] Error: did not throw exception: index {a: 1} was rebuilt in spite of --noIndexBuildRetry : [js_test:index_no_retry] 2018-02-13T22:06:10.241+0000 doassert@src/mongo/shell/assert.js:18:14 [js_test:index_no_retry] 2018-02-13T22:06:10.242+0000 assert.throws@src/mongo/shell/assert.js:365:13 [js_test:index_no_retry] 2018-02-13T22:06:10.242+0000 @jstests/noPassthrough/index_no_retry.js:56:1 [js_test:index_no_retry] 2018-02-13T22:06:10.242+0000 @jstests/noPassthrough/index_no_retry.js:7:2 [js_test:index_no_retry] 2018-02-13T22:06:10.242+0000 failed to load: jstests/noPassthrough/index_no_retry.js If we want to fix this, should we just open a separate ticket to track the work, given that we won't revert the other commit (which is the main point of this ticket)? kyle.suarez commented on Wed, 14 Feb 2018 01:37:16 +0000: I'm glad I committed them separately siyuan.zhou, could you please link any build failures caused by that commit to this ticket? xgen-internal-githook commented on Wed, 14 Feb 2018 00:32:39 +0000: Author: {'email': 'siyuan.zhou@mongodb.com', 'name': 'Siyuan Zhou', 'username': 'visualzhou'} Message: Revert "SERVER-33093 include unready indexes for rebuilding when reconciling catalog" This reverts commit 91b0a8c599cae1a130f8450e37681b3a134b5dd6. Branch: master https://github.com/mongodb/mongo/commit/c4e444a62ce32eb37928396699fc7aaa0536226e xgen-internal-githook commented on Tue, 13 Feb 2018 21:32:03 +0000: Author: {'email': 'kyle.suarez@mongodb.com', 'name': 'Kyle Suarez', 'username': 'ksuarz'} Message: SERVER-33093 include unready indexes for rebuilding when reconciling catalog Branch: master https://github.com/mongodb/mongo/commit/91b0a8c599cae1a130f8450e37681b3a134b5dd6 xgen-internal-githook commented on Tue, 13 Feb 2018 21:32:02 +0000: Author: {'email': 'kyle.suarez@mongodb.com', 'name': 'Kyle Suarez', 'username': 'ksuarz'} Message: SERVER-33093 rebuild all indexes in one pass in openCatalog() Branch: master https://github.com/mongodb/mongo/commit/17e104e78f5682d4caf7a1ee39788201fd619ec0

Steps to Reproduce