Info

​​Issue Status as of Nov 15, 2022 ISSUE DESCRIPTION AND IMPACT In MongoDB 4.4 and earlier, TTL indexes with an expireAfterSeconds value of NaN (not-a-number) were functional as indexes for queries but did not expire any documents. When such a config is in place, the following error is logged by MongoDB 4.2 and 4.4 primary replica set nodes: 2022-08-05T21:00:00.000+0000 E INDEX [TTLMonitor] Error processing ttl index: { v: 2, key: { created: 1 }, name: "created_1", ns: "dbname.collname", expireAfterSeconds: nan.0, background: true } -- DurationOverflow: Cannot negate the minimum duration In MongoDB 5.0.0-5.0.13 and 6.0.0-6.0.1, as part of SERVER-56676, TTL indexes began treating NaN as 0, and the listIndexes command necessary for initial syncs and mongodump backups began returning 0 instead of NaN. Because of this change in unspecified behavior, when a TTL index with this improper configuration exists, the following can trigger the sudden expiration of TTL-indexed documents in a collection: On MongoDB 4.4, when: upgrading to MongoDB 5.0.0-5.0.13 initially syncing from a 5.0.0-5.0.13 or 6.0.0-6.0.1 node On MongoDB 4.2, when initially syncing from a 5.0.0-5.0.13 or 6.0.0-6.0.1 node On MongoDB 5.0.0-5.0.13 or 6.0.0-6.0.1, When restoring from a mongodump of a 4.2 or 4.4 collection that has a TTL configured with expireAfterSeconds: NaN When initially syncing from a version 4.2 or 4.4 node that has a TTL configured with expireAfterSeconds: NaN Now that this issue is addressed, the following case may still present concern: On MongoDB 4.2 and 4.4, when initially syncing from a 5.0.14+ or 6.0.2+ node that still has a TTL index with an invalid config, a newly synced node will end up with a system-dependent TTL configuration of expireAfterSeconds:. Use the collMod command to set a reasonable value, or remove the TTL index. WORKAROUNDS AND REMEDIATION In general, avoid this issue by avoiding expireAfterSeconds: NaN as a configuration and correct this config anywhere it exists. The following script for the mongosh shell reports any TTL indexes with an expireAfterSeconds: NaN configuration: Note: Do not use the legacy mongo shell for this operation. function getNaNIndexes() { const nan_idx = []; const dbs = db.adminCommand({ listDatabases: 1 }).databases; dbs.forEach((d) => { const listCollCursor = db .getSiblingDB(d.name) .runCommand({ listCollections: 1 }).cursor; const collDetails = { db: listCollCursor.ns.split(".$cmd")[0], colls: listCollCursor.firstBatch.map((c) => c.name), }; collDetails.colls.forEach((c) => db .getSiblingDB(collDetails.db) .getCollection(c) .getIndexes() .forEach((entry) => { if (Object.is(entry.expireAfterSeconds, NaN)) { nan_idx.push({ ns: `${collDetails.db}.${c}`, index: entry }); } }) ); }); return nan_idx; }; getNaNIndexes(); Once identified, correct any TTL indexes with the expireAfterSeconds: NaN configuration and establish an unambiguous, valid configuration with a specified behavior. The collMod command allows you to modify the expireAfterSeconds value for an existing index. MongoDB intends to help protect against this behavior change by: In this ticket, SERVER-68477, MongoDB versions 5.0.14+ and 6.0.2+ now render badly configured TTL indexes ineffective, rather than applying a meaning of expireAfterSeconds: 0 to these indexes. Releasing SERVER-68522 in MongoDB 5.0.11. With this fix, MongoDB 5.0.11+ will refuse to start if a TTL index with expireAfterSeconds: NaN exists, to ensure that the normal upgrade path from 4.4 is protected from this unexpected change in behavior. See SERVER-68522 for additional details. Original description Currently listIndexes, and subsequently initial sync, do not properly handle NaN values for expireAfterSeconds. This can result in unexpected TTL behavior, especially when upgrading from MongoDB 4.4 to MongoDB 5.0 or when migrating earlier index definitions to MongoDB 5.0 or 6.0.

Top User Comments

xgen-internal-githook commented on Wed, 31 Aug 2022 10:59:42 +0000: Author: {'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'} Message: SERVER-68477 add unit test for TTLCollectionCache (cherry picked from commit f32f550aff1def5709ea68f42dc9d4bf9321d5e6) Branch: v6.1 https://github.com/mongodb/mongo/commit/559d9f496e6e3ee5262e64cf3d00f5f741239d4b xgen-internal-githook commented on Tue, 30 Aug 2022 00:30:02 +0000: Author: {'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'} Message: SERVER-68477 createIndexes overwrites NaN expireAfterSeconds before starting index build Branch: master https://github.com/mongodb/mongo/commit/58796facf40c99ddf8bb537adf563dc43aa0a863 xgen-internal-githook commented on Mon, 29 Aug 2022 12:52:34 +0000: Author: {'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'} Message: SERVER-68477 TTLMonitor fixes NaN expireAfterSeconds on TTL indexes during step up Branch: master https://github.com/mongodb/mongo/commit/b8eb75f99a18a8651e2315998cf16f53e48917bb xgen-internal-githook commented on Mon, 29 Aug 2022 01:01:20 +0000: Author: {'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'} Message: SERVER-68477 remove epoch restriction on ttl indexes Branch: master https://github.com/mongodb/mongo/commit/eb2f7f03f8c0522f85a9cae2c61bec4673251103 xgen-internal-githook commented on Sun, 28 Aug 2022 13:32:33 +0000: Author: {'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'} Message: SERVER-68477 listIndexes repairs TTL indexes with NaN expireAfterSeconds Branch: master https://github.com/mongodb/mongo/commit/d6528bf96f08b79ca850902b2d1d81264fa7baa1 xgen-internal-githook commented on Sat, 27 Aug 2022 12:28:30 +0000: Author: {'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'} Message: SERVER-68477 TTL monitor skips indexes with NaN expireAfterSeconds (cherry-picked from commit 14c07b53186ee88497abd0ba492d2d9157891718) Branch: master https://github.com/mongodb/mongo/commit/7d76329f8915ec80ba07c07e4a0adbc7fd187107 xgen-internal-githook commented on Fri, 26 Aug 2022 19:29:22 +0000: Author: {'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'} Message: SERVER-68477 add startup warning for TTL indexes with NaN 'expireAfterSeconds' (partially cherry-picked from commit abdedd367e2f331816354563f8ae95f6bb35c51d) Branch: master https://github.com/mongodb/mongo/commit/5d21e0115edb120873c8beac2901e03a76ccfc00 xgen-internal-githook commented on Fri, 26 Aug 2022 14:39:39 +0000: Author: {'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'} Message: SERVER-68477 add TTLMonitor::onStepUp() Branch: master https://github.com/mongodb/mongo/commit/3b17e5f7916fdf1a21dd162a079aeb9d06ca9425 xgen-internal-githook commented on Fri, 26 Aug 2022 13:07:04 +0000: Author: {'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'} Message: SERVER-68477 include 'expireAfterSeconds' type information when registering TTL indexes with the TTLCollectionCache Branch: master https://github.com/mongodb/mongo/commit/cc3ae631bce7943fbda5182ff3b9d93d1125be40 xgen-internal-githook commented on Thu, 25 Aug 2022 19:17:40 +0000: Author: {'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'} Message: SERVER-68477 add unit test for TTLCollectionCache Branch: master https://github.com/mongodb/mongo/commit/f32f550aff1def5709ea68f42dc9d4bf9321d5e6

Steps to Reproduce