OPERATIONAL DEFECT DATABASE
...
...
Got a value of -1 for the currentActive transactions when running serverStatus against a replica set secondary. "transactions" : { "retriedCommandsCount" : NumberLong(0), "retriedStatementsCount" : NumberLong(0), "transactionsCollectionWriteCount" : NumberLong(0), "currentActive" : NumberLong(-1), "currentInactive" : NumberLong(1), "currentOpen" : NumberLong(0), "totalAborted" : NumberLong(1), "totalCommitted" : NumberLong(0), "totalStarted" : NumberLong(1) }
xgen-internal-githook commented on Fri, 21 Dec 2018 15:10:13 +0000: Author: {'username': 'tessavitabile', 'email': 'tess.avitabile@mongodb.com', 'name': 'Tess Avitabile'} Message: SERVER-37189 Record metrics correctly for transactions that fail prior to unstash (cherry picked from commit 06dd323b423deccb5fbb650ddbed0135c00191d2) Branch: v4.0 https://github.com/mongodb/mongo/commit/0bf668f3336ce9fd7da430a788405fca47158416 xgen-internal-githook commented on Thu, 15 Nov 2018 15:44:24 +0000: Author: {'name': 'Tess Avitabile', 'email': 'tess.avitabile@mongodb.com', 'username': 'tessavitabile'} Message: SERVER-37189 Record metrics correctly for transactions that fail prior to unstash Branch: master https://github.com/mongodb/mongo/commit/06dd323b423deccb5fbb650ddbed0135c00191d2 william.schultz commented on Wed, 19 Sep 2018 21:40:54 +0000: Yep, looks like there is an issue here with how we update internal transaction metadata when the first command of a transaction fails against a secondary. On secondaries, we don't actually fail the transaction until the first command is executed. Before this, we treat the transaction as if it was started inside TransactionParticipant::_beginMultiDocumentTransaction. This would increase the currentActive count to 1. When the command invocation fails with NotMaster, we don't actually throw an exception, but return an ok:0 response. This means that we don't try to abort the transaction here. So, even though the transaction appears to fave "failed" to begin, the internal transaction metadata indicates that there is a transaction in-progress, and the metrics counters reflect that. I need to investigate a bit more to determine the solution to this, but we may need a better way of failing transactions early if they are run against secondaries. I attached an additional repro that demonstrates the issue, against master branch as well. isabella.siu commented on Wed, 19 Sep 2018 15:24:55 +0000: william.schultz I do not have test commands enabled, and it does not occur if it is run against a primary. bruce.lucas@10gen.com commented on Wed, 19 Sep 2018 15:20:31 +0000: Working from her repro, problem occurs in 4.0.2 as well does only occur on secondary --setParameter enableTestCommands=1 does not seem to make a difference Note that even though this only occurs on a secondary where transactions are not allowed, if that secondary were then promoted to primary I assume the accounting error would persist. william.schultz commented on Tue, 18 Sep 2018 21:49:16 +0000: isabella.siu In 4.0, we explicitly disallow transactions on replica set secondaries. See SERVER-33580 and SERVER-34557. Does this issue still occur if the transaction is run against a primary? Note: Transactions will be allowed to run against secondaries if test commands are enabled, so it might be worth checking that in your setup. isabella.siu commented on Tue, 18 Sep 2018 17:28:05 +0000: william.schultz
on a secondary: sessionOptions = {causalConsistency: false} sess = db.getMongo().startSession(sessionOptions) sess.startTransaction() db.serverStatus({ repl: 0, metrics: 0, locks: 0, wiredTiger: 0 }) db.abc.insert({_id:213}) sess.getDatabase("test").abc.insert({_id: "insert-4"}) // this had a "TransientTransactionError" db.serverStatus({ repl: 0, metrics: 0, locks: 0, wiredTiger: 0 }) sess.getDatabase("test").abc.insert({_id: "insert-4"}) // this had a "TransientTransactionError" db.serverStatus({ repl: 0, metrics: 0, locks: 0, wiredTiger: 0 })
MongoDB Integration
Learn more about where this data comes from
Bug Scrub Advisor
Streamline upgrades with automated vendor bug scrubs
BugZero Enterprise
Wish you caught this bug sooner? Get proactive today.
