Info
Quiesce mode throws ShutdownInProgress, so in SERVER-48492 we made the reconfig hook (run_reconfig_background.js) ignore ShutdownInProgress errors. However, we've still observed a failure, on commit ca65f33c on master, on this line in run_reconfig_background.js:
const conn = connect(TestData.connectionString);
The script logs:
connecting to: mongodb://localhost:20500,localhost:20501,localhost:20502,localhost:20503,localhost:20504/?replicaSet=rs
Error: can't connect to new replica set primary [localhost:20500], err: The server is in quiesce mode and will shut down :
connect@src/mongo/shell/mongo.js:372:17
@jstests/hooks/run_reconfig_background.js:121:18
@jstests/hooks/run_reconfig_background.js:11:2
failed to load: jstests/hooks/run_reconfig_background.js
I hypothesize that the "connect" line threw an error with a code that is not ShutdownInProgress.
Indeed, the client-side code that threw this error replaces the code with FailedToSatisfyReadPreference, in dbclient_rs.cpp:
const std::string message = str::stream()
<< "can't connect to new replica set primary [" << _primaryHost.toString() << "]"
<< (errmsg.empty() ? "" : ", err: ") << errmsg;
monitor->failedHost(_primaryHost, {ErrorCodes::Error(40659), message});
uasserted(ErrorCodes::FailedToSatisfyReadPreference, message);
I think the reconfig hook should ignore error messages like "can't connect to new replica set primary" in addition to its current error-suppression rules.
Top User Comments
xgen-internal-githook commented on Mon, 2 Nov 2020 00:29:08 +0000:
Author:
{'name': 'A. Jesse Jiryu Davis', 'email': 'jesse@mongodb.com', 'username': 'ajdavis'}
Message: SERVER-51893 Ignore "can't connect to new replica set primary" in reconfig hook
Branch: master
https://github.com/mongodb/mongo/commit/2c8c742c516cbca9a8724eda610d3b01b0b3d62c
