...
After an invariant failure, the powercycle_replication_smalloplog task fails to find the core dump on the remote host. From the task log: [2018/07/18 10:42:23.228] Return code: 1 for command ['buildscripts/remote_operations.py', '--verbose', '--userHost', 'ubuntu@10.122.3.1', '--operation', 'copy_from', '--sshConnectionOptions', '-i /data/mci/df66a34b85dac21a0a2b25a4431a40f9/powercycle.pem -o GSSAPIAuthentication=no -o CheckHostIP=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectTimeout=10 -o ConnectionAttempts=20', '--retries', '10', '--file', '/log/powercycle/*.core', '--file', '/log/powercycle/*.mdmp'] [2018/07/18 10:42:23.228] scp: /log/powercycle/*.core: No such file or directory [2018/07/18 10:42:23.228] scp: /log/powercycle/*.mdmp: No such file or directory [2018/07/18 10:42:23.231] Command failed: command [pid=27775] encountered problem: exit status 1
xgen-internal-githook commented on Fri, 21 Sep 2018 16:45:30 +0000: Author: {'name': 'Jonathan Abrahams', 'email': 'jonathan@mongodb.com', 'username': 'hptabster'} Message: SERVER-36842 Trap error when printing informational message (cherry picked from commit 790cdfebf736064d3ab1368ce3a5c9bde71ef7c0) Branch: v3.6 https://github.com/mongodb/mongo/commit/ba151950b1faff07afb1302bf770569f4eb9c7d0 xgen-internal-githook commented on Fri, 21 Sep 2018 16:45:28 +0000: Author: {'name': 'Jonathan Abrahams', 'email': 'jonathan@mongodb.com', 'username': 'hptabster'} Message: SERVER-36842 Core dump not generated after invariant failure in powercycle hosts (cherry picked from commit 9f634fbb3b49e1a591f054b097caf862f192f0c2) Branch: v3.6 https://github.com/mongodb/mongo/commit/9947327e88bc43f187d235c44f79492dfb00bf0f xgen-internal-githook commented on Thu, 20 Sep 2018 21:39:23 +0000: Author: {'name': 'Jonathan Abrahams', 'email': 'jonathan@mongodb.com', 'username': 'hptabster'} Message: SERVER-36842 Trap error when printing informational message (cherry picked from commit 790cdfebf736064d3ab1368ce3a5c9bde71ef7c0) Branch: v4.0 https://github.com/mongodb/mongo/commit/693520e52214c72a12efd132d2274d4d32bc2f21 xgen-internal-githook commented on Thu, 20 Sep 2018 21:39:22 +0000: Author: {'name': 'Jonathan Abrahams', 'email': 'jonathan@mongodb.com', 'username': 'hptabster'} Message: SERVER-36842 Core dump not generated after invariant failure in powercycle hosts (cherry picked from commit 9f634fbb3b49e1a591f054b097caf862f192f0c2) Branch: v4.0 https://github.com/mongodb/mongo/commit/30e55781d8e4e2860b778b4b57c31323f5a774a5 xgen-internal-githook commented on Tue, 18 Sep 2018 12:52:13 +0000: Author: {'name': 'Jonathan Abrahams', 'email': 'jonathan@mongodb.com', 'username': 'hptabster'} Message: SERVER-36842 Trap error when printing informational message Branch: master https://github.com/mongodb/mongo/commit/790cdfebf736064d3ab1368ce3a5c9bde71ef7c0 xgen-internal-githook commented on Mon, 17 Sep 2018 19:35:15 +0000: Author: {'name': 'Jonathan Abrahams', 'email': 'jonathan@mongodb.com', 'username': 'hptabster'} Message: SERVER-36842 Core dump not generated after invariant failure in powercycle hosts Branch: master https://github.com/mongodb/mongo/commit/9f634fbb3b49e1a591f054b097caf862f192f0c2 jonathan.abrahams commented on Fri, 14 Sep 2018 15:05:16 +0000: The ulimit has to be set to unlimited on the spawned EC2 hosts.