Launching soon: The world's first vendor agnostic bug scrubLearn more & join waitlist
OPERATIONAL DEFECT DATABASE
...
...
Got a duplicate symbol error for some functions in WaitForMajorityServiceWaitForMajorityService. This is the SCONS command I had used (note that I did not regenerate Scons after git pulling): ./buildscripts/scons.py --variables-files=etc/scons/mongodbtoolchain_v3_clang.vars --ninja generate-ninja ICECC=icecc CCACHE=ccache And the Ninja command I invoked: ninja -j400 install-core I was at git commit 6c0ac0678f3c159c65c497acf155e964dec43f61 on branch master of mongodb/mongo. I've also attached my build.ninja file to the ticket. Here's the full error: FAILED: build/opt/mongo/mongod PATH=/opt/mongodbtoolchain/v3/bin:/usr/local/bin:/opt/bin:/bin:/usr/bin ICECC_CLANG_REMOTE_CPP=1 CCACHE_NOCPP2=1 ICECC_VERSION=/home/ubuntu/mongo/build/scons/icecc/opt_mongodbtoolchain_v3_bin_clang_opt_mongodbtoolchain_v3_bin_clang++.tar.gz CCACHE_PREFIX=/usr/bin/icecc /usr/bin/icerun /opt/mongodbtoolchain/v3/bin/clang++ @build/opt/mongo/mongod.rsp ld.lld: error: duplicate symbol: mongo::WaitForMajorityService::waitUntilMajority(mongo::repl::OpTime const&) >>> defined at wait_for_majority_service.cpp:101 (src/mongo/db/s/wait_for_majority_service.cpp:101) >>> wait_for_majority_service.o:(mongo::WaitForMajorityService::waitUntilMajority(mongo::repl::OpTime const&)) in archive build/opt/mongo/db/s/libtransaction_coordinator.a >>> defined at wait_for_majority_service.cpp:101 (src/mongo/db/repl/wait_for_majority_service.cpp:101) >>> wait_for_majority_service.o:(.text+0x510) in archive build/opt/mongo/db/repl/libwait_for_majority_service.ald.lld: error: duplicate symbol: mongo::WaitForMajorityService::_periodicallyWaitForMajority(mongo::ServiceContext*) >>> defined at wait_for_majority_service.cpp:145 (src/mongo/db/s/wait_for_majority_service.cpp:145) >>> wait_for_majority_service.o:(mongo::WaitForMajorityService::_periodicallyWaitForMajority(mongo::ServiceContext*)) in archive build/opt/mongo/db/s/libtransaction_coordinator.a >>> defined at wait_for_majority_service.cpp:145 (src/mongo/db/repl/wait_for_majority_service.cpp:145) >>> wait_for_majority_service.o:(.text+0xA50) in archive build/opt/mongo/db/repl/libwait_for_majority_service.ald.lld: error: duplicate symbol: mongo::WaitForMajorityService::get(mongo::ServiceContext*) >>> defined at wait_for_majority_service.cpp:61 (src/mongo/db/s/wait_for_majority_service.cpp:61) >>> wait_for_majority_service.o:(mongo::WaitForMajorityService::get(mongo::ServiceContext*)) in archive build/opt/mongo/db/s/libtransaction_coordinator.a >>> defined at wait_for_majority_service.cpp:61 (src/mongo/db/repl/wait_for_majority_service.cpp:61) >>> wait_for_majority_service.o:(.text+0x3C0) in archive build/opt/mongo/db/repl/libwait_for_majority_service.ald.lld: error: duplicate symbol: mongo::WaitForMajorityService::setUp(mongo::ServiceContext*) >>> defined at wait_for_majority_service.cpp:65 (src/mongo/db/s/wait_for_majority_service.cpp:65) >>> wait_for_majority_service.o:(mongo::WaitForMajorityService::setUp(mongo::ServiceContext*)) in archive build/opt/mongo/db/s/libtransaction_coordinator.a >>> defined at wait_for_majority_service.cpp:65 (src/mongo/db/repl/wait_for_majority_service.cpp:65) >>> wait_for_majority_service.o:(.text+0x3D0) in archive build/opt/mongo/db/repl/libwait_for_majority_service.ald.lld: error: duplicate symbol: mongo::WaitForMajorityService::shutDown() >>> defined at wait_for_majority_service.cpp:73 (src/mongo/db/s/wait_for_majority_service.cpp:73) >>> wait_for_majority_service.o:(mongo::WaitForMajorityService::shutDown()) in archive build/opt/mongo/db/s/libtransaction_coordinator.a >>> defined at wait_for_majority_service.cpp:73 (src/mongo/db/repl/wait_for_majority_service.cpp:73) >>> wait_for_majority_service.o:(.text+0x130) in archive build/opt/mongo/db/repl/libwait_for_majority_service.ald.lld: error: duplicate symbol: mongo::WaitForMajorityService::~WaitForMajorityService() >>> defined at wait_for_majority_service.cpp:57 (src/mongo/db/s/wait_for_majority_service.cpp:57) >>> wait_for_majority_service.o:(mongo::WaitForMajorityService::~WaitForMajorityService()) in archive build/opt/mongo/db/s/libtransaction_coordinator.a >>> defined at wait_for_majority_service.cpp:57 (src/mongo/db/repl/wait_for_majority_service.cpp:57) >>> wait_for_majority_service.o:(.text+0x0) in archive build/opt/mongo/db/repl/libwait_for_majority_service.ald.lld: error: duplicate symbol: mongo::WaitForMajorityService::~WaitForMajorityService() >>> defined at wait_for_majority_service.cpp:57 (src/mongo/db/s/wait_for_majority_service.cpp:57) >>> wait_for_majority_service.o:(mongo::WaitForMajorityService::~WaitForMajorityService()) in archive build/opt/mongo/db/s/libtransaction_coordinator.a >>> defined at wait_for_majority_service.cpp:57 (src/mongo/db/repl/wait_for_majority_service.cpp:57) >>> wait_for_majority_service.o:(.text+0x0) in archive build/opt/mongo/db/repl/libwait_for_majority_service.a clang-7: error: linker command failed with exit code 1 (use -v to see invocation) ninja: build stopped: subcommand failed.
xgen-internal-githook commented on Tue, 13 Oct 2020 06:07:06 +0000: Author: {'name': 'Ryan Egesdahl', 'email': 'ryan.egesdahl@mongodb.com', 'username': 'deriamis'} Message: SERVER-50363 Merge --build-tools=next into stable Merging the following fixes into the stable version of the build tools and migrate fully from --ninja=next to the --build-tools=next: SERVER-47598 SERVER-50010 SERVER-47943 SERVER-50125 SERVER-50376 SERVER-49457 SERVER-49493 SERVER-49036 SERVER-48966 (cherry picked from commit 78bb3f3c8a658a5a9fec8d55864e426382f68bd0) Branch: v4.4 https://github.com/mongodb/mongo/commit/e7f6696a96db5438a269c5e67a06ff9e05cb897a xgen-internal-githook commented on Wed, 7 Oct 2020 23:01:35 +0000: Author: {'name': 'Ryan Egesdahl', 'email': 'ryan.egesdahl@mongodb.com', 'username': 'deriamis'} Message: SERVER-50363 Merge --build-tools=next into stable Merging the following fixes into the stable version of the build tools and migrate fully from --ninja=next to the --build-tools=next: SERVER-47598 SERVER-50010 SERVER-47943 SERVER-50125 SERVER-50376 SERVER-49457 SERVER-49493 SERVER-49036 SERVER-48966 (cherry picked from commit 78bb3f3c8a658a5a9fec8d55864e426382f68bd0) Branch: v4.4 https://github.com/mongodb/mongo/commit/7b93770e0835087c023ae3e3f36eeb46d204767f xgen-internal-githook commented on Tue, 6 Oct 2020 02:57:51 +0000: Author: {'name': 'Ryan Egesdahl', 'email': 'ryan.egesdahl@mongodb.com', 'username': 'deriamis'} Message: SERVER-50363 Merge --build-tools=next into stable Merging the following fixes into the stable version of the build tools and migrate fully from --ninja=next to the --build-tools=next: SERVER-47598 SERVER-50010 SERVER-47943 SERVER-50125 SERVER-50376 SERVER-49457 SERVER-49493 SERVER-49036 SERVER-48966 Branch: master https://github.com/mongodb/mongo/commit/78bb3f3c8a658a5a9fec8d55864e426382f68bd0 xgen-internal-githook commented on Sat, 15 Aug 2020 02:58:08 +0000: Author: {'name': 'Daniel Moody', 'email': 'daniel.moody@mongodb.com', 'username': 'dmoody256'} Message: SERVER-49457 Ninja tool static links should delete existing archive before linking" Branch: v4.4 https://github.com/mongodb/mongo/commit/3bfeaa9d487709a2169aa57d33ebbe7055444c1f acm commented on Fri, 14 Aug 2020 17:57:52 +0000: daniel.moody - I queued this up for v4.4 backport. Feel free to self-approve the backport and take care of it in this sprint. xgen-internal-githook commented on Fri, 14 Aug 2020 05:07:21 +0000: Author: {'name': 'Daniel Moody', 'email': 'daniel.moody@mongodb.com', 'username': 'dmoody256'} Message: SERVER-49457 Ninja tool static links should delete existing archive before linking" Branch: master https://github.com/mongodb/mongo/commit/a78139a155bedfd8200e155db66f59837d5796c9 JIRAUSER1253549 commented on Wed, 5 Aug 2020 19:36:24 +0000: Ninja ran into this itself when ninja use to use ninja to build itself: https://github.com/ninja-build/ninja/commit/bbc111010163fab044a4782b771fdd6dfa289c66 Seems the recommended solution from ninja is to manually delete the archive yourself. acm commented on Mon, 13 Jul 2020 19:25:13 +0000: The issue here is extremely subtle. SCons always removes target files before rebuilding them (unless they have been marked .Precious). Ninja on the other hand does not do this. In a build mode where we produce archive files, an out-of-date archive file, like libtransaction_coordinator.a becomes across this transition, remains on disk as-is when the archiver is re-invoked. Since ar archives are accumulative, this means that the existing symbol information from the prior state is retained. We will either need to figure out how to convince Ninja to remove archive files before rebuilding them (this may be hard or impossible), or make the archive action in Ninja compound and start with a removal of the archive file (this may also be hard). This situation only arises when building with --link-model=static and across commits where object files move between archives. The straightorward workaround is to build with --link-model=dynamic. acm commented on Mon, 13 Jul 2020 18:14:22 +0000: A clean build at 6c0ac0678f3c159c65c497acf155e964dec43f61 doesn't reproduce this issue for me, but that wasn't too surprising. If there is an issue here, it is going to be related to ninja regeneration with respect to the change made for SERVER-49316 in 2a8f91f240f9ec5cb5e47fcbb78a613985de6632. I checked out the commit before 2a8f91f240f9ec5cb5e47fcbb78a613985de6632 and built, and then checked out 2a8f91f240f9ec5cb5e47fcbb78a613985de6632. And indeed, duplicate symbols appear: Generating: build.ninja scons: done building targets. [10/11] Linking build/opt/mongo/mongod FAILED: build/opt/mongo/mongod PATH=/opt/mongodbtoolchain/v3/bin:/usr/local/bin:/opt/bin:/bin:/usr/bin ICECC_CLANG_REMOTE_CPP=1 CCACHE_NOCPP2=1 ICECC_VERSION=/home/andrew/Documents/10gen/dev/src/mongodb/build/scons/icecc/opt_mongodbtoolchain_v3_bin_clang_opt_mongodbtoolchain_v3_bin_clang++.tar.gz CCACHE_PREFIX=/usr/bin/icecc /usr/bin/icerun /opt/mongodbtoolchain/v3/bin/clang++ @build/opt/mongo/mongod.rsp ld.lld: error: duplicate symbol: mongo::WaitForMajorityService::waitUntilMajority(mongo::repl::OpTime const&) >>> defined at wait_for_majority_service.cpp:101 (src/mongo/db/s/wait_for_majority_service.cpp:101) >>> wait_for_majority_service.o:(mongo::WaitForMajorityService::waitUntilMajority(mongo::repl::OpTime const&)) in archive build/opt/mongo/db/s/libtransaction_coordinator.a >>> defined at wait_for_majority_service.cpp:101 (src/mongo/db/repl/wait_for_majority_service.cpp:101) >>> wait_for_majority_service.o:(.text+0x510) in archive build/opt/mongo/db/repl/libwait_for_majority_service.a ld.lld: error: duplicate symbol: mongo::WaitForMajorityService::_periodicallyWaitForMajority(mongo::ServiceContext*) >>> defined at wait_for_majority_service.cpp:145 (src/mongo/db/s/wait_for_majority_service.cpp:145) >>> wait_for_majority_service.o:(mongo::WaitForMajorityService::_periodicallyWaitForMajority(mongo::ServiceContext*)) in archive build/opt/mongo/db/s/libtransaction_coordinator.a >>> defined at wait_for_majority_service.cpp:145 (src/mongo/db/repl/wait_for_majority_service.cpp:145) >>> wait_for_majority_service.o:(.text+0xA50) in archive build/opt/mongo/db/repl/libwait_for_majority_service.a ld.lld: error: duplicate symbol: mongo::WaitForMajorityService::get(mongo::ServiceContext*) >>> defined at wait_for_majority_service.cpp:61 (src/mongo/db/s/wait_for_majority_service.cpp:61) >>> wait_for_majority_service.o:(mongo::WaitForMajorityService::get(mongo::ServiceContext*)) in archive build/opt/mongo/db/s/libtransaction_coordinator.a >>> defined at wait_for_majority_service.cpp:61 (src/mongo/db/repl/wait_for_majority_service.cpp:61) >>> wait_for_majority_service.o:(.text+0x3C0) in archive build/opt/mongo/db/repl/libwait_for_majority_service.a ld.lld: error: duplicate symbol: mongo::WaitForMajorityService::setUp(mongo::ServiceContext*) >>> defined at wait_for_majority_service.cpp:65 (src/mongo/db/s/wait_for_majority_service.cpp:65) >>> wait_for_majority_service.o:(mongo::WaitForMajorityService::setUp(mongo::ServiceContext*)) in archive build/opt/mongo/db/s/libtransaction_coordinator.a >>> defined at wait_for_majority_service.cpp:65 (src/mongo/db/repl/wait_for_majority_service.cpp:65) >>> wait_for_majority_service.o:(.text+0x3D0) in archive build/opt/mongo/db/repl/libwait_for_majority_service.a ld.lld: error: duplicate symbol: mongo::WaitForMajorityService::shutDown() >>> defined at wait_for_majority_service.cpp:73 (src/mongo/db/s/wait_for_majority_service.cpp:73) >>> wait_for_majority_service.o:(mongo::WaitForMajorityService::shutDown()) in archive build/opt/mongo/db/s/libtransaction_coordinator.a >>> defined at wait_for_majority_service.cpp:73 (src/mongo/db/repl/wait_for_majority_service.cpp:73) >>> wait_for_majority_service.o:(.text+0x130) in archive build/opt/mongo/db/repl/libwait_for_majority_service.a ld.lld: error: duplicate symbol: mongo::WaitForMajorityService::~WaitForMajorityService() >>> defined at wait_for_majority_service.cpp:57 (src/mongo/db/s/wait_for_majority_service.cpp:57) >>> wait_for_majority_service.o:(mongo::WaitForMajorityService::~WaitForMajorityService()) in archive build/opt/mongo/db/s/libtransaction_coordinator.a >>> defined at wait_for_majority_service.cpp:57 (src/mongo/db/repl/wait_for_majority_service.cpp:57) >>> wait_for_majority_service.o:(.text+0x0) in archive build/opt/mongo/db/repl/libwait_for_majority_service.a ld.lld: error: duplicate symbol: mongo::WaitForMajorityService::~WaitForMajorityService() >>> defined at wait_for_majority_service.cpp:57 (src/mongo/db/s/wait_for_majority_service.cpp:57) >>> wait_for_majority_service.o:(mongo::WaitForMajorityService::~WaitForMajorityService()) in archive build/opt/mongo/db/s/libtransaction_coordinator.a >>> defined at wait_for_majority_service.cpp:57 (src/mongo/db/repl/wait_for_majority_service.cpp:57) >>> wait_for_majority_service.o:(.text+0x0) in archive build/opt/mongo/db/repl/libwait_for_majority_service.a clang-7: error: linker command failed with exit code 1 (use -v to see invocation) ninja: build stopped: subcommand failed. Note that build.ninja was in fact regenerated. Since libwait_for_majority_service.a doesn't exist in the commit before, the problem must lie with the build of libtransaction_coordinator.a across the transition.
Run ninja -j400 install-core at commit 6c0ac0678f3c159c65c497acf155e964dec43f61 on master branch of mongodb/mongo.