Info

As of this morning, running `ninja -j500 install-core` on my dev box (ubuntu 18) for a  `scons --ninja` build results in the build completing successfully, but with multiple warnings spammed to the output: [729/4612 ( 15%) 64.160s] Compiling build/ninja/mongo/util/options_parser/value.dyn.o Can't read /proc/cpuinfo: No such file or directory Can't read /proc/cpuinfo: No such file or directory [731/4612 ( 15%) 64.240s] Compiling build/ninja/mongo/s/catalog/type_lockpings.dyn.o This pattern is repeated between each target object ad nauseum. Since my system certainly does have a world readable /proc/cpuinfo, I assume this is an icecream related issue where the build of that object is occurring on another machine (perhaps non-linux?). My scons invocation is: SCONS_INVOCATION_W_TARGETS = /usr/bin/python3 "buildscripts/scons.py" $ "CC=/opt/mongodbtoolchain/v3/bin/clang" "CXX=/opt/mongodbtoolchain/v3/bin/clang++" $ "MONGO_VERSION=0.0.0" "MONGO_GIT_HASH=unknown" "CCFLAGS=-gsplit-dwarf" "--modules=enterprise" $ "--ssl" "--link-model=dynamic" "VARIANT_DIR=ninja" "--modules=enterprise" "-j30" $ "--allocator=system" "ICECC=icecc" "--modules=enterprise" "--ninja" "build.ninja"

Top User Comments

acm commented on Fri, 10 Jul 2020 17:12:20 +0000: Tagging this for backport to v4.4 so we can keep the build system in sync once that branch opens up again. xgen-internal-githook commented on Fri, 10 Jul 2020 17:04:03 +0000: Author: {'name': 'Daniel Moody', 'email': 'daniel.moody@mongodb.com', 'username': 'dmoody256'} Message: SERVER-49016 added check for icecream version and handle missing cpuinfo Branch: master https://github.com/mongodb/mongo/commit/b9f0132fc51fe6c821a8ca2de6b34cfd0a6b8465 JIRAUSER1253549 commented on Wed, 8 Jul 2020 16:53:53 +0000: in newer icecreams, it will actually force an empty proc/cpuinfo file. I made a PR to support remapping any arbitrary file and allowing custom cpuinfo to be added: https://github.com/icecc/icecream/pull/548 JIRAUSER1253549 commented on Wed, 1 Jul 2020 20:36:24 +0000: After some more investigation with acm, it does appear --addfile /proc/cpuinfo fixes the issue, however that was not observed at first because of issues related to SERVER-47598. The added /proc/cpuinfo file was not getting redistrubuted because the name of the compiler package remained the same. One downside to using --addfile /proc/cpuinfo is that it will always have different contents because it includes instantaneous info like current CPU freq, so the resulting compiler package will always have a different hash. This can cause unnecessary redistribution of the compiler package. It looks like icecream could easily support --addfile /proc/cpuinfo=/some/other/file so that any file could be remapped in place in the compiler package, so an feature request will be opened in the icecream repo. If and when that is implemented, our addfile can be updated to make a static copy of cpuinfo extracting the instantaneous info thats not necessary, resulting in compiler packages that hash match if nothing else changed. acm commented on Wed, 1 Jul 2020 16:07:07 +0000: OK, the plot thickens. I'm building install-platform-test as a smaller target. When I use my local workstation system clang, which ls clang-9, I get a ton of these errors. When I use toolchain clang, I get only two instances! The two instances appear to be assembly files. The use of /proc/cpuinfo in clang appears to be related to interpreting what -m[arch|cpu]=native (or some clang-y equivalent) ought to mean, but adding -mcpu=westmere to fix a target CPU also doesn't seem to make it not do that. The use of --addfile doesn't seem to help, even though it appears in the tarball. acm commented on Wed, 1 Jul 2020 15:41:30 +0000: I tried the above suggestion, but it doesn't seem to work for me. I still see the errors. I confirmed that the generated tgz contained the file: $ tar tvf build/scons/icecc/opt_mongodbtoolchain_v3_bin_clang_opt_mongodbtoolchain_v3_bin_clang++.tar.gz | grep proc -r--r--r-- 1000/1000 25698 2020-07-01 11:37 proc/cpuinfo JIRAUSER1253549 commented on Wed, 1 Jul 2020 14:35:46 +0000: acm in icecream.py, add --addfile /proc/cpuinfo to the create_env string: https://github.com/mongodb/mongo/blob/51d7101f5eeee0c1b26dd0c035be47effc6373b7/site_scons/site_tools/icecream.py#L58 create_env = "ICECC_VERSION_TMP=$$($ICECC_CREATE_ENV --$ICECC_COMPILER_TYPE $CC $CXX --addfile /proc/cpuinfo | awk '/^creating .*.tar .gz/ { print $$2 }')" acm commented on Wed, 1 Jul 2020 14:11:05 +0000: daniel.moody - That would be a nice simple fix and also wouldn't require altering icecc-create-env, as a bonus. I am able to repro this locally so if you have a proposed fix I can give it a try. JIRAUSER1253549 commented on Tue, 30 Jun 2020 21:50:45 +0000: possible we can add --addfile /proc/cpuinfo to our icecrean-create-env for clang? Maybe even based on version? I don't think that file matters for the remote host, so addfile will copy the local file for compiling on the remote. See: https://github.com/icecc/icecream/issues/176#issuecomment-299484912 I tested addfile and it worked on ubuntu 18 host, but I was not able to reproduce in the first place so I don't know. Could sara.golemon update this ticket with info about the host the command was run on acm commented on Mon, 29 Jun 2020 18:58:19 +0000: This seems relevant: https://github.com/icecc/icecream/blob/0c7ef30dddb9ef813f281e9a51d703857721878e/README.md#clang-tries-to-read-proccpuinfo-and-fails

Steps to Reproduce