Launching soon: The world's first vendor agnostic bug scrubLearn more & join waitlist
OPERATIONAL DEFECT DATABASE
...
...
Consider including a particular versions range into pip components specification for setuptools https://github.com/mongodb/mongo/blob/v7.0/etc/pip/components/compile.req#L6 because mongo-tooling-metrics == 1.0.7 https://github.com/mongodb/mongo/blob/v7.0/etc/pip/components/tooling_metrics.req#L1 requires setuptools >= 58.1.0, Without that restrictions, if an incompatible (with mongo-tooling-metrics) version of setuptools (e.g. 45.2.0) is already installed on the system, we observe such behavior: user@ubuntu20.04:~/mongo$ sudo python3 buildscripts/scons.py DESTDIR=/opt/mongo_uttam CCFLAGS="" SHCCFLAGS="" --linker=gold LINKFLAGS="" install-mongod Checking required python packages... Requirements list: Cheetah3 PyYAML=3.0.0 cryptography==2.3; platform_machine == "s390x" or platform_machine == "ppc64le" cryptography==36.0.2; platform_machine != "s390x" and platform_machine != "ppc64le" mongo-ninja-python==1.11.1.4; platform_machine == "x86_64" and sys_platform == "linux" mongo-tooling-metrics==1.0.7 ninja>=1.10.0; platform_machine != "x86_64" or sys_platform != "linux" packaging psutil pymongo==4.3.3 pypiwin32>=223; sys_platform == "win32" and python_version > "3" pywin32>=225; sys_platform == "win32" and python_version > "3" regex requests=2.0.0 requirements_parser setuptools types-PyYAML~=6.0.5 typing-extensions>=3.7.4 (setuptools 45.2.0 (/usr/lib/python3/dist-packages), Requirement.parse('setuptools=58.1.0'), {'mongo-tooling-metrics'}) Try running: /usr/bin/python3 -m pip install -r etc/pip/compile-requirements.txt user@ubuntu20.04:~/mongo$ /usr/bin/python3 -m pip install -r etc/pip/compile-requirements.txt /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead from cryptography.utils import int_from_bytes /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead from cryptography.utils import int_from_bytes Ignoring pypiwin32: markers 'sys_platform == "win32" and python_version > "3"' don't match your environment Ignoring pywin32: markers 'sys_platform == "win32" and python_version > "3"' don't match your environment Ignoring cryptography: markers 'platform_machine == "s390x" or platform_machine == "ppc64le"' don't match your environment Ignoring ninja: markers 'platform_machine != "x86_64" or sys_platform != "linux"' don't match your environment Requirement already satisfied: cryptography==36.0.2 in /usr/local/lib/python3.8/dist-packages (from -r etc/pip/components/platform.req (line 6)) (36.0.2) Requirement already satisfied: mongo-ninja-python==1.11.1.4 in /usr/local/lib/python3.8/dist-packages (from -r etc/pip/components/platform.req (line 8)) (1.11.1.4) Requirement already satisfied: psutil Requirement already satisfied: pymongo==4.3.3 in /usr/local/lib/python3.8/dist-packages (from -r etc/pip/components/core.req (line 3)) (4.3.3) Requirement already satisfied: PyYAML=3.0.0 in /usr/lib/python3/dist-packages (from -r etc/pip/components/core.req (line 4)) (5.3.1) Requirement already satisfied: types-PyYAML~=6.0.5 in /usr/local/lib/python3.8/dist-packages (from -r etc/pip/components/core.req (line 5)) (6.0.12.9) Requirement already satisfied: requests=2.0.0 in /usr/lib/python3/dist-packages (from -r etc/pip/components/core.req (line 6)) (2.22.0) Requirement already satisfied: typing-extensions>=3.7.4 in /usr/local/lib/python3.8/dist-packages (from -r etc/pip/components/core.req (line 7)) (4.5.0) Requirement already satisfied: Cheetah3 Requirement already satisfied: packaging Requirement already satisfied: regex Requirement already satisfied: requirements_parser Requirement already satisfied: setuptools in /home/ubuntu/.local/lib/python3.8/site-packages (from -r etc/pip/components/compile.req (line 6)) (45.2.0) Requirement already satisfied: mongo-tooling-metrics==1.0.7 in /usr/local/lib/python3.8/dist-packages (from -r etc/pip/components/tooling_metrics.req (line 1)) (1.0.7) Requirement already satisfied: cffi>=1.12 in /usr/local/lib/python3.8/dist-packages (from cryptography==36.0.2->-r etc/pip/components/platform.req (line 6)) (1.15.1) Requirement already satisfied: dnspython=1.16.0 in /usr/local/lib/python3.8/dist-packages (from pymongo==4.3.3->-r etc/pip/components/core.req (line 3)) (2.3.0) Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging-r etc/pip/components/compile.req (line 3)) (3.0.9) Requirement already satisfied: types-setuptools=57.4.4 in /usr/local/lib/python3.8/dist-packages (from requirements_parser-r etc/pip/components/compile.req (line 5)) (57.4.18) Requirement already satisfied: pydantic=1.8.2 in /usr/local/lib/python3.8/dist-packages (from mongo-tooling-metrics==1.0.7->-r etc/pip/components/tooling_metrics.req (line 1)) (1.10.7) Requirement already satisfied: gitpython=3.1.29 in /usr/local/lib/python3.8/dist-packages (from mongo-tooling-metrics==1.0.7->-r etc/pip/components/tooling_metrics.req (line 1)) (3.1.31) Requirement already satisfied: distro=1.5.0 in /usr/local/lib/python3.8/dist-packages (from mongo-tooling-metrics==1.0.7->-r etc/pip/components/tooling_metrics.req (line 1)) (1.8.0) Requirement already satisfied: pycparser in /usr/local/lib/python3.8/dist-packages (from cffi>=1.12->cryptography==36.0.2->-r etc/pip/components/platform.req (line 6)) (2.21) Requirement already satisfied: gitdb=4.0.1 in /usr/local/lib/python3.8/dist-packages (from gitpython=3.1.29->mongo-tooling-metrics==1.0.7->-r etc/pip/components/tooling_metrics.req (line 1)) (4.0.10) Requirement already satisfied: smmap=3.0.1 in /usr/local/lib/python3.8/dist-packages (from gitdb=4.0.1->gitpython=3.1.29->mongo-tooling-metrics==1.0.7->-r etc/pip/components/tooling_metrics.req (line 1)) (5.0.0)
JIRAUSER1270035 commented on Thu, 11 May 2023 12:26:20 +0000: Hey tausif.rahman@mongodb.com, thank you for your prompt response! Sure, on my setup I solved that issue by manually upgrading setuptools when `pip install -r etc/pip/compile-requirements.txt` failed to do that; My setup is: python v3.8.10, pip v20.0.2, no virtualenv; After your comment I tried to run `pip install -r etc/pip/compile-requirements.txt` using virtualenv v20.0.17 with the same "bad" results: Requirement already satisfied: pycparser in /home/user/test/venv/lib/python3.8/site-packages (from cffi>=1.12->cryptography==36.0.2->-r etc/pip/components/platform.req (line 6)) (2.21) Requirement already satisfied: gitdb=4.0.1 in /home/user/test/venv/lib/python3.8/site-packages (from gitpython=3.1.29->mongo-tooling-metrics==1.0.7->-r etc/pip/components/tooling_metrics.req (line 1)) (4.0.10) Requirement already satisfied: smmap=3.0.1 in /home/user/test/venv/lib/python3.8/site-packages (from gitdb=4.0.1->gitpython=3.1.29->mongo-tooling-metrics==1.0.7->-r etc/pip/components/tooling_metrics.req (line 1)) (5.0.0) ERROR: mongo-tooling-metrics 1.0.7 has requirement setuptools=58.1.0, but you'll have setuptools 45.0.0 which is incompatible. Installing collected packages: mongo-tooling-metrics Successfully installed mongo-tooling-metrics-1.0.7 But after upgrading pip from v20.0.2 to your v23.1.2, I ended up with your "good" results: Requirement already satisfied: pycparser in /home/user/test/venv/lib/python3.8/site-packages (from cffi>=1.12->cryptography==36.0.2->-r etc/pip/components/platform.req (line 6)) (2.21) Requirement already satisfied: gitdb=4.0.1 in /home/user/test/venv/lib/python3.8/site-packages (from gitpython=3.1.29->mongo-tooling-metrics==1.0.7->-r etc/pip/components/tooling_metrics.req (line 1)) (4.0.10) Requirement already satisfied: smmap=3.0.1 in /home/user/test/venv/lib/python3.8/site-packages (from gitdb=4.0.1->gitpython=3.1.29->mongo-tooling-metrics==1.0.7->-r etc/pip/components/tooling_metrics.req (line 1)) (5.0.0) Installing collected packages: setuptools Attempting uninstall: setuptools Found existing installation: setuptools 45.0.0 Uninstalling setuptools-45.0.0: Successfully uninstalled setuptools-45.0.0 Successfully installed mongo-tooling-metrics-1.0.7 setuptools-58.5.3 So the difference in behavior is in pip versions (v20.0.2 vs v23.1.2); Regarding the boundaries of the dependency’s versions range – I suppose the wider the range will be (of course, without regression and side effects), the more users setups it will cover. About the upper bound - currently the latest version of setuptools is 67.7.2, which also has the `get_distribution` function. JIRAUSER1258055 commented on Wed, 10 May 2023 13:27:46 +0000: We have the dependency in mongo-tooling metrics because it uses the setuptools `get_distribution` function. I kind of just chose that setuptools version, but we haven't had any issues for the past few months. Can you upgrade your setuptools vesrion? I tried to repro your issue w/ python3.8, pip 23.1.2, virtualenv, & all packages installed except mongo-tooling-metrics. I also downgraded setuptools to 45. When I run pip install -r etc/pip/compile-requirements.txt, it downloads mongo-tooling-metrics and upgrade setuptools to 58. I see this at the end of the output: Requirement already satisfied: pycparser in ./test-venv/lib/python3.8/site-packages (from cffi>=1.12->cryptography==36.0.2->-r etc/pip/components/platform.req (line 6)) (2.21) Requirement already satisfied: gitdb=4.0.1 in ./test-venv/lib/python3.8/site-packages (from gitpython=3.1.29->mongo-tooling-metrics==1.0.7->-r etc/pip/components/tooling_metrics.req (line 1)) (4.0.10) Requirement already satisfied: smmap=3.0.1 in ./test-venv/lib/python3.8/site-packages (from gitdb=4.0.1->gitpython=3.1.29->mongo-tooling-metrics==1.0.7->-r etc/pip/components/tooling_metrics.req (line 1)) (5.0.0) Installing collected packages: setuptools Attempting uninstall: setuptools Found existing installation: setuptools 45.0.0 Uninstalling setuptools-45.0.0: Successfully uninstalled setuptools-45.0.0 Successfully installed setuptools-58.5.3 We can loosen the dependency as well – I've tried with as old as setuptools 20 & it looks like this works. But this will require changes across 3 repos (mongo-tooling-metrics, mongo-ninja-python [uses mongo-tooling-metrics] & mongo[uses mongo-tooling-metrics]).
// Install setuptools (version that is out of mentioned range) // Try to build Mongo Server user@ubuntu20.04:~/mongo$ sudo python3 buildscripts/scons.py DESTDIR=/opt/mongo_uttam CCFLAGS="" SHCCFLAGS="" --linker=gold LINKFLAGS="" install-mongod // Try to check & [re]install improper packages as suggested by failed installation output user@ubuntu20.04:~/mongo$ /usr/bin/python3 -m pip install -r etc/pip/compile-requirements.txt