👷 Enable deptree hash-checking in the infra 🔗 - #420
Open
webknjaz wants to merge 4 commits into
Open
Conversation
This sorta fixes pypa#279, though not fully, perhaps. We may want to discuss it more in the future. It became possible to pip v26.1 [[1]] fixing the issue I reported when Pradyun was first battle-testing the new resolvelib-based replacement back in 2020 and it turned out to crash whenever the new resolver would see constraint files with entries having `--hash` attached [[2]]. This was quite a popular use for the users of `pip-tools` which delayed the adoption. It's finally been fixed this May [[3]] so we can use it here. This version of pip got bundled into CPython and backported into 3.13 that our container uses. So I'm bumping pip to the corresponding version to match that in our pip-tools managed files too (since we've been downgrading pip to a version that doesn't for two months LOL). With that, the patch also includes hashes for whatever bits of the tree have been present in the constraint "lock" files already without re-resolving them or bumping anything else transitive. Thanks to Pradyun for finally fixing the thing in pip upstream, and to Seth, and Thomas for periodically poking me about this ☮️ [1]: https://pip.pypa.io/en/stable/news/#v26-1 [2]: pypa/pip#9243 [3]: pypa/pip#13887
$ podman run \
--rm \
-it \
-v $(pwd)/requirements/:/requirements:rw,Z \
--entrypoint=/bin/bash \
--workdir=/requirements \
ghcr.io/pypa/gh-action-pypi-publish:v1.14.2
> python3 -m pip install --upgrade 'pip < 26.1.1' pip-tools
> pip-compile --config=../.pip-tools.toml -o runtime-prerequisites.txt runtime-prerequisites.in
> pip-compile --config=../.pip-tools.toml -o runtime.txt runtime.in
webknjaz
requested review from
Copilot,
facutuesca,
miketheman,
sethmlarson,
sirosen and
woodruffw
July 31, 2026 21:05
Member
Author
|
I've tagged multiple people here mainly to see if anybody has concerns regarding bumping a lot of deep transitive deps together… |
There was a problem hiding this comment.
Pull request overview
This PR updates the container build and dependency lockfiles to support hash-based dependency verification by generating pip-tools outputs with hashes, and aligns the build environment to newer pip behavior that supports hashed constraints.
Changes:
- Enable
pip-toolshash generation (generate-hashes = true) and regeneraterequirements/*.txtwith hashes. - Bump and hash-pin
pipinruntime-prerequisites.*to a version compatible with hashed constraints. - Copy
.pip-tools.tomlinto the Docker image to support in-container lockfile maintenance workflows.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| requirements/runtime.txt | Regenerated lock/constraints file with --generate-hashes and updated pinned versions + hashes. |
| requirements/runtime-prerequisites.txt | Regenerated pip prerequisite lock with hashes and updated pip pin. |
| requirements/runtime-prerequisites.in | Document and require a pip version new enough to support hashed constraints. |
| Dockerfile | Include .pip-tools.toml in the image; continues installing dependencies during image build. |
| .pip-tools.toml | Enable hash generation in pip-tools configuration. |
Suppressed comments (1)
Dockerfile:24
- The image build installs from the unpinned
.infiles, while the hashed/pinned.txtfiles are only used as constraints. This likely does not enforce hash-checking during installation, which is the central goal described in the PR/issue. Consider installing directly from the hashed lock files (and optionally enabling--require-hashes) so the build fails if hashes don’t match.
PIP_CONSTRAINT=requirements/runtime-prerequisites.txt \
pip install --user --upgrade --no-cache-dir \
-r requirements/runtime-prerequisites.in && \
PIP_CONSTRAINT=requirements/runtime.txt \
pip install --user --upgrade --no-cache-dir --prefer-binary \
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+2
to
+3
| # that prevented us from using `--require-hashes` in constraint files since | ||
| # roughly year 2020. But now we can get back to using it. |
This shouldn't be important right now and can be addressed later.
webknjaz
commented
Jul 31, 2026
| run: python3 -m pip install build twine | ||
| env: | ||
| PIP_CONSTRAINT: test/requirements/runtime.txt | ||
| PIP_NO_REQUIRE_HASHES: 1 # FIXME: incomplete lock file explodes |
Member
Author
There was a problem hiding this comment.
(looks like I'll have to deal w/ this before merging)
This was referenced Jul 31, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The main theme here is that this kinda fixes #279, perhaps not fully. I'm not planning to increase the scope, but people are welcome to send in follow-ups. We may want to discuss more in the future.
This specific piece of work became possible due to pip v26.1 [1] fixing the issue I reported back when @pradyunsg was first battle-testing the new resolvelib-based replacement in 2020, and it turned out to crash whenever the new resolver would
see constraint files with entries having
--hashattached [2]. This was quite a popular use for the users ofpip-toolswhich delayed the adoption here. It's finally been fixed this May [3] so we can use it here.This version of pip got bundled into CPython and backported into 3.13 that our container uses. So I'm bumping pip to the corresponding version to match that in our pip-tools managed files too (since we've been downgrading pip to a version that doesn't for two months 🤣).
With that in mind, the central patch also includes hashes for whatever bits of the tree have been present in the constraint “lock” files already without re-resolving them or bumping anything else transitive. Plus it configures
pip-toolsto always include hashes from now on. There's also a small convenience commit before this for including thepip-toolsconfig into the container image for future constraints management that wasn't there for some reason.The last commit in the series bumps the “lock file” constraints separately+transitively:
Thanks to Pradyun for finally fixing the thing in pip upstream, and to
@sethmlarson for the initial exploratory PR, and to @graingert for poking me periodically about this ☮️