Introduce compile_pip_requirements rule #373
Conversation
Thanks for upstreaming. To give others some added context, we at Canva also use Over the year we've used |
pip==9.0.3 | ||
setuptools==44.0.0 | ||
wheel==0.30.0a0 | ||
# |
thundergolfer
Oct 25, 2020
Collaborator
We'd be able to get rid of this requirements file once piptool
is fully removed from the rules. Would we then remove the macro usage here, or is there a reason to keep it?
We'd be able to get rid of this requirements file once piptool
is fully removed from the rules. Would we then remove the macro usage here, or is there a reason to keep it?
alexeagle
Oct 26, 2020
Author
Collaborator
sure if the requirements file goes away, then so should the rule that keeps it updated.
sure if the requirements file goes away, then so should the rule that keeps it updated.
d01ae74
to
74f80f0
ah, now the CI failure is because it's running python 2 and one of the requirements varies with python version
|
9f192e0
to
6bfa3c0
a800010
to
d6984b1
This uses pip-tools to compile a requirements.in file to a requirements.txt file, allowing transitive dependency versions to be pinned so that builds are reproducible. Fixes #176
Hey, I tried to be sneaky and try this out. Did I do it wrong?
diff --git a/WORKSPACE b/WORKSPACE
index 06724b178..15306f4af 100644
--- a/WORKSPACE
+++ b/WORKSPACE
@@ -23,10 +23,10 @@ register_toolchains("//python/interpreter:bfe_py_toolchain")
### Load rules_python first so our version wins.
http_archive(
name = "rules_python",
- sha256 = "95ee649313caeb410b438b230f632222fb5d2053e801fe4ae0572eb1d71e95b8",
- strip_prefix = "rules_python-c8c79aae9aa1b61d199ad03d5fe06338febd0774",
+ sha256 = "72800a67f1aa25d13dfeb8269e1d92c2dfb044792893605238db90b6a0c39a3c",
+ strip_prefix = "rules_python-7175da78d6d032094ea69ab418bc7f8e6f15573a",
# equivalent SHA to that of 0.1.0 release, except the archive has experimental stuff like wheel.bzl
- url = "https://github.com/bazelbuild/rules_python/archive/c8c79aae9aa1b61d199ad03d5fe06338febd0774.tar.gz",
+ url = "https://github.com/bazelbuild/rules_python/archive/7175da78d6d032094ea69ab418bc7f8e6f15573a.tar.gz",
)
load("@rules_python//python:repositories.bzl", "py_repositories")
diff --git a/python/BUILD b/python/BUILD
index ffd0fb0cd..020ee0cc5 100644
--- a/python/BUILD
+++ b/python/BUILD
@@ -1 +1,3 @@
package(default_visibility = ["//visibility:public"])
+load("@rules_python//python/pip_install:requirements.bzl", "compile_pip_requirements")
+compile_pip_requirements()
I got this error:
FWIW, we are using a custom python interpreter (3.7.9) and our python runtime has no python2 allowed. Also, sorry for jumping in randomly... don't want to distract too badly from the point of the PR, but if this is a bug (missing dep?) it might be helpful. |
I've now test-driven this myself and run into the same error as @dhalperi. As yet I haven't investigated the issue deeply, but I can create the same import problem in my workspace's Nix shell with Unrelated, but I it was a bit finicky figuring out whether to create the
So at some point bootstrap instructions would be good to tell users how to smoothly onboard into this. |
This uses pip-tools to compile a requirements.in file to a requirements.txt file,
allowing transitive dependency versions to be pinned so that builds are reproducible.
Fixes #176