in

A tool to build reproducible wheels for you Python project or for all of your dependencies


pip-tools project.

pip-compile --generate-hashes --allow-unsafe --output-file=requirements.txt requirements.in

Please make sure that you note down all the build dependencies of any given dependency, otherwise during the build process, pip will download from PyPI and install them in the build environment. If you are building from a requirements file, during download and extracting each source tar ball, you can notice if the dependency has any build time dependency or not. Otherwise, you can manually look at the build time dependencies.

For example in the following text you can find a few packages with build time dependencies.
Look at the lines with Getting requirements to build wheel.

Collecting build==0.7.0
  Using cached build-0.7.0.tar.gz (15 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting click==8.0.1
  Using cached click-8.0.1.tar.gz (327 kB)
Collecting packaging==21.0
  Using cached packaging-21.0.tar.gz (83 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting pep517==0.11.0
  Using cached pep517-0.11.0.tar.gz (25 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done

Bootstrapping the build environment

For any production use, you should also bootstrap the build environment, and create the initial virtual environment to build all dependencies in that environment only. You can store the wheels in any place you want (S3, or git-lfs), and start from there during creating the environment next time.

In following commands, we will create a set of wheels for such bootstrap environment where the build requirements are mentioned in bootstrap.in

python3 -m venv .venv
source .venv/bin/activate
python3 -m pip install pip-tools # This is coming directly from pypi
pip-compile --generate-hashes --allow-unsafe --output-file=bootstrap.txt bootstrap.in
asaman -r bootstrap.txt

This will create all the wheels in the ./wheels directory.
From next time, one can install them from the ./wheels directory directory.

But, first we will create a new requirements file with only the hashes from our reproducible wheels, the output file name will be verified-bootstrap.txt.

asaman-generate bootstrap.txt

Now we can use this file to create the environment.

python3 -m venv .venv
source .venv/bin/activate
python3 -m pip install --no-index --find-links ./wheels --require-hashes --only-binary :all: -r verified-bootstrap.txt 

GitHub

https://github.com/kushaldas/asaman


Leave a Reply

Your email address will not be published. Required fields are marked *

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

GIPHY App Key not set. Please check settings

The React Lifecycle – DEV Community

JavaScript this Keyword Explained for Beginners