25

I am thinking about switching from pip & virtualenv to pipenv. But after studying the documentation I am still at a loss on how the creators of pipenv structured the deployment workflow.

For example, in development I have a Pipfile & a Pipfile.lock that define the environment. Using a deployment script I want to deploy

  1. git pull via Github to production server
  2. pipenv install creates/refreshes the environment in the home directory of the deployment user

But I need a venv in a specific directory which is already configured in systemd or supervisor. E.g.: command=/home/ubuntu/production/application_xy/env/bin/gunicorn module:app

pipenv creates the env in some location such as /home/ultimo/.local/share/virtualenvs/application_xy-jvrv1OSi

What is the intended workflow to deploy an application with pipenv?

chicks
  • 2,393
  • 3
  • 24
  • 40
caliph
  • 1,389
  • 3
  • 27
  • 52
  • in case you solved this issue, would you mind refferencing a list of commands for the workflow? i have this cheatsheet would be amazing if you could commit to it https://github.com/elfelround/ec2-deployment-cheatsheet/blob/master/2.7%20remastered%20for%203%20with%20pipenv – Mr-Programs Jan 26 '19 at 14:36

5 Answers5

25

You have few options there.

  1. You can run your gunicorn via pipenv run:

    pipenv run gunicorn module:app

This creates a slight overhead, but has the advantage of also loading environment from $PROJECT_DIR/.env (or other $PIPENV_DOTENV_LOCATION).

  1. You can set the PIPENV_VENV_IN_PROJECT environment variable. This will keep pipenv's virtualenv in $PROJECT_DIR/.venv instead of the global location.

  2. You can use an existing virtualenv and run pipenv from it. Pipenv will not attempt to create its own virtualenv if it's run from one.

  3. You can just use the weird pipenv-created virtualenv path.

rczajka
  • 1,810
  • 14
  • 13
  • may i ask how you solved this issue in production? https://stackoverflow.com/questions/54378879/pipenv-on-production-workaround-with-user-is-it-viable – Mr-Programs Jan 26 '19 at 14:37
12

I've just switched to pipenv for deployment and my workflow is roughly as follows (managed with ansible). For an imaginary project called "project", assuming that a working Pipfile.lock is checked into source control:

  1. Clone the git repository:

    git clone https://github.com/namespace/project.git /opt/project

  2. Change into that directory

    cd /opt/project

  3. Check out the target reference (branch, tag, ...):

    git checkout $git_ref

  4. Create a virtualenv somewhere, with the target Python version (3.6, 2.7, etc):

    virtualenv -p"python$pyver" /usr/local/project/$git_ref

  5. Call pipenv in the context of that virtualenv, so it won't install its own:

    VIRTUAL_ENV="/usr/local/project/$git_ref" pipenv --python="/usr/local/project/$git_ref/bin/python" install --deploy

    The --deploy will throw an error, when the Pipfile.lock does not match the Pipfile.

  6. Install the project itself using the virtualenv's pip (only necessary if it isn't already in the Pipfile):

    /usr/local/project/$git_ref/bin/pip install /opt/project

  7. Set a symlink to the new installation directory:

    ln -s /usr/local/project/$git_ref /usr/local/project/current

My application is then callable e.g. with /usr/local/project/current/bin/project_exec --foo --bar, which is what's configured in supervisor, for instance.

All of this is triggered when a tag is pushed to the remote.

As the virtualenvs of earlier versions remain intact, a rollback is simply done by setting the current-symlink back to an earlier version. I.e. if tag 1.5 is broken, and I want to go back to 1.4, all I have to do is ln -s /usr/local/project/1.4 /usr/local/project/current and restart the application with supervisorctl.

karlson
  • 5,325
  • 3
  • 30
  • 62
  • so i have been following the commands listed here but I am having the issue of pipenv making pip unusable which is also listed on the commands linked, however the answer i saw somewhere of installing pipenv under specific user alert me https://github.com/elfelround/ec2-deployment-cheatsheet/blob/master/2.7%20remastered%20for%203%20with%20pipenv – Mr-Programs Jan 26 '19 at 13:43
  • @Mr-Programs WDYM pipenv makes pip unusable? What does not work? – karlson Jan 28 '19 at 10:00
  • https://i.stack.imgur.com/Ok09j.png or https://stackoverflow.com/questions/54378879/pipenv-on-production-workaround-with-user-is-it-viable – Mr-Programs Jan 28 '19 at 11:46
  • I see. This is not specific to pipenv though, but a general problem of using `sudo pip install ...` alongside your package manager to install python-packages. You should install `pipenv` for some specific user instead (with `pip install --user pipenv`) – karlson Jan 28 '19 at 14:38
8

I think pipenv is very good for managing dependencies but is too slow, cumbersome and still a bit unstable for using it for automatic deployments.

Instead I use virtualenv (or virtualenvwrapper) and pip on the target machine.

  • On my build/development machine I create a requirements.txt compatible text file using pipenv lock -r:

      $ pipenv lock -r > deploy-requirements.txt

  • While deploying, inside a virtualenv I run:

      $ pip install -r deploy-requirements.txt
Udi
  • 29,222
  • 9
  • 96
  • 129
  • as oposed to a pip freeze? – Mr-Programs Jan 26 '19 at 13:45
  • @Mr-Programs `pipenv lock -r` as opposed to `pip freeze` gets you sha256 hashes for packages, which I prefer as my code is more reproducible. From [the docs](https://pipenv.readthedocs.io/en/latest/basics/#example-pipfile-pipfile-lock): "Note If you’d like a requirements.txt output of the lockfile, run $ pipenv lock -r. This will include all hashes, however (which is great!). To get a requirements.txt without hashes, use $ pipenv run pip freeze." – Jason R Stevens CFA Dec 30 '19 at 23:52
5

Just do this:

mkdir .venv
pipenv install

Explanation:

pipenv checks your project directory for a sub directory named .venv. If it finds it, then pipenv creates a local virtual environment (because then it sets automatically PIPENV_VENV_IN_PROJECT=true)

So now if you want you can either activate the virtual environment with:

source .venv/bin/activate

Or config you app.conf for gunicorn with something like this:

exec /path/to/.venv/bin/gunicorn myapp:app
sunwarr10r
  • 4,420
  • 8
  • 54
  • 109
4

To create virtual environment in the same directory as the project set the following environment variable doc

PIPENV_VENV_IN_PROJECT=true

This installs the dependencies to .venv directory inside project. Available from PipEnv v2.8.7

salmanulfarzy
  • 1,484
  • 14
  • 18