Note to self from dependency hell

venv

If you aren’t using virtual environments yet, you’re missing out. This is the most important way to ensure that you are installing a package the exact same way, and with the exact same versions as the maintainer of the packages do. If you have a virtual environment and get stuck in dependency hell, you can hit the reset button and reinstall everything. In most cases pip is smart enough not to redownload any packages that you already waited for in your first install attempts.

For example, on your PC, right now, you can probably find the source code for the pytest package like this:

$ find ~/.cache/pip -name '*pytest' -type d
unzip -l ~/.cache/pip/wheels/01/46/f3/310aa7f479fbf2e11aa3660c5224ac65ff74d78130c2193151/pytest-2.6.0-py2.py3-none-any.whl 
Archive:  /home/hobs/.cache/pip/wheels/01/46/f3/310aa7f479fbf2e11aa3660c5224ac65ff74d78130c2193151/pytest-2.6.0-py2.py3-none-any.whl
  Length      Date    Time    Name
---------  ---------- -----   ----
      498  2014-07-20 09:06   pytest.py
       24  2014-07-20 09:06   _pytest/__init__.py
     3760  2014-07-20 09:06   _pytest/_argcomplete.py
    13449  2014-07-20 09:06   _pytest/capture.py
    33518  2014-07-20 09:06   _pytest/config.py
    13877  2014-07-20 09:06   _pytest/core.py
     5264  2014-07-20 09:06   _pytest/doctest.py
     2667  2014-07-20 09:06   _pytest/genscript.py
     6817  2014-07-20 09:06   _pytest/helpconfig.py
    10265  2014-07-20 09:06   _pytest/hookspec.py
     7931  2014-07-20 09:06   _pytest/junitxml.py
    25501  2014-07-20 09:06   _pytest/main.py
    10591  2014-07-20 09:06   _pytest/mark.py
     7655  2014-07-20 09:06   _pytest/monkeypatch.py
     2532  2014-07-20 09:06   _pytest/nose.py
     2576  2014-07-20 09:06   _pytest/pastebin.py
     3190  2014-07-20 09:06   _pytest/pdb.py
    24644  2014-07-20 09:06   _pytest/pytester.py
    77994  2014-07-20 09:06   _pytest/python.py
     3203  2014-07-20 09:06   _pytest/recwarn.py
     3392  2014-07-20 09:06   _pytest/resultlog.py
    16514  2014-07-20 09:06   _pytest/runner.py
     9819  2014-07-20 09:06   _pytest/skipping.py
     2025  2014-07-20 09:06   _pytest/standalonetemplate.py
    19498  2014-07-20 09:06   _pytest/terminal.py
     2496  2014-07-20 09:06   _pytest/tmpdir.py
     6528  2014-07-20 09:06   _pytest/unittest.py
     6005  2014-07-20 09:06   _pytest/assertion/__init__.py
    12928  2014-07-20 09:06   _pytest/assertion/newinterpret.py
    18272  2014-07-20 09:06   _pytest/assertion/oldinterpret.py
     1921  2014-07-20 09:06   _pytest/assertion/reinterpret.py
    25417  2014-07-20 09:06   _pytest/assertion/rewrite.py
     9999  2014-07-20 09:06   _pytest/assertion/util.py
     1061  2025-02-07 21:00   pytest-2.6.0.dist-info/LICENSE
     2862  2025-02-07 21:00   pytest-2.6.0.dist-info/METADATA
      109  2025-02-07 21:00   pytest-2.6.0.dist-info/WHEEL
       67  2025-02-07 21:00   pytest-2.6.0.dist-info/entry_points.txt
       15  2025-02-07 21:00   pytest-2.6.0.dist-info/top_level.txt
     3039  2025-02-07 21:00   pytest-2.6.0.dist-info/RECORD

pdm

Pros

pdm does a great job as a backend engine for building Python packages. The pdm init command is vital, if you want to create a pyproject.toml file quickly. And creating virtual environments with pdm will ensure that all your Python packages are colocated with the code that uses them, making it easier to deploy and manage multiple projects on remote virtual machines.

Cons:

  1. site-packages/ are redownloaded or copied from cache inneficiently (slow and bulky)
  2. pdm install complies with pypi’s restriction disallowing installation of dependencies from direct source (git) URLs (must manually pip install -e local clones)

uv

From the makers of ruff, uv is Rust wrapper for your virtual environment package installer. It’s designed to replace pipx, so it does a good job isolating your virtual environments for command line apps that you want to be available everywhere. Like pipx if you uv pip install a command line app, that command will be available in your path, and it will use a virtual environment for that one package when it runs in your global environment.

As a bonus if you run pip within uv you will typically be done 100x faster, and with 10x disk space filled up. UV doesn’t get its speed and space savings from Rust but from the smarter algorithm it uses for caching packages downloaded from PyPi. And you don’t have to install rust to install uv. It comes as a standalone binary for linux, Mac, and Windows, though it is a bit more complicated to install the binary on Windows. And if you don’t like installing binaries on your PC, you can use pip instead.

UV is fast because:

  1. it uses the same download cache that pip and pdm use
  2. It hardlinks between your different site-packages directories

So what is the main difference between uv and its predecessors.

  • pdm gives Python packaging and dependency management a UX and API like npm
  • pipx is pip + venv for command-line apps in your global $PATH, each with it’s own venv
  • uv replaces pipx and works on non-cli Python packages and is faster more space-efficient (shares packages across .venvs)

The loopwerk.io blog reviewed and used all 3 packages. They switched all their packages over to UV a year ago and it’s only kept getting better and better. I’ve been using pdm for a year, and pip for 10+ years, and I’m switching to uv on my projects, because I can’t fit everything I need on my laptop without it. The only downside I can see is that it’s built in Rust so the builders may not be as Pythonic as you’d like.

Disadvantages

UV doesn’t create .venv/ with a name that reflects the project or python version.

.workon_

source .venv/bin/activate || ( pdm venv create 3.11 && source .venv/bin/activate && uv pip install -e . )

This way pdm creates the virtual environment and the initial pyproject.toml and UV handles dependency management and install packages to virtual environments.

bash_aliases.sh

alias pip='uv pip'
alias venv='pdm venv create'
$ source .venv/bin/activate
(mesa_python-3.11) hobs@ryz:~/code/mesa_python/mesa_python$

References