There are 2 seperate package universe. We can view them as 2 different libraries.
1 Highest Level Overview - Ecosystem
1.1 Python Ecosystem
This is the Python ecosystem. Packages are "wheels". Understood by pip/uv.
1.2 Conda Ecosystem
Packages are "conda packages" which can contain any language. Understood by conda/pixi.
2 The Libraries inside the Ecosystem
2.1 PyPI
- Stands for Python Package Index
- This is the one dominant library in the Python Ecosystem
- It is shipped as wheels/sdists
The tools/librarians used to access the python libs in this package universe are pip and uv. These can only install python packages.
2.2 conda-forge
- This is one of the many packages (formally known as channels) in the Conda Ecosystem
- It can install compiled C/C++/Fortran libs, the Python interpreter itself, a C++ compiler, CUDA, R, and UHD.
The tools/librarians used to access these libs are conda and pixi. Packages come from channels (repositories).
3 Tools for these Libraries
3.1 pip
pip installs Python Packages. It doesn't manage envs. We need to pair it with venv which is the old and universal method. pip freeze can be used as a lockfile but not a true resolved lock.
3.2 uv
uv is a fast (made with Rust) installer + venv + Python-version manager + project/lockfile. Lockfile is done with uv.lock automatically created and managed using uv sync, and is a true multi-platform lock.
3.3 conda
conda installs anything from python to system libs and allows for managed environments. It is powerful but historically a slow solver. A slow solver meaning when we run conda install pandas, the tool has to find versions of pandas and every dependency to ensure that every other package are also compatible. This is a slow solver because conda's original implementation of this process was inefficient. The fix was mamba, which is a C++ reimplementation using libsolv, which anaconda adopt it as libmamba. We can install add-ons like conda-lock to enable lock management.
3.4 pixi
pixi is a fast (built with Rust) project manager for Conda Ecosystem. It can manifest + lockfile + task runner.
Manifest:
- this is the
pixi.toml - the hand-written file where we can declare what we want at a high level
- It contains
[workspace]which is the project metadata with name, version, which channels to pull from (conda-forge) and which platforms to support (linux-64,osx-64,osx-arm64,win-64). - It contains
[dependencies]which has all the pacakges we need, with loose constraints such as[python = ">=3.11"oruhd = *which means "any version"
Lockfile:
- This is machine-generated, resolved solution. When we run
pixi install, the solver takes those loose constraints and computes the exact version of EVERYTHING. Including hundreds of transitive dependencies we never listed, that satisfies them for every platform we declare. It writes all out with exact versions, download URLs, and hashes.
Tasks:
[tasks]contains named commands tied to their actual full terminal command line for simpler running of different commands using just a single "task word"- for example,
app = { cmd = "python app/main.py", depends-on = ["build"] }allows us to simply run this in terminal line:pixi run app - tasks run inside the lockfile
How it all comes together:
- We edit the manifest (
pixi.toml) - when we use
pixi installto install any new dependencies, it will automatically read the manifest, solves and writes the lockfile, then builds that exact environment - when we run
pixi run <task>, pixi will execute oru named commands inside it