Simplify Your Bioinformatics Workflow with Pixi: A Fresh Take on Conda
/ 3 min read
Recently, Anaconda has decided to start charging users for access to the default channel. This change in policy has significant implications for the data science and machine learning community, as Anaconda is a widely used platform for managing Python and R packages.123 Prefix.dev has a nice article summarizing the whole thing.
The Python ecosystem has recently seen a proliferation of attempts to completely rework its package managers. Several new projects have emerged, each aiming to address in the existing package managers. An issue that isn’t really due that “pip and conda are bad”. pip
was released April 4th, 2011 and Anaconda
was first release July 17th, 2012. That’s long before I even took an intro CS course my freshman year of college, so I’m not gonna judge what everyone else was doing then.
The space is ripe for disruption. There have been some exciting complete rewrites that have come out recently with ruff and uv from Astral in the Python ecosystem. In the conda ecosystem, I’ve lost track of the different ways to install a conda package, Anaconda, miniconda, Mamba, micromamba, mambaforge. I can’t keep up.
I recently started using Pixi
from prefix.dev. It’s been really nice. I want to forget about a package manager. Pixi let’s me do that.
Pixi for Bioinformatics
Set up on a server/locally
Just thought I’d add a quick TL;DR here. This is how easy it is to get started. Works on your server, locally on your laptop. Makes me think of some other great bioinformatics tools…
Things every Bioinformatician loves to see
One way to install it just about everywhere
Problem: Which way should you install conda?
- Anaconda
- miniconda
- Mamba
- Micromamba
- mambaforge
Solution:
Environment activation is automatic
Problem: In every new terminal users have to run conda activate applied-genomics
.4
Solution: Every project has the same commands
pixi shell
pixi shell-hook
pixi run
Tasks are built-in
Problem: snakemake --cores 4
Solution:
Sure, you could have a Makefile, or reach for just. But that’s just one more tool everyone you collaborate with has to learn.
Storing environment details is taken care of for you
Problem: conda install
doesn’t update the environment.yml
5
Solution: pixi add
updates the pixi.toml
for you.
But wait there’s more! It locks the version to avoid major version bumps automatically!
In case python4 sneaks up on you.
No built in lock file
Problem: Versions in the environment.yml
are only half the battle…6
Solution: pixi add
updates the pixi.lock
file as well behind the scenes!
It’s gotta be difficult to migrate to, right?
With pixi you can import environment.yml
files into a pixi project.
This will create a new project with the dependencies from the environment.yml
file.
Footnotes
-
https://www.theregister.com/2024/08/08/anaconda_puts_the_squeeze_on/ ↩
-
https://www.linkedin.com/feed/update/urn:li:share:7229549722070310912/ ↩
-
Remembering the environment name in every project is really the issue. ↩
-
Which makes it really hard to reproduce research code as we’ve learned firsthand… ↩
-
Going to avoid going off into the weeds here, there is conda-lock. Reach out Jonathan Manning for his upcoming TED talk on conda lock files. ↩
Webmentions for this post
2 Mentions
Edmund Miller
Vist the source of this webmentionTL;DR my quick #bioinformatics #pixi setup commands: ``` curl -fsSL https://pixi.sh/install.sh | bash # source ~/.bashrc or logout pixi config append default-channels conda-forge --global pixi config append default-channels bioconda --global # Examples pixi global install -c bioconda nextflow pixi g i rclone nextflow ``` bioinformatics pixi
Christian Meesters
Vist the source of this webmention@emiller Thank you. Interesting summary! Side note: Wanted to try pixi on our new(!) cluster. Turns out, it cannot work with a kernel <5.10, but we run 4.18.0. ????
Responses powered by Webmentions