In GUIX, how to use a old version of a package, no longer in the channel?

Question

Part of what attracted me to GUIX is that various different versions of packages can be 'installed' at the same time without interfering with each other. But I can't figure out how to actually use those different versions.

E.g. recently, the pyyaml package was upgraded from 5.4.1 to 6.0. For various reasons, I want to keep using 5.4.1. (I'm just using pyyaml as an example here.) I do have the older versions in my store:

$ ls -d1 /gnu/store/*pyyaml*
/gnu/store/22v8l25b33vs65wjd9ap28n772bvlih3-python-pyyaml-5.4.1/
/gnu/store/2j2s1jd6y8x7mlqjp968955misx1qw1c-python-pyyaml-6.0/
/gnu/store/54imz4x65s3xbjrgrfswgk815gfkhk4b-python-pyyaml-5.4.1/
/gnu/store/6537a8na1rbilffqqi642q0lipqfi2zg-python-pyyaml-5.4.1.drv
/gnu/store/6flrrmhq203vg6awdw7r2lsmzix4g2rh-python-pyyaml-6.0-guile-builder
/gnu/store/73k3qdz9rdh64pl7a0f0951zm2pbx5s2-python-pyyaml-5.4.1.drv
/gnu/store/7bcbwi93ihz8v2sdzmj6l9vhjqaxr14l-python-pyyaml-5.4.1-builder
...

How can I use these older versions?

It would be fine to use such an older version only in isolation. For example, I was hoping something like this could work:

$ guix shell "[email protected]" python
guix shell: error: python-pyyaml: package not found for version 5.4.1

This error is expected, because that older version is not available in my channels. So maybe it is possible to somehow specify an older version of the channel to be used, but I cannot figure out how.

---

Side-node about the XY-problem, the immediate cause for this question is that docker-compose now does not work anymore:

$ guix shell docker-compose
guix shell: error: build of `/gnu/store/8qhvnw5mwra9i6ji24xlywcpdl0rdznn-docker-compose-1.29.2.drv' failed
$ zcat /var/log/guix/drvs/8q/hvnw5mwra9i6ji24xlywcpdl0rdznn-docker-compose-1.29.2.drv.gz
...checking requirements: ERROR: docker-compose==1.29.2 ContextualVersionConflict(PyYAML 6.0 (/gnu/store/igfl4023dzvl8vi6xs1m96lcsr4fdw07-python-pyyaml-6.0/lib/python3.9/site-packages), Requirement.parse('PyYAML<6,>=3.10'), {'docker-compose'})

However, I do not care particularly about docker-compose (w.r.t. this question). If anything, this question is part of my journey to replace that with GUIX-native tools.

(Also, I'm aware that pyyaml 6 forces some safety features on its users, so pyyaml 5 should not be used anymore; pyyaml is just used as an example.)

Answer 1

Third time is a charm, using a manifest with inferiors seems to be the best.

See info guix Inferiors for details. Here a snapshot of the info page. (Interesting that their use-case is very similar to mine: using an older version of a json library for the guile interpreter versus using an older version of a yaml library for the python interpreter.)

     Note: The functionality described here is a “technology preview” as
     of version 0.0-git.  As such, the interface is subject to change.

   Sometimes you might need to mix packages from the revision of Guix
you’re currently running with packages available in a different revision
of Guix.  Guix “inferiors” allow you to achieve that by composing
different Guix revisions in arbitrary ways.

   ...

   When combined with channels (*note Channels::), inferiors provide a
simple way to interact with a separate revision of Guix.  For example,
let’s assume you want to install in your profile the current ‘guile’
package, along with the ‘guile-json’ as it existed in an older revision
of Guix—perhaps because the newer ‘guile-json’ has an incompatible API
and you want to run your code against the old API.  To do that, you
could write a manifest for use by ‘guix package --manifest’ (*note
Invoking guix package::); in that manifest, you would create an inferior
for that old Guix revision you care about, and you would look up the
‘guile-json’ package in the inferior.

It continues with an example. Below is the same example adapted to this particular use case:

$ cat mypyyamlmanifest.scm
(use-modules (guix inferior) (guix channels)
             (srfi srfi-1))

(define channels
  (list (channel
         (name 'guix)
         (url "https://git.savannah.gnu.org/git/guix.git")
         (commit
          ;; The commit with the python-pyyaml 5.4.1
          "d3e1a94391a838332b0565d56762a58cf87ac6b1"))))

(define inferior
  (inferior-for-channels channels))

(packages->manifest
 (list (first (lookup-inferior-packages inferior "python-pyyaml"))
       (specification->package "python")))

$ guix shell -m mypyyamlmanifest.scm 
...

$ python3 -c "import yaml; print(yaml.__version__)"
5.4.1

---

A reflection on why it took so long to find this information. (Maybe GUIX developers would read this, maybe I can use this to provide a patch to the documentation myself.)

guix info guix package was my starting point for using manifests. The --manifest description has short examples on how to make manifests, and does not talk about the channels. There is a link to the --export-manifest description that I followed to learn how to recreate the environment created for my first answer. That section explains that this export does not include the channel information, which led me to conclude that manifests do not contain channel information at all and require a second file (my second answer). The --export-manifest descriptions links to --export-channels right below it, which at first I didn't read because I just adapted my existing channels.scm. However, that section finally explains that inferiors are needed.

It would perhaps have been easier for me to understand if the --manifest section of guix package would explain that it is possible to define channels in the manifest directly.

If I understand correctly, inferiors are technically different from channels, so the last sentence above is wrong and it is not possible to define channels in manifests. However in practice it seems that one could just use the inferior for all the packages in the manifest, thereby effectively hardcoding the channel in the manifest. This makes me wonder if it wouldn't be simpler to just allow channel specification in the manifest directly.

---

While answering the question, this solution is still not enough to solve the initial problem that triggered the question: running docker-compose, because docker-compose still uses the python-pyyaml from the original channel. It is possible to let a package use an inferior as input using modify-inputs. This makes it possible to let only docker-compose use the outdated python-pyyaml from the inferior, and let the rest of the profile use the newer python-pyyaml from the original channel.

Answer 2

Thanks to the excellent info guix page, I figured that this works:

guix shell \
    --with-git-url=python-pyyaml="https://github.com/yaml/pyyaml.git" \
    --with-branch=python-pyyaml="release/5.4.1" \
    python-pyyaml python

But it feels a bit hackish if there is already a perfectly fine package definition in https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/python-xyz.scm?h=d3e1a94391a838332b0565d56762a58cf87ac6b1#n3907.

As in, pyyaml is quite a simple package, which is nicely hosted on github, following standard git branching and Python installation procedures. But some packages might be more complex where such a simple substitution might not work.

Is there some way to specify a specific commit on a channel git url to use for a package? (Or for a complete guix shell command?)

---

Edit: another reason this solution is not great because it doesn't work with --export-manifest:

$ guix shell \
    --with-git-url=python-pyyaml="https://github.com/yaml/pyyaml.git" \
    --with-branch=python-pyyaml="release/5.4.1" \
    --export-manifest \
    python-pyyaml python > mymanifest.scm

$ guix shell -m mymanifest.scm
guix shell: error: the source of [email protected] is not a Git 

with mymanifest.scm:

$ cat mymanifest.scm
;; What follows is a "manifest" equivalent to the command line you gave.
;; You can store it in a file that you may then pass to any 'guix' command
;; that accepts a '--manifest' (or '-m') option.

(use-modules (guix transformations))

(define transform1
  (options->transformation
    '((with-git-url
        .
        "python-pyyaml=https://github.com/yaml/pyyaml.git")
      (with-branch . "python-pyyaml=release/5.4.1"))))

(packages->manifest
  (list (transform1
          (specification->package "python-pyyaml"))
        (transform1 (specification->package "python"))))

Answer 3

This is perhaps more GUIXy:

• Create a channels file from the commit that still has pyyaml 5.4.1.
• Create a profile from that channels file.
• Activate that profile.
• Create a manifest with the desired packages.
• Create a shell with that manifest.

$ cat mypyyamlchannels.scm
(list (channel
        (name 'guix)
        (url "https://git.savannah.gnu.org/git/guix.git")
        (branch "master")
        (commit
          "d3e1a94391a838332b0565d56762a58cf87ac6b1")
        (introduction
          (make-channel-introduction
            "9edb3f66fd807b096b48283debdcddccfea34bad"
            (openpgp-fingerprint
              "BBB0 2DDF 2CEA F6A8 0D1D  E643 A2A0 6DF2 A33A 54FA")))))

$ guix pull -C mypyyamlchannels.scm -p mypyyamlprofile

$ GUIX_PROFILE="$(realpath mypyyamlprofile)"

$ . "$GUIX_PROFILE/etc/profile"

$ guix shell python-pyyaml python --export-manifest > mypyyamlmanifest.scm

$ cat mypyyamlmanifest.scm 
;; What follows is a "manifest" equivalent to the command line you gave.
;; You can store it in a file that you may then pass to any 'guix' command
;; that accepts a '--manifest' (or '-m') option.

(specifications->manifest
  (list "python-pyyaml" "python"))

$ guix shell -m mypyyamlmanifest.scm

This has two drawbacks though:

• It is kinda verbose. It requires 2 files (channels and manifest) and 4 commands (guix pull, set GUIX_PROFILE, source profile, guix shell) and 2 symlinks (mypyyamlprofile, mypyyamlprofile-1-link) to recreate the same environment. The not-working transformation solution would require only 1 manifest file and 1 guix shell command. Is there perhaps a way to combine the channels and manifest file into a single thing that then can be used to instantiate a shell in a single command?
• It doesn't allow mixing and matching from the main channel. I suppose that mixing would require creating a channels file that combines the default channel and then somehow cherry-picks the python-pyyaml package from the channel with that specific commit. This could then introduce inconsistencies, but those could probably be resolved.

---

Edit: Apparently Inferiors would allow mixing and matching easier:

   Sometimes you might need to mix packages from the revision of Guix
you’re currently running with packages available in a different revision
of Guix.  Guix “inferiors” allow you to achieve that by composing
different Guix revisions in arbitrary ways.