18

I'm a bit of a newcomer to Arch Linux and still getting used to some of its paradigms. I come with a lot of habits from another distro that was quite a bit more structured and in some ways predictable.

One thing I would like to do with a few systems¹ is enable a cron job to do automatic updates of all system packages. This seems like it would be pretty easy except I also want some meaningful feedback from the system telling me how things went that was not so verbose that I end up ignoring it until I discover the system is acting up.

  • The full output of pacman is unnecessary. I don't care how long the downloads took or that it is on update 46 of 53.
  • For the most part I don't care about successes.
  • I do care about errors. If an update run fails I want to know about it and any specific error messages should be patched through.
  • I do care about the "notices" given off during install. For example today's systemd update said:

    :: coredumps are no longer sent to the journal by default. To re-enable:  
    echo >/etc/sysctl.d/50-coredump.conf \  
        "kernel.core_pattern=|/usr/lib/systemd/systemd-coredump %p %u %g %s %t %e"
    

    Or filesystem produced this:

    warning: /etc/group installed as /etc/group.pacnew  
    warning: /etc/passwd installed as /etc/passwd.pacnew  
    warning: /etc/shadow installed as /etc/shadow.pacnew
    

    This last category is actually what prompted me to ask this question because it seems these are very inconsistent across the package set. Some of these seem to be generated by post_upgrade(), others by install(), etc. Sometimes they are written to stdout, sometimes to stderr. The format of the messages varies widely: sometimes a whole block is prefixed on indented in some way, other times there is just a bare echoed string.

I want to know about things that may require my intervention on a system but not be bothered otherwise. Is there a tool out there that intelligently manages this data and simplifies the system administration? Is there any way to get the output generated by packages separately from the pacman process installing them? Or am I on my own to write some kind of parser that filters out benign stuff from the install log?

¹ Before you jump all over how stupid this is please note that I'm smart enough to not do this on production servers and not without snapshot based full system backups that would make recovery easy in the event of a catastrophe.

HalosGhost
  • 4,732
  • 10
  • 33
  • 41
Caleb
  • 69,278
  • 18
  • 196
  • 226
  • The first thing I would do is remove the `yaourt` tag, both from your question and your conceptual model: trying to do this with the official repos will be difficult enough, adding random packages from the AUR to the mix will make it quixotic... – jasonwryan Jun 25 '14 at 08:20
  • Oh, and this isn't a stupid idea; but it very much pushes against the way Arch works. Using `checkupdates` and `-Syu`ing once a day is a pretty trivial task; I don't see any actual benefit for all the work your proposal would entail. – jasonwryan Jun 25 '14 at 08:25
  • @jasonwryan I meant to expand on that, but I do actually have some AUR packages I would like to fit in the same workflow. I currently make extensive use of yaourt and sorely miss the fact that there is no dry-run support for package management operations! :/ – Caleb Jun 25 '14 at 08:26
  • 2
    The DNA of Arch is getting your hands dirty: this is just not the way it is intended to be run; you may get there, but it will be a Pyrrhic victory and, if documented here, will just lead a whole lot of newbies to the edge of a cliff that they are completely unprepared for... – jasonwryan Jun 25 '14 at 08:29
  • 2
    I spent a while trying this last year and came to the conclusion that the only way to achieve upgrades without regularly killing systems was to build new images (which seems to be the base testing point and more reliable in Arch) then transplant service's with a config management tool. Maybe updates have settled down a bit but there were 3-4 in a 6 month period that would hose a system without manual intervention. – Matt Jun 25 '14 at 08:42
  • The other process I had was an internal pacman cache built from successful test installs to release from (ala Satellite), but then I realised I was doing a lot of work other distro's do for you, and moved on. – Matt Jun 25 '14 at 08:42
  • I don't have a lot of experience nor expertise with it, but I have a feeling Gentoo's [portage](http://dev.gentoo.org/~zmedico/portage/doc/man/portage.5.html) can do something like that easily, as emerge has many different quiet flags for instance. There is also re: config files... [dispatch-conf](https://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=3&chap=4) which is in line with some of your points. But that's no Arch! –  Jun 27 '14 at 03:39
  • @mikeserv my point is that in documenting an approach here, people who are not familiar with Arch (or Linux in general) will think that it is a) a good idea, and b) a viable approach. It is clearly neither... – jasonwryan Jun 27 '14 at 09:47
  • @Caleb - sorry. removed. wait a minute, though - I don't see any mention of the Arch Build System... That was the crux of what I was suggesting. – mikeserv Jun 28 '14 at 00:25
  • @mikeserv If the ABS provides a way to deal with this administiation issue, then please explain how in an answer. – Caleb Jun 28 '14 at 20:52
  • The thing is that this is a *really* broad subject, and it can be handled *many* ways. `yaourt` has the `-c` option for merging configs; `pacbuilder`, `pacman`, `pacmatic` and/or `yaourt` can all be configured to build and maintain local repos - and `yaourt` can automatically backup and restore `alpm` databases. Both the `aur` and `abs` tree alike can be handled with something as simple `jq` in the cli if you use the `json` interface. `systemd-nspawn` can spin up testing containers at any time, and if you use `btrfs` snapshotting is a breeze. `systemctl` can snapshot config state anyway. – mikeserv Jun 28 '14 at 21:09
  • But whether you can or will configure all or any of those things - and the probably dozens of other possible methods that dont come to mind at the moment - for yourself depends mostly on what its worth to you. Does a local repo for a typical desktop user have any kind of roi? Doubtful, not counting the learning experience. I highly recommend `btrfs`, but that usually means a disk wipe for others, though the same could be effected with dm, i guess. The journal, configs, alpm backups and similar all come down to commitment i guess. And im just not committed enough to answer. – mikeserv Jun 28 '14 at 21:19
  • Oh - i dont mean *can* like *capable*, its just that engineering this kind of solution amounts to a highly personal and systemic endeavor with a lot of interlocking dependencies - i meant *can* like *feasible*. I only now reread it. – mikeserv Jun 28 '14 at 22:23

3 Answers3

13

I want to be clear for both you and any other person who might happen upon this thread. What you want to do is not possible. Arch makes no secret of the fact that it expects you to administer your system. Part of that responsibility is being present for the update cycle.

Now, there are steps of the update process that you can automate responsibly, but it will never amount to an automated -Syu. You simply need to be there for the upgrade.

What you can do, for example, is to set up a cron job that downloads any packages that need to be installed (but not install them). The following is an excerpt from the pacman documentation:

SYNC OPTIONS
...
-w  --downloadonly
    Retrieve all packages from the server, but do not install/upgrade anything.

You could then, setup a cron job to run sudo pacman -Syuw. I'm not terribly familiar with cron or it's scheduling abilities. However, if it is capable of only running a job after a particular condition has been met (for example, a command returning a particular value), then it would be fairly trivial to trigger the downloading of new packages as needed.

I would imagine running a check against checkupdates | wc -l, for example. If it returns any number above zero, you could trigger the download. Again, however, this will not replace running an -Syu when you are present and ready to deal with the upgrade.

For the final part of your question, where to get output of what happened during an upgrade, you're looking for /var/log/pacman.log. It includes all error information that might interest you.


There is a tool that is meant to make pacman a little more friendly for unfamiliar users. If I'm not mistaken, it also provides a notification about news items if there should be an update that requries manual intervention.

This tool might make your life a little simpler in the short run, but it will never be a replacement for sensibly and diligently following the Mailing Lists.

HalosGhost
  • 4,732
  • 10
  • 33
  • 41
  • 2
    Actually, for notices about official packages that will require manual interventions, I would recommend that you follow [arch-dev-public](https://mailman.archlinux.org/mailman/listinfo/arch-dev-public) and [arch-general](https://mailman.archlinux.org/mailman/listinfo/arch-general). – HalosGhost Jun 27 '14 at 02:22
  • Also, @jasonwryan is entirely correct. Doing this with `pacman` might be possible. Doing it with `yaourt` is not feasible. – HalosGhost Jun 27 '14 at 05:07
  • `yaourt` actually does a slightly better job of this than `pacman`, though, as i think you mean, its default configuration as little more than an aur helper would almost certainly wreck things in the long run. It is, however, extremely configurable, and can be made to maintain source trees very effectively. Both `yaourt` and `pacman` only wrap `alpm` anyway. – mikeserv Jun 29 '14 at 04:47
  • Or, i guess it is truer to say, `yaourt` wraps `pacman` and some additional `alpm` functionality and `pacman` wraps `alpm`. In any case though, `yaourt` extends `pacman`. – mikeserv Jun 29 '14 at 05:02
  • As noted [here](https://bbs.archlinux.org/viewtopic.php?pid=1315813#p1315813), `-Syuw` is potentially quite dangerous. It has the ability to produce partial updates if the updates are not applied, and software is later installed with `-S`. – Sparhawk Oct 04 '19 at 00:32
1

There is an old script which does a really good job; it is called safepac.

EDIT: This script is still available on github here: https://github.com/bencahill/binfiles/blob/master/safepac

What does it do and how does it work?

The way I usually update Arch is to read the news and then do pacman -Syu, or to just do pacman -Syu and if anything goes wrong, read the news. Now this script does nothing else: It gets the latest news entries from the RSS feed, does some sed magic, and compares the names of all packages that have updates available to the news. It matches " package-name " and "package-name-" in the news, and if the package name appears somewhere in the news, it will ignore the package and only update all the others. This way, anything that might go wrong will be postponed until one has the time to inspect the matter by hand, while non-critical updates will just be done automatically!

Usually, once you have taken care of a possibly non-smooth update, you will not want to have safepac still ignore a certain package because it's still in the news. That's what the ignorelist is for. So every time you've updated manually, you can add the respective news entry to the ignore list with safepac -Ia xxx, where xxx is the number of the news entry. You can also specify the number of latest news you want to include in its parsing with the -n switch. The script is also smart enough to skip non-existing entries. If you encounter problems with packages that have very short names that might arbitrarily appear in the news (like "yes") you can add them to the whitelist with safepac -Wa so that they will always be installed even if they appear in the news.

Please note that you need a working mail setup as this script also sends you a digest mail (with full log?), and with salient parts of the update:

parsing of pacman output to look for these words and add it to the digest:
"note" "pacnew" "error" "important" "warning" "exists".

The script is old, but the idea is good and probably still works with minor update, if any.

Lesto
  • 169
  • 3
  • one possible issue is major (or breaking) update are not told in the news, but you have to track by yourself. Case in point, update from python2 to python3 – Lesto Aug 29 '18 at 19:47
  • Still, you could use checkupdates (pacman-utils) and get the list of packages, and then remotely whitelist them, good if you have a pool, but then maybe is better to setup something like a local repo – Lesto Aug 29 '18 at 20:35
-2

I use my own untility to run yaourt by cron.

vadzim
  • 101
  • 1
    Yaourt [is effectively dead](https://github.com/archlinuxfr/yaourt/issues/382): this is a spectacularly bad idea. – jasonwryan Dec 13 '18 at 17:57
  • I used `yaourt` for many years myself. It was a good tool, but the things it was build to "wrap" have changed shape and it does several things quite wrong now. At this point it is so problematic that the authors of the tool have said even they are not using it themselves any more. If you want an AUR helper I suggest you try one of the actively maintained ones. I switched to `yay` myself and am quite happy. – Caleb Dec 14 '18 at 13:30
  • That utility works with `yay` for me as well. – vadzim Dec 14 '18 at 20:29