0

I have a command line utility which is able to receive an image via stdin and convert it to a different format and output it to stdout.

I use this from a program, but every time I run it, a subprocess needs to be invoked which is very slow.

So I wonder is there a way, ideally using systemd, to turn this command line program into a "server" that remains in memory and somehow is able to receive image data and output it?

Duke Dougal
  • 965
  • 3
  • 16
  • 27
  • If you are familiar with how to create a daemon with `systemd`, then yes. There are plenty of examples on Google that you can use as a template. – Nasir Riley Mar 21 '22 at 00:05
  • @NasirRiley are you sure you understand the question? – Duke Dougal Mar 21 '22 at 00:12
  • It might help to tell us what this command line program is. – frabjous Mar 21 '22 at 00:15
  • The question is, do you understand how what you are trying to do would function? You that you have a command line utility and that you want to run it `ideally using systemd`. If you want to use it with `systemd`, then you need to create a daemon for it. You can also add more information on how it functions and what it's supposed to do to even determine if `systemd` is viable for it. – Nasir Riley Mar 21 '22 at 00:16
  • @frabjous the command is cavif https://github.com/kornelski/cavif-rs it converts images to avif format. – Duke Dougal Mar 21 '22 at 00:19
  • That program doesn't really seem that slow. Does using `--speed=7` degrade the image too much? For me, it speeds things up about 4x over the default speed. – frabjous Mar 21 '22 at 00:44
  • @frabjous - cavif isn't slow. What is slow is when my Python program calls it as a subprocess 20 times a second. I'm wondering if instead I can keep cavif in memory as some sort of daemon/service which Python can then call. – Duke Dougal Mar 21 '22 at 00:46
  • Wouldn't it still require a subprocess for Python to "call" the service/daemon? Marcus's suggestion has more merit; maybe checkout [PyO3](https://github.com/PyO3/pyo3) and/or other ways of using Rust libraries in python. – frabjous Mar 21 '22 at 00:53
  • 2 ways: **Way # 1.** Use `inotifywait` [example1](https://unix.stackexchange.com/a/323919/286615), put it in a script & *daemonize* by `nohup script &`; **Way # 2.** Create a `systemd` 'Path Unit' [example2](https://www.putorius.net/systemd-path-units.html) – Seamus Mar 21 '22 at 09:23

1 Answers1

0
  1. you could of course start the program ahead of time, but not send anything to its stdin yet; then, when you need it, it's already started. Of course, after you've "used" it, the process would be dead, and you need to start another one. But assuming conversion and writing to a file isn't instantaneous either, you could already start the next process and hold it ready for a subsequent image. In fact, you could have a pool of waiting processes. Pretty standard technique from the early days of Linux webservers – "worker processes". (This all has nothing to do with systemd.)
  2. This is a library. The right thing to do is not to hand of data to a separate process, if that handoff is costly (somehow I doubt that's the case – yes, process creation is costly, but Linux will cache your executable file…). Instead, simply use the cavif library from your program directly to convert the data. That's what libraries are for.
Marcus Müller
  • 21,602
  • 2
  • 39
  • 54