0

I'm trying to curl a txt file and pipe it to pip.

Example:

curl -s URL | pip install -r -

It doesn't work beucase the last - is not the curl output as I'd expect. I often have this issue with other commands too.

How to fix this with bash?

Freedo
  • 1,205
  • 6
  • 31
  • 57
  • pip does not accept requirements from pipe/stdin – RomanPerekhrest Nov 04 '19 at 07:00
  • What do you mean does not accept? I thought this was a bash / linux thing, and transparent to whatever command you are running – Freedo Nov 04 '19 at 07:01
  • I remember I once had the same issue downloading a .sql file stored on the web (it's just plain text) and using it with mysql command to restore a backup. I think I had to use something like << together with - to work. I don't remember exactly and I lost the code because of power loss – Freedo Nov 04 '19 at 07:03
  • 1
    I'm not entirely sure what you're expecting to happen, but some of this is certainly *not* transparent to the command. In your command, `pip` receives the arguments "`install`", "`-r`", and "`-`". It is completely up to the `pip` executable how to interpret these. It also has the output of `curl` available on its standard input ("stdin"), so if it reads from that it'll get the downloaded file. If it doesn't read from stdin, it'll never see the downloaded file at all. – Gordon Davisson Nov 04 '19 at 07:53
  • Is there a way to force it to read frm stdin? with some unix magic like redirection or something? I really love building 1 line commands with curl + storing stuff on URLs – Freedo Nov 04 '19 at 07:55
  • Again, I'm not entirely sure what your goal is, but take a look at: ["Using data read from a pipe instead than from a file in command options"](https://unix.stackexchange.com/questions/16990/using-data-read-from-a-pipe-instead-than-from-a-file-in-command-options) – Gordon Davisson Nov 04 '19 at 08:08
  • 2
    @Freedo, using `-` to mean _stdin_ is indeed a feature of the program in question, not that of the shell. The usual workaround is to use `/dev/stdin`. – ilkkachu Nov 04 '19 at 09:57
  • @ilkkachu This is the answer I needed, thanks. using /dev/stdin works. It's sad they closed my question since I think a lot of people have the same issue but don't know how to google it very well – Freedo Nov 04 '19 at 10:20
  • @Freedo, well, closing as a duplicate basically just links the existing questions to this. They already have answers on doing basically the same thing (not with `pip`, I guess, but this isn't specific to `pip` anyway.) It shouldn't mean this question would disappear. – ilkkachu Nov 04 '19 at 11:35

1 Answers1

3

Using - as a method to pipeline information which could also come from a file is a fairly common way for things to work, but it is not a given.

I’m assuming your URL contains only a flat list of packages, IE no html:

pip install -r <(curl -s URL)

Or

pip install $(curl -s URL)
bxm
  • 4,561
  • 1
  • 20
  • 21
  • Yes this doesn't work for pip. But you can use /dev/stind as @ilkkachu said – Freedo Nov 04 '19 at 10:18
  • Apologies, I didn't have `pip` to hand to verify how `-r` behaves. Should be able to do this via `<(...)` -- answer edited. – bxm Nov 04 '19 at 10:57