Pipe demon output to a function

Question

I'm receiving a MQTT message with mosquitto_sub and I want to redirect the output (two strings) to a function.

CLIENT:

mosquitto_pub -d -t hello/world/ -m "Alessio,base64"

On the server I have a bash program

SERVER: (it works)

function pipeTee(){
        tee -a mqtt_broker.log
}
mosquitto_sub -t +/# | pipeTee

but if I try t pipe to another function too, the program doesn't work.

#!/bin/bash
function testPipe(){
        echo "va"
}

function pipeTee(){
        tee -a mqtt_broker.log
}
mosquitto_sub -t +/# | pipeTee | testPipe

mosquitto_pub from client:

mosquitto_pub -d -t hello/world -m "Aless,base64"
Client mosqpub/1841-raspberryp sending CONNECT
Client mosqpub/1841-raspberryp received CONNACK
Client mosqpub/1841-raspberryp sending PUBLISH (d0, q0, r0, m1, 'hello/world', ... (12 bytes))
Client mosqpub/1841-raspberryp sending DISCONNECT

Server when not working:

pi@raspberrypi:~ $ ./mqtt_broker.sh 
va
^C

and the file log is empty

Does `mosquitto_sub` exit back to the shell after running? What is the output of that command? — Kusalananda, Jul 07 '16 at 11:19
yes, but the problem is in in the shell script. It0s normal for mosquitto_pub to exit after the command — Mitro, Jul 07 '16 at 11:20
your `testPipe` function exits after its echo. This means any writes from the tee will receive a signal SIGPIPE broken pipe, and so tee will die. You would need, eg, to add a `cat` to the end of your function. — meuh, Jul 07 '16 at 12:01

Kusalananda · Accepted Answer · 2016-07-07T14:22:18.037

Consider the following shell script:

function thetee {
    tee data.out
}

function thepipe {
    echo "Hello!"
}

while true; do
    echo "Hi there!"
done | thetee | thepipe

echo "Done."

The while-loop here is a generator of unlimited lines with the text "Hi there!".

Running it:

$ sh script.sh
Hello!
Done.

Then look at the number of lines in data.out:

$ wc -l data.out
     274 data.out

Again:

$ sh script.sh
Hello!
Done.

$ wc -l data.out
     236 data.out

Issues:

The infinite loop isn't infinite.
The number of lines of output differs between runs.

Reasons:

The thepipe function in my example simply echoes a string, then exits, causing the pipe to fail. No further input can be processed by the last stage of the pipeline and the loop exits.
The number of lines actually written to data.out depends on how quickly the shell is able to start up the pipeline and how quickly the thepipe function can do its thing and exit.

In your case, this happens so quickly that no input from the data generator (mosquitto_sub) is able to arrive to the tee function.

Solution:

Make thepipe function actually consume its input:

function thepipe {
    echo "Hello!"
    cat
}

This will make the code in my example output a single "Hello!" followed by infinite "Hi there!" (which are also saved into data.out). The echo at the end will never be executed.

Therefore:

Change the definition of you testPipe function in a similar manner. For example:

function testPipe(){
        echo "va"
        cat
}

This will have the effect of consuming the input, and sending it on to the next stage of the pipeline, or, if it's at the end of a pipeline, to wherever standard output is redirected after it (with a line containing the string va at the start of the stream).

Pipe demon output to a function

1 Answers1

Linked