0

Considering:

cat filename | while read i
do
    echo $i
done

all other answers failed, this one works! this one will cut text file to line by line.
I want to know more about the key part read. But I cannot found the source code. which read no output. what is "read" in this context?

roaima
  • 107,089
  • 14
  • 139
  • 261
jian
  • 539
  • 3
  • 15
  • The first link you provided. – Marcus Müller Aug 13 '23 at 06:59
  • See `info bash read` and [Understanding "IFS= read -r line"](https://unix.stackexchange.com/q/209123) and [Why is using a shell loop to process text considered bad practice?](https://unix.stackexchange.com/q/169716) and [Why is printf better than echo?](https://unix.stackexchange.com/q/65803) – Stéphane Chazelas Aug 13 '23 at 07:06
  • @StéphaneChazelas thx. my use case is cut text line by line then copy to database. since sql have delimiter and quote other messy case. while read i successfully copied to database. will try printf. – jian Aug 13 '23 at 07:23
  • 1
    You'd be better of generating the whole SQL code from the input using things like awk bearing in mind that it's very hard to avoid SQL injection vulnerabilities that way (same with your read+echo approach), or use proper programming languages like `perl` that have modules to talk to databases. – Stéphane Chazelas Aug 13 '23 at 07:30
  • Most database systems have a configurable fast-loader tool (which will for example fix up indexes all at once at the end, instead of updating after every commit). First time I used Oracle's SQL*Loader, I ran it about five times because I simply did not believe it had done anything useful that fast. – Paul_Pedant Aug 13 '23 at 08:12

2 Answers2

3

this one works

Actually it doesn't. The read will parse the line and rebuild it according to shell rules. For example,

echo '  hello  world  ' |
    ( read x; echo ">$x<" )

The captured $x has neither leading nor trailing space:

>hello  world<

(Change read x to IFS= read -r x and compare the result.) Further, by not double-quoting the variable $x when you use it, it is subject to further processing. For example,

touch a b c    # Create three empty files
x='here is my star: *'

echo "$x"      # Outputs: here is my star: *
echo $x        # Outputs: here is my star: a b c

In your case the files a, b, c will also be supplemented by the set of files you have in your current directory. (I should point out that touch only creates empty files as a side effect; if the files already exist it will simply update the last-modified time to now and will most definitely not truncate or recreate them.)

After all that, you won't be able to use the value of $x outside the loop as you've got the loop in a subshell. (Set x='demo' before the loop starts and then echo "$x" after the loop ends. It will have been unaffected by the loop.) You don't need the cat in this case as there's only one file:

while IFS= read -r x
do
    printf '%s\n' "$x"
done <filename

But even after all that, typically there is little need to loop over input like this as many tools process files line by line anyway. c.f.

cat filename
roaima
  • 107,089
  • 14
  • 139
  • 261
  • Back in SunOS 4 days, `cat` was in `/usr/bin` and so not available at boot time. So in `/etc/rc.boot` we could find a function definition for `shcat` which was "good enough". Basically `shcat() { while test $# -ge 1 do while read i do echo "$i" done < $1 shift done } `. Was sufficient to allow boot processes to read config files (eg `hostname.??0`) :-) – Stephen Harris Aug 13 '23 at 12:28
  • @StephenHarris while I liked SunOS 4 at the time (and SVR3, just to play fair) I don't think I'd choose to go back to it now :-) – roaima Aug 13 '23 at 12:38
2

This is the read shell builtin command, which is specified by POSIX. The Bash implementation is described in the Bash manual, and its source code is part of the Bash repository.

Stéphane Chazelas
  • 522,931
  • 91
  • 1,010
  • 1,501
Stephen Kitt
  • 411,918
  • 54
  • 1,065
  • 1,164