How to expand this awk column 2 selector with Bash parameter $1?

Question

I am trying to make my bash script a function with Bash input parameter but AWK's syntax is causing a problem. Original AWK code

http://stackoverflow.com/a/19602188/54964
awk -F "\"*,\"*" '{print $2}' textfile.csv

Pseudocode with Bash parameter $1

file=$(awk -v colN="$1" -F "\"*,\"*" '{print $"${colN}"}' "${input}") 
# http://stackoverflow.com/a/19602188/54964 
# http://stackoverflow.com/a/19075707/54964

The problem is the part print $"${colN}".

Current output fails to catch the second column and takes the whole line etc

-0.21,-0.245
-0.205,-0.22

Having only print $colN is not correct, since it takes then always the first column regardless of the value in $1.

Example of the use case where I call it by bash code.bash 2; or complete script here which works if you do not hard-code which column to choose (1/2) in all two-column CSV files for the joined result of second columns

#!/bin/bash
ids=(101 118 201)
dir="/home/masi/Documents/CSV/"
index=0
for id in "${ids[@]}";
do
        input=$(echo "${dir}P${id}C1.csv")
        # take second column of the file here
        file=$(awk -v colN="$1" -F "\"*,\"*" '{print $colN}' "${input}") # http://stackoverflow.com/a/19602188/54964 # http://stackoverflow.com/a/19075707/54964

        Ecgs[${index}]="${file}"
        index=$index+1
done

Inputs multicolumn 1.csv 2.csv 3.csv

-0.21,-0.245
-0.205,-0.22

Wanted output

101,118,201
-0.245,-0.245,-0.245
-0.22,-0.22,-0.22

OS: Debian 8.5
Bash 4.30

Answer 1

Your example input has the same values in the first and second field for all files (and the same values for all files), which doesn't really help understand the exact use case. After all, if you really want the same value three times and you can get it from any field of any input file, you don't even need to check the other two files. You can just use:

cut -d, -f2 input.csv | paste -d, - - -

Of course this doesn't work for real input, just your example input. (Work on improving your example input/output for this type of question, it helps a lot.)

---

If we make the assumptions that:

• You always have exactly three input files
• Called input1.csv, input2.csv, input3.csv
• With exactly two columns each
• And you want the second column from each file

You can do this most easily with a combination of Awk and paste (and shell file globbing):

paste -d, input[123].csv | awk -F, -v OFS=, '{print $2, $4, $6}'

---

If those assumptions are wrong, blame poor input/output examples. ;)

Answer 2

To answer your question as stated, given

$ cat file
a,b,c
d,e,f
g,h,i
j,k,l

and a simple test script

$ cat col.bash
#!/bin/bash

awk -F, -vcol="$1" '{print $col}' file

you can verify that $col indeed references the desired column i.e.

$ ./col.bash 2
b
e
h
k

If that's not working in your case, then there are other factors at play. Regardless, there are far simpler ways of extracting columns from multiple files.

Answer 3

Using Bash and AWK in the case will be very hard. I could not solve the problem by the solutions proposed here. You are going to have much problems with "/'/... so a single tool is necessary here.

Use just gawk as discussed in the thread ECG Bash selection tool.

# https://codereview.stackexchange.com/a/146370/122105
#!/usr/bin/gawk -f

# https://www.gnu.org/software/gawk/manual/html_node/Join-Function.html
@include "join.awk"

BEGIN {
    FS = "\"*,\"*";
    last_row = 0;
}

BEGINFILE {
    rows[0][ARGIND] = gensub(".*P([0-9]*)C.*", "\\1", "g", FILENAME);
}

{
    rows[FNR][ARGIND] = $col;
    if (FNR > last_row) { last_row = FNR; }
}

END {
    for (r = 0; r <= last_row; r++) {
        print join(rows[r], 1, ARGC - 1, ",");
    }
}

Please, read the complete answer of 200_success here with excellent explanations.