0

I have a list of directories. Some of them have a pattern in their names. I made an array for those patterns.

Then I should loop over a find result, and compare each result with each pattern and if it matches skip that item.

Here's my code:

declare -a patterns=("*Api" "*Panel" "Common" "Site*" ".*")
while read folder; do
    if [[ $folder == "" ]]; then continue; fi
    for pattern in "${patterns[@]}"
    do
        echo  "Checking $folder $pattern ..."
        if [[ $folder == $pattern ]]; then
            echo "Matched $folder $pattern"
        fi
    done
done <<< "$({ find /some-path -mindepth 1 -maxdepth 1 -type d | cut -d'/' -f3 | sort })"

But it does not work. For example AdminApi as a directory should match *Api.

If I do it statically like if [[ AdminApi == *Api ]]; then echo 'yes'; fi it works.

But when I do it as a loop over an array, it does not work.

What is the problem here?

Saeed Neamati
  • 537
  • 4
  • 17
  • Don't loop over a `find` result of all you need is select files from that which match the pattern. Use the pattern directly instead of `find` to get the list of files. – Marcus Müller Feb 25 '23 at 08:01
  • @MarcusMüller, this is a small code from a larger code. Though that's the option, let's say we can't do that now. The array is dynamic and I can't create a dynamic `find` command. – Saeed Neamati Feb 25 '23 at 08:04
  • Hmmm, as a software developer, I'd say that sounds like an unwise architectural choice is being made. Anyways, you're just using the wrong comparison operator here. – Marcus Müller Feb 25 '23 at 08:06
  • @MarcusMüller, I'm a software developer myself :D Can you please show me the correct operator? Any why `==` works for static values, but not for dynamic variables? – Saeed Neamati Feb 25 '23 at 08:07
  • Because == is the equality operator, it works for equality, not pattern matching. See the bash manual for the other *comparison operators*. – Marcus Müller Feb 25 '23 at 08:09
  • @MarcusMüller, the point is that it works for `AdminApi == *Api`. So, technically it's not an `equality operator`. But thank you, I'll check those operators. – Saeed Neamati Feb 25 '23 at 08:18
  • ah, you're absolutely right there, sorry. – Marcus Müller Feb 25 '23 at 08:28
  • @StéphaneChazelas, yeah that was a typo. I updated the question. Thanks. – Saeed Neamati Feb 25 '23 at 09:49
  • Does running your script with `bash -x` reveal why it doesn't work? – Stéphane Chazelas Feb 25 '23 at 10:01
  • 1
    You don't need a group command `{` or `}` inside command substitution. Sure, it works, but only because almost any shell syntax will work inside `$()` - it's not necessary and it adds nothing of any value here. In fact, your group command shouldn't even work because the list of commands need to be terminated by either a newline or a semi-colon (see `man bash`, search for the section headed "Compound Commands"). i.e. there should be a `;` between the `sort` and the `}`. – cas Feb 25 '23 at 10:01
  • @StéphaneChazelas, it printed out so many pages, I can't trace it. – Saeed Neamati Feb 25 '23 at 10:12
  • @StéphaneChazelas, after simplifying a lot of code, that `bash -x` saved me. Thank you. It was because of a stupid typo in my code. But I learned how to debug bash at least. – Saeed Neamati Feb 25 '23 at 10:51
  • If that's too big, run as `BASH_XTRACEFD=7 7> file.log bash -x ./the-script` and look for relevant entries in `file.log`. Or just add `(set - o xtrace; ...)` wrapping around the parts you want to trace. – Stéphane Chazelas Feb 25 '23 at 10:51

1 Answers1

1

Using zsh instead of bash would make this a lot easier (and reliable):

typeset -A patterns=(
  '*Api'   .
  '*Panel' .
  'Common' .
  'Site*'  .
  '.*'     .
)
for dir in /some/path/*(ND/:t); do
  matched_patterns=( ${(k)patterns[(K)$dir]} )
  if (( $#matched_patterns )) print -r -- $dir matched ${(j[, ])matched_patterns}
done

Gives for instance:

.Panel matched .*, *Panel
SiteApi matched Site*, *Api

That uses the K subscript flag which for associative arrays causes the expansion to return the elements for which the key as a pattern matches the subscript. And with the k parameter expansion flag, it's the key as opposed to the value that is returned. You could remove it and define the associative array as:

typeset -A patterns=(
  '*Api'   'API pattern'
  '*Panel' 'panel pattern'
  Common   Common
  'Site*'  'site pattern'
  '.*'     'hidden file'
)

To get:

.Panel matched hidden file, panel pattern
SiteApi matched site pattern, API pattern

For instance.

If the aim is just to get the list of directories that match either of these patterns, then, it's just:

patterns=( '*Api' '*Panel' Common 'Site*' '.*' )
dirnames=( /some/path/(${(j[|])~patterns})(ND/:t) )
print -rC1 -- $dirnames

Or for those that match none of them:

set -o extendedglob
patterns=( '*Api' '*Panel' Common 'Site*' '.*' )
dirnames=( /some/path/^(${(j[|])~patterns})(ND/:t) )
print -rC1 -- $dirnames

As for your approach, you may want to read:

Though none of those would explain why it doesn't work for you.

It works for me where I see:

Checking .Panel *Api ...
Checking .Panel *Panel ...
Matched .Panel *Panel
Checking .Panel Common ...
Checking .Panel Site*, ...
Checking .Panel .* ...
Matched .Panel .*
Checking SiteApi *Api ...
Matched SiteApi *Api
Checking SiteApi *Panel ...
Checking SiteApi Common ...
Checking SiteApi Site* ...
Matched SiteApi Site*
Checking SiteApi .* ...

You may want to run your script with bash -o xtrace (same as bash -x) to see what's going on.

Or:

BASH_XTRACEFD=7 7> file.log bash -o xtrace ./the-script

To save the tracing output in a file.

Or add some set -o xtrace (+o to disable) in chosen places to enable/disable that tracing.

Stéphane Chazelas
  • 522,931
  • 91
  • 1,010
  • 1,501
  • I always get astonished at the length of your explanations and the number of details you go into. Thank you so much. I hope you get the bests in your life. You deserve a lot of good things for the help you provide for people like me. – Saeed Neamati Feb 25 '23 at 09:19
  • I read the `why looping over file is bad practice`. We do have a lot of internal policies that check our codes for a lot of rules and one rule is that our files should be only composed of ASCII characters and nothing more. So, does this mean that in our special case, looping over files is OK? I disagree with you (with all respect) regarding the **easiness** of `find ... -exec` command. It's not easy at all. For us as a developer team who are used to loops in JavaScript and C# and Python, loops are much more easier to read and maintain. – Saeed Neamati Feb 25 '23 at 10:03
  • 1
    @SaeedNeamati, no, space, newline, `*`, `?`, tab, etc are all ASCII characters and are a problem. Also unless you modify the kernel, policy won't help you if there are bugs or malicious actors creating those files. Best is to use correct code and correct APIs (those that treat file names for what they are: arbitrary sequences of bytes) to process file names. – Stéphane Chazelas Feb 25 '23 at 10:57
  • the argument `malicious actors creating those files` is solid and very true. I would confer it with the team. We have a lot of internal scripts. It would take some time I guess. But that argument convinced me completely. – Saeed Neamati Feb 25 '23 at 11:05