using find, sed, and xargs to fix filenames from data recovery on a mac

Question

Okay, I'm going in circles. I'm using this command

find . -print0 -name '*.1.*' | sed -e 'p;s/\.1//' | xargs -0 -n2 mv

To try and rename hundreds of files that had ".1." added just before the file extension when they were archived after someone accidentally deleted 200GB worth of data.

I'm caught between the mac xargs interpreting spaces in filenames as separate arguments, and not being able to set the delimiter to only newlines, not spaces. I cannot figure out how to have find print the '\0' character as well as newlines between. Any ideas on how to get this to work? I've been searching in circles and it seems simply being on a mac environment is making this more complicated than necessary.

Alternatively trying rename command but still having issues

 find . -name '*.1.*' -type f -exec rename -n 's/\.1//' '{}' \;

ANSWER as per @Wildcard below

find . -name '*.1.*' -type f -exec sh -c '
  for f do
    suf="${f##*.1}"
    new="${f%.1.*}$suf"
    if [ -e "$new" ]; then
      printf "Cannot move file <%s>\n" "$new"
    else
      mv -n "$f" "$new"
    fi
  done
  ' find-sh {} +

This sounds like it might be a tool for [rename](https://metacpan.org/release/File-Rename). This is available on the linux systems I've used through the package manager. — Att Righ, Dec 05 '17 at 00:38
Alternatively, this sounds like a great fit for a scripting language like python: for when you hit up against the "programming" limits of bash. — Att Righ, Dec 05 '17 at 00:40
@AttRigh I agree - however I shouldn't have to install developer level tools on a system I am administering after simple backup restores — FaultyJuggler, Dec 05 '17 at 00:41
Well rename isn't really developer level :). Umm, so I imagine the issue that you need to solve is getting sed to split on nulls rather than newline. Apparently there is a `-z, --null-data` argument for precisely this purpose. — Att Righ, Dec 05 '17 at 00:44
@AttRigh I started trying this command, and I'm getting "no file exists" errors find . -name '*.1.*' -type f -exec rename -n 's/\.1//' '{}' \; — FaultyJuggler, Dec 05 '17 at 00:45
@AttRigh it would appear macOS terminal does not have rename available — FaultyJuggler, Dec 05 '17 at 00:47
Hmm... that's odd. You might try the -v argument. Otherwise perhaps try wrapping this in `strace -e execve -F` to work out precisely what is being executed. It might be choking on the newlines. I am pretty hopeful about the -z argument for sed by the way. — Att Righ, Dec 05 '17 at 00:48
@AttRigh - that's not odd. There is no `rename` on OSX. Also, there's no `sed -z` either, that's a `gnu` extension. This is an easy job for any shell that supports `${var/string/replace}`. — don_crissti, Dec 05 '17 at 00:51
@don_crissti yeah looks like a gnu extension to me. From the fact that is is [in the gnu documentation](https://www.gnu.org/software/sed/manual/sed.txt) but not in [this man page](https://linux.die.net/man/1/sed). — Att Righ, Dec 05 '17 at 01:08
May I suggest you use a unix computer :P . [Rename can be installed with brew](https://superuser.com/questions/152627/renaming-many-files-in-mac-os-x-batch-processing) as [can gnu-sed](https://stackoverflow.com/questions/30003570/how-to-use-gnu-sed-on-mac-os-x) — Att Righ, Dec 05 '17 at 01:11
@AttRigh - lol, if you bother scrolling down you'll find out that those are both `gnu sed` man pages... The one in your 2nd link is just too old that's why it doesn't mention `-z`... Anyway, [here's the official Options section](https://www.gnu.org/software/sed/manual/html_node/Command_002dLine-Options.html#Command_002dLine-Options) — don_crissti, Dec 05 '17 at 01:14
Anyway, more or less a duplicate of [Remove next-to-last extension in a file name](https://unix.stackexchange.com/questions/175530/remove-next-to-last-extension-in-a-file-name) — don_crissti, Dec 05 '17 at 01:16
Why so hostile? I'm bothering to trying to work out the causes for the differences without access to a mac machine and trying to include sources for my assertions. Anyway some digging shows that mac is using [freebsd sed](https://stackoverflow.com/a/40669131) which is [documented here](https://www.unix.com/man-page/FreeBSD/1/sed/) — Att Righ, Dec 05 '17 at 01:40

Wildcard · Accepted Answer · 2017-12-05T05:02:03.427

3

The following should be safe always and should work on a Mac:

find . -name '*.1.*' -type f -exec sh -c '
  for f do
    suf="${f##*.1.}"
    new="${f%.1.*}.$suf"
    if [ -e "$new" ]; then
      printf "Cannot move file <%s>\n" "$new"
    else
      mv -n "$f" "$new"
    fi
  done
  ' find-sh {} +

Note the mv -n exits successfully without renaming the file, if it would overwrite an existing file. You would probably like it reported on. So this does so.

Also, in the odd edge case where there is a directory with the target new name, mv would move the file into it without the safety check I added (even with -n). That wouldn't be desirable either.

If a file has more than one .1. string in its name, this command will remove the last one only. (Which is probably closer to what was intended than removing the first one would be.)

Theoretically you don't need the -n switch at all with the safety check if block, but I left it in to safeguard data in event of race conditions (if some other process creates a target file just before you move a file on top of that name).

Most importantly, this won't blow up no matter how bizarre your file names are. Even if they have embedded newlines, single quote characters, asterisks and all sorts of other things.

Come to think of it, though, I'm not sure how well bash parameter expansion works on Unicode.

edited Dec 05 '17 at 05:02

answered Dec 05 '17 at 01:14

Wildcard

35,316
26
130
258

"unexpected token 'done' " though I assume maybe I need to put this in a shell script? – FaultyJuggler Dec 05 '17 at 03:31
@FaultyJuggler oops! Left out `fi`. Try it now. (What I get for not testing. I still didn't test, though.) – Wildcard Dec 05 '17 at 03:33
Okay, one small error which I can probably figure out, it's also deleting the trailing dot before the file extension – FaultyJuggler Dec 05 '17 at 03:40
Got it find . -name '*.1.*' -type f -exec sh -c ' for f do suf="${f##*.1}" new="${f%.1.*}$suf" if [ -e "$new" ]; then printf "Cannot move file <%s>\n" "$new" else mv -n "$f" "$new" fi done ' find-sh {} + – FaultyJuggler Dec 05 '17 at 03:44
@FaultyJuggler, not quite. I've just added in the period. – Wildcard Dec 05 '17 at 05:02

using find, sed, and xargs to fix filenames from data recovery on a mac

1 Answers1