Reformatting a large number of XML files

Question

I am manipulating a large number of XML files scattered throughout a nested directory structure.

I tried the following:

$ find . -name "*.xml" -type f | xargs -- xmllint --format

The problem is that generates the formatted XML output on the screen, but doesn't change the file.

How can I change this command so that the actual file contents are changed?

score 29 · Accepted Answer · edited Jan 02 '18 at 12:02

29

This can be done from find directly using -exec:

find . -name "*.xml" -type f -exec xmllint --output '{}' --format '{}' \;

What's passed to -exec will be invoked once per file found with the template parameters {} being replaced with the current file name. The \; on the end of the find command just terminates the line.

The use of xargs isn't really necessary in this case because we need to invoke xmllint once per file as both the input and output file names must be specified within the same call.

xargs would be needed if the command being piped to from find was working on multiple files at a time and that list was long. You can't do that in this case, as you need to pass the single filename to the --output option of xmllint. Without xargs you could end up with a "Argument List too long" error if you are processing a lot of files. xargs also supports file replace strings with the -I option:

find . -name "*.xml" -type f | xargs -I'{}' xmllint --output '{}' --format '{}'

Would do the same as the find -exec command above. If any of your folders have odd chars in like spaces you will need to use the -0 options of find and xargs. But using xargs with -I implies the option -L 1 which means only process 1 file at a time anyway, so you may as well directly use find with -exec.

edited Jan 02 '18 at 12:02

Jeff Schaller

66,199
35
114
250

answered Oct 08 '12 at 07:43

didster

799
5
5

@manatwork thanks for the edits - sticky fingers ;o) – didster Oct 08 '12 at 07:46
I just ran this and it appears to work a treat! Many thanks for the prompt and concise response! – Harry Oct 08 '12 at 08:03
2

“This will fail if the file list is too large”: No, it won't fail (it's processing a single file at a time), and in fact `find … -exec` is the most direct way to do this. – Gilles 'SO- stop being evil' Oct 08 '12 at 23:02
@Gilles Good point! I have updated my answer accordingly. – didster Oct 11 '12 at 07:31
1

This works due to the fact that `xmllint` firstly loads full xml document into memory and only then parse / write out. This allows document in-place processing. – gavenkoa May 28 '18 at 20:26
@gavenkoa Reference? – akhan Dec 23 '18 at 03:36

score 8 · Answer 2 · answered Oct 08 '12 at 07:43

8

I typically attack these problems with a layer of indirection. Write a shell script that does what you want, and call that. I'd suggest as a start

#! /bin/sh
for file
do
   xmllint --format $file > $file.tmp && mv $file.tmp $file
done

The try it out on a file or two by hand, then you can replace it in the xargs

find . -name "*.xml" -type f | xargs -- xmltidy.sh

answered Oct 08 '12 at 07:43

Julian

899
5
5

This looks like a good approach if I should need to do more complex manipulation in the future. Thanks for the response. – Harry Oct 08 '12 at 08:05

Reformatting a large number of XML files

2 Answers2