10

I have two files have same data but in different lines.

File 1:

<Identities>
    <Identity>
        <Id>048206031415072010Comcast.USR8JR</Id>
        <UID>ccp_test_79</UID>
        <DisplayName>JOSH CCP</DisplayName>
        <FirstName>JOSH</FirstName>
        <LastName>CCP</LastName>
        <Role>P</Role>
        <LoginStatus>C</LoginStatus>
    </Identity>
    <Identity>
        <Id>089612381523032011Comcast.USR1JR</Id>
        <UID>94701_account1</UID>
        <DisplayName>account1</DisplayName>
        <FirstName>account1</FirstName>
        <LastName>94701</LastName>
        <Role>S</Role>
        <LoginStatus>C</LoginStatus>
    </Identity>
</Identities>

File 2 :

<Identities>
    <Identity>
        <Id>089612381523032011Comcast.USR1JR</Id>
        <UID>94701_account1</UID>
        <DisplayName>account1</DisplayName>
        <FirstName>account1</FirstName>
        <LastName>94701</LastName>
        <Role>S</Role>
        <LoginStatus>C</LoginStatus>
    </Identity>
    <Identity>
        <Id>048206031415072010Comcast.USR8JR</Id>
        <UID>ccp_test_79</UID>
        <DisplayName>JOSH CCP</DisplayName>
        <FirstName>JOSH</FirstName>
        <LastName>CCP</LastName>
        <Role>P</Role>
        <LoginStatus>C</LoginStatus>
    </Identity>
</Identities>

If I use diff file1 file2 command I am getting below response:

1,10d0
<     <Identities>
<         <Identity>
<             <Id>048206031415072010Comcast.USR8JR</Id>
<             <UID>ccp_test_79</UID>
<             <DisplayName>JOSH CCP</DisplayName>
<             <FirstName>JOSH</FirstName>
<             <LastName>CCP</LastName>
<             <Role>P</Role>
<             <LoginStatus>C</LoginStatus>
<         </Identity>
20a11,20
>     <Identities>
>         <Identity>
>             <Id>048206031415072010Comcast.USR8JR</Id>
>             <UID>ccp_test_79</UID>
>             <DisplayName>JOSH CCP</DisplayName>
>             <FirstName>JOSH</FirstName>
>             <LastName>CCP</LastName>
>             <Role>P</Role>
>             <LoginStatus>C</LoginStatus>
>         </Identity>

But I need to get no difference, because these files having same data in different lines.

Rui F Ribeiro
  • 55,929
  • 26
  • 146
  • 227
user32026
  • 101
  • 1
  • 1
  • 3
  • By sorting them linewise and comparing, you can check if they are *not equal*. Of course, equal after sorting does not mean that they are really equal as sorting destroys the XML syntax. – jofel Feb 08 '13 at 17:18
  • Don't know how to solve it. they differ by order in file1 a then b and in file2 b then a. you may expose question with diff -y -B -Z -b --strip-trailing-cr file1 file2 – Yurij73 Feb 08 '13 at 18:14
  • 2
    You could try `xmldiff`, but I think that will still notice the order changing, as order is relevant in generic XML. I think your best approach is to use an XML parser & generator to put each file in a canonical order and format, then use `xmldiff` or `diff`. A job for your favorite scripting language (Perl, Ruby, Python, etc.). – derobert Feb 08 '13 at 19:47

4 Answers4

6

You can achieve what you want with the help of a small Python script (you'll need Python installed, as well as the lxml toolkit).

tagsort.py:

#!/usr/bin/python

import sys
from lxml import etree

filename, tag = sys.argv[1:]

doc = etree.parse(filename, etree.XMLParser(remove_blank_text=True))
root = doc.getroot()
root[:] = sorted(root, key=lambda el: el.findtext(tag))
print etree.tostring(doc, pretty_print=True)

This script sorts the first-level elements under the XML document root by the content of a second-level element, sending the result to stdout. It's called like this:

$ python tagsort.py filename tag

Once you've got that, you can use process substitution to get a diff based on its output (I've added one element and changed another in your example files to show a non-empty result):

$ diff <(python tagsort.py file1 Id) <(python tagsort.py file2 Id)
4a5
>     <AddedTag>Something</AddedTag>
17c18
<     <Role>X</Role>
---
>     <Role>S</Role>
4

I had a similar problem and I eventually found: https://superuser.com/questions/79920/how-can-i-diff-two-xml-files

That post suggests doing a canonical xml sort then doing a diff. The following should work for you if you are on linux, mac, or if you have windows something like cygwin installed:

$ xmllint --c14n File1.xml > 1.xml
$ xmllint --c14n File2.xml > 2.xml
$ diff 1.xml 2.xml
Community
  • 1
VenomFangs
  • 354
  • 2
  • 12
0

It's tagged shell, but honestly I prefer using a scripting language with a parser. In this case perl with XML::Twig.

It goes something like this:

#!/usr/bin/env perl
use strict;
use warnings;

use XML::Twig;

sub compare_by_identity {
   my ( $first, $second ) = @_;
   foreach my $identity ( $first->get_xpath('//Identity') ) {
      my $id = $identity->first_child_text('Id');

      print $id, "\n";
      my $compare_to =
        $second->get_xpath( "//Identity/Id[string()=\"$id\"]/..", 0 );
      if ($compare_to) {
         print "Matching element found for ID $id\n";
         foreach my $element ( $identity->children ) {
            my $tag  = $element->tag;
            my $text = $element->text;
            if ( not $element->text eq $compare_to->first_child_text($tag) ) {
               print "$id, $tag has value $text which doesn't match: ",
                 $compare_to->first_child_text($tag), "\n";
            }
         }
      }
      else {
         print "No matching element for Id $id\n";
      }
   }
}

my $first_file  = XML::Twig->new->parsefile('test1.xml');
my $second_file = XML::Twig->new->parsefile('test2.xml');

compare_by_identity( $first_file,  $second_file );
compare_by_identity( $second_file, $first_file );

I'm explicitly comparing one 'Identity' element at a time, and checking that all the fields in one, exist in the other, with the same value.

And then reversing that, because the second file might have extra entries.

Sobrique
  • 4,404
  • 14
  • 24
0

Using xq from https://kislyuk.github.io/yq/:

xq -sS '.[].Identities.Identity |= sort | .[0] == .[1]' file1.xml file2.xml

This reads in both XML files and translates them into JSON. Each file appears as separate elements of the top-level array, so in the example above, file1.xmlwill be found in .[0] and file2.xml will be found in .[1] (this is done courtesy of the -s option). The keys of each file will be automatically sorted due to the -S option.

The identities of each document are sorted and then the two documents are compared.

The output will be the string true for the data provided in the question, or false if a single change is made to either document.

You may use this in a shell test like so:

if xq -e -sS '.[].Identities.Identity |= sort | .[0] == .[1]' file1.xml file2.xml >/dev/null
then
    echo 'documents are the same'
else
    echo 'documents are different'
fi

The -e option causes the exit status of xq to be zero if the last statement has a true boolean value.

For reference, the JSON document created internally for the two files in the question will look like this:

[
  {
    "Identities": {
      "Identity": [
        {
          "DisplayName": "JOSH CCP",
          "FirstName": "JOSH",
          "Id": "48206031415072010Comcast.USR8JR",
          "LastName": "CCP",
          "LoginStatus": "C",
          "Role": "P",
          "UID": "ccp_test_79"
        },
        {
          "DisplayName": "account1",
          "FirstName": "account1",
          "Id": "089612381523032011Comcast.USR1JR",
          "LastName": "94701",
          "LoginStatus": "C",
          "Role": "S",
          "UID": "94701_account1"
        }
      ]
    }
  },
  {
    "Identities": {
      "Identity": [
        {
          "DisplayName": "account1",
          "FirstName": "account1",
          "Id": "089612381523032011Comcast.USR1JR",
          "LastName": "94701",
          "LoginStatus": "C",
          "Role": "S",
          "UID": "94701_account1"
        },
        {
          "DisplayName": "JOSH CCP",
          "FirstName": "JOSH",
          "Id": "048206031415072010Comcast.USR8JR",
          "LastName": "CCP",
          "LoginStatus": "C",
          "Role": "P",
          "UID": "ccp_test_79"
        }
      ]
    }
  }
]
Kusalananda
  • 320,670
  • 36
  • 633
  • 936