2

I have a large binary file (nearly binary file !) which has many \r\n in it [ gets Generated in windows env and doesn't have controle over generating application ]. I want to remove \r from this big file [ =~ 1Gb ] to do further processing in another legacy app that shouts at \r's

Is there any easy way to do it the unix way?

Jeff Schaller
  • 66,199
  • 35
  • 114
  • 250
Prashant Bhate
  • 261
  • 3
  • 8
  • Are you sure it's a binary file? If you open it in a text editor does it have `\r\n`s visible? It's easy to process a text file and change those up, but if you are really looking at a binary application that is quite different. What "further processing" do you have in mind? If this is a compiled binary you can't just muck around in the binary stuff, if it's not actually binary then the question is just how to change line feed types in a text file. – Caleb Jun 30 '11 at 10:34
  • it's complicated, yes is binary file , and yes it does have \r\n file has proprietary structure, binary file with headers separated by \r\n – Prashant Bhate Jun 30 '11 at 10:56
  • In that case the solutions for text files should work, although you might want to be more careful. The perl solution I mentioned only removes the carriage returns before newlines, so that might be the best option. You could also script perl to run on an in-place file instead of producing a new one if it's really 1G, but this way is saver. – Caleb Jun 30 '11 at 10:59
  • 1
    @Caleb `sponge`, `perl -ip` and friends still write a new file then replace the original below the hood. See [Is there a way to modify a file in-place?](http://unix.stackexchange.com/questions/11067/is-there-a-way-to-modify-a-file-in-place/11074) for how to truly act on the existing file directly. – Gilles 'SO- stop being evil' Jun 30 '11 at 20:52

1 Answers1

7

Either it's binary or it isn't. In the event that your thing really isn't binary but just complicated text, here are a couple solutions.

First, you many Linux distros come with a utility called dos2unix that you can run on a file to convert the line ending style:

$ dos2unix original_file converted_file

You could also do this with tr like so:

$ tr -d '\r' < orignal_file > converted_file

Or with perl:

$ perl -pne 's/\r$//g' < orignal_file > converted_file
Caleb
  • 69,278
  • 18
  • 196
  • 226
  • Liked perl syntax, how to replace it in file, instead of generating new one ? – Prashant Bhate Jun 30 '11 at 11:06
  • The `-i` argument makes it handle files in place. You can supply an argument to make it make a backup. `perl -p -i -e 's/\r$//g' filename` for no backup, or `perl -p -i.bak -e 's/\r$//g' filename` for with a backup. – Caleb Jun 30 '11 at 11:15