Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [parsing]

46 questions
152
votes
9 answers

How to remove empty/blank lines from a file in Unix (including spaces)?

How do I remove empty/blank (including spaces only) lines in a file in Unix/Linux using the command line? contents of file.txt Line:Text 1: 2:AAA 3: 4:BBB 5: 6:CCC 7: 8:DDD output desired…
Michael Ellick Ang
  • 2,039
  • 3
  • 14
  • 15
76
votes
5 answers

How do I grep through binary files that look like text?

I have binary files that should be text (they're exported logs), but I can't open it with less (it looks ugly - it looks like a binary file). I found that I could open it with vi and I can cat it (you'll see the actual logs), but what I'd really…
Robyn Smith
  • 861
  • 1
  • 7
  • 5
74
votes
12 answers

Do you have any useful awk and grep scripts for parsing apache logs?

I can use log analyzers, but often I need to parse recent web logs to see what's happening at the moment. I sometimes do things like to figure out top 10 ips that request a certain file cat foo.log | grep request_to_file_foo | awk '{print $1}' | …
deadprogrammer
  • 1,691
  • 7
  • 24
  • 25
8
votes
7 answers

Get list of user-agents from nginx log

I have nginx log file, and I want to find out market share for each major version of browsers. I am not interested in minor versions and operating systems. I would like to get something like this: 100 IE6 99 IE7 20 IE8 200 FF2 300 FF3 I know how…
Željko Filipin
  • 223
  • 1
  • 3
  • 12
7
votes
2 answers

Parseable NGINX accesslog files with delimiters

The default NGINX format is this: log_format combined '$remote_addr - $remote_user [$time_local] ' '"$request" $status $body_bytes_sent ' '"$http_referer" "$http_user_agent"'; Which is a bit hard to parse. I am…
Kasper Grubbe
  • 311
  • 4
  • 9
5
votes
3 answers

Discover intended Foreign Keys from JOINS in scripts

I'm inheriting a database that has 400 tables and only 150 foreign key constraints registered. Knowing what I do about the application and looking at the table columns, it's easy to say that there ought to be a lot more. I'm afraid that the current…
Jason Kleban
  • 806
  • 3
  • 9
  • 20
4
votes
4 answers

AWS - Script to create EC2 snapshots and automatically rename them

I'm currently trying to setup a script (using AWS CLI from an Ubuntu server) that will do the following: Create a snapshot of every existing volume. Those volumes already have a NAME tag (Like SERVER1, SERVER1DATA, SERVER2, SERVER2DATA etc). Rename…
Nicolas
  • 53
  • 1
  • 1
  • 4
4
votes
2 answers

PowerShell - Add multiple strings to the same element in an array

So I have a array that I need to search through line by line and divide up by interfaces. My code loops through this file line by line. I want to split up the interfaces by the "!" character and add the the strings to an element in an array so I can…
runcmd
  • 255
  • 2
  • 4
  • 11
4
votes
1 answer

How to handle many daily emails from cron efficiently?

With each new server comes at least one more daily crontab email. The FreeBSD servers even send two: a daily and a security run output. Thanks to my obsession with virtualisation the number of emails I need to read (and then often discard without…
drumfire
  • 1,699
  • 3
  • 14
  • 21
4
votes
1 answer

Parsing iometer results

How do your visualise your iometer results?
romant
  • 526
  • 5
  • 21
3
votes
0 answers

I am looking for software that can parse emails and expect emails within a grace period

I manage a diverse set of backup systems, the only commonality is email notifications, There status being Failed, Success, Success with exceptions and unknown(where the email isn't sent). Case 1) Expect email from [email protected] every day, at…
Jacob Evans
  • 7,886
  • 3
  • 29
  • 57
3
votes
1 answer

Is there a way to determine the log format if given a ".log" file?

Is there a way to determine the kind of log (so that it can be parse correctly) if I have no prior information about the type (for eg. syslog, apache log, IIS log) of log it is? I am trying to write a Grok filter for the logs but I have no idea what…
shruti gupta
  • 33
  • 1
  • 4
3
votes
2 answers

Scripts or tools to parse email message to remove attachment of particular name

I have configured Redmine email integration and while it's awesome, a major annoyance is that people have signatures that include a company logo, which then gets posted to every ticket they update via email. I know this is not a perfect solution but…
tacos_tacos_tacos
  • 3,250
  • 18
  • 63
  • 100
3
votes
3 answers

How can I limit the output of the whois command?

How do I get only the owner/orgname from a PTR whois lookup? I don't need all the other entires, just the part about ownership (google in this case). Edit /usr/bin/whois 74.125.236.52 or any ip get OrgName # # Query terms are…
Rajeev
  • 251
  • 1
  • 4
  • 10
2
votes
2 answers

Ansible parsing a JSON output

I'm using URI in order to get a JSOn output to a register called "vchosts" TASK [debug]…
Vidanez
  • 21
  • 1
  • 2
1
2 3 4