1

I want to copy the whole log that are stored in the AWS S3 bucket if the following line is present:

\"Key\" : 951332,\n

I've tried escaping by trying this:

aws s3 ls s3://bucket_name | grep "/\"Key/\" : 951332,/\n" --recursive

but not getting anything back, does anyone know how I can run the grep in this manner?

Bart
  • 2,151
  • 1
  • 10
  • 26
Samosa
  • 71
  • 2
  • 10
  • Your `grep` is getting the output of `aws s3 ls` as input. The `--recursive` option makes no sense as you don't actually want to run that locally. You would have to look inside the files on the AWS side of things. I don't know how to do that as I've never used AWS. – Kusalananda Sep 02 '19 at 12:08
  • About that line. Are you wanting to match that _literally_, including the backslashes? – Kusalananda Sep 02 '19 at 12:13

1 Answers1

0

Mount S3 using s3fs, then do..:

grep -r Key /PATH/TO/MOUNT/POINT/

... then pipe it through grep 951332 and check if that's enough resolution for your case.

It might take a while and incur in AWS DataTransfer cost on you if you run this locally away from AWS, so you ideally want to run this from an EC2 instance in the same VPC.

If you are anyway bold to run locally away from AWS with this approach even with its costs, you might want to redirect stdout and stderr to some docs in order to check them later if necessary, instead of running the costly command line again.

Wrapping up, I might go with:

grep -r Key /PATH/TO/MOUNT/POINT/ | \
        grep 951332 \
        >/LOCAL/PATH/TO/grep_stdout \
        2>/LOCAL/PATH/TO/grep_stderr
#   In /LOCAL/PATH/TO/grep_stdout should be the paths to the docs you
# were searching.

Alternatively:

grep -r Key /PATH/TO/MOUNT/POINT/ | \
        grep 951332 &>/LOCAL/PATH/TO/all_grep_outputs
#   In /LOCAL/PATH/TO/all_grep_outputs should be the paths to the docs you
# were searching.
41754
  • 85
  • 1
  • 7
  • 19