Command Line Goodness Part II

OK...so fortunately for me (and you), I am working a case right now that will help me to illustrate my point about using the command line perfectly. Now, let me preface these next few posts, and simply state that this is MY way of using the command line to parse data. There are SO many tools, and SO many different command switches, that you could literally skin the proverbial cat a dozen different ways. My way works for my purposes. If you have a command syntax that is more efficient, please let us know! I'm all about getting better!

Now, In this case, I have six images from a Point of sale (POS) network. My first goal is to find if I have track data in the clear on any of the images. Since this the fraud in this case is "card present", meaning that the bad guys jacked track data and used it to create new cards, I am anticipating finding some. For the purposes of data reduction, I JUST want to work with the systems with clear text track. So...I mount my images read only with ImDisk (This is pretty basic, so I am not going to cover how to do this in this post. Basically, you just need a single raw image, since ImDisk will not work with split images.) Once mounted, I use this command:

z:\>egrep -r "\;?[0-9]{12,19}[\=][0-9]{12,32}\.?" * > c:\cases\track2hits

This regex will provide me with all of the matches, along with their full paths. Now, I anticipate finding data in log files, or backup files. However, what I am looking for is a totally abnormal location. For example, in this case, I found a file we'll call "winprc.dll". Now, dll files should NOT contain track data...at least none that I have ever seen, so this is automatically extremely suspect and likely a dump file for a credit card sniffing malware. How do I know this?

Well...think about it for a second. You have track data...in a dll...which should absolutely not be there. Something put it there...it had to to get there in the first place. So, in less than 30 minutes, I know can make the logical jump to the fact that I have an intruder, I have malware (since I have a dump file), and I will logically have to have an exfiltration point....bad guy has to get his goodies off the network so he can make his counterfeit cards right? All this from a single regex...pretty kewl huh.

Now, I can start creating my "dirty word list" or set of keywords to search on later. The name of this dll is now at the top of my list.

OK...so next, I am interested in who connected to this box. So, I export my event logs in hopes of finding some IP addresses. Once I export them, I convert them into flat text files with Event Log Explorer and stick in them my working case directory.

Now, I JUST want the IPs from the logs...nothing else. Now, since I am working with customer data, I am not going to show the actual event logs, but here is a screenshot of the event logs from my local machine.


OK...a quick check with word count and I see that I have 2394 lines in JUST this one log. (cat filename | wc -l). Now, I could export the logs to a csv and go through it in a spreadsheet, but that would require me to manually pick out IPs. WAY too time consuming. I want answers now. So...back to my regexes. I can use the regex for IP addresses and let's see how much smaller we can make this sample.

egrep -o "\b(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\b" | sort | uniq

SO...this regex pulls out anything that matches an IP address, the "-o" option shows me ONLY the hits for the regex (as opposed to the entire matching line), "sort" puts them in numerical order, and uniq removes the duplicates. Leaving me with this...

0.0.0.0
10.32.8.162
10.96.33.21
10.96.33.23
192.168.40.2
2.6.1.62
255.255.255.0
5.3.1.0
82.0.174.0
82.0.188.0

A whopping 10 IP addresses...or at least what appear to be IP addresses, but that' for me to figure out. Now again, this is not the data from the images, but from my local machine, but it clearly illustrates my point. I went from over 2000 lines of log files, to 10 in a few seconds by using the command line. Now, if you want to get a count say of the number of time each IP appears, simply add the -c (for count) option. In this example, my results looked like this...

12 0.0.0.0
2 10.32.8.162
3 10.96.33.21
2 10.96.33.23
6 192.168.40.2
3 2.6.1.62
6 255.255.255.0
1 5.3.1.0
1 82.0.174.0
1 82.0.188.0

The number off to the left shows the number of times each IP appears in the log. Pretty slick.

Now, you can use these commands to parse anything...not just IPs! In my current case, working with these two commands gave me an idea of what was going on, and which IPs may have interacted with this system, all in under 30 minutes. Not going to solve the case by any stretch, but as you can see I can perform significant data reduction using these methods.

In my next post I will cove how to use cut and awk to carve more complex log files to extract JUST the data you want.