Monday, June 29, 2009

Y the 101?

As some of you have noticed, the first few posts on TheDigitalStandard are centered around foundational principles of investigation. I did this intentionally because I believe whole heartedly that without integrating these critical building blocks of investigative theory into your personal methodology, you will not - and I really mean this - WILL NOT be anything more than an average investigator.

I started with "Semper Gumby" where I wrote about being flexible and allowing the evidence guide you rather than your theories dictate what the evidence said. I followed up with, "Trust" where I wrote about trusting your instincts, and never being satisfied with simply doing the minimum, but allowing yourself to be woken up at zero-dark-thirty to work on a case. I finished up with "The Alexiou Principle" in which I wrote about having clearly defined expectations, you establish an investigation plan, and you follow through with the plan by asking and answering quantifiable questions.

Now, I realize that this may sound rather harsh, so please let me assure you - I intended it to be. I assume that if you are taking the time to read this blog, you are trying to improve your forensic and incident response skills. It was not too long ago, that I was where you most likely are now...interested, overwhelmed, inundated. Interested in this line of work...how to break into the field, where to begin, who to contact, who will give you a chance? Overwhelmed with tools, techniques, terms like MD5, non-repudiation, analysis, and file carving. Inundated with tools, EnCase, FTK, TSK, Helix, SMART, F-Response, and MFL (just to name a very few).

So you being where you are, I would also assume that you want to get better. You want to HAVE these tools, and more importantly you want to know how to USE these tools! How do you parse a registry? What is file carving anyway? How do I use a regex to find credit card data? What does all of this look like? AHHHHHHH!!! There is TOO MUCH!

Let me assure you, you are correct - there is too much for an one person to know. Be very very wary of anyone who tells you the contrary. I don't care they've "been doing this for 20 years". They don't know everything. None of us do. So what? Are we doomed? Are we all bound to mediocracy? The answer is a resounding, "No"...no you are not...IF and only IF you have a solid foundation of understanding upon which to build.

Build your foundation on critical concepts like "Locard's Exchange Principle", "Occam's Razor", "The Alexiou Principle". Use critical tools like organization, clear concise questions, and sound logic. Develop useful habits like taking good notes, using a buddy to discuss your case work, and remaining mentally pliable.

Listen, I have heard it said that the Peace Corps is the toughest job you'll ever love. I think that is crap. Being a Computer Forensic Investigator is BY FAR the toughest job you'll ever love. So if you are like me, and you really love this line of work, then why not be the best at it? Be the best there is! To do that, you have to develop the core skills necessary for greatness. Look at some greats in the world of sports...Tiger Woods, Curt Schilling, Walter Payton, Michael Jordan...they were/are the best at their particular sport because they worked hard. Harder than anyone else. They were the first on the field, and the last to go home. They hit just one more putt, reached for one last yard, and threw just one more batter. They were anything but average and each of them shared one thing in common - mastery of the basics that made up the game.

The core message here is that if you are going to do this job, don't be average. Master the basics...get SO good at those that you could do them in your sleep. Then, you will be ready to truly be great! It's a long journey which I have yet to complete. I work hard everyday to get better, learn more...get that one last regex working before calling it quits.

Master the basics, and be great! I hope to be there someday too!

Saturday, June 27, 2009

The Alexiou Principle

One of the most common mistakes I see made by computer forensic investigators of all levels of experience, Law Enforcement, and Corporate, is the lack of a clear direction. They know there is "bad guy" stuff out there somewhere, so they go attack it without first establishing an investigation plan. In my experience, this methodology NEVER yields solid results, and ALWAYS leads to confusion, wasted time, and questions going unanswered. As forensic investigators using such absolute terms like "never", and "always" is a NOGO, but in this case I find that it's true. Please comment you have experienced the contrary.

In the spirit of coming to you with solutions and not problems, I would like to share the investigative theory that I use and that have been proven to work EVERY time..."The Alexiou Principle". Named after its creator Mike Alexiou, the Alexiou Principle states four questions for the investigator to answer:

1. What question are you trying to answer?
2. What data do you need to answer that question?
3. How do you extract that data?
4. What does that data tell you?

Every case has a question - in fact most cases have multiple questions. Regardless of what the circumstances are, there is something you are trying to answer. I encourage my team to make sure these questions are clear with their customers to ensure that expectations have been set appropriately on both sides. Few things are worse than coming to what you believe to be the conclusion of an investigation and have the customer ask you, "What about X"? and you're all, "What? We never discussed X"! To avoid this hammer out the questions the client wants to have answered and review it with them! That way, they know what you are going to do, and more importantly, YOU know what the expectations on you are! I also encourage my team to log their questions in their Case Notes. Doing this not only helps them to keep track of what it is exactly that they are doing (preventing scope creep and analysis paralysis), but it also allows them to track what the answers to those questions are and how they came about finding those answers (making the writing of their final reports SOOOOOO much easier).

Once you have established what it is that you are trying to answer, it's time to figure out what data is going to give you the answers you are looking for. Depending on the question and the operating system there are likely several places that you will need to look. The key here is being able to phrase precisely and clearly what it is that you are looking for. For example, I have worked several cases involving AS/400s. Now although my first job out of the Army was as an AS/400 administrator, that was more than 10 years ago and I was only in that role for about six months. So any knowledge I had about the AS/400 or Z/OS is totally gone (probably replaced by important facts like which restaurants have the best bread pudding, and how to help my daughter hold a "proper" tea party)! But who cares!? Not me IF I can either ask the customer's admin, or I have Internet access. Say for instance I wanted to know user activity on an AS/400. A quick search on the web showed me that I would need to check the DSPSECAUD to see if security auditing was activated, and then I could display the audit journals by running DSPJRN JRN(QAUDJRN). I know that is kind of an obscure example, but you get the idea.

Once you have identified your data, you need to extract it. How you do this is really up to you and will depend greatly on your case, the evidence you have on hand, the data you have already collected, and the tools in your toolbox. I won't go much more into this question, because I think it's pretty self explanatory. The one thing I will say is that you should treat this question like a when we did math problems in like 5th grade. SHOW YOUR WORK. State in your notes and in the final report that you extracted data X, Y, and Z for the purposes of BLAH.

Finally, and most importantly (in my opinion) is what the data tells you. Some common mistakes investigators (including myself!) make are jumping to conclusions about the data, misinterpreting the data, or forcing the data to fit into their theories. When you are looking at your data and think you have an answer, go back to your notes and outline the process you just took to get there. Seriously, this takes all of 3 minutes, but will prove to be invaluable to your case. I will state my question, indicate where the data exists, indicate how I extracted that data to include the tool(s) and version(s), and what I believe that data is telling me. Once I have all of that information written down, I will read it over a few times to make sure it says what I want it to say (as opposed to simply being the first thing I wrote down - which is often nothing more than a series of jumbled thoughts and comments). Then I show it to a buddy and ask him if it makes sense. A peer review of your conclusions is a great tool you can use to ensure you are not making incorrect assumptions, or misinterpreting data. Then you would repeat this process for every question you have been asked to answer by your customer.

If this seems really easy to you, that's because it is. Too many investigators introduce unneeded complexity into their investigations by not focusing and having a clear investigation plan. Maybe these people think they need to make things seem complex so that they appear to be smart, or like "real" pros...seasoned and hard core? I don't know, in my opinion having an attitude like that is counter productive and a huge waste of time.

Being a good investigator has nothing to do with how many $5 words you can throw out, or how many years of experience you have! It's about figuring out what happened! I promise you, if you apply the Alexiou Principle and go into your cases with a PLAN, you will be a better investigator, regardless if this is your first case or your 500th.

Tuesday, June 23, 2009

Trust

So, as you are probably aware, I am working on a case right now that has an interesting pair of malicious binaries. In a previous post, I stated that I was not able to get one of the binaries working either in a VM or on my dirty box. Well, after re-reading that post, and sleeping on it (or at least trying to), I couldn't stop thinking about it...something was not sitting well.

My instincts told me that I was missing something. So rather than simply dismissing it, or chalking it up to indigestion (which I hear Peanut Butter Cup cereal will do to you), I got out of bed at 0400, started the coffee pot, and decided to trust my instincts.

After trying to run the malware again unsuccessfully, I realized that I had not "Googled" the error message. That is something I usually would not have forgotten, but for whatever reason, I had. So I did...and what came back had something to do with the Microsoft .Net framework. I then found the download page from MSDN (which if you don't have bookmarked, you REALLY should) and grabbed "dotnetfx.exe" - let's remember that name for later (in a post I am working on for tomorrow)! So, I ran the executable, installed .Net v2, and kicked off the malware again. This time NO ERROR! While I was not sure what it was doing that this point, I was confident it was doing something as the error message that I had become so accustomed to was no longer plaguing me! The point here is that I trusted my gut...something was not sitting right and I was determined to find out what...which I did! And THAT ended up being HUGE in this case!

Next, I fired up Process Explorer and restarted the malware. It appeared for like a second...maybe two, then disappeared. Well, I had already opened the binary in both a hex editor and PEDump without any interesting results. I had to get at the process as it ran! I tried to dump RAM, but I couldn't do it quickly enough to catch the process as it ran. After about 45 minutes of launching the binary (which I could do from the cmd line) and trying to click on the process when it popped up in Process Explorer, I got it! What I saw in the strings as the process ran in memory was that it was trying to open and FTP connection to a remote IP address...AND...it gave me something that looked a whole lot like a username and password. Again, by trusting my instincts, I not only got the malware working, but I got an IP, a username, and password (later validated by the USSS...so I can state with confidence that I got it!)

So as I sat on my back porch celebrating, my neighbor (Glen Painter) came over. As I talked to him about my victory, I told him I still had one question. The malware definitely FTP'd the customer data off the network, but how did it get it in the first place? Well, my neighbor happens to be a very good .Net Developer. So we downloaded the free version of Red Gate's .Net Reflector and decompiled the binary into C#. By taking this last step, I was able to see that the malware was listening on the TCP port that the Point of Sale (POS) software used to transmit credit card numbers from the POS terminals to the back of house (BOH) server. As it listened, it created a file that contained the results of the traffic sniffing! This file format was the exact same as the format of the files on my forensic image that I suspected were being exported! By trusting my friend, I was able to learn something new, put a new tool in my arsenal, and really bust this case wide open. Thank you GLEN!

In this particular situation, the technology was not my obstacle...it rarely ever is. Truth be told, I hit a rut. I thought what I had done was good enough, and I was prepared to go on. So that made me wonder...how many other investigators do the same thing? Get stuck on something, but instead of waking up at 0Dark30 and knocking it out, they slap whatever they have into their report and call it a day.

I also have to give credit to Harlan. He was up around 0500, and I was able to bounce some ideas off of him. This brings me to my last point...trust your friends (#2). Look, we have a tough job. Every case I get is difficult in some way, but that's what makes it fun. It's a challenge! The key is to look that challenge straight in the eye, and kick it in the teeth...not fold at the first sign of adversity. Having a trusted friend that you can bounce your thoughts off of is a great tool. You need someone you can talk candidly to, who will tell you when you are way off, and who you won't lose any face with when you make mistakes.

Since Harlan and I don't work together anymore, we obviously don't share customer data, so we just stick to the forensics. After about 30 minutes of taking to him, feeling a bit sheepish a time or five, and gathering my thoughts, I was back on track, and able to find what I was looking for and then some.

Look, if you don't get woken up in the middle of the night thinking about case work, I will go out on a limb and say you may be in the wrong line of work. I have solved many many cases between the hours of midnight and 0600 when something was bothering me so much that I couldn't sleep...I HAD to figure it out. I'm sure there are investigators out there, who can be in the middle of difficult case, have some unresolved issues...throw some hash sets around hoping for a hit, and when they don't get anything they go to bed and sleep like babies. I am just not one of them.

Trust your instincts, Trust your friends, and Trust your abilities. Trust really is one of the best tools in your toolbox!

Monday, June 22, 2009

Semper Gumby

I am taking a brief detour from my Live Analysis series to post something that I feel is extremely in CF and IR - the need to be flexible. I LOVE the line popularized by the character Gill Grissom the TV Show "CSI"..."Let the evidence guide you".

Having been an investigator for a number of years now, and approaching 100 cases (yes...I keep track) I have seen a very common and very frightening mistake made by both veteran and rookie investigators. They generate some kind of idea about the case early on with really looking closely at the data and then they try to force the evidence to fit into their theory. When this starts happening, it doesn't matter if the data supports their theory or not...I think it becomes a pride issue...they simply cannot be wrong!

Having worked with some outstanding folks like Harlan Carvey, Don Weber, and Colin Sheppard we have always said..."Leave your ego at the door". Seriously...having an overinflated sense of self or having the lack of humility will NOT serve you well in this business.

In the case I am working on right now, I had two thoughts early on...one was Malware and the other was a packetsniffer. Well, in the first case, I found a process through Live Analysis using Sysinternal's Process Explorer that made me suspicious. I noted it in my Case Notes and moved on. When I got back to the lab, I extracted the binary and took and MD5 checksum of it and compared it against that from a known good (I dowloaded the binary by the same same from Microsoft). Well, the MD5s matched. Theory blown.

Next I found a process that I thought for SURE was packet sniffer (heh...it was even named packetsniffer.exe)! The problem was that when I ran Promisc Detect during my volatile collection process the NIC was NOT in promisc mode. Now regardless of what that process was supposed to do, it was not doing it at the time I collected the data. So, when I got back to the lab I extracted the binary and ran it on a VM with the same OS as the customer. It failed to run, and gave me an error. Thinking maybe it had a red pill, I put it on my dirty box and tried it again...same thing...errored out. Just to make sure it was not trying to "trick" me into thinking it was an error when it was really working, I opened the binary with a hex editor and extracted the strings to see if the error message was present...it was not. Also, I ran Promisc detect on my host to see if my NIC was in promisc mode...it was not. Theory blown. The best I could do in this instance was indicate that I found the process, but that I was unable to duplicate a scenario in which the binary was actually sniffing packets.

The real takeaway here is to not let your theory guide the evidence - like Forrest Gump putting one of those round pegs into a square hole. Rather, let the evidence guide your theory and always be prepared and be comfortable with being wrong. You need to be able to switch directions and quickly. You CAN be wrong, so leave the ego at the door.

Semper Gumby!

Saturday, June 20, 2009

Live Analysis Part I - Changing of the Guard

There has been a drastic paradigm shift in the forensic community that folks like myself, Harlan Carvey, Jibran Ilyas, Colin Sheppard, Cory Altheide, and Don Weber have been a major catalyst for. The days of "image everything" and letting tools like EnCase and Gargoyle sort them out are gone. If you expect to remain relevant in the forensic community, and have an impact on the industry, you NEED to know how to perform live analysis.

The first thing we need to look at is the benefit of analyzing a live system. You are dealing with the compromised system! You may never get a chance to lay your hands on that keyboard again and there is a wealth of information that can be learned from real time analysis. I will cover some of the tools I use and why, but there is a major point which needs to be understood before we proceed. I know how all of the tools I use interact with the several Windows platforms (2000, XP, 2003, 2008, and Vista). I have tested them and know what kind of fingerprint they are going to leave. MAKE SURE you do the same for any tools you use, and document what you are doing, when, and why. This will allow you to identify any "tracks" you make on the system should a forensic image be later required, for data reduction purposes.

As you look at a live system, you need to have something in mind that you are looking for...ie...go in with a plan. Simply looking at a system and "trying to find bad guy stuff" is both unrealistic and illogical. As malware (simply defined as any program used for malicious intent) has become more advanced, its presence has become less obvious. You WILL NOT see processes called "hacked.exe", "P0wn3ed", or "slkjdhfbc.exe" - you WILL see processes called, "lsass.exe", "svchost.exe", and "winlogon.exe". The later are processes you would normally see on a typical Windows system, and would likely not draw any suspicion from an admin or your "average" investigator. This is why you need to go in with a plan.

The most successful model I have used (an still use) is the Alexiou Principle (named after it's creator Mike Alexiou). It asks the following questions:

1. What question are you trying answer?
2. What data do you need to answer that question?
3. How do you extract that data?
4. What does that data tell you?

By employing the Alexiou principle, you can sit in front of a live system with a goal in mind instead of just "willy nilly" tromping all over your suspect system.

In the next post, we will cover some live response tools, what the output looks like, and what that output means.

Windows Forensic Analysis 2/e Review

I recently purchased and read (thanks to two cases back to back with a nice layover in Chiacgo due to weather) Harlan Carvey's new book, "Windows Forensic Analysis Second Edition".

In my tenure as a security professional I have read a LOT of books. I can honestly say that without question, this is the best, most useful forensics book I have read. I am not going to go into a breakdown of chapters, however I will say that the chapters on Live Response, Memory Analysis, Registry Analysis, File Analysis, and Executable File Analysis are fantastic. My copy is full of hi-lights and margin notes! I have also already started working in some of the tools from the DVD into my volatile collection scripts.

What Harlan does really well is makes very complex topics (like the Windows Registry) very understandable. He also is able to tie in why it's important to you as an investigator, and what kinds of things you need to be on the lookout for.

Quite frankly, if you don't have this book yet - you need it! If you are serious about your trade, and want to learn more about how to conduct a comprehensive analysis of Windows based systems - you need this book! Seriously, the chapter on Registry analysis ALONE is worth the price of the book.

Buy it now...seriously...I cannot emphasize that enough!

SANS Forensic Summit 2009

Don't forget that the SANS Forensic Summit is coming! There will be some fantastic speakers including yours truly!

Also, I found out recently that Syngress will be sending out multiple copies of my book, Harlan's new book, "Windows Forensic Analysis, Second Edition", and Eoghan's Casey's "Malware Forensics"! You can come get your copies signed and meet the authors, and ask questions.

This is going to be a great event and I am really looking forward to it! Hope to see you there!

Welcome

Welcome to The Digital Standard! After much prodding from my good friend Harlan Carvey, I decided to create this blog.

A little bit of background on me, and why you should care that I have a blog in the first place. I am a Senior Security Analyst for the SpiderLabs at Trustwave (www.trustwave.com). I have been in the security industry for going on 10 years, and been working in the computer forensics field for the past four years. I have a BS in Applied Management, an MS in Information Security, the CISSP, CEH (Certified Ethical Hacker), CREA (Certified Reverse Engineering Analyst), and QSA (Qualified Security Assessor) certifications, am on the HTCIA Board of Governors, and am a former US Army Signal Corps Warrant Officer...oh ya, and I am the principle author of "Unix and Linux Forensic Analysis" by Syngress.

This blog will be dedicated to tips, tools, techniques, interesting case work, emerging threats and attack vectors, and anything else dealing with computer forensics which I think is applicable.

So why the name, "The Digital Standard"? Well, I think the "Digital" piece is pretty self explanatory, so I will skip over to the term "Standard". As a soldier, I learned at a very young age that "the standard is the standard". Being somewhat about such a underwhelming statement confused, I continued my search for truth. I quickly learned that the "standard" was a basis for measurement to which I was being assessed on a number of things from my haircut to the score on my PT test. It was the "bar", and I was to meet or exceed it, or expect consequences.

For the purposes of this blog, I think the name fits since I strive to be the best forensic investigator I can be. I read, study, and research all in an effort to get better with every case. I feel that THIS is the standard...constant improvement without a destination...just continually getting better.

Thank you for visiting and I hope you will return often!

Chris