Monday, June 22, 2009

Semper Gumby

I am taking a brief detour from my Live Analysis series to post something that I feel is extremely in CF and IR - the need to be flexible. I LOVE the line popularized by the character Gill Grissom the TV Show "CSI"..."Let the evidence guide you".

Having been an investigator for a number of years now, and approaching 100 cases (yes...I keep track) I have seen a very common and very frightening mistake made by both veteran and rookie investigators. They generate some kind of idea about the case early on with really looking closely at the data and then they try to force the evidence to fit into their theory. When this starts happening, it doesn't matter if the data supports their theory or not...I think it becomes a pride issue...they simply cannot be wrong!

Having worked with some outstanding folks like Harlan Carvey, Don Weber, and Colin Sheppard we have always said..."Leave your ego at the door". Seriously...having an overinflated sense of self or having the lack of humility will NOT serve you well in this business.

In the case I am working on right now, I had two thoughts early was Malware and the other was a packetsniffer. Well, in the first case, I found a process through Live Analysis using Sysinternal's Process Explorer that made me suspicious. I noted it in my Case Notes and moved on. When I got back to the lab, I extracted the binary and took and MD5 checksum of it and compared it against that from a known good (I dowloaded the binary by the same same from Microsoft). Well, the MD5s matched. Theory blown.

Next I found a process that I thought for SURE was packet sniffer ( was even named packetsniffer.exe)! The problem was that when I ran Promisc Detect during my volatile collection process the NIC was NOT in promisc mode. Now regardless of what that process was supposed to do, it was not doing it at the time I collected the data. So, when I got back to the lab I extracted the binary and ran it on a VM with the same OS as the customer. It failed to run, and gave me an error. Thinking maybe it had a red pill, I put it on my dirty box and tried it again...same thing...errored out. Just to make sure it was not trying to "trick" me into thinking it was an error when it was really working, I opened the binary with a hex editor and extracted the strings to see if the error message was was not. Also, I ran Promisc detect on my host to see if my NIC was in promisc was not. Theory blown. The best I could do in this instance was indicate that I found the process, but that I was unable to duplicate a scenario in which the binary was actually sniffing packets.

The real takeaway here is to not let your theory guide the evidence - like Forrest Gump putting one of those round pegs into a square hole. Rather, let the evidence guide your theory and always be prepared and be comfortable with being wrong. You need to be able to switch directions and quickly. You CAN be wrong, so leave the ego at the door.

Semper Gumby!


  1. Semper Gumby - Always Flexible
    Semper Fidelis - Always Faithful
    Sic Semper Tyrannus - Hey, lady...get off my chest!

  2. Interesting how this post ties right in alongside HogFly's What do you seek post...

  3. Exactly :)

    p.s. Awesome job with this blog so far. I am loving it!

  4. I've just discovered your blog and read through your posts, really interesting!

    However, being a network forensics guy, I can't help commenting on the "packetsniffer.exe" issue. The absence of promiscuous mode does not in any way say that the process isn't doing network sniffing. You can for example my tool, NetworkMiner, to sniff a host's traffic using Raw Sockets without entering promiscuous mode.

    The widespread use of switched networks today actually makes the promiscuous mode redundant; a switched network will simply not forward ethernet frames with non-matching destination MAC addresses than a NIC.

    It is only when sniffing through a network tap, monitor port (i.e. SPAN port) or plain old hub that the use of promiscuous mode actually is of any use. Well, there is of course also the issue of MAC flooding, which effectively turns a switch into a hub, but such malicious activity can easily be detected through network forensics (i.e. just sniff some network traffic).

  5. are correct. Fortunately for me in this case (if you read the next post) I was not satisfied with my answer either. I honestly had not dug deeply enough into the malware. I got up very early one morning and did not move from in front of my analysis server until I figured it out. Turns out it was a serial sniffer (no promisc needed, just like you said)!

    Thank you for the feedback! Please keep reading and keep me honest! It's the ONLY way to improve.