Sunday, July 13, 2014
Linux forensics/incident response is a new thing for me. I've never had occasion thus far to conduct a "real" investigation into a Linux machine. This "intrusion" into my honeypot inspired me to conduct my own attack and investigation so I could learn more about the subject. I'm a noob to this stuff, so if you see things I did wrong or could have done better, I'd be delighted to hear constructive criticism from you.
Given my lack of experience with Linux based investigations, I've searched for information on the subject but haven't found a lot. I read the excellent book by Chris Pogue, Unix and Linux Forensic Analysis DVD Toolkit a few years ago and still refer to it from time to time. A note to Chris, I'd still love to see a second edition of the book! But I digress...
I began by creating a new Ubuntu Server 12.04 server virtual machine. I installed openssh-server so I could connect to the VM from the host and placed the VM network settings to Host Only. I also set a password for the root user, so I could log in immediately with root privileges, just as my honeypot attackers had done.
I installed Inetsim on my host machine and placed a copy of the file my attackers had downloaded to my honeypot to the "fake" files so I would be able to download it in the VM. Prior to starting my attack, I configured Inetsim to be listening on the Host Only network and started the program. I also started tcpdump, using the -i option to listen to the host only, vmnet1 adapter and -w to output a .pcap file. Finally, the time to attack had arrived.
With my virtual machine running and logged out, I opened a terminal on the host machine (Ubuntu 14.04 LTS) and typed the command:
I was greeted by the usual warning that the system can't identify the authenticity of the host I'm trying to connect to and wanting me to verify I really wanted to do it. I hit Y and the host was added to the .ssh/known_hosts file. After entering the password, I was in and ready to do my nefarious deeds.
I decided to type in all the same commands my honeypot attacker had, with a couple of exceptions. When I got to the point of downloading the file, I decided to just make up a fictional URL instead of using an IP address with port number, as the real attacker had done. I also decided to simply execute the file once downloaded instead of using the nohup command so I could be a little more sure I would see everything in my memory capture. I may do it again later with the nohup command and compare the results of memory analysis.
After going through all the commands the real attacker had, including downloading the .bash_root.tmp3 file to the /tmp directory and running it, I entered the ps x command, just to see what I could see before logging out. I noted the downloaded file was running in memory as expected. I then logged out and disconnected from the ssh session.
My initial plan was to conduct the attack on a virtual machine, then pause it and collect the .vmem file for memory analysis, followed by grabbing the .vmdk file for disk analysis. However, that didn't seem very realistic, so I changed my plan.
For the memory collection, instead of using the .vmem file, I decided to use Hal Pomeranz' Linux Memory Grabber script (lmg). lmg makes use of Joe Sylve's excellent LiME, a loadable kernel module which allows the collection of RAM from a Linux or Android system. lmg also makes use of Volatility and dwarfdump to create a Linux profile of the system for use with Volatility. I have used LiME by itself before with good results, but I wanted to try lmg and see how it worked automating the collection of memory and the creation of the Volatility profile.
I followed the directions for setting up lmg on an 8 gb thumb drive, but had some problems getting lmg to setup properly. Hal was kind enough to help me with it and I was able to use it for this investigation. I won't go into how to set it up here. You can read the install directions on the lmg Github page. I added a static version of dwarfdump and a freshly downloaded copy of Volatility 2.3.1 to the thumb drive and tested it on a non-infected system with positive results.
I went back to the virtual machine, logged in as root and mounted the lmg thumb drive. The simple command ./lmg -y started the script and it executed with no problems. lmg captures RAM and saves it in a capture folder on the thumb drive. As noted previously, it also automatically creates a Volatility Linux profile and saves it to the thumb drive as well. The VM only had 1 gb of RAM, so this process didn't take long. Once it was complete, I shut the VM down.
I next went to the VM folder on the host machine and used dc3dd to make a copy of the .vmdk file. Kinda silly, I guess, but I wanted to do things semi-realistically. I could have booted the VM again to a live CD and imaged it that way, but decided I would save a little time.
So now, I have a RAM capture, a disk image and a .pcap file. I'll be taking a look at the RAM capture first with the awesome Volatility and post about that in part 3. I will also post the Linux profile to my Linux Volatilty Profiles Github page soon.
Sunday, July 6, 2014
As I mentioned in my last post, Brett Shavers is offering a free course on the Windows Forensic Environment (WinFE). If you've never heard of WinFE, it's a Windows Forensic boot CD and it's highly customizable for your individual needs. In the past, the build process was a bit cumbersome, but several different improved ways of building it have since been created. Brett has been a champion of WinFE for quite a while now, so I was sure his course would be very good. I signed up as soon as I first learned about the it and completed it in a few days. It could be done all in one day, but I preferred splitting it up a little.
The Windows Forensic Environment course covers the history, building and usage of WinFE. The course consists of 30 modules, including 27 video lessons, a wrap-up video, a qualification exam and a course downloads page. The vast majority of the videos are done by Brett himself, while a couple videos by others are included as well. All of the videos are short enough that you can sit down and watch a few without spending your whole day at it. Registration is easy enough, either by creating a new account through skilljar.com or signing in with your Google or Facebook account.
The course begins with a brief introduction to WinFE and its history. Something I really liked was Brett starts out from the beginning making it clear what WinFE is and what it's good for, as well as talking about when it might not be the best choice. He talks about potential pitfalls when using it along with ways you can screw up by not using it right (accidentally boot the evidence drive, etc).
After the introductory section, Brett covers when it's a good idea to use a boot cd and suggests other times when you're better off just removing the evidence hard drive and imaging through a hardware write blocker. As he points out, if you've got all the time you need and have no reason to boot the machine on-site, you may as well remove the hard drive in the lab and do things the "traditional" way.
An overview of forensic boot systems is next. WinFE and various Linux forensic boot CD's are talked about and compared. I like that Brett doesn't tell you that WinFE is the answer to all your needs and that you'll never need Linux. Rather, he says right up front that sometimes WinFE isn't going to work for you for some reason and a Linux boot CD might be your best option. He suggests having both available when you go on-site so you're ready for any situation.
WinFE development is next, followed by the use of DiskPart and the WinFE Write Protection Tool. Demonstrations are given of DiskPart and the Write Protection Tool after the lecture on each. Following this, he talks about the importance of tool validation and that you must validate tools yourself and not simply rely on the word of others.
One of the cool things about WinFE is that there are multiple ways you can build your own. Some are a bit cumbersome, while others are strikingly easy. Brett covers them all and demonstrates how to use each method to build your own version of WinFE. Two of the videos in this section were created by other people, while Brett takes care of the remainder.
Next up, quite a few different use cases are presented for WinFE. Suggestions for each of those use cases are given and some other tips are given at the end of this section.
After the wrap-up, you get your chance to take the course exam. The exam consists of 25 questions and you must score an 80% or above to pass. You get two chances to take it, so if things don't go well the first time, you can still take another shot at it. I'm happy to say I passed on the first try.
The final section of the course is a downloads page. Links to lots of different WinFE related materials are here, including materials referenced throughout the course.
I was very pleased with this course. I thought Brett did a great job presenting the material. He speaks with the voice of experience and you can tell the suggestions he makes throughout the course are taken from his own use of WinFE. This course is free, but it is definitely worth paying for.
Speaking of which, Brett has also come out with another course that does cost a little money. This course is titled the X-Ways Forensics Practitioners Guide Online Course. As you may know, Brett and Eric Zimmerman wrote the book of the same name. The course costs $195, but if you sign up before July 17 and use discount code xwf1, you'll get a 25% discount. Not only that, but if you sign up before the 17th, you'll also get part 2 of the course for free! I hope to come up with some cash and take these courses, as I've been an X-Ways user for over 4 years. I learned a lot from the book and I'm sure the class will be good as well.
Wednesday, July 2, 2014
Next, I want to say congratulations and awesome job to David Cowen for completing 365 straight days of posting to his Hacking Exposed Computer Forensics Blog. I find it difficult to get 10 or 11 posts up a year, so I can't imagine coming up with a new one every single day. His Forensic Lunch and Sunday FunDay contests have also been an excellent way to learn more about CF and I thank him for his hard work.
Corey Harrell has an fantastic post on his Journey Into Incident Response blog: Improving Your Malware Forensics Skills. He goes through his process for conducting forensic analysis of an infected system and points out that establishing the process you will use should be the first step in the investigation. He talks about tools, setting up your test environment and describes various ways of infecting your test systems. It's a great post and I learned from it. I suspect you will too.
I'm excited that The Art of Memory Forensics, written by the core developers of the Volatility Framework will be released later this month. The front and back covers along with the table of contents and preview are already available to view on the Amazon page for the book. Without even seeing it yet, I will predict right now this book will be a strong contender for a Forensic 4cast Award next year.
From all the talk I've heard, it sounds like this year's SANS DFIR Summit in Austin, Tx was another first class event. I've been fortunate to attend twice and found the conference to be an excellent venue for learning and networking. Congrats to Rob Lee and everyone at SANS for another successful Summit. Wish I could have been there, but my funding didn't come through. Also, congrats to Lee Whitfield on another successful 4cast Awards. Congratulations to the nominees and winners as well!
A new learning opportunity has come about and I'm already signed up. Brett Shavers announced a new, FREE course on using the WindowsFE boot environment. From the course web page, "This course will give you everything you need to know in order to fully understand, build, use, and testify to the use of the Windows Forensic Environment." You can learn more and sign up on the Windows Forensic Environment course page.
Brett also announced a separate course he'll be releasing soon on the use of X-Ways Forensics. Brett, along with Eric Zimmerman, wrote the award winning X-Ways Forensics Practitioners Guide, so I know he knows what he's talking about when it comes to XWF. Follow the book website or Twitter account for details.
UPDATE: The class is now available at http://courses.dfironlinetraining.com/x-ways-forensics-practitioners-guide
UPDATE 2: Use discount code "xwf1" between now and July 17 to get a 25% discount on the course tuition.
That's all I have for now. Take care and thanks for reading!
Saturday, May 31, 2014
After a long hiatus from blogging, I have finally returned. To be honest, I just haven't had the time for it, but I've recently had some fun stuff going on that I wanted to write about.
As you may know, I am quite interested in both malware and honeypots. I've run Linux SSH honeypots in the past and recently started one up again. Within a few days, I started seeing a repeat visitor who tried to change the settings of the system as well as downloading one file per visit.
Based on viewing the logs "replayed", it looks like the attacks on my honeypot are automated. There are no typos or other signs of actual human interaction. In all cases, the attacking IP address and the IP address from which files were downloaded were based in China (shock!). All the downloaded files were of the Executable and Linkable Format (ELF) type.
The IP address I have seen these attacks coming from is 184.108.40.206. The whois data for this IP is:
inetnum: 220.127.116.11 - 18.104.22.168
descr: CHINANET anhui province network
descr: China Telecom
descr: A12,Xin-Jie-Kou-Wai Street
descr: Beijing 100088
status: ALLOCATED PORTABLE
changed: firstname.lastname@example.org 20040721
There is one other IP that has conducted an almost identical attack and it too was of Chinese origin.
I am including a video recording below of one of the log replays. Below that, I'll talk about what the attacker was trying to accomplish with each command. One note: I am not even close to being an expert on Linux or on attacks. If you see that I am wrong about something I post here, please let me know. I love to learn and will be happy to be corrected.
descr: CHINANET YUNNAN PROVINCE NETWORK
descr: China Telecom
descr: No.31,jingrong street
descr: Beijing 100032
remarks: service provider
status: ALLOCATED PORTABLE
remarks: This object can only be updated by APNIC hostmasters.
remarks: To update this object, please contact APNIC
remarks: hostmasters and include your organisation's account
remarks: name in the subject line.
Thursday, January 9, 2014
Digital Forensic Blog of the Year
Digital Forensic Article of the Year
Digital Forensic Book of the Year
Computer Forensic Hardware Tool of the Year
Computer Forensic Software Tool of the Year
Phone Forensic Hardware Tool of the Year
Phone Forensic Software Tool of the Year
Digital Forensic Examiner of the Year
Digital Forensic Organisation of the Year
The awards ceremony has really become a much anticipated event each year after the whole thing started a few years back as a sort of tongue in cheek thing. I have to admit, it really brightened my day to be nominated for one of these awards 3 years ago and then to actually win the Digital Forensic Blog of the Year award last year. I hope everyone reading this post will head over to the Forensic 4cast site and nominate people for each of the categories.
The hardest part of this for me is trying to decide who to nominate and later vote for. Some categories aren't hard, but the blog and blog post of the year awards are always tough for me. There are so many high quality blogs out there that it's tough to just pick one. Corey Harrell, Jack Crook, Harlan Carvey and David Cowen all stand out this year with some excellent posts that I've learned a great deal from.
Likewise, the Digital Forensic Examiner of the Year is tough. I know so many people in the field whom I believe deserve an award like this that it's hard to pick just one.
I want to thank Lee Whitfield for all the hard work he puts in to these awards each year. Lee is a great friend and a great supporter of the DFIR community. I know he takes some guff from people who seem to think he should automatically include them or their product as nominees whether or not anyone actually nominates them and that's a shame. He deserves praise and thanks for bringing the awards to us each year instead of complaints from sore losers. These are the "people's choice" awards for DFIR and Lee just manages the show, he doesn't decide who gets nominated or who wins.
Okay, I'm off the soapbox now. So, get yourself on over to the Forensic 4cast Nominations announcement page and make your nominations.
Tuesday, December 3, 2013
After successfully imaging and examining RAM, I decided to make several profiles for machines I regularly interact with. After that, I decided I may as well share them with others. Therefore, I have created a Github page with the four profiles I've created so far. I will be creating and posting more very soon. It isn't much, but I've wanted to find some way to contribute back to the community and thought this would be a good start
Saturday, November 30, 2013
The course was written by Andrew Case, Vico Marziale and Joe Sylve. All three are well known in the forensics field and have contributed much. Andrew is one of the core developers of Volatility, as well as a co-developer of RegDecoder with Vico and Joe. Vico is also a co-developer of Scalpel. Joe is also the developer of LiME. These guys know their stuff.
The course is a combination of video lectures along with hands on labs and quizzes. There are 14 course modules, though some of them are broken down into multiple parts, for a total of 21 lecture videos. The course slides from the videos are also provided in PDF format. Other supplemental reading materials are also provided in PDF format. Some of the supplemental reading is necessary to complete the labs and quizzes, while others are simply to help you understand a topic better but not required.
The majority of modules include a hands-on lab and a quiz. The lab is used to answer the quiz questions. I'll describe the lab setup in a moment. Each lab is accompanied by a guide that walks you through how to complete the exercise in case you get stuck. These PDF format lab guides not only help when you get stuck, but they make great reference material for those times later on when you can't quite remember how to solve a particular problem.
The labs themselves are all performed in online virtual machines accessed through your web browser. One is a Windows 7 virtual machine, while the other VM is Ubuntu 12.04 LTS. All the required tools and lab files are pre-loaded on these VM's and ready for use. I really enjoyed working with the labs and felt they added a great deal to the course. I'll provide some examples later in this review. Here are a couple screenshots from the lab VM's.
Module 1 is an introduction to the course and the Windows Registry. The goals of the class and other material are covered, followed by a description of just what the Registry is and why we should care about it. I suspect most people taking the course will have some idea of what the Registry is, but I think there will be many who have no idea just how much they can actually gain from it.
The core hives are all briefly covered, followed by coverage of the NTUSER and UsrClass hives. Finally, methods of acquiring registry hives from running and "dead" machines are covered. There is no lab with this module, per se, but complete instructions for accessing and using the labs is covered in the written material.
Module 2 is when we start diving in to the nuts and bolts of the Registry. Hive structure is covered first, followed by discussion of the various parts of the hives, including keys and values. Records and their headers are also described in detail and help those new to the material to understand how the Registry is put together, so to speak.
Module 3 is the first one to be broken down into multiple lectures. 3a covers the Software hive, 3b covers the System hive and 3c teaches about NTUSER. The purpose of each of these hives is described and many keys of interest are given. It is acknowledged that no one course could cover every possible key that might be valuable to an investigation, but many of the most frequently valuable keys are given. In this module, as well as most all of the others, real world "war stories" are given to help drive the point of the lesson home. 3d covers the "other" Registry hives, being the UsrClass, SAM and Security hives.
Next up, Module 4 talks about recovering Registry data from various Windows backup facilities. Registry data from Windows XP Restore Points, Vista/Win7 Volume Shadow Service and Windows 8 VSS and File History are covered. Using historical Registry data from backups is talked about, such as incorporating them into timelines.
Registry analysis tools is covered in 5a, b and c. 5a talks about several tools, including Access Data's Registry Viewer, RegLookup, MiTeC Windows Registry Recovery and libregf. As with the other modules, there is a lab that gives you the opportunity to try out some of the tools.
5b and 5c cover the "big dogs" in Registry analysis tools: RegRipper and RegDecoder. The use of each is covered in detail, along with other RegRipper related tools, RipXP and Auto_Rip.
I have taken a few halfhearted stabs at learning Perl and Python in the past, but never really got anywhere. I always thought it would be nice to come up with my own RegRipper plugin, but hadn't ever tried. Modules 6a and 6b teach about scripting RegRipper and RegDecoder. After completing 6a, I finally did my first RegRipper plugin. It even worked! ;-)
Deleted Registry hives and keys was next in the course. What happens when keys and values are deleted and information about recovering deleted data is the focus here. Very interesting stuff and something I think could be very useful to many analysts.
While I'm no expert, I greatly enjoy learning about memory forensics. I was pleased to see Module 8 covered the Registry as it appears in RAM. Volatile hives and data are covered here, along with using the great Volatility Framework to access Registry data in a RAM image are all part of this excellent module.
Modules 9 and 10 cover Registry timelines and baselines respectively. Methods of using RegRipper and RegDecoder to create timelines from the registry are explained.
Module 11 is about one of my favorite subjects, malware. This module is all about how malware uses the registry for persistence, as well as the ways it may alter registry settings (such as turning off the firewall, etc.) to allow certain behavior. Also discussed is the way to find evidence of program execution in the Registry to help track down malware.
Anti-forensics is the focus of Module 12. I'm getting ready to go testify in a case next week in which ccleaner was used to attempt to foil forensics data recovery, so I found particular interest in this module. The module covers the various means of anti-forensics and their effects on the Registry very well.
Module 13 is the "final test" so to speak. A scenario is presented and you must use the skills and tools you've learned over the preceding modules to answer the quiz. The instructors did a really nice job putting this together and I think you'll find it was done very well. Finally, Module 14 is simply the end of class wrap-up.
You can probably guess from all that precedes this paragraph, but yes, I do highly recommend this course. At $399, you definitely get your money's worth. The lecture videos are all very good, getting the point across with easy to understand lessons. The labs are also very good. I thought they were both educational and actually kinda fun to work in. I believe the course has something to offer both new analysts and veteran's alike. I've been doing forensic work for about 5 years now and make use of the Registry in most of my investigations. Still, I learned a great deal in this course and think most others will too.