Detailed Forensic Investigation of Malware Infections
Instructor Mike Danseglio
How does an investigator hunt down and identify unknown malware? In this IT Security training video, Security expert Mike Danseglio (CISSP / CEH) will perform several malware investigations on infected computers and identify symptoms, find root cause, and follow the leads to determine what’s happening. He’ll demonstrate his preferred tools including systems where he has no idea of the source or scope of infection. The types of malware Mike analyzes include rootkits, botnets, viruses, and browser toolbars.
We’re going to talk a little bit about forensic investigation. It’s kind of an interesting one in that, based on feedback from my last webcast How Do You Know You’ve Been Hacked? I found out that a lot of folks wanted a little bit more hands‑on, a little more technical, not as much overview or broad‑spectrum stuff. They wanted to dig right in.
We took that feedback and I bounced it off a few folks here at Interface Technical Training. We came up with the idea of “What does Mike do when Mike’s looking at malware, a system that’s probably infected, possibly infected, looking at a system that’s known to be bad. What do I do? What are my techniques?” It’s kind of hard to write those down. That’s what we’re going to be doing today. Some of it’s really nasty. Some of it’s not quite as nasty.
I’ve got three cases for you that I’m going to show you, and we’ll walk right through them.
Three Compromised Systems
- Windows 7 SP1 + Updated
- Typical Configuration
What I did was I took three Window 7 machines, Window 7 Service Pack 1 all updated, all patched up, all fixed, ready to go. I configured them as a lot of corporate environments would configure them.
I put the network that they’re connected to on the “work network,” which actually is very profound. I disabled firewall because a lot of organizations do that for application reasons, for different things. They say, “Well, we have a firewall at the perimeter,” or, “We have two firewalls between the computer and the Internet. Therefore, we don’t need firewalls on the host.” I disabled that.
Some of the machines that I’ve got, I’ve configured with malware scanners. Some I haven’t. Some configured themselves with malware scanners, which is a really interesting case. We’re going to look at that in a moment.
These computers are actually physical computers. I’m going to talk a little bit as I get into this on how I configured them to be isolated, yet still have live malware.
I had to worry about protecting the rest of the company, the rest of our network from malware while still being able to broadcast live malware for you today.
I’m going to be remoting into those with VNC. VNC because I like it, because in this environment it’s great. You could use RDP. You could use whatever. It doesn’t really matter.
The three machines are:
Tom, Dick, and Harry are ready to go. Each of these is a different user scenario. Each of these has a different back story, a little bit different philosophy around what kind of malware they got, where they got it, how they got it. Before I go forward on that, I wanted to talk a little bit about Tom, Dick, and Harry as a group.
I tried as best I could to emulate what typical users will do in a corporate environment. I didn’t get this construed, “well I extracted this particular piece of malware and injected it this way,” and did this stuff. Users typically don’t really want malware. Users are looking to solve a problem, or fill a need, or perform a task.
These users, unfortunately for us a lot of times, are not as well versed in the evil set or malware in the Internet, and attachments, and things like that, and they wind up with malware. I tried to really focused when I was getting this things infected on the different stories, the different scenarios that would make this result happen. In Tom and Dick, that’s definitely what I’ve done very, very tightly.
Harry is a little bit of an exception, just because Harry is the most dangerous of the three that we’re going to be going through. That’s why I’m leaving him for the end. I wanted to show you all three of those. I bounced these scenarios off students, off other instructors, off other staff, off a bunch of the history that I’ve got working these problems.
If you think they’re not realistic, or there not the ones you’re looking for, make comments and chat about what you want to see next time. Definitely, I’d be happy to do that for you.
The Process I’m Following
- Understand the Problem Report or Flag
- Examine the System
- Determine Probable Scope of Impact
- Identify Remediation Options
What I’m going to do is first off, I’m going to fire up one at a time Tom, Dick, and Harry. We’ll start with Tom. I want to understand what the user problem is. What the concern is from the user about what’s going on in the system.
After I understand what’s happening because if there’s not a problem, I’m not really going to solved anything. I want to examine the system and see what’s going on with the system. I want to take a look at it without actually breaking anything, without actually fixing anything, examine the system.
I’m going to spend some time looking around seeing what’s going on.
The third bullet is an important one. Identify how bad the problem is. Is it just this computer? Is it all computers? Is this particular piece of malware, or set of malware growing in size? Is it trying hide itself, is it trying to infect other computers? Is it just giving us basic pop‑ups? Is it a browser‑based thing? Is it an OS‑based thing?
Get an idea of the scope, and potentially the impact. Is it destroying data? Is it destroying the company network? Is it just really pissing off one particular user at a time? Those are very different things with different kinds of responses that we might formulate. At a basic level, I’m going to probably dabble a little bit, because the questions will inevitably go there, into remediation options. How would we fix this? How would I get rid of this stuff that’s happening?
We actually do have Tom, Dick, Harry. I physically can figure this. These are old slack machines or scrap machines that we had laying around Interface. They’re Dell 490’s. They’re not bad. For this purpose, they’re actually perfect. They’ve got most of my four gig of RAM. They’ve got an old rotational like a 250 gigabyte or 500 gig hard drive. They’ve got an old Zeon processor.
They weren’t doing anything, so it didn’t really matter if I absolutely destroyed them. I was worried yesterday that I damaged the hardware on Harry. It looks like I only damaged software on Harry inadvertently, not the hardware. Just as a quick note because I’m over here, what I have also done is I have configured Tom, Dick, and Harry to have two network interfaces.
The wired NIC runs along the floor over to the console where I’m VNC’ing, and they’re on one switch altogether, nothing else. They’re not connected to any backbone, any Internet connection at all. They are connected to each other, could just easily be a hub. The other network connection it has is a wireless network card. I’ve got a wireless network card, an Alpha, on all three of these.
The reason that I put an Alpha on all three of these is I needed an Internet connection to really be able to show you what this malware is doing live. I needed an Internet connection, but I didn’t want it to be talking to the rest of our network here at Interface. These three wireless NIC’s connect up to a dedicated access point that is tunneled out just to the Internet, that doesn’t touch anything else.
It is isolated enough that these things can get to the Internet, but not get to any other part of our network theoretically. If you’ve been in my hacking class, you know that theoretically is the operative word there. Could we decimate that? Could we blow away the switch? Make it into a hub? Break the VLAN? Of course, we could.
Most of the malware we’re looking at today, I don’t expect it to be of that level, but there is always the possibility. There is always that threat. Let us get started on Tom. Tom’s an interesting user. This is Tom’s computer that we’re remoted into, that I’m remoted into. “Stop with the royal we, Mike.” That I’m connected into.
Tom is a big fan of cats. He’s one of those crazy cat guys. If he had rollers, he’d be a crazy cat lady. Tens of cats. Hundreds of cats at home. He talks to them. He gives them odd names. Stuff like that. It’s an interesting situation for Tom, because at work he has administrative privilege on his computer. He wanted to get cat screensavers and cat desktop wallpapers. He wanted them to change. Sounds like a reasonable user, right? Great.
What did Tom do? Tom went out to the Internet and searched for “free wallpaper kittens,” “free wallpaper cats,” and “free desktop background Windows 7 cats, kittens,” that kind of thing. Because I configured this machine, he took whatever the default prompt was. I believe he clicked three times across two different websites and allowed those websites to install software that would change the background, that would install screen savers.
Of interesting notes, since this is a forensic investigation discussion. When I started with this machine, it didn’t have the wallpaper. I didn’t happen to have that wallpaper handy, but also of note, there were no icons on the desktop at all when I started. The first thing I notice when I walk up to a computer like this is, “Wow, that’s a lot of icons on the desktop.”
If you are like me, if you’ve been doing this for any length of time, you immediately see some red flags in which icons are on the desktop. The first thing I notice is, “Holy moley, WeatherBug is installed.” We don’t really going to have to go any further into this machine. We know what’s going on here. We know that there’s almost certainly going to be some malware on here.
WeatherBug is one of those stereotypical, carries‑a‑bunch‑of‑other‑stuff‑with‑it, packaged‑with‑other‑pieces‑of‑software, nasty things. We don’t have just to stop there. Apparently, when Tom was downloading these free wallpapers and screensavers, he allowed these free whatever they were, to install whatever other software they wanted, like a 24/7 help. A Help Desk application.
What do we know just from looking at that icon? Well, 24/7 Help sounds like it would allow remote desktop access. It sounds like it would allow someone to take remote control of this computer. It sounds like it would allow people to transfer files back and forth. Who? Who knows? Does Tom know? Does Tom know who’s connecting to this? Guess what? The answer is, no, probably not. It installed a security piece of software. Some kind of malware scanner.
What kind of malware scanner? Who the hell knows? Is it real? Is it fake? We’ll find out in a minute. It installed GeekBuddy. I don’t even know what GeekBuddy is yet. Again, I’m doing this live, so I’m going to find out what GeekBuddy is. Probably use my old friend, the Google, to find out, and all kinds of other lovely software ‑ rebate informer, rebateblast.com, which is a shortcut. That’s lame. All kinds of nasty stuff.
Down here on the tray, is the next place I look, and wow, all the software is already running. He’s got some kind of inbox toolbar, which is probably a web browser toolbar, I’ll fire up a browser in a minute. It installed some backup software, it installed some security software, it installed whatever the hell AppGraffiti is.
By the way, I would recommend if you’re doing this, you’re taking notes. What the hell is on the machine? You’re either capturing the screen like what we’re doing here, using Camtasia, or using Fraps, something like that, or you’re grabbing snip, using the snipping tool, or SnagIt, or you’re just writing all this stuffs down, because you want to know what’s going on in the machine.
The next thing I’ll typically do, is before I even get in to any technical tools, I just go and fire up the browser. Let’s fire up Tom’s browser and see what this thing looks like, see what this astrocity has in store for us. Well, first thing I noticed there, even before any web pages, we have a tab here called 24/7 Help. That app over here seems to have installed an attachment to every app that launches, where you can ostensibly click on tech support.
OK, great. Do I really want to do this? I can imagine Chad is probably giving me an answer. I don’t think I want to do it. We have a slight delay. I mean, I actually have had a webcast in the past entitled, “Why are all toolbars are evil?” I’ve got three of them up here already right now.
This is horrible. I mean these stuff is just, what the hell? Three toolbars. My home page has been changed, or rather Tom’s home page has been changed, to inbox.com, whatever that is. Did Tom intend to do that? No. Remember, Tom only really wanted to install cat backgrounds and cat screensavers.
All this garbage are on there. What can toolbars do? Why are they evil? They can monitor all web traffic. A lot of times, they don’t just monitor the traffic through the web browser they’re installed in. They’re actually launched as background processes, and they can monitor all web traffic.
You don’t have to be surgeons here to figure out, this already tells me, we’ve probably got a relatively infected computer. Let’s get a little bit deeper on this now and see what’s going on. We’ll close that garbage. I’m going to bring up both Task Manager, and I’m going to bring up Control Panel programs and features.
Generally speaking, most pieces of malware are resource pigs. They tend to suck up a lot of memory, tend to suck up a lot of CPU. A lot of times, all I have to do really is sort by amount of memory getting used, or sort by CPU getting used to actually see some of the more prevasive malware. VNC is what I’m running to actually do this presentation. We can ignore that, but we immediately see this live PC support component.
Unit.exe is actually taking up a ton of memory, relative to the rest of the stuff. Is that over here? No, it’s not. How interesting is that? We have some software that’s running that’s not listed in the add/remove programs list, or the programs and features list. We do however see that all of the stuff probably started right around here, right around 420. It started with 360 Total Security, MyPC Backup Super Optimizer, Unchecky, WeatherBug.
Interestingly enough, because I saw this machine get infected, most of the installs actually stopped being visible right about here, right about WeatherBug. It stopped prompting Tom to actually say, “OK, or click yes in order to agree to the terms of service.”
The rest of the stuff just got funneled into the background. What happens with most malware infections is, you get a cascade effect, where one piece of malware installs all the others. Do we know which piece of malware installed all the others? Actually, no. It’s not easy to see from here which particular piece of malware did because it might not have installed itself.
It might have been an executable, it might have been a browser component that plugged in and actually installed all of this, funneled all this stuff down. It could be just bad as anything that actually funneled all these traffic in it. Usually, I look for the first one. Does this tell me anything? Really not.
This doesn’t tells me that 360 Total Security installed MyPC Backup and Super Optimizer, and Unchecky. Sorry, I just got a little heeby jeeby there. That creeps me out. [pause] All of this stuff, as you probably can already guess, is garbage. We don’t want them on the system. It’s all malware.
That actually gets to the concept of what is malware. Malware, if you watched the previous session I did, is malicious software. Whatever software is undesired, whether it’s undesired that the user or the administrator didn’t want it installed, or undesired in the sense of doing something in the background that shouldn’t be getting done. This stuff is malware.
There’s a ton of malware here. Ton and ton. One, two, three, four, five, six, seven, eight, nine, ten, eleven, twelve, thirteen, lucky 13, of malware. I assure you, if I left this machine on for another day or two, it would go up. The number would just keep going up and up and up. It’s a cascade effect.
You got to get rid of this stuff as quickly as you can. That’s why, to quote the old adage, “An ounce of prevention is worth the pound of cure,” here. Preventing the software from getting in on the first place, this malware from getting on in the first place, is how you essentially make sure the system doesn’t get hosed up in the first place.
I have a 24/7 tab installed in task manager, how did that get installed?
It’s a cool trick. Windows allows us to actually have software that will hook every app that runs. It depends on where the app is running, depends on how deep in the OS the app is hooked. We can actually see this with non‑malware, we see this with regular software.
For example, Smart Software that we used for smart boards here, hooks every application. There’s a lot of different features of Windows Glass or Windows Aero, that allows a programmer that knows what he’s doing to simply hook the Win32 API, and make a call down that says, “Hey, whenever an application, a user mode application is launched, go ahead and launch me with it, and put me on the title bar.”
The interesting thing about the 24/7 help tab, is it has to be written by an experienced programmer. This is actually has to be a relatively good code to a, make it do what’s it doing, and then b, to make it not crash every app, because it has the potential to crash every single app.
Arguably, this is one of the more dangerous pieces of malware. But how does it get installed? It’s get installed just like any other piece of software. There’s many pieces of software that can hook other Windows as long as the person that installs it is a local administrator which brings me to…I’ll do it to the camera, I’ll do the Hitchcock thing…”Are you letting users be local administrators? Hmm?”
There we go. Answer that question for yourself and you’ll know whether this is possible. Cool, next question.
Why isn’t the app showing up in the install list? Why are they not showing up in the install list?
That’s a great question, actually. Apps sometimes will show up in the programs and features list and sometimes they won’t because the programs and features list is not a security feature. Programs and features list is not a security feature. It is a convenience feature to users and administrators that allows users and administrators to figure out what programs were nice enough to tell Windows, “Hey, I’m installed and here’s how you get rid of me or manage me over time.”
Do apps have to do this? No, no they don’t. It is an honor system. If apps don’t want to list themselves here, they just install themselves without listing themselves here. Most of the apps we want to use, most of the apps we are happy to play with, do actually list themselves there. Most apps do.
Most installers like Wise installer, all those kind of standard installers, Visual Studio, Packager, they actually list themselves here because that’s the right thing to do. Malware doesn’t do the right thing, does it? There you go, it’s trivial to get an app installed without having it listed there.
Do you recommend any malware apps?
Anti‑malware apps? OK, yes I recommend whatever anti‑malware app you can installed because you’ll see it’s infinitely harder, not impossible, to get malware on a system that has a malware protection software. If you’re asking me which one’s the best, I actually have an answer for that. It took me years and years to come up with an answer.
The best anti‑malware product is any anti‑malware product that’s up‑to‑date and on because I’ve actually seen infections first hand where there is a great anti‑malware product in place but it was disabled or snoozed or wasn’t up‑to‑date. Whether it’s Semantic, Kaspersky, Barracuda, Sophos, or Microsoft, I don’t actually care. All I care about is it’s on and it’s up‑to‑date and I’m happy.
Will one maybe be a little bit better for this particular piece of malware at this point or that point? Yeah, they all play that game. Microsoft found this one first and Semantic didn’t find it for two weeks. Then, the next piece of malware, Semantic found it immediately and Microsoft took a month to find. They play that game. They all play the little numbers game, but at the end of the day, it all averages out.
Cool? All right, good, good, good. I’m right on time. Thank you, Jeff. That is Tom in a nutshell. By the way, as a curious point, because I haven’t actually looked, I’m going to look to see if Tom wound up getting his cat screensaver or not because I’m just curious. It’s under control panel, display. Oh no, sorry, it is under customization. I’ll do it this way, personalized. I don’t know, that doesn’t seem like it’s worth all the malware to me.
We don’t even have to wait around for Dick, unfortunately. No pun intended. We can see that, first of all, this installation of Windows, actually got, somehow, unactivated.
I don’t know if that’s actually a component of how the software was installed, but I don’t believe I was getting this message before. Malware sometimes makes enough modifications to Windows where Windows says, “You’re not licensed anymore.” Makes enough changes to the kernel, makes enough changes to the Windows activation engine. Tries to extract keys to actually send back to the mother ship where it will actually unactivate a copy of Windows.
Immediate red flag. I’ve got a Windows activation prompt. I didn’t even notice that before I noticed the Comodo GeekBuddy, a GeekBuddy performance issue flag.
This is a vanilla installation of Windows 7. Vanilla installation of Windows 7. Before we even get into what Dick is doing on this machine, we’ve got a vanilla installation of Windows 7 saying, “Hey, you’ve got performance issues. You should fix it.” That doesn’t sound likely to me at all. It just doesn’t.
The scenario we’ve got here with Dick is Dick wanted to actually browse the Internet without his manager being able to see what he was doing. Dick wanted to install Firefox. He went out to a popular search engine and typed, “Download Firefox,” and took the first link that he could find that looked relatively legit. Then he followed a few prompts and actually said, “OK, great. I’ve got Firefox on my machine now.” That’s what happened.
Interestingly, the first thing I noticed is we’ve got a lot of very, very similar icons here. Tom, looking for screensavers and kitties. Dick looking for some kind of private way to browse the Internet. They wound up, it looks like, with the same packager.
In malware universe, a lot of times the people that package up malware, that package up the spyware, the adware, stuff like that, they get paid per install. They’ll package up a bunch of these chunks of malware into one big installer and then stick on top of them whatever semi legit software they think people are looking for.
They’ll probably stick a copy of Firefox in there with all of this spyware, adware, spamware, all that kind of garbage, and then present it to the user. That’s the first thing I know. I’ve got two columns and change of garbage on the machine. I’ve got this GeekBuddy pop up that’s actually quite annoying. We’ve got slightly different malwear here. We’ve definitely got, good old weather bug is here.
PC RX System Optimizer, that should be interesting. PC RX System Optimizer, I’ve actually encountered this thing in the past. What it does is optimize the registry. I love that term. I can see my database guy in the corner actually rolling his eyes a little bit. How does that work, Jeff? How does the registry get optimized?
Get rid of stuff.
Mike: Get rid of stuff, yeah, because more stuff in the registry must be bad, right? That’s now how Windows works at all, but it preys on people and their weakness and their lack of understanding of how Windows works, because most of the people on the webcast understand that registry cleaners don’t do anything. But most users don’t get that, where they say, “Well, my computer’s running slow.”
In fact, that was the other complaint that Dick had, was, “My computer’s running slow. I need to optimize it.” Other people are getting newer computers. My computer is old. My computer is lagging. My boss won’t get me a new computer. I tried to break it, and he just gave me another one of the old computers, so I’ll install some stuff to speed it up.
This is the kind of stuff they see when they try to speed up their computer. They’ll probably click “fix issues now.” I’ve not ever clicked “fix issues now” before. I would wager that this is going to do virtually nothing for me, that this will not really change anything significant in the registry, and that it will probably want me to pay for it.
Because, look… [cough] Pardon me. Even though this is a brand new fresh install of Windows 7 with Office, I think I installed on it. I’ve got 11,073 redundant files. I’ve got 22 registry errors, whatever the hell that means, because those aren’t real things. I need to tweak a bunch of stuff. Don’t you think that’s important? Well, I do.
Matt, you’re running the camera, should I click on the big green button? I think, yeah, he’s giving me a thumbs up. I think I should click on the big green button. Why would I not click on the big green button if I’m a user? I’ll get away from the 360 Internet protection that wants to scan all of my IO , then come over to the tab that it intended to open, which is the “how much do you want to pay” tab.
They’ve only got, the cheapest version is 40 bucks for one year, where they’ll keep this software on my machine for a year. Then, I can also have a backup CD that will actually…Oh isn’t that nice. They’ll burn a CD, and send it to me with all of the software that they detect on the CD already. That’s pretty miraculous, isn’t it? How do you think they get that software? I’ll just leave that exercise for the user. Sorry, throat is drying out. I’ll drink a little bit more water.
Secure multi‑license. We have blah, blah, blah and blah. They want some money from us. Actually, one interesting bit’s down here. How does the webform know I’m in Arizona? Think about that. Because it’s right, I am in Arizona.
That’s not just a lucky guess. A lot of these websites look at geofencing data. They try to get geolocation from source IP, or from other data that they actually harvest, either from the machine or the network connection to actually try to populate this. That will vary, the success will vary depending on what information they extract, and how they’ve been installed.
If I let this thing get installed…Yeah, sure, why not? This is Dick’s machine. We’ll assume Dick would want all of this stuff to get done.
Well then, we’ll almost certainly install something, a piece of malware or actually a piece of legitimate software that’s being used as malware called, “Fiddler,” which will actually do a debugging session of the browser and actually track all IL. All Port 80 and 443 traffic, and log it to a place where can actually send that up to the attacker.
This piece of browser‑based malware can be pretty nasty. But even still, let’s assume that that’s not malicious. This certainly is doing things that in a corporate environment that we don’t want it to do. We certainly don’t want some third party piece of malware scanning. We almost certainly don’t want something called, “Driver Update,” because if…
Dick is actually installing these on his machine, even inadvertently. He’s going to wind up clicking on some of this stuff at some point. Actually, this is a great segway into the fact that a lot of these malware is unstable. A lot of this malware just doesn’t work right. It’s crashing on me.
That’s OK. I don’t mind if the app crashes. Yep?
I want to go kind of behind the scenes now. What’s going on behind the scenes?
I think back into the Task Manager, what’s going on? How do we detect what’s going on, because I just seen what’s on the screen?
All right. I took the liberty bringing up this particular Task Manager. I actually added the command line column to Task Manager because I wanted to see where all of this stuff is launching from. You’ll notice that we’ve got, actually, a bunch of different software launching from a bunch of different locations.
If you did already know, program files is typically where apps install. But as I said, apps install based on the honor system most of the time. They don’t have to install in Program Files.
This app, Third One Down, no.exe did not install in Program Files. It actually installed under the User Profile. It actually installed under the Admin User Profile. It’s the OkayFreedom application, and it’s…The executable is called, “no.exe.”
We’ve actually got a bunch of different apps here. My favorite technique for using this particular window for using Task Manager in malware forensics is actually to just sort by command line. Why? Because anything that’s generally, generally, not universally, but generally, anything that’s in Windows is probably going to be something that is something I want to have. It’s less interesting to me. Not immune, not perfect, but less interesting to me.
Anything above that, anything that’s loading at a Program Files, x86 or Program Files, which, if you have a 32‑bit system, you won’t see x86. You’ll just see them all as Program Files.
These are all suspect. These are all things that loaded up. I haven’t done anything since the machine booted. These are all things that loaded at boot time and are running currently.
I can immediately see, don’t want that. Don’t want that, Don’t want that. Don’t want this. Don’t want this. Don’t want this. Don’t want most of the stuff that is up here. Absolutely not.
US tech support. I haven’t even got it into the fact that there’s a live tech support thing going on on the machine. Forgetting the fact that these annoying popups kept coming up.
Let’s look at killing one of these. Well, I can remediate that. I’ll just kill the process. That is so awesome that people think that, isn’t it? Let’s get rid of GeekBuddy. Let’s get rid of Driver Update. Wait, I thought I got rid of GeekBuddy. Hold on. Let me right click it again and choose, “End Process.”
I could have sworn that I just got rid of GeekBuddy like 10 seconds ago. I don’t think…Oh. What an interesting coincidence that is. It launches itself again, and again, and again, and again, and again.
There’s GeekBuddy reminding me that it launches itself. It’s taunting me, teasing me. Do you see that? Isn’t that horrible? How does that happen? I know the question’s going to come in.
How does this app keep getting killed and then launching itself? There’s a bunch of different ways that software can actually hook itself. It could be a restart on crash service. It can actually be a scheduler thing. It could be a separate process that’s watching for the existence of GeekBuddy as a process or a service, and if it’s failed, will re‑launch it automatically.
There’s a bunch of places in the registry that we can actually hide entries that will tell Windows, “This thing has to be running all the time, and if it isn’t, launch it again.” We can tell Windows to do that, or the software can manually do it, or both. I would wager since GeekBuddy is so persistent and so quick, it’s probably doing both.
Did you have a virus tracker?
Oh, that will be coming up in just a sec, actually. We can try to kill these things. Actually, the hallmark of malware, typically, is the fact that it likes to be persistent. It likes to come back to itself over and over and over again. Is that bad? Yeah, that’s actually really, really, really bad.
While I’m bringing up control panel, because I wanted to take a look at what it looks like in there, Jeff, any questions coming in while I’m doing this?
How does malware disable running ending malware software?
That’s actually kind of an awesome question. How does malware disable anti‑malware software? For that, I really need to go over to the other monitor and show you how Windows works in a real brief way. I won’t do a lot of architecture on this, but I wanted to make sure to call this out, because it’s really an interesting bit.
Windows is a layered modular operating system. We have the Win 32 apps, the typical 32‑64 bit apps that run up here. We have some other modules that sit down here. Then, we have user mode and kernel mode. Here we actually have services, like service control monitor. We have file system. We have GDI, the graphics display. We have a bunch of IO, etc.
Under that, we have kernel and the hardware abstraction layer. This is not a perfect description, but this is good enough for this conversation. Down here, we actually have the hardware. Everything above the hardware layer is software, everything. If you’ve got a malware scanner running here…Let’s say you have, at this layer, you’ve actually got Semantic. Ah! What the hell?
All right, let’s say you’ve got Semantic running up here as a Win 32 app or as something that plugs directly into Win 32. If the malware plugs in down here, let’s say as a service, as a kernel mode service, the malware can actually affect anything higher up on the stack than itself. It can actually either fool Semantic, and tell Semantic, “Yeah, there’s no malware here. Everything is a notepad text document with ‘hello’ as the only content,” or it can actually kill processes and services that are running above it.
The question is how deep did the malware infect the system? The deeper it gets infected, the harder it is to remove, the harder it is write, and the easier it is to actually fool the anti‑malware software or uninstall it, or disable it entirely. Cool, other questions?
We’re not getting rid of malware yet. We’re still investigating malware. I promise at the end, I’ll have a couple of minutes about getting rid of malware, but for now, let’s continue looking at what the malware is doing, and how it’s working. Other questions?
Is one browser safer than another?
No, all the browsers are vulnerable to different types of attacks, different types of malware, different types of infections. Most of the infections now, or a dominant amount, either use PDF as an attachment, as a vector to infect the machine, or they use Java. Browsers pretty much all support PDF and Java because they kind of have to. There’s not really one great browser, one crappy browser. That was OK to say like six or seven years ago. Now it’s not.
It’s also not one operating system or another. Mac’s have equal problems. Do Mac’s run Java? Of course they do. Do they get infected? Yes. Do Mac’s open PDFs? Yes, they do. Do they get infected? Yes. I’m using Windows here because most of us in corporate environments use Windows as a standard desktop, but I could just as easily do an entire session on destroying Macs, and the creative people behind the cameras would all weep into their double tall lattes with their Fedoras and their ironic beards.
They’d go to the Apply store and they’d ask the genius bar, “Hey, I heard that the Mac can get infected.” The genius would say, “Oh no, you can’t get infected.” Then I would just be garbage, so don’t take that bait. Macs are no safer than PCs. There no worse, but they’re no better. Other questions before we move on to Harry?
There’s one, a long time ago… Is this app, and I think they’re referring to hidden malware apps, hidden within DLL files or the registry?
Mike: OK, are any of these pieces of malware hidden within DLLs or within the registry? Well, if DLLs can be executed, which they can, then malware can hide in a DLL. They can hide in just about anything, so DLLs, absolutely. A file can be very clean, an executable can be clean, but it can call a DLL that’s infected, so yes. The answer is yes to that.
Are they hidden in the registry? The registry doesn’t include executable code. The registry is a database. Can you include references to malware in the registry? Absolutely. Can malware hide in the registry? Not really, it depends on how you look at that term. Malware can be referenced by the registry. It can be called at Start Up. It can be loaded as a service, but it can’t actually exist in the registry.
Cool, now it’s time for the scary part. We’ll leave Dick for just a moment because we probably all agree at this point. We can come to the conclusion Dick is pretty well infected with all kinds of nastiness. Again, we have this kind of cascade of software which is very, very, very similar to the software we saw on Tom. Now, we have something that’s going to be a little bit different.
We have Harry. Harry over there looks pretty innocent, and it looks just like all the other ones. However, Harry’s the one that scares me the most. Why? Because Harry goes out and downloads torrents. Harry likes to look at movies at work, and likes to listen to music at work.
He downloads MP3s, he downloads MP4s. He downloads books, PDFs, and all that kind of stuff, at work, because that’s what he does. He’s into entertainment. Here we’ve got… let’s see, under Documents he’s got some PDFs. Under Music he’s got a bunch of different music. Under Videos he’s got a bunch of different videos.
As soon as I see this kind of thing, I question, first of all, does my corporate side have a policy against Harry downloading all this stuff or having all this stuff on his machine? This is all obviously personal stuff. Not a big fan of that at work. But even more so, how many of these files can be containers for malware?
Can MP4s? Yes. Can MP3s? Yes. Can AVIs? Yes. Can WMVs? Probably the worst of all of them. Yes, absolutely. Can those PDFs? Absolutely they can.
I haven’t infected Harry yet. This one’s a little bit different than the rest, because Harry will have certainly come in here and double clicked on all of these different attachments. He’s reading a book called, “Fifty Shades of Some Color.” Didn’t want to get arrested, didn’t want to go to jail for copyright infringement, so we’ll call it “Fifty Shades of Some Color.”
When he double clicks that, he doesn’t realize that he can be running active code. The PDF may actually contain active code. He’s running an old version of Adobe Reader, and immediately his malware scanner comes up and says, “Holy crap.”
This one I did put a decent anti‑malware on there. I kind of picked a middle of the road malware protection software. Not high end, not low end, not useless, not perfect. I picked something middle of the road. AVG I like because it’s great for home, it’s free, and it’s very, very average.
It’s immediately coming up and saying this PDF is going to be a problem. If we’ve got a user that’s downloading torrents at work, do you think they’re going to say, “No don’t open this.” Do you think they’re going to want to be protected?
And I can see my cameraman saying, “No. Of course not.” Harry wants to read, “Fifty Shades of Some Color.” He’s going to say, “Nope let me do that, that’s cool. Accept. Uh‑oh, error opening. I’d better try it open again. Oh‑oh, I can go beyond Adobe Reader. Look! Big flashy button.”
You know I don’t even need to go this far to get infected, because Harry, he’s definitely going to get nasty. I’m worried, actually, to be honest, while this is loading up that this machine will not be running much longer. Because the longer I leave this running with active malware, the more likely it is to actually destroy itself. To blue screen out or something like that.
Most likely what this did, this PDF did…When I say, “Ignore threat.” Is it’s actually going to throw this dialogue. Does this look like a PDF? No. Does it look like “Fifty Shades of Some Color.” No. This is actually a piece of malware that’s being launched by the PDF that wants to install some more software.
That’s the interesting trick about this kind of malware. A good malware attack will not contain the nastiest parts in the initial chunk. Harry’s malware scanner may not have gone off initially. This might not have been flagged.
However, the subsequent download stuff that he downloads and installs may actually trigger his malware scanner, or it may not.
There’s a couple more pieces of software here, crackdownload.xxx, malware1.xxx. If I rename those executables to .exes, which is what they are, that’s when it gets a little dangerous.
When I’m doing the research, I usually just leave them as .xxx, but let’s go ahead and make them .exes. That immediately comes up with an installer icon, so it’s definitely an installer of some type. Wow, AVG does not like crackdownload.exe at all, does it?
This is a generic piece of malware that I actually downloaded this morning. This is very fresh. This particular piece of malware is exceptionally fresh. I just got it off the vine this morning, and AVG is already picking it up.
However, AVG did not pick up malware1 yet, did it? I’m going to say, “Yeah, protect me for the moment.” It gets rid of it, quarantines the file. Let’s assume that Harry wants to install whatever this malware1 is. It could be called any file. I just chose malware1 so that I would remember what it is.
Oh man, it didn’t run. It didn’t run at all. There’s nothing on my screen. I wonder if it’s not running anymore. Let’s find out, shall we? Let’s hit a browser. This is another interesting tool that I use, I think, for malware investigation.
If you believe malware is on a system, or if I believe malware is on a system, and I know specifically what file or files I’m concerned about, I’ll actually scan them. At a website called virustotal.com.
Local scanners are good, but you only really get a limited amount of information about what’s going on on a system with a local scanner. With something like virus total, which is a free service that is out there on the intertubes, you can actually upload a file to the website, and it will scan the file using multiple malware scanners.
When it comes back, it will actually tell me, “For these various malware scanners I found this, I didn’t find that, this was OK, that was not OK and so forth.”
While we’re waiting for virus total to come up I will start task mangler. Let’s see if there’s anything interesting running. Options…I’m sorry, view, select columns, command line, enter… I will sort by path. I’ve got a bunch of AVG stuff here.
Interestingly, I don’t really see that malware that we launched a few minutes ago. Does that mean it’s not running? No. No it doesn’t mean it’s not running.
Can task manager see everything that’s running on the system? No. Stuff can hide from task manager. It’s harder, to be honest. it’s harder to hide from task manager than it is from control panel programs and features, but applications can do it.
One particular category of application that can do it is actually called the root kit, and while I’m uploading this file, I’ll tell you what a root kit is. Then we’ll probably be just about at time.
I’m going to have virus total scan malware1.exe. It’s already scanned this. Interestingly, this is a really particularly nasty piece of malware. Why? Because out of the 35 malware‑scanning engines that virus total used this morning to take a look at this file, one of those malware scanning engines, just one, said, “This is probably malware, and I would block it.”
The other 34 either said nothing, or they were offline at that moment. They said, essentially, “This file is just fine. I would let it pass.” If your malware scanner isn’t on the list of stuff that found it, that could be a major, major, major problem.
You could do this with any file you like. You could do this all day long if you like. There’s a reasonable amount of limit here. Let’s see. Reanalyze, which will actually force that file to get reanalyzed by all the engines again.
I’ve seen this actually do some interesting stuff when I’ve seen live malware outbreaks. I’ll take a piece of very fresh malware, upload it to virus total, and it’ll come back clean. No malware scanners found malware in this particular file.
Then a day later I’ll upload it again, and 30 of them find it. Then I’ll upload them a day later, and 50 of them will find it. Then it becomes an interesting question of when did the user, or Harry…When did this actually get…When did this piece of malware get launched?
So you can see already that I’ve got more malware scanners reporting that this is malware than I did a couple of hours ago when I was preparing for this session. I’ve got…I had one hit earlier. I’ve got at least four now that are reporting back.
What if I used bit defender? Yep, bit defender says, “That’s not a piece of malware.” What if I use Microsoft software? Microsoft software says, “Yep. that’s clean no problem.” What if I use malware bites, says, “It’s clean.”
Kaspersky says, “Dangerous.” McAfee says, “Dangerous.” These others say, “Dangerous.” The rest of them say, “Clean.” So how do we know if it’s malware or not? We have to be diligent. We have to look for the behavior that we think, or we have to look at users and determine based on their behavior, based on their patterns. What have they been downloading? What have they been launching?
Do they have permission to launch this kind of stuff, and does it look like malware packages have actually been downloaded? A lot of it is behavior. With Harry, that’s why Harry is so dangerous of a scenario. I don’t see any direct indications that anything bad is happening on Harry. I assure you it’s infected. It’s nasty infected at this point.
Even nastier because I don’t see it. Right? Nothing there. I’ve sorted by date, so nothing there from today except for Flash Player and Reader.
Does a clean install clear everything in the kernel?
Does a clean install clear everything in the kernel? Yes, absolutely it does. Is a clean install guaranteed to wipe out all malware? No, it’s fairly sure to, but there are a thing called bootkits. Rootkits are pieces of software that hide from malware scanners. Bootkits are ones that persist over reboots, and sometimes can persist across operating system reinstalls.
The only way to be absolutely sure is to take the hard drive and a pick axe, or fire, or a revolver, or an automatic hand gun and destroy it. That’s the only way to be absolutely sure. As I’ve said many, many times in the past the only way to be sure is nuke the site from orbit.
Another question, is memory forensics used as a tool in malware investigation?
Mike: Memory forensics is sometimes used in investigations. However, it’s generally only used by law enforcement because it’s really touchy and really hard to do. In a corporate environment and most user environments, memory forensics is a little beyond the scope of what we’re going to do.
File and folder forensics as well as running applications are usually enough to determine is there a malware outbreak, is there some kind of infection? Memory forensics usually reserved for law enforcement, for embezzlement, for fraud, for murder investigations, actual evidence gathering kind of stuff. Next question.
What is ransomware?
Ransomware is software that installs ‑‑ it’s another form of malware ‑‑ and typically is very, very quiet initially.
What it does is it actually holds the computer to ransom or the system to ransom, meaning it installs itself. It does some configuration, changes, or modifications to files, to systems, to data, and then it will prompt the user for, “Hey, I see that you like data. I have destroyed all of your data and have a backup on a server in Russia. Would you like it back? If so, pay me some bitcoin. Pay me some money.”
That’s ransomware. It’s another form of malware. It’s particularly vicious because of how it behaves. It doesn’t look like malware a lot of times. It looks like just another piece of encryption, decryption software, or another useless type of scanner registry piece of software, but it doesn’t really look like malware to many malware scanners. It’s really, really, really horrible stuff. That’s what ransomware is.
Does this machine have ransomware? I don’t know yet. It might. It might very well. CryptoLocker was one of the more famous ones. It was theoretically shut down about a year ago, little less than a year ago by law enforcement, but shut down is a really, really, really subjective term. CryptoLocker version two is nearly impossible to shut down.
CryptoLocker version two fixed a lot of problems that CryptoLocker one had by doing things like using peer‑to‑peer networking instead of central server networking, using a distributed system for command and control, instead of a centralized system, using better cryptography.
CryptoLocker version two actually uses elective curve cryptography. It makes it much faster to encrypt files with a key that the remote server has but you don’t have. So, ransomware is actually really, really nasty. You prevent it though, the same way you’d prevent any piece of malware. You have to prevent it from getting in, because once it’s in, you don’t know what it’s infected. You just don’t. There’s almost no way to know.
As we’re wrapping up, I know we’re a couple of minutes over time and I don’t want to keep you too long. I want to be respectful of your time. Harry is probably not the best one to look at live network forensics on.
Alt, F4, we will get out of it that way. Harry I’m taking offline, but I want to bring up Tom, mostly because I like the picture of the kitty. I am going to fire up Wireshark on Tom and because I’m sniffing locally, I’m not go into promiscuous mode sniffing, I can actually sniff the wireless adapter. The one that I mentioned is actually connected to the Internet and we can see whether it’s phoning home or not, which is another technique.
We just don’t have time for all the techniques unfortunately in one session, but we can look at which of these applications are actually talking to the Internet and which are not. It looks like it’s pretty quiet actually.
It’s just doing a little bit of SSDP traffic, nothing terribly exciting. However, if I start kicking off Comodo, or 24 by 7 search, or 24 by 7 support, all of that kind of stuff, I may actually use that to figure out what it’s talking to.
Look, there is an awful lot of interesting traffic going there. Stop that for just a moment. I’ve got some filtering and ICMP traffic, but that doesn’t really stop this from getting out. We probably also have on the firewall side, based on what I’m looking at here, we’ve probably got our administrator, our network administrator blocking known malicious sites.
He’s probably using one of the firewall contraptions that actually has a black list, white list configuration. I know these sites are malware so don’t allow them to connect. That’s great.
These pieces of malware typically just go find another server. This thing will just be trying, and trying, and trying, and trying and trying as long as it needs to try, in order to find a server that it can connect to, to download some more malware.
How do we use this as our forensic investigation? Typically, what I can do here is actually use this list to figure out what we’re communicating with and what we expect to communicate with.
Also, as I’m killing apps, if I don’t want to try to reinstall the OS for immediation, if I want to try to kill all the different apps that are causing havoc, what I can do is actually look at the network traffic before and after I kill those processes, or remove that software to determine is that traffic stopping?
But, you have to be careful. Malware authors know that we look at this stuff. What they’ll do is actually go offline for a day, or two, or three. They won’t phone home for a while and then they’ll wake right back up.
Is this an absolute assurance that that software is not still running somewhere in the background? No. Whether it won’t get launched by another app and fire up again? No. No assurance at all.
The only way to be sure you don’t have any malware on a system any more is to nuke the site from orbit. You’ve got to decimate it and start again. It’s really the only way to be absolutely sure. It’s so much easier to prevent it. Preventing it is almost simple compared to getting rid of it. That’s all I got.
Can malware block an install of anti‑malware software?
Yes, absolutely. It comes back to, where is the malware running, and if it’s able to block processes from launching? Yes. It absolutely can. Not a problem at all.
Any thoughts on EMET.
Mike: EMET, it’s another piece of malware protection, that plus minus, plus minus. If you like it, that’s great. Keep it up to date. If you don’t like it, get rid of it. Get something that you do like and keep it up to date.
It’s fine. I don’t have any particular nasty opinion. The only one I don’t really like is ClamAV. ClamAV kind of stinks on ice, it’s really, really, really horrible, but it’s not really meant to be great anyway. It’s a bottom tier malware scanner. It’s the only one I’d never recommend, but I’ve never seen a corporation use it widespread. It’s usually used for real basic point defense.
What classes do we get into remediation?
In the CompTIA Security + class, we actually spend a good amount of time talking about ripping malware out, scanning systems, removing it, that kind of thing. In the Ethical Hacking class, we actually dissect how malware gets on a system. With the intent of, now you understand how malware gets on a system, how it functions in more depth than this, so you can figure out how to remediate it.
We don’t directly remove malware. We just talk about how to install it, because it’s a hacking class. CompTIA Security + class, is probably the best course for actually learning about remediation and practicing it.
Can Combo fix malware or any malware? He’s asking about different, I don’t know if this one’s Combo, ComboFix?
I’ve heard of Zongo com, but I have never heard of ComboFix. I don’t know anything about that one. Sorry.
Can you comment about air‑gap malware?
Air‑gaping, interesting concept. If I walked over to Harry over there, and unplugged the network connection and unplugged the wireless nick, there would be no logical connection to any other system.
That’s air‑gap system, meaning keyboard, mouse, video, local to the machine, and no way for that machine to communicate with anything else. In theory that machine, now that it’s not logically connected, no data connection, now that it is air‑gaped is invulnerable to malware.
That’s theoretical. However, is that machine truly air‑gaped? No, because watch. Watch what I’m going to do. Matt, I’m going to switch cameras for a second. I’m going to stroll over here with my handy dandy USB stick and I’m going to come right over here, and look, it’s an air‑gaped machine.
I’m scared, air‑gap machine, I can’t possibly…Yeah, there you go. Now it’s not air‑gaped any more, is it? Now I’ve got a USB stick throwing data on it, no longer air‑gaped enough to protect against malware. The malware has to be crafted to support this kind of transfer.
Most malware rely on Internet connectivity, network connectivity, some type of logical connectivity. However, air‑gap malware can actually can just be transported on USB stick or, God forbid, floppy disk or optical, DVD, that kind of thing, whatever.
The next zip‑drive, if you’re really old, or jazz drive. Remember those things? They were great for storage. Air‑gap machines are not invulnerable. It’s just an additional layer of complexity that an attacker has to overcome.
Are they better defended than the same system with a network connection? Generally, yes. Are they invulnerable? Generally, no. Other questions, Jeff?
How about Bluetooth? Can Bluetooth be used to infect?
Can Bluetooth be used to infect a computer or phone? Absolutely. Bluetooth is just another data transfer. In fact Bluetooth 4.0 uses the same frequencies as WiFi, so you have one less radio that you need to worry about.
Yes, Bluetooth, data transfer. Bluetooth transfers files. Bluetooth transfers information. Absolutely, it’s actually pretty easy.
Do you use any tools, Linux tools to identify malware?
Do I use any Linux tools to identify malware? I don’t need to. I can. I can make it really look fancy. I could fire up Collie Linux and use some forensic software there. I don’t need to. I can just, generally what I just did there, on that other camera, and I’m going to just switch back for convenience sake.
What I generally did over here 99.9 percent of the time I can find the malware just by doing this, so why would I need to fire up Collie, other than to look really cool for my boss and get paid more money. Crap, I said that in front of him. Sorry.
I meant Collie is an important tool that requires many years of extensive experience. No, you don’t really need it. It’s nice to have. It’s interesting to have as a chunk of knowledge, but most of the time the investigation doesn’t need to go that far.
What I would recommend that we use, and we can talk about remediation in more depth in another session, but I actually do want to show, I know we’re way over. Do I have like one more minute? Then were done? OK.
One more minute, I will show you what I generally keep on my USB stick. Generally, I have a USB stick for looking at all of this, or for my tools that I need to keep around handy. At a minimal level I gather a bunch of tools over time, but I like the tools that I’ve got up on the board, as my typical set of go‑to tools.
I’ve already used Wireshark and VirusTotal. For remediation I keep Spybot, Revo Uninstaller, Malwarebytes. I keep these three handy. These are all three remediation tools trying to remove software.
Again, these aren’t the only ones. They are not necessarily even the best ones. They’re the ones that I prefer, because I’m comfortable with them and I use them often enough to be very familiar with them.
You might use other stuff, totally cool and AccessData for forensic tool kit imager and autopsy. AccessData, I’ll move the mouse, if I know that a system is really infected to the point where I actually want to take a deeper look or I’m worried about espionage or some type of illegal activity.
The forensic tool kit imager is a free tool that I’ll download. All of these are free actually. It’s a free tool I’ll download to snap an image of a hard drive and save it, pristine, perfect image of a hard drive, like bit by bit, sector by sector.
Then I’ll actually feed that data into Autopsy, which is an open source forensic investigation tool that will actually ingest that disk image, process it, look for signs of malware and actually let me do things like walk through the file system, look at slack data, look at all of that kind of stuff to figure out is there something bad going on or not?
Mike Danseglio teaches IT Security classes at Interface Technical Training. His security work has included protecting militaries, government agencies, and private industry around the world. He has developed and taught extensive security training on topics including cryptography, security technology, and attacks and countermeasures. Mike is a Certified Information Systems Security Professional (CISSP), Certified Ethical Hacker (CEH) and Langevin Master Trainer.
Mike Danseglio Teaches
AZ-500 - AZ-500: Microsoft Azure Security Technologies
WIN740 - WIN740: Advanced Windows Troubleshooting and Support
WIN600 - WIN600: Introduction to Windows Troubleshooting and Support
BITLOCK - BITLOCK: Planning and Deploying BitLocker Drive Encryption Training
CISSP - CISSP: Certified Information Systems Security Professional
CCSP - CCSP: Certified Cloud Security Professional
WIN600 - WIN600: Introduction to Windows Troubleshooting and Support
BITLOCK - BITLOCK: Planning and Deploying BitLocker Drive Encryption Training
AZ-500 - AZ-500: Microsoft Azure Security Technologies
WIN740 - WIN740: Advanced Windows Troubleshooting and Support
CEHv12 - CEHv12: Certified Ethical Hacker (CEH) v12
CISSP - CISSP: Certified Information Systems Security Professional