January 29, 2020
  • 6:52 pm Trump’s Press Conference MELTDOWN
  • 6:52 pm Celine Dion – French Press Conference w/ English Subtitles (Montreal, July 31st 2016)
  • 6:52 pm Lyndon LaRouche Webcast, April 11th, 2014
  • 6:52 pm KAMC HD Weather Webcast November 22nd, 2019
  • 6:52 pm Record A Call With Wildix
SANS DFIR Webcast – Detecting Evil on Windows Systems – An In Depth Look at the DFIR Poster

– [Benjamin] Hello everyone and welcome to today’s
Sans DFIR webcast, detecting evil on
Windows systems, an in depth look
at the DFIR poster. My name is Benjamin White,
with the Sans Institute and I’ll be moderating
this webcast. Today’s featured speakers are
Rob Lee, Sans faculty fellow, and Mike Pilkington,
Senior Security Consultant. If you have any questions
throughout the presentation, please enter them into
the questions window, located on the Go To
Webinar interface. We’ll be answering them
during the Q&A session at the end of the webcast. And with that, I’d like to hand the
presentation over to Rob Lee. – [Rob] Good morning
everyone, my name’s Rob Lee, I’m the Sans curriculum lead for digital forensics
and incident response. My co presenter today
is Mike Pilkington and he’s gonna be
joining us very shortly and we’re both gonna be
presenting the Sans brand new digital forensics and
incident response poster that we titled the finding evil
poster that we have at Sans. Really happy today,
this is turning into a really nice spring day
here in the East area for a couple reasons,
for those who know me, tonight I’m gonna go
see Captain America, the new movie that is
coming out with my wife so I’m really excited about that and then tomorrow is opening day for the Washington Nationals and we’re taking the kids
out there to see that so it’s definitely– Spring is kicking
into high gear. And at the end of the weekend I’m traveling down
to Sans Orlando to begin teaching our
advanced response course down in the magic kingdom. So really really a lot of really fun things
happening for Sans and for me personally
over the next couple days so I’m really excited about it. To kick things off I
wanted to touch upon why the poster like
this is important because this is
our second poster and we actually just
got notification that we’re in the process
of creating a third poster which is gonna come out
later this year as well but these posters, we’ve gotten
a lot of great feedback on. Finding that they’re very useful
across many of our courses. I’m not gonna go through
our specific curriculum but it kinda gives
you an outline of all of the
courses that we have in the Sans Institute line up but I wanna do a
special shout out for some of our newer courses that are joining
our ranks in 2014. Specifically in starting
off with forensics 526 which is our brand new
memory analysis class. A lot of cool things
going on with this class, we have Jake Williams
and Alyssa Torez who are heading up
the authorship of
this course right now and are doing an excellent
job of creating a class that is aimed at the
practitioners who
need to be able to use memory forensics
in an environment, whether they’re looking for
standard criminal type evidence or they’re trying to use it for advanced intrusion
investigations. Also on a special
note to forensics 572, our new advanced
network forensics class. This class is sold out,
not only at Sans Orlando, it just sold out in Sans
Europe, Security Europe as well and this class is
headed up by Phil Higgin who is an exceptional former United States Air
Force trigger puller. Did trigger pull in a sense for
cyber search trigger pullers that are out there and he does
a significant amount of work in making this
class truly awesome. His co author is
Steve Armstrong, Royal
Air Force in the UK, this class is extremely popular so if you’re interested in this I would definitely take a look into signing up later this year. And then we have
smartphone forensics, brand new again this
year forensics 585 headed up by our all
female author team Heather McCollick, Cindy
Murphy, and Lee Cromwell. This class again started
off on a very strong footing and it’s gonna be showing
up at Security West next, that’s gonna be in early May and of course our
new debut class which is in data
that over the summer which’ll be debuting in the fall is our new Mac forensics class
headed up by Sierra Edwards. Our San Jose class by the
way from the second beta is not sold out
yet but it’s close. You can take the
class at half off if you sign up for
one of the betas because as long as you’re okay
with some of the scaffolding and being able to help
out improve the class before we do the
official launch, the betas are a good way to get a pretty good term
before they head out. So we’ve gone from four
classes over the past year into a total of nine
classes this year so I certainly hope there’s
a lot of new things out there to potentially take on Sans. Also down on the left hand side is a lot of free utilities and free information
we have at Sans. So before we begin
officially into how do you– About the poster in depth, I wanna talk quickly about
how do you get the poster? You can get it
online in a PDF form which makes it
really easy for you to potentially see the
inner details of it if you wanna put
up on your screen. But if you want something to
post on your wall in your labs, in your security operation
centers, or other locations, maybe in your bedroom,
depending on how you roll, we have the link here, the
dfir.to/Get-Find-Evil-Poster. It is cap sensitive so
you can put it in there. There’s a sign up form online
that you’ll be able to go out and request a poster and
Sans will mail one to you over the next couple weeks. So if you want another copy
or a couple extra copies, have yourself and someone
else in your organization sign up and you’ll receive an actual paper copy
of the actual poster. So a little bit
background on this poster, this actually is not entirely, I’m not going to
take 100% credit for the idea behind this poster because actually
the idea from it stemmed from our
students in our courses. One of the biggest requests
that we’ve had over the years when we go through
incident response, we’d always discuss that
hard earned knowledge that a lot of incident
responders have specifically, how do we know the FCC codes and how do we know that
execution of certain programs on a system might end
up being a trigger for a potential compromise? So the biggest cheat sheet that we’ve ever been requested
was we need a process list, we need a way to
know what is good and what is potentially bad, specifically on Windows systems and an enterprise environment. So we’ve kinda toyed around
this idea for a little bit, we actually had long
discussions with Jason Fossen, he’s the author of the
spearing Windows track. He actually has a talk
called process hacker that kinda steps an individual
trough the Windows processes. But even with that talk people still wanted a
more in depth cheat sheet. So went to the individuals on
our authorship for the courses and found that Mike’s
background and knowledge was perfect in order
to get him rolling on creating a poster like
this and just as a side note, while we were literally in our
first draft of this poster, I wanna give a shout
out to Patrick Olson who also released a blog
article in a very similar way of what is the Windows services,
what is good, what is bad? And it’s funny when he
released his blog article I sent him a draft
of our poster saying hey great minds think alike. And Patrick kinda
laughed and says we were kind of moving
toward the same direction and he actually helped
contribute to this poster and helped us make it better by commenting on what he would
add, what he would take away and he helped edit the
overall poster to begin with. So what we’re gonna do here
in this presentation is kinda go through both
sides of this poster of the know normal,
how do you find evil, by knowing what is normal. And then the opposite
side, the knowing abnormal, how do you use
artifacts on a system it’s potentially saying
this is not right, this should not be there,
and it might be a trigger for a much more
interesting situation and it could be an indicator that it is a
definitive compromise, so I’m gonna pass it off
to my co presenter Mike, he is gonna kick
this off with the, let me see, went too far,
Know normal find evil side. So Mike, all yours. – [Mike] Awesome, thanks Rob. Well I wanna go through what
we consider side one here which is actually, if
you downloaded the PDF, you’ll notice it’s the
second page of the PDF but this is really the core, what we focused on
initially and like Rob said, we wanted to provide
something to the community but specifically to
our students as well who had been asking for this, it’s the ability to
see what’s normal and in particular this
is on a process level but see what’s normal
on a Windows system so that the abnormal
can bubble up. So this is what we came up with, what we’re showing
on the slide there, like I said this is our
main page of the poster, is the process hacker tool and this is an open source
version of process explorer from Sysinternals. But it’s actually running a
default installation windows 7 and all we did was fire up
windows 7 Enterprise and log on and start Internet Explorer, so we actually do see Internet
Explorer at the bottom and we’ve got one little
box on Internet Explorer and how it behaves
and things like that. The overall idea is to point out the key characteristics
of these processes that are typically running
on the Windows system. So things like what is
the proper legitimate, what is the proper image path
for legitimate processes? So where’s the process
executable found? What are the normal parent
processes for these processes? And relative start time. So most of these
that we’re showing are gonna start at boot
or very quickly after boot and so if you start
to see a process that looks like it’s legitimate but it’s got an unusual
process start time or it’s starts well after
boot, it might raise a flag. Also what are the user accounts that are typical
for these processes? So these are the things
that we wanted to highlight and at a large level that’s
what we’re showing here. As we have on the bottom
left here of the screen, let me make sure my
mouse is working. Wanna go ahead and
mention again as Rob said, Jason Fossen’s talk
on process hacker actually I think the title is exploratory surgery
with process hacker. It’s a wonderful presentation, if you get a chance to see him you gotta definitely
go check him out ’cause he’s got a lot of energy and is pretty funny in
presenting this information, he does a great job presenting which is sometimes
hard to present this low level information. But he’s also written
it up really well, he’s got it very well documented so definitely check out
this link and his PDF. Actually I’m gonna go back
one and just show you, at the top there’s
where this blue, and what we’re gonna be showing is mostly screenshots
from the poster and just kinda diving
into each one of them. But at the top is this blue box. What we wanted to do is give you a checklist of
things to look through as you’re trying to
spot the anomalies. What we’re looking for,
and when you think about what malware is doing,
malware wants to hide, it doesn’t want to be found, and so there’s
typically two ways that it’s gonna try to hide. It’s either gonna just try
to blend in and look normal, so just kinda hide
in plain sight, or it’s gonna try to do
more stealthy techniques and not even be found within
a normal process listing or things like that. So that’s more into the code injection and
the root kit technology which we’ll talk about
in a later slide, we’ve got a little bit
of information on it on the other page. But when we’re
looking for things that are hiding in plain sight that are just
trying to blend in, these are the types
of characteristics
we wanna look for to help us bubble out the
things that are not normal. So we wanna understand
what the proper, for example the proper
parent processes for these normal
Windows processes. And that way we
can look for ones that are not in a
proper parent process. We wanna look for, I’ll just
kinda go through the list here, look for processes that are
in the wrong path for example. Like I said, the wrong parent
process, that are misspelled, that’s very common, we see that a lot with VC
host and LSASS and others. Maybe running under
the wrong user account, that’s definitely
something to keep in mind. Unusual start times, unusual
command line arguments, and then packed
executables as well, this is one that bubbles up particularly when we’re
looking at a live system and we can use something
like process explorer, really highlights
packed processes for us. This is kinda the
checklist we go through when we’re trying to
find the anomalies and if we know what’s normal we can easily spot these
potentially malicious processes. Alright so what we’re gonna
do is dig into a few of these, I’m gonna start
out with SvcHost, this is undoubtedly
malware’s favorite target. One of the reasons is because
there’s just so many of them. Again, what we’re
showing here on the left is a typical default
installation of windows 7, and when you fire up a default
installation of Windows 7, there’s 10 instances of SvcHost, I always have to catch myself ’cause I end up saying
it the wrong way. SvcHost, service host. Now you may ask, why are
there so many of these? Basically it boils
down to a decision, kind of a balancing
act at Microsoft is making between
performance and stability versus system resource usage. So the idea is on a default
installation of Windows, there’s almost 150 services. Now most of those are disabled, only about 50 of
them are running but still that’s 50 services
that are running by default. Many of these actually have
their code coded as a DLL and that DLL is running within
one of these service hosts. And the idea is Microsoft, they could have each one of
the processes be its own… (static) …Sorry, each one of the
services be its own process, that’s completely possible. But there’s a certain
amount of overhead that goes with each process so basically what they do
is they try to group them up and find the right balance between the right
privilege levels and the right number of
services per service host. So essentially they’re trying
to help speed up the system, help minimize the resources and the drag on the
overall work station. So that’s why there’s
a bunch of them. Now of course malware
looks at this and says well what’s one more, if we just add another one
people aren’t gonna notice. So typically what
malware’s gonna do is it’s either going to
misspell it slightly which I keep doing by accident and they’ll misspell it to
S-C-V host instead of SvcHost. Or they may spell it SvcHost but they put it in
different paths, so it couldn’t
reside in system 32 where the legitimate one is ’cause they can’t
have the same name but it might be found in the
Windows directory, for example. So that’s where you’ll
often find this. But then also just keep in mind, I’ll talk about this
a little bit later too that malware is going
to legitimately use
services as well specifically for
persistence mechanisms and it might be that
legitimate malware is found within one of these real
instances of SvcHost. So we’re actually
gonna break down a service here in a little bit
and see what that looks like. Alright so let’s
look at another one, and this one’s really
interesting, again it’s CSRSS, this is the client
server runtime subsystem. It’s another one that’s
commonly targeted by malware because it’s found on
all instances of Windows so admins and other people,
techies see it and recognize it. It’s another one where they might simply just
slightly misspell it by adding an S or moving an S or something along those lines. There’s typically two
instances of CSRSS. The first is for session
zero where the services run. And the second is for the
interactive console session which is session one. Now there can be
multiple sessions, so if someone’s logged
in with remote desktop, then that can
create a new session and that would
create a new CSRSS but typically most work stations
you’re gonna find is two. A little bit of history too, so this is kind of
neat for two reasons. One is because something
that you wanna look for and spot anomalies that maybe
there’s an extra instance and it’s spelled wrong or
something along those lines. But from an advanced
malware perspective and advanced
attacker perspective, CSRSS is interesting because
it actually is responsible for drawing the
command prompt window. Going back historically it
actually had a lot more duties and was overall an
overarching process for managing the
windows on the desktop. Much of that’s been deprecated and now actually with
Windows XP and 2003, it really was just responsible
for the command window. And with Windows 7 that’s
even been further deprecated and now Conhost which is
a child process of CSRSS, Conhost is now the
one that’s responsible for drawing that command window. But what’s really
cool about it is when you’ve got an attacker that’s interactively
on that system and running commands on that
system within a command shell, those commands have to be
stored in this process. Again, depending on
which version it is, if it’s Windows 7 or 2008 or
2 then it’s gonna be Conhost. Older versions are
gonna be CSRSS. It turns out that we can analyze the
memory of these processes and see the commands
that some would run within the command shell. What we’ve got here is just
a little example of this. Now typically what
you’re gonna do is use memory forensics to kind
of dig out this information. You could either, there’s a
nice plug in from Volatility. Even just old
fashioned string search can be really useful in this
case against memory image. But what I’m doing is
showing a live instance and a quick demonstration
here of how this works. This is using the
Sysinternals tool VMMap which will allow you to basically view the
memory of a running process so the top left here I’ve
attached to the Conhost process and I’m specifically
looking at the heap for this instance of Conhost. And you have to dig around, there’s a lot of information
here, it’s kind of organized, well the organizations not
easy to find, but at any rate, the data’s in here, the data’s
in here for the commands that were typed into the console for this particular
running process. I just wanna point
out a few things here. Right kind of in the middle here we’ve got the administrator
command prompt which is actually the
title for this CMD window. And then also you see multiple, what I’m showing in the command
window is the doskey/history so these are the
commands, the last, I think it does 30 commands
but I’m not positive. At any rate this shows
a number of commands that had run in
this command window. And a few of them are
popping up within the heap, within this small section of
the heap that I’m looking at so ipconfig/all and
ipconfig is showing up. Also this SC command which I’ll talk about
a little bit later is a great command for querying the configuration
of running services as well as services
that are stopped but at any rate what I did
here is I ran sc queryex which is like an extended
query on the wsearch service. And what’s really
cool about this one is we’re seeing that okay yeah this is what I ran
in the command window and you can see it
up in the top right. But also you see the return data so this is what this command
actually returned back to me. So again this is
really a neat feature that’s an artifact that’s usable and if you get to a
machine quickly enough and you’re getting
a memory image you got a great chance
of pulling out this data. – [Rob] Hey Mike, can I
add something to that? – [Mike] Oh sure. – [Rob] Let me go back
one slide, thanks. – [Mike] There we go. – [Rob] So on this piece
of information here I really wanna point out, and this is Rob Lee
speaking right now, this is a very
interesting artifact and over the years
I’ve done case work. We found a ton of useful
information in here and I wanted was, this is
where they talk about– In regards to finding
strings in these processes there is a great tool
within Volatility that allows you to
do the extraction of a lot of these elements. One of the things
that I found is that that tool often misses
many of them too and I think that if you’re
doing a memory analysis and you’re doing
extraction of this data or dumping it from
the process itself you have to be really careful that sometimes it over
relies on the tools and I actually teach this in
508 and I know they go into it in our memory forensics
class too even more so that if you don’t actually do the string’s
extraction manually, there’s still a
tendency of pretty to potentially miss
additional keystrokes that the attacker came in
and typed on the system. We actually have several
demos of that in the classes and I’m really not trying to say hey Volatility’s a
poor tool or anything but it’s like with any tool,
an over reliance on that as the only thing
that you’ll check for, you end up missing some
of those key artifacts. For example, in one of the
previous courses I just taught I was teaching someone who’s actually extremely
experienced that actually used to work at Teramark
with the Volatility team and one of the
things that he found, and he’s very
known in Volatility and I said well notice you
have to do strings against it you actually get more data out and especially when
you start seeing that he went back to one of his
cases in class and went back and literally did a
fist punch in the air when he saw key data for that
case that he’d been missing simply because he wasn’t
doing strings against it and mainly just using the
over reliance on the tools so I want to make sure
that this one specifically people actually still go down
to the old school technique. I know it seems old school but it really is gonna pay off for you to do an
additional check to make sure there’s
no additional strings that you might be missing
or command line input you might be missing
from these two processes. I’ll hand it back to
Mike at this point. – [Mike] Okay thanks
Rob great great point. Alright so we’re gonna look
at a couple more processes. This next one is LSASS, another
commonly targeted command, it’s not spelled LSSAS or LSASSS but you’ll see this often and as you do more
malware analysis. The thing about the
LSASS process is it’s extremely important
to the Windows system. Among its many duties, one
of the primary ones is that it handles account
authentication. It also handles password
policy enforcement and even writing
to the event log. Now because it’s very
common to all Windows hosts, malware does often target it. One of the more interesting
pieces of malware that’s targeted LSASS in the
past was the Stuxnet worm. A true nation state
piece of malware. And it actually did
something interesting, it used a technique
called process hollowing which we’ll talk about
a little bit more here shortly when we
talk about code injection but basically it
actually starts up a legitimate instance of LSASS, it takes out the
legitimate code, and shoves in its malicious code and then continues to run LSASS and it tricks the
Windows system. But the one thing
that really stood out with this particular
piece of malware is that it actually invoked
multiple instances. One of the main markers here
that you really wanna focus on is that there should
always only be one running instance of LSASS. So Rob I think you were
gonna talk some to Explorer? – [Rob] Yeah explorer.exe
is one of the processes that a lot of people assume
that it’s a Windows system 32 but in fact it’s not. Explorer.exe is one
of the processes that has the deep
history of being one of the first
exploited processes for the DLL search
order hijacking and basically what
that does is that an executor will process
that loads DLLs initially will look in the present
working directory, in this case in the
Windows directory, where explorer would be found before it switches over
to Windows system 32. So bad guys exploit
processes like explorer by putting rogue DLLs
into the same directory that would load them before
the official DLL is found in Windows system 32. So this is one of those that
explorer basically runs desktop and obviously is the main
gooey interface for the users but it’s exploited
because of the fact that in some cases it’s
not exactly found in the standard Windows
system 32 directory and as a result it pairs
access control lists or specific monitoring of
the system 32 environment by a lot of external processes and byte listing and so forth. They might miss that someone
could still get around that by doing the DLL hijacking, dropping the DLLs
into the Windows pack and then when the individual
user logs on the system, the torturized DLLs would
then be loaded into explorer. So it’s really a crafty trick
and our team at Mandiant when I was working there
initially found this, they spent I think
it was about 24 hours trying to figure out how the
process was being loaded. They knew that the rogue
process was being loaded they just couldn’t figure out how it was achieving
its persistence but explorer.exe is one of
those that because it’s not in the core Windows system
32 directory location was initially targeted
by our bad guys for this. There’s other examples
since that point but it has that notoriety
that’s associated with it. Mike do you have any
additional comments on this before I switch over
to the abnormal side? – [Mike] No I’ll
let you move on. – [Rob] Sweet alright. So on the next phase here
we’re gonna switch over to talking about some
of the artifacts that we consider
that are odd to find on a standard Windows
system in an enterprise. Now I need to do a caveat here. One of the assumptions
that we’re gonna make, a broad assumption, is that these are on
standard work stations. Standard users,
not high end users, not technical users, not
power users environments. So you’re thinking
that your standard, this is 90% of, the majority of the
systems in your enterprise are gonna be your standard users who have probably never
dropped to a command prompt. If you said anything
about the Windows registry they would look at you blankly. They’re primarily using web, office documents,
email, and more. And so unless they
are doing coding, unless they are a power user, typically they’re not gonna
be utilizing the system the way you and I
probably are gonna utilize a standard Windows machine. Now who would this entail? Obviously HR, potentially
anyone in finance, anyone on the executive side are typically probably
not doing that. We do have, a lot of these users are highly targeted
by our adversaries. Obviously our system
administrators are also targeted by
their adversaries. It would be hard to differentiate a system admin
from adversary type impact but if you start looking
at your standard systems, a lot of these artifacts are
gonna be found as anomalies because of the fact
that these normal people just will not create
with similar artifacts that an adversary would when
you start taking a look at it. So let’s dive into
a couple of these and we start with the
unusual OS artifacts. This was often requested list while we were going through
the advance response courses at Sans 508 and 526 that we would notice that
a lot of the artifacts are generally created when you have a power user
or sysadmin on a system but a lot of those
systems are not managed with the administrator actually
going out and touching them. Usually administrators do some group policy to do
updating of these machines and will not be equivalent
to using a scalpel or going out and manually
upgrading the system or touching it remotely. When you start taking a look
at some of these artifacts, initially it’s kind of striking
when you end up doing so because you’re saying
what is the likelihood that a user in HR is going
to drop to a command prompt and start doing things? So for example if you
see cmd.exe execution, on most systems that are
non system administrator, this is an extremely
rare occurrence. You would see adversaries
drop to it all the time because they’re
using their shells, they’re using their
backdoor, the malware, or even they’re trying to install processes or
services on the system but again the typical
execution of this, it would cause my eyebrow, the Spock eyebrow to
go up just a little bit if it was a standard user in
your enterprise environment. How do you prove
execution though? If you go back to our first
poster, the evidence of chart, you would actually be able to show evidence of
command line execution, for example looking
for the prefix files associated with the user
assist keys, the shim cache, jumpless, and others
that’ll be able to show that this process was
actually executed. Another one that is
really interesting is the you just don’t see
the standard users dropping to a command prompt and then running the
next command for example which taps us to
remote system share. Now the net command
in and of itself is an easy way for an adversary to connect to your remote system and be able to access
their remote share. Again, that would be in a
log on type three event, a network based log on
which again normal users simply would only be doing
if they’re using explorer to attach to someone
else’s machine. So you could actually
differentiate from an adversarial type attachment to a remote system
in your environment versus the standard user by seeing how they
actually go about doing it. Typically explorer is not
gonna invoke net.exe execution, that’d be someone from
the command prompt. Now nearly following execution you would see them attach
to remote share of a system, they’ll upload their
malware to the other system and then execute it. How would they do that? Well again you would see
additional two artifacts on your at.exe execution. Now typically only a scheduled
tasks are gonna be executed but when you ever
see a scheduled task being executed from a non
system administrator machine to another non system
administrator machine, that would be something
that would again cause a Spock eyebrow
to go up a little bit and saying why would that
user drop to a command prompt, run at.exe from one
system to another? Again, enough of it is
rare in these environments you just typically don’t see it. So another way to do that
is that you execute psexact which leaves residue
in the registry in addition to the prefix
files, the shim cache and more and PS exact, just the fact
that it’s being executed in many cases will actually start a service on
the remote machine which also leaves
its own trail behind. So there’s a lot of
these little artifacts that we potentially see that when you take a step
back and you think about it from a system
administrator perspective that the standard users
are simply not gonna do and if you see these executions or the existence of these
programs on a system in any way, then that would
potentially be an indicator that something
weird is going on, and potentially warrants
further investigation. Now as I said before a lot of
this stuff is very gray area, it’s not black and white, you might have a legitimate
usage of some of these, however it is more rare
on your non power user non system administrator
type systems. Mike, wanna throw
something else in that? – [Mike] Yeah I actually
just wanted to kinda hit on one of the ones I thinks
really interesting too is this use of Sysinternals
tools, right in the middle, I think it’s the fourth bullet. There’s psexact,
Psloggedon and procdump are the three that
we pointed out. Psexact is not a surprise,
it’s been used for a long time, it’s used by admins
legitimately many times but it’s also
usually by attackers to remotely run commands and we’ve seen that and
talked about it quite a bit. Psloggedon if you
haven’t come across this, this is a way that attackers look for
systems in the environment that have interactive logons and they’re basically able
to scan the environment if they have the
right credentials to query the remote machine but they’re able to determine who’s interactively
logged onto that machine. As they get to know
your environment and get to know
who are the people and what are the accounts
that have a lot of privileges, it’s an easy way for
them to figure out I need to go after this machine and that user on this machine. Within in active log on,
the users credentials. Both the password and
the password hash. The actual password
and the password hash are stored in lss we
talked about earlier, and a great way to get
that from attackers stand point now,
is to use ProcDump which is another microsos
Sysinternals tool, signed by microsar, most AV systems and most adment, you can notice it. They can use ProcDump just
do a dump of the memory, per lss is got all
those credentials in it, then they pull it
back to their system and run the tool mimicats which will just present them
with the users password. SOLs password hash. So I think those are
really interesting coz basically is full garment
of how users can target an attack and gain credentials within an environment, and they are all
signed by microsar. They are all tools
that are used by admins and not often flagged by AB. I guess one other point I had and this is really
just a side note but there was an
interesting email thread that went through
the DFIL mailing list recently about this book rtfm, read at, sorry red team, I was thinking of
the other rtfm. Red team fied manual and it is basically… I don’t know how many pages, probably about 90 pages or so of just a mass of cheat
sheet from a Pinterest that’s been doing, been in the business
for 10 years or so. Is really good book,
I actually just got it in the mail three days ago and just spent the entire
night kinda combing through it. A lot of stuff like this,
commands you’re seeing here, talked about, it’s no thrill, it’s basically like say,
it’s kinda of a cheat sheet with just a one
line with a command and just a simple explanation, very simple explanation
what it does, but it’s nine bucks
on the Amazon, might be worth checking out. I liked it a lot and
gives you some good ideas some things that the
attackers are doing and things that you can look for from a forensinc standpoint. Alright so, let’s move
on and talk about, code injection and
Rootkit behavior. So in three minutes
or so I have, certain amount of
depth we can get into but we wanna to just
kinda highlight it here and discuss it in the sense that this is the other technique. Really the other of the
two primary techniques for hiding malware. The other one is code
injection and Rootkit behavior, and this is really
ways that it can, that malware can become stealth and hide behind the scenes. Now, basically between the two here, we’ve got code injection
which is relatively easy way to run this within
a legitimate process. So I talked about
one of the techniques which is process hollowing, that’s what (mumbles)
did with lsass, where he just ripped
up a legitimate code and put its code in
the process itself. The other one that is
probably more common is an easier really, is a dll injection. So let us just use, basically API functions that
are available within windows to inject a dll into
a running process, and you will see this
with lots and lots of different pieces of malware, because it’s simple, it’s
stable and it’s effective. So, I mean, why not use it. Now Rootkit is actually
very effective as well but they’re more
difficult to implement because now you’re getting
into the corner level of the Operating System. So you have to have
a malware author that’s familiar with
basically device driver programming. So that they can
create one that’s not only effective but stable, because if it’s not stable, it can easily
blue-screen its machine and not only be detected but you know, a blue screen
machine is probably not gonna help attacker
that much in most cases. So Rootkits are important and there is something
we need to be aware of, but they are not as common as
the code injection techniques. So really for both of these, this is where the
stealth comes in it’s not hiding in plain sight, it’s not that you can just look through a process
list and detect it, but most of our tools are
not gonna detect either of these methods. So this is really
where malware analysis, sorry, memory
analysis really shine and allows us to look beneath the covers and see
what’s actually going on at the Operating System level. Now there area couple of tools that we’ve mentioned here, Rootkit Revealer and GMER. So those are both tools
that basically do. Sort of what we do
in memory analysis, where they look at both
what the Operating System is telling us is
running that machine, as well as looking at
it at a lower level and basically doing a dif and if there is a difference
then it reports it, and often that’s what is
actually hiding from us. So those are two tools you
might run on a running machine, but really memory analysis
is where you wanna look. So, Rob I think you were gonna have
some thoughts on this card. – [Rob] Yeah, and just to, going back on slightly onto, finalize the discussion
on code injection. You just mentioned
memory analysis how important it is, how its now become a stable and any
answer response team. A lot of people when I
initially talked to them about code injection,
Rootkit behavior, they give that blank stare
that, “Oh I’m not a coder,” and granted that a
lot of people feel that you need to be
at the coding level or you need to be
like somehow coding plugins for the variety of
tools that are out there, and honestly you don’t. One of the things I’ve
always tried to aim at in the standard response classes that we offered since and I know that
(mumbles) would a lot and the brand new 526 class , (mumbles) on memory analysis it really aiming at
a practitioner level so that in practition who’s
doing a lot of investigations can fly into a system and you
will be able to identify it, where there is rootkit
behavior and code injection, and you don’t need to
be able to understand debugging or you know, maybe you’re an
expert at (mumbles) make that happen, but granted, that does help and I will tell you that you will be
a lot better off if you have that plug. For those who are
kinda like worried about, “Hey memory
analysis is a step beyond “my capabilities.” I would highly take or
recommend, take a look at some of recent
courses coming up. Specifically the
memory analysis one, just you see how
that will potentially pertain to you and
your own environment, really into the
practitioner side. Diving into the next slide, talking about network activity. This is another one
that a lot of people turn to when they top
network activity out, and they have this bewildered
look on their face, like, “How do I know what
I should be looking at, “how do I know
what’s legitimate, how do I know it’s
not legitimate?” And so I created this rule list as a way to potentially
help them out in identifying things that
would cause my eyeballs to attract to that process, because you can have a lot
of network connections. You’re saying, “Well they’re
all meaningless to me. “Maybe do I need to look up “on who is every IP
address that I see, “look out every socket?” And the answer to
that question is no, and part of the reason
why is truly interesting how the adversaries
are communicated out of your environment, because it is barely noticeable by the network operation center and screw the operation centers. Anything that is not
report any activity, which is required, tends to turn their
eyeballs to it. So what adversaries do, is that they tend to hide in
the large amount of traffic that is web-based that is
leaving your environment. So what are first things
that I tell individuals when you’re looking
at a whole system, something that draw my eyeballs is any process communicating
over port 80, 443 or 8080 or any other
traditional browser ports, that is not a browser. Now having said that there is obviously updates
and other processes that will do it, but you will be able
to quickly investigate, interrogate those processes and determine that
that’s normal activity, but you will also
notice immediately that why is explorer that I
see communicating over port 80. That would be something
that, should stop (mumbles) that’s not right. It should be Firefox,
it should be Chrome, it should be something
that should be a legitimate browser based type of connectionality
of your environment and when you see, especially a traditional
windows process like svchost communicating
over browser ports, that should stop you
in your tracks sensing, that’s wrong, that’s evil, we can see you now. Now the opposite side of this, is when you see
Internet Explorer or any other browsers,
not communicating over one of their traditional ports. This could be an indicator
that someone might overlook i.e Explorer process or
Firefox process and assume, “Well that’s a normal
process of the user,” and all of a sudden
it’s communicating and doing rtp connections
in your environment. (mumbles) now I know
something like that. There is no way that
I’m probably doing that maybe there is a
plugin out there that has rtp back under, something too at
Firefox and Chrome, but you, helps you to
investigate that fairly quickly. The other thing that you see is, take no part of the traditional connections in your environment. For example, if you’re
in an environment that offers remote
desktop to home users, more likely or not
coming in through a concentrator of some point. So majority of the
traffic originates, or 3389 is going to originate
from that single location. when you have Jo and HR system connect rtp to the CFO system, that should cause the
network operations center and screw the operation centers eyebrows to go up a little
bit since that would be weird. Why would an rtp
traffic originate from a system that
traditionally does not do rtp and turn to environment. Since admin systems and even the (mumbles) probably sets into these rules, they potentially need to go
do a fixing on the network, but even then, you would see that those systems
are traditionally clustered around
the sane location and are not like Jo
an HR communicating with the excecutive systems in the TFO system altogether. So this is just a
couple of the ones that I typically tell people
about, take a look at, and when you’re starting just using simple rules, you immediately see processes
that are not legitimate that are trying to communicate. That hide in normal web traffic, then simply you would
see them as suspicious, and you even dump
out your memory and look at residual
network connections and still see those identifiers. This is something that
would potentially stick out and say that is odd. I need to investigate
that a little bit more. – [Mike] Awesome. Alright, well let’s talk
about services once again. We hit on svcs
specifically earlier because it is so commonly
targeted by malware, just trying to blend in, but the fact is malware
often uses services for the persistent mechanism. Whether it’s named
svchost or not once it installs
itself as a service or it’s available upon due. So its one of two most common, the other very common
mechanism is to use on the run keys
within the registry, but in most investigations you wanna take a close
look at the services and see what’s there
and what’s running. So what we’ve done here is some ideas about how to
investigate that service or the list of services
that are on the system. On the dead box,
you’re going to have to look at the registry. So right there in the
middle is the registry or location, HQ alarm system, current control services, and all the services
and in fact drivers are actually the
configuration information within these keys. On a live system, you can
actually use the SD command which is really robust and has a lot of
different arguments you can use to get
more information so that the extended query
and the regular query. QC is the configuration
information that talks about the
paths and the executable and then the arguments
give the executable. Privileges and they
queue trigger info. So this trigger info
is kind of interesting because it turns out that while most services are
going to start at boot it doesn’t have to,
it actually can start, based on the trigger. And so let’s take a look at
an example of one of these. Now most services, most
default services and windows, just start at boot or
actually or even disabled. Very few of them actually
use a trigger to start but it is possible and then, and there were some that do. This is one example of service
that’s built into windows that starts upon a trigger. We just with that and look
towards the bottom here using specifically the
AC command trigger info on W32 time, this is the
Windows Time Service. It gets started once the
machine is joined to the domain. So we’re seeing over
here in the right, a domain joint event or trigger is what starts this service, and when the system
leaves the domain, if it’s taken off the domain,
it actually stops the service. So all that trigger information is within this trigger info key, it’s not nearly as nicely
spelled out for us here as what it does,
but it turns out there’s a lot of
different triggers that could start a
service or stop a service from device that’s being
connected or disconnected, firewall events,
group policy changes, network IP availability
is kind of common one and custom events as well. Mark Bargurd actually
did a pretty interesting blog article or
actually series of blogs over on the internet
storm center site and if you look at Mark
Bargurd in stealthy persistence to find this one and several quite a few others
he documented. But one idea he came up with is you can create these
custom triggers is what if you had a trigger to where a certain
wireless network became in range of the victim and the the malware
or the service, the most services
would start up. So kind of neat and not
something you’d probably have to worry about too much but just be aware
that it is there. But services in general is
definitely something you need to pay close attention to, and just kind of going over
a few of the highlights here toward the top,
within the registry if you’re looking at
a particular service, the main key here
is going to have quite a few values of interest. In this particular case,
it’s actually the executable that runs the code as svchost with a dash K so it’s in this
group called local service and that’s a very common
grouping for the first spc host. We have some of the
more common groups which is designated
with this minus k listed on the poster. And then within the
parameters here, you got the actually
service Dll that’s used. This is the real code that
runs this particular service. It’s a system, where
it’s system 32w32time. So, if you get a
suspicious service you’re not aware about
and not aware off and you know, I think
it might be malicious, you can check the
digital certificates or check the digital signatures to make sure it’s signed
and from a legitimate source and kind of continue with
your investigation from there. – [Rob] All right for
the evidence persistence and I’m not going to, there’s, this can can be at
it’s own discussion in itself but it
goes without saying that to be able to use survive, what we do is
incredibly important when you’re an advisory
attacking a system in an enterprise. So these are just
some of the areas that we typically see our adversaries try to take advantage of and work from the low end going
to auto-start registry keys. You know from the higher end, modifying local group policy to Microsoft Office
add-in bio slashing even and others. This is, it really does take a little bit more
knowledge out there to eventually identifying this. And in most cases, you
potentially need to go into more of an examination; how you determine that
something is bad or not, we have that on our first poster using the malware
funneling capabilities but this again, if you’re
going to survive reboots, you’re going to see some of
the traditional locations and our adversaries
are going to typically use the lowest capable
method that will succeed because in most environments, they’re going to be able
to remain persistence for months until
they’re finally detected and then they start to use more of advanced
capabilities later on because they like to
hold those in reserve but the schedule tasks or
service replacement or creation, glide back in and seem as
we talked about earlier modifying the sseek command
or using that sseek command to find another,
artifacts like that or things that we
potentially try to identify. A lot of tools out
there are using (mumbles) at k, all we gotten to you know Mac
loans, auto banner and others have allowed you to
go look for a lot of that evidence manually. You know even if you’re
traditional for indicator all lead to using
innovised capabilities, to able to pull these
data off the systems. So once you head back to Mike to close out and
open up the questions and among the questions we had
is how do you get the poster? Here is the link and this is the order in our line. One that I already saw one
of the questions out there ask specifically about if you’re overseas, how
do you receive the poster? Unfortunately, Sand
is only doing this for it’s United States, for United States’s mail. If you’re looking to
getting the poster and a physical copy of
the poster oversees, any other stands events in Asia or in Europe, are the rare where the
forensic class is running, we’re going to be sending
a ton of these there, so you could always pick that up at that location. Being able to send
out a hundred of these because it’s just the way
snail mail works at this point, it’s just not possible. Now if you do want to contact me and receive permission
from your printer to print it locally, you could do that as well. We don’t mind that, as long
as you’re not taking it and removing all of
our stuff from it and it becomes your
poster instead, but you know, we will allow you to print your own posters If you want professional
poster credit place, Wikipedia end user, we know people who have created in big advancements
of the poster that way and in some cases, the poster, it was goin to print the poster we might need to talk less and if you’ve just
cc them on the email we’ll send back
the most (mumbles) that kind of thing We’ll be able to do that. So I do apologies
that won’t be seeing this (mumbles) and give
you physical copy of it but from now this
is only, US only. – [Mike] Okay, well,
I think let’s see. I can take over, there we go. I mean that basically
wraps up the formal part of the presentation
with some Q and A hopefully here is a second. I just want to say
thanks for joining, also hopefully this
is helpful to you. I think it’s really
going to be helpful to our students like
Rob’s mentioned. They’ve been asking for it and I think it’s really
going to help clarify some of the exercises we do in our forensic 50A class, as well as actually 610 and of course, 526, which is the memory
and deaf class. They are all going to find
this poster really useful for digging through and in terms of processes and find the needle
in the hay stock. So I’ll leave it at that. I think Ben would
probably chime in here and see if we have
any questions. – [Ben] Thanks mate, yeah. We do have questions,
those up here. I guess I’ll just
jump right into them. One question we had was a couple of times there was, the explorer (mumbles) was
also see a science case, it’s called a (mumbles) Why does it change
to sentence case or what scenario does it change. This was particularly
seen in the case of (mumbles) – [Mike] Can you, I’m sorry, Actually just… I had to turn up
my volume there, can you repeat it one more time? – [Ben] Sure, one question. A couple of times
it was observed that explore.esc is also seen as a sentencecaseexplore.esc. Why does it change sentencecase or which in there
that does it change? This is staggeringly seen
in the case of (mumbles) – [Mike] I can tell
you and tell you two possibilities here
that I can think of, one is there’s
something wrong with it, so it could be, have this
process hollowing technique. That happened with lsass, so that it actually is a
legitimate instance of explore but yet the legitimate
code has been ripped up and the Mariposa
code was put it, so that’s a possibility. It could be slightly misnamed or in the wrong location,
it could be the right name but the path is in
the wrong location. The other possibility
is that there can be multiple
incidence of explorer.exe running a system
and that’s probably not what’s going on
here based on the fact that this is the
analysis of Miraposa but you can’t have
multiple incidences and that is when you have
multiple interactive logins. So if someone has performed
a fast user switching and then another interactive
login with a different account, I see you have two
in org at the logins or remote desktop is
the other common time where you’ll see two instances of explorer.exe. – [Ben] Okay. We have one more question here. It’s about unusual artifacts, hindering code team
browsers running net.exe in the background. Have you noticed this? – [Mike] I have not. – [Rob] (mumbles) would say no. – [Mike] That sounds
pretty interesting. I am wondering if when
you set the background, I am wondering if it’s a
child process of the browser. If so, that seems
really interesting and I haven’t seen that
and would not expect that. – [Ben] Okay, Mike. You referenced a book earlier, could you possibly give us
a title of that one again and maybe where we can find it? – [Mike] Yes, for sure. It’s called RTFM for Red Team Field Manual. It’s sold on Amazon
and it’s only $9, I think it was self-published, so it’s worth 9 bucks
for sure in my opinion and it’s a really neat, kind of handy massive chichi,
essentially is what it is. So RTFM, Red Team Field Manual. – [Rob] (mumbles) It’s, I just pick mine up for as cheap as it is, it’s a really nice
book and user manual. However, the only thing
I would recommend is that it’ll be really nice if
this was Matronic Quantum 2 because having this, you know, something so small on your iPad or Kindle or something like that to be able to quickly
browse through would be specially nice. I think they are
trying to figure out; figure if we use online or not but and the KPF version of it but it basically has, you know, everything that you
would find on MSTN and you know, SIFT
internal website and how potential
attackers might use it. For example, one of the
biggest ones I saw in there which was really nice was how do you use mimicats? Having a, or you’re running, mimicats and then
do the (mumbles) Do the inaxsi in the (mumbles) Exact Manti had to type in order to drop your passwords
in plain text It was only like
three commands there but I can’t tell you
the amount of times I always had look that up in order to remember, what
was that sequence again? And I had to go back
in there and do it because tools in French and there’s you know
unless you be French, it’s not something
that is going to be entirely partsy when
you’re using it. So RTFM is well
worth the 9 bucks that you get off of it. – [Ben] I also have
the amount of time for one more question here. The question is, since
now where byte is a staple in many operations environment, does code injection
and group techniques be identified or does it require manual memory analysis for our techniques and tools. – [Mike] I would say Mower,
Mower Bites is pretty effective. It’s probably another one. It’s not designed necessarily. You know just for
Rickets by any means signature based and there’s
a lot of different signatures that are rootkit or
certainly code injection could leave behind
besides just the fact that it’s injecting code. So I’d say it’s still
an effective tool and actually, it’s certainly
on of the best out there but it’s a, it’s not as
effective as doing memory now. It’s because that’s the truth, so that’s where the code
has round up it’s… It may try to hide
but it has to run and when it’s running,
it’s running in memories. So that’s getting it
the root of the problem. – [Rob] And
cynically, with Mike, there’s a kind of it’s problem is a lot of AD
tools that are out there. Even the good ones
like you know, Malabytes because (mumbles)
that do a lot more inspection of like
weird things going on, RAM is you still
potentially need to have something that
it targets memory Now one of the biggest things
that I see the downfall of the memory analysis
community right now is that you almost
need to image bram and then be able
to then inspect it. So capabilities that scale
are a few and far between that allow you to
do memory expression across hundreds of systems without having a first
dump and memory image and then examine it. So tools like Indian
Samear is a good example, but also using it
on the low end side if you want to use that
response with others, helps you do that scalability is quite useful. There’s a really good utility written by Micheal Carlon we’re talking about sure why
the people seen carbontinmen which will lie to you,
get direct access to ram on a running widows system so you could then still
run your memory analysis to or against it
even on a live host. Red line also has that
kind of capability but it’s always scalable so you know one thing that I
see a lot of people saying, “Yeah, I want to
do memory analysis but in an enterprise
environment, it’s becoming more and more
difficult topic coming in. It is difficult right now, so I imagine over the next
few years is more vendors get into the space that you’ll see a lot
more stalable solutions for a lot cheaper
price start coming out which will allow
even the EV community might start getting
into the slogan. Since they’re already
running on the system, all they have to do
is connecting the rams in order to run some of these but because it’s not something
you want to do routinely because there’s a lot that
of process background stuff that you have to
do on these agents, but it’s still something that I’m expecting the
community to respond either commercial
capabilities or others. We’ve just got to the point
where we can at least see it a lot better than
we have ever done but now we need to get it back and the more the operational, how do we actually use this
in enterprise environment and so we don’t have
download 1000 memory images just so that we can do
club injection inspection. As we could say (mumbles)
as Mike (mumbles) pointed it out here, if you can detect
it, it’s great, but you know, yeah, 10,000
costing your enterprise, you cancel your code
injection memory analysis on every single one of those without some additional help. – [Ben] All right. Great, I think we’re
just out of time now. I’d like to thank
Mike and Wobbly, both for your great
presentations. And then special
thanks to our audience for listening in today. We greatly appreciate it. (mumbles) upcoming (mumbles) the sensor or forward
slash webcast. Until next time, take care and we hope to have
you back again, in that Sans webcast.

Robin Kshlerin



  1. Ken Pryor Posted on April 20, 2014 at 12:18 am

    Great webcast! The SANS DFIR posters are awesome.

  2. 이응규 Posted on April 20, 2014 at 11:52 am

    Very bad video quality

  3. Sev Meow Posted on September 10, 2015 at 6:17 pm

    In addition to the RTFM (Red Team Field Manual), Blue Team Handbook: Incident Response Edition is also a great buy. Another compact book which is essentially a huge cheatsheet including things such as useful Wireshark filters, tips for Windows forensics, tips for Linux forensics etc.

  4. Jakob Oberascher Posted on April 18, 2018 at 5:28 am

    Excellent session, thank you!