Journey Into Incident Response

In the Fall I was staring out my back window seeing my yard covered in orange leaves. This sight is one I see each year and I have always viewed as my yearly chore. The chore of cleaning up the leaves that have fallen from the trees. At times I would see some joy the leaves would bring as my kids would play in them but mostly I viewed the leaves with disdain. Knowing I would be spending hours upon hours cleaning it up. I came to accept this yearly chore as something that doesn’t change since it came with the territory of owning a property with trees. This was until I became more knowledgeable about a subject and this knowledge changed my perspective on how I see these leaves.

For over the past year I took some time to get refocused in life. During this time I was reflecting on different things; one of those things was I have never grown my food. My food typically came from stores, farmer markets, or local farmers. Thinking about it I realized my food has always came from someone else’s labor. I had no clue how to grow food nor what was involved with growing food. I decided I wanted to change this and I jumped head first into becoming more knowledgeable about organic gardening.

I won’t go into detail about my approach; basically I read books, researched on websites, spoke to friends who garden, and I spoke to local farmers who I buy food from. I tried to cover all of my bases to know as much as I could about the entire plant life cycle. My goal is to be fully self-reliant so to avoid having to constantly buy compost I started to learn about composting. As I went deeper into the art of composting by reading and seeing what others have done before me, the more knowledgeable I started to become. The more knowledgeable I started to become the more my perspective started to change. Staring out of my back window each Fall I only saw a chore. However, this year as I was staring out of the window I saw something else. I saw enough brown material that I could use to make compost the next spring. To create the rich compost loaded with nutrients to feed my vegetable plants. I saw the potential for cover material I could put on my raised beds to protect the soil during the winter months. I saw what a blessing each Fall is since it is when nature provides you with a wealth of material you can use to improve your soil to grow better vegetable plants.

As I stared out the window I also reflected on the similarities between my journey into composting and a security analyst’s journey into DFIR. When I’m building up a security analyst to do DFIR work the approach is the same. The first few months I allow them to be paid to learn; there job is to gain knowledge so their perspective looking at data changes. I want to give them knowledge about what they are looking for, different files and folders on the system, different log sources, and the analysis process. I try to give them enough knowledge to change how they see data and what that data means. To change them from seeing just a bunch of files and folder names to seeing select artifacts and log files. To change them from seeing just a bunch of activity to seeing the malicious activity. To change them from seeing alerts and alarms to seeing what the exact attack vector is.

Knowledge is the key to changing one’s perspective; applying the knowledge is what makes the change reality.

"Knowledge without application is like a book that is never read"

~ Christopher Crawford

I was digging a hole to plant my blackberries plants when I kept hearing a noise of something moving around the corner of my house. I stopped digging and walked around the house to see what was making the noise. I didn’t see anything anywhere so I shrugged it off and went back to digging the hole. Shortly thereafter I heard the noise again so I went back to look around the corner. Again, I didn’t see anything so I went back to work thinking maybe it was the wind. After a few minutes I heard the noise for a third time and this time I was determined to figure out what was making the noise. I went around the corner of my house but I still didn’t see anything. Then I looked down to my right to my basement window well that sits below ground and saw what was making the noise. Sitting next to my window inside the window well was a squirrel, which wasn’t moving since it saw me standing right above it.

I walked a few feet away so the squirrel couldn’t see me but I could still see it. I stood on top of my air condition unit to see what the squirrel was doing. After a minute, the squirrel started to move around. Not just in any manner but it started to walk the boundary of the window well making a circle. As I stood there watching the squirrel I realize what occurred. I built up the soil on that side of my house to prepare for our garden but this caused the soil to be close to the top of my window well. The squirrel must had been walking and fell into the window well before I was able to buy window well covers. The trapped squirrel searching for a way out turned it into a routine. The routine of walking in circles trying to find a way to escape but not finding one. The squirrel keeps walking searching for a way out. In the end, the squirrel is just walking in a small circle. As I was watching the squirrel I could see it had been trapped for some time; maybe for hours or maybe the entire day.

I thought about how I could help the squirrel escape without it biting me. My first attempt was to put a branch into the window well. This way the squirrel could climb up the branch to escape. I dropped the branch down into the window well and went back to my spot to watch what happens. The squirrel started to walk the circle and approached the branch. Then the squirrel walked over the branch and continued looking for a way out. My first thought was maybe the branch was too small so I replaced it with a piece of lumber. The same thing occurred with the squirrel walking right over the lumber and not seeing that the wood was its way out from being trapped. I stood there watching the squirrel and thought to myself the squirrel is trapped in its own routine. For hours the branch and lumber were not there so the squirrel was walking right past it since it was not expecting it. My neighbor came over to help me get the squirrel out. It took a few minutes but he was able to manage to lift the now freaked out squirrel out of the window well with the shovel. The squirrel panicked and jumped right back down into the window well. However, this time the squirrel was no longer trapped in its routine since the experience with the shovel was a jolt to its senses. My neighbor now struggled to get the squirrel on the shovel so he decided to set a brick on the bottom of the window well. The squirrel immediately saw the brick and used it to jump out of the window well to free itself.

At times we can find ourselves trapped in our routines and this is especially true when performing analysis for security monitoring, digital forensics, or incident response. Routines make our job easier because we can perform certain actions without having to think really hard about how to do it. The downside of routines is they tend to put us on auto-pilot, which blinds us to seeing something new that is right in front of us. Similar to the squirrel’s routine blinding it to seeing the way to escape. Every now and then when you are performing routine analysis tasks take the time to stop and think about what you are doing, what you are trying to accomplish, and what you are seeing. If you don’t then you may never see what you are missing because we don’t have the luxury of someone giving us a jolt to break us out of our routines.

As we marched across the parade deck from the side we looked as one. The sound of about 70 Marines' heels hitting the pavement but sounded as one. The sound of the hoarse drill instructor's voice echoed throughout the 3rd Battalion. The sight from the side must had been one to see. 70 Marines appearing as only a few walking in a single line. In one instant, in one brief moment the few became many. The drill instructor echoed one command followed by quickly correcting himself with a different command. The 70 Marines who were marching as one became many as they tried to adjust. The stress of making a mistake on his first platoon must have added to the pressure. As the Marines marched across the parade deck the drill instructor kept echoing the wrong commands forcing the Marines to adjust. The stress of the Marines striving to take first place must have added to the pressure. They lost their focus and were no longer in sync with the Marine standing next to them. It must have been a sorry sight from the side seeing close to 70 arms and legs marching with the sound of 70 heals hitting the parade deck at different times. Cluster is the most G-rated description one can give seeing the Marines march across the parade deck that afternoon.

The evaluation was over and the 70 Marines filed back into their barracks. The brief moment of reflection in their minds was broken as the sound of a footlocker being kicked broke the silence. The roar of the two other drill instructors’ hoarse voices followed the loud bang of more footlockers being kicked. The blame for the cluster on the parade deck was placed squarely on the recruits. That afternoon the Marines spent quality time doing sandpit hopping across 3rd Battalion in Parris Island. For those not acquainted with this tradition the following is what occurs. Recruits are forced exercise in what seems like a giant sandbox by following the orders barked by their drill instructor. Jumping jacks, mountain climbers, jumping jacks, push ups, mountain climbers, etc.. This goes on for a period of time before the recruits then run to the next sandbox to be smoked again in the same manner before running to the next sandbox. This continues until the drill instructors get bored or the recruits need to be somewhere. Words don’t do justice describing getting smoked so please take a few minutes to see a Pit Stop in action. In the sweltering heat of South Carolina, the recruits had sweat powering down their faces as they were covered in sand with sandfleas biting them. As much as they tried to ignore it they could only focus on the feeling of bugs feasting on them and not being able to do anything about it (one scratch typically ends with a lot longer time being smoked). That afternoon the recruits (me being one of them) thought to ourselves why are we being punished when our drill instructor messed up.

It was easier to blame even though it was hard to tell what even happened. It was easier to blame then it was to take responsibility so it wouldn't happen again. It was easier to blame then it was to admit we messed up; despite the circumstances we lost focus and resembled nasty civilians instead of Marines marching in sync. It was easier to blame to distract us from our current reality of shit.


Moral of the Story

It is wise to direct your anger towards problems - not people; to focus your energies on answers - not excuses.

- William Arthur Ward

Things have been quiet on jIIr since I over committed myself. The short version is I had zero time for personal interests outside of my commitments, $dayjob, and family. Things are returning back to normal so it’s time to start working through my blog idea hopper. In the meantime, this post is sharing some of my recent random thoughts. Most of these thoughts came in response to reading an article/email, seeing a tweet, hearing a presentation, or conversing with others. 


~ We need to stop looking to others (peers, vendors, etc) to solve our problems. We need to stop complaining about a lack of resources, information, training, tools, or anything else. We need to start digging into our issues to solve them ourselves instead of looking for the easy answers.

~ As we work to better defend our organizations, we need to take to heart R. Buckminster Fuller's advice. "You never change things by fighting the existing reality. To change something, build a new model that makes the existing model obsolete." Our focus needs to be on building and improving the new reality while ignoring the haters who are stuck in the past.

~ We need to stop saying we don't have enough resources. We need to focus on our workflows and seek out ways to improve, automate, and become more efficient. Slight changes on existing workflows can free up resources for other areas.

~ We need to start using this new technology called Google. There is no such thing as a stupid question but there are questions that can be easily answered by doing a simple Google search.

~ Let's get our current generation tools working properly before talking about next generation. If we can't properly configure and use our current tools then getting a so called “next generation” tool won't solve anything.

~ We need to stop saying we need more training. We need to stop saying for me to do task X I need to be sent to training Y. We just need to realign our priorities to spend time on self-development. Turn off the TV, buy a book, build some virtual machines, conduct some tests, and analyze the results.

~ How about we talk more about detecting and responding to basic security threats. If we can't alert on commodity malware infections or web server compromises and have effective workflows triaging those alerts then hunting shouldn't even be in our vocabulary. Forget about hunting and focus on the basics.

~ Let's stop generalizing by saying if company X was monitoring their logs then they would had detected the compromise sooner. That is until there is more published practical information telling organizations how they can actually set up their security monitoring capability. If there is very little practical information or assistance about building a security monitoring program then we shouldn't be surprised when organizations struggle with the same complicated process.

~ On the same note and while we are at it. Let's also stop saying if company X looked into their alerts then they would had seen there was a security issue. We need to start providing more published information instructing others how to actually triage and build workflows to respond to those alerts. If we don’t share and publish practical information about triaging workflows then we shouldn’t be pointing out the failures of our peers.

~ Let's stop focusing our security strategy on the next new product instead of looking at how to better leverage our existing products. New products may address a need but we might be able to address the same need with existing products and use the money we save to address other needs.

~ Let's stop with the presentations and articles pretending to tell other defenders how to do something while the author says they are not saying how exactly they do it to prevent threats from knowing. This serves no purpose and is counterproductive since it’s actually not telling other defenders how to do something. What’s the point of saying anything in the first place?

~ Please let's stop adding noise to the intelligence sharing echo chamber. Whether if its products or conferences, most say we need more threat intelligence and we need to start sharing more. No other specifics are added; just that we need it and others need to share it. In the end we are just echoing noise without adding any value.

~ We need to stop saying how we have a shortage of talented security staff to hire. It is what it is. We need to start talking about how we can develop highly motivated people who want security as their career. We may not be able to hire talented security staff but we can definitely grow them to meet our needs.

~ We need to expand our focus on detecting and responding to threats from being primarily end point focused to server focused. A good percentage of articles, intelligence sources, and products talk about end point clients with very little mention about servers. How about detecting and responding to compromised web servers? How about database servers? How about CMS servers such as Joomla, WordPress, and Drupal? Our conversations are only talking about a part of our IT infrastructures and not the entire infrastructures.

~ We need to stop complaining that our management just doesn't get or take security seriously. The issue can be two things. Maybe we aren't communicating in a way for them to care. Maybe security really is not a high priority. Either way, we need to either: fix it, move on to another organization, or just accept it and stop complaining about it.

I saw the excitement in my son's eyes as the biggest smile was stretching from ear to ear. He slowly stretched out his arm to show me what he got at camp that day. He was extremely excited and I could sense his happiness as I heard him say "I won it with only one dollar. I did it on my first try. Can we keep it?" My eyes focused on what was in his hand. It was a plastic bag with a small goldfish swimming around. "I won it at the fair today. Can we keep it?" In that split second I quickly ran through what owning a fish might entail and it was very similar to the picture used in this post. I then said "yes, we can keep it". My son excitedly ran to his summer camp counselor with so much excitement to tell her the fish was going home with him.

As we were walking to pick up my youngest son I realized the first thing I didn't think about. My five year old would be upset seeing his brother with a goldfish and knowing he doesn't have one. I thought problem solved; we'll just buy him one at the pet store while we are there getting supplies. We reached my five year old in his camp and his eyes grew bigger and bigger as he saw the bag. "Is that a fish" he asked and my seven year old replied "Daddy is getting you one too".  At that moment both kids had smiles as they kept staring at the little fish swimming in the bag. As we were walking down the hall we walked past another parent. She saw the bag with the fish and nervously said "Oh lucky you". I laughed and I could see she was a bit nervous walking down the hall to pick up her kid.

On the drive home, I remembered what my wife said at one point. Dam, my wife. Make that item number two that didn’t cross my mind when my son asked me if we could keep the fish. She has been dead set against owning a fish and this time playing like I misunderstood or didn’t hear her won’t work. “Absolutely no fish" is pretty clear. I knew I wasn’t getting out of this one so I thought I might as well get something out of it. I sent her a text message saying the boys had a big surprise for her. Despite her continued texts trying to guess the surprise on my drive home I wouldn't answer them and I only deflected saying she had to wait.

As my wife opened the door both of my sons went running up to her. They said guess what a few times trying to gather their thoughts from their excitement. Then my seven year old says "at the fair I won a fish on my first try. I did it with only one dollar. Daddy said we could keep it and he is getting Gab one too." She started to give me that stare until she walked over and started watching the fish swim around in its bag of water. Maybe she ran through what a fish would entail too but maybe not. Whatever it was I wasn't going to ask when she said it looks like we are making a trip to the pet store.

On the drive to the pet store my wife and I were on the same page. We would get to the store then buy a basic tank, a second fish, and some food. As we walked up and down the aisle there were tanks of all sizes. Not sure what size we needed we asked the store for assistance. The cashier said he would send over the fish lady. I gave him a puzzled look and was like "fish lady?" He said that's what we call her since she knows everything about fish.

We continued walking up and down the aisle waiting for the fish lady while continuously stopping my boys from wrestling each other. A younger girl was walking towards us and I asked if she was the fish lady. She laughed and then explained all the tanks and fish she owns. I told her we were looking for a tank to hold two goldfish. She said each fish should have least 10 gallons of water and then I glanced at the shelf. At that moment I knew getting the small basic tank we thought that would work was no longer an option. Nope, we had to get a real fish tank. As we continued listening to the fish lady she started going down the list of things we would need. Water conditioner, food, gravel for the bottom of the tank, filter, vegetation (fake or real), a stand for the tank to keep it level, structures for the fish to hide in, and the list went on. My wife and I both reached for our phones to confirm what she was saying without her noticing (we research everything before buying something). We were making sure she wasn't trying to pull a fast one on us and our quick research confirmed what the fish lady said. I even saw the weekly work that owning a fish entails. I stopped counting all of the things I didn’t think about when I quickly ran through the list of what I thought owning a fish entails.

After hearing the fish lady I said that's a lot more than I was expecting. My kid won a fish at the fair and we thought we would only need a basic tank. She cracked a smile and then said "oh, a fair fish huh". After she helped us and was walking away I got the feeling this must happen a lot. Parents getting a fish at the fair, going to the fish store, and then getting hit over the head with what it really entails to own a fish. We grabbed a shopping cart, grabbed all of our supplies, the fish my five year old picked out, and selected the stand for our 20 gallon tank. As we left the store I kept thinking about the dollar fish that just cost us hundreds of dollars. That evening I spent hours putting together the stand and tank while my wife was cleaning all the items going into the tank (another thing we weren't expecting).

What I thought owning a fish entailed was nothing close to what is actually involved with owning a fish. Spending a dollar to win a fish was nothing compared to the hundreds of dollars needed to take care of the fish. The weekly work I envisioned was a lot less than the actual work I have been doing for weeks.

If I could do it again knowing now what I didn't know when we sent our son to camp that day. I would do things differently. I would had told him to save his dollar and do not bring home any fish. Mommy and I are doing some research and then next weekend we will go get the supplies and fish to set up a nice tank. It will be better than just watching two goldfish swimming around in a 20 gallon tank. This is the approach I would had taken. The approach of not trying to make things work with a dollar fish because in the end I still paid the same amount as I would had going with the better option in the first place.

My guess is this story plays out every year at a lot of organizations. The only exception is organizations are not dealing with goldfish but tools.

“You never change things by fighting the existing reality. To change something, build a new model that makes the existing model obsolete.” —Richard Buckminster Fuller

It's very easy to accept the way things are and say in response "it is what it is". It's easy to say I tried and give up when others push back against the things you want to change. It's easy to say this is how we always did it so why change anything now. Now let's put this into context of information security. It's easy to accept the thinking "that no one gets security" and then take on the mentality of not doing anything to change it by saying "it is what it is". It's easy to say I tried and give up when you make an attempt change how people approach security but then get push back by others. It's easy to say this is how we always approached securing our organization so why change anything now.

The quote I opened this post with nicely summarizes how you can go against the grain and put an organization on a better path to addressing their security risks. How you can change the existing security strategy focused on prevention to one focused on a balance between prevention, detection, and response. Start building the better approach (model) to enable others to see the value it adds. Continue building out the better approach and showing value to others. Showing the value and benefits results in people buying into the new approach. Eventually the change will take hold putting the organization on the better path. Building the better approach is more effective than fighting against the existing reality and those who are complacent with the way things are. Changing the security strategy won't occur without some resistance. There will be remnants of those who resist your changes and will fight to make things go back to the way things were. Those remnants won't be as successful in influencing change because they will be fighting against a new reality and they will lack the motivation and/or determination to go against the grain to build a better model.

We are overwhelmed with data and are not sure what to look at or collect? I came across this paraphrased comment in a SIEM forum and it echoes a sentiment I have seen about SIEM. Deploying the technology results in a ton of noise and alerts making it hard to make sense of. Some organizations struggle using SIEM effectively and at times their staff are drowning in a sea of logs and alerts. The comment is also one foreign to me. I’ve read about it and seen others say this but I never witnessed it for myself. My path, my journey was a different one. This post reflects on my SIEM journey for the past year in hopes that it can help others who are either taking their first step or are already on their SIEM journeys.

Disclosure: jIIr is my personal blog and is not affiliated with my employer. This post only covers my personal experience and does not go in to details related to SIEM implementation in my employer’s environment. Any questions along these lines will unfortunately go unanswered. Some lines are not meant to be crossed and this is one of them.

Start with Why It Is Needed

My journey didn’t start when the SIEM was acquired but it occurred long before then when my perspective about security strategies changed. The security strategy is critical to explore since the strategy is the force pushing organizations down the SIEM path in the first place. Let’s go back in time to when I was in a unit performing vulnerability assessments against other public sector organizations. Over time I began to see fundamental problems with various security strategies I encountered.

Some strategies were completely centered on prevention. Almost every security resource – including funding, staffing, etc. – was applied to tasks and projects related to preventing attacks. In these organizations we always found something; in every organization we always found something. With each finding came the task of explaining to auditors on my side the cause of the finding. Auditors see things in black and white but security findings are not clear cut. The truth is we probe the target’s environment and security program moving from well protected areas to areas not well protected. These were the areas our findings came from. Organizations cannot protect everything in a manner to prevent everything; it’s an impossible task and even with an unlimited budget it is not achievable.

Some strategies were centered on compliance. Security resources and priorities are focused on the findings in the latest audit. As time goes by the security strategy is to address and reduce those findings without taking in to consideration areas posing the highest risks to the organization. At times during our engagements we were welcomed. The expectation was we would highlight areas they should focus on and help the security folks convince management to allocate the appropriate resources to address those areas. For a while I thought the work we were doing accomplished this as well. In time I became to see things differently. No matter how effective an audit is, this security strategy will never work since there is a fundamental problem. Audits only confirm if something complies with criteria outlined in a regulation, policy, or standard. If something has no criteria then it is very difficult for an audit to list it as a finding since each finding needs to be tied back to a criteria. If the criteria (regulation or policy) doesn’t address something that is a high risk to the organization then the audit/assessment will miss it as well. Making matters worse is it is very difficult to determine how effective something is. If something exists, has supporting documentation, and passes some tests in most cases this satisfies the audit requirement. Audits don’t truly evaluate how effective processes and controls are. They check a box as something being present then move on to other areas not well protected (remember it’s impossible to protect everything). This results in significant gaps in security strategies driven by compliance and the news headlines of breaches for organizations compliant with regulation X highlights this.

Then some security strategies were reactive. This is when the security strategy is based on the latest event; both inside the organization and outside of it. With each new event the organization switches focus and resources to address it even if it is not the highest risk to the organization. This leads down a path of random controls put in place to combat random threats and what little security resources are available is used in an ad-hoc manner. Reactive security strategies in my opinion are not even a strategy and are doomed for failure.

Over time, the fundamental problems with various security strategies I encountered made me ask myself a single question. If I ever found myself in a position to lead an organization’s immature security program. How should I approach building their program from scratch? Exploring this question brought me to various information security resources. It even lead me to obtaining my Masters of Science in Information Assurance. In time I came to the following conclusion:

1. There are fundamental problems with security strategies based on prevention, compliance, and reactive.

2. Most information security decisions I witnessed in my entire career were not based on actual data to support the decisions. Decisions were based on experience, intuition, what someone else recommended, or shiny new objects. At times, decisions not based on actual data resulted in bad choices, wasted what little security resources are available, and didn’t address the actual threats the organizations faced.

3. What security strategies need is “an intelligence-driven approach, whereby decisions are made based on real-time knowledge regarding the cyber adversaries and their attack methods, and the organization’s security posture against them”.

The fundamental problems with security strategies based on prevention, compliance, and reactive would be addressed in an intelligence-driven approach. Security decisions would not only address the most significant risks an organization faces but the decisions would be influenced by facts and information. The path to intelligence-driven security needs to start with the foundation mentioned in the quote below.

“To be ready to take on an intelligence program, the organization needs to have a foundation in place for monitoring the network for intrusions and a workflow process for responding to incidents.” 

This became my perspective on how security strategies needed to be and I ended up in a security office who agreed with the strategy. This strategy started me down the SIEM path as part of laying a foundation for security monitoring. The driving force behind SIEM was security monitoring and it influenced what logs were collected and how they were analyzed.

Expect a Significant Time Commitment

I knew there was going to be a significant time requirement at my $dayjob but I didn’t know about the impact on my personal time. I had a well-rounded background to take on a SIEM project but I never built the equivalent of a SOC. I read the often quoted percentage that “somewhere between 20% and 30% of SIEM deployments among his client base fail, meaning not only do they not meet predefined goals, but also that many organizations don't even bother using the product”. I also read the articles and comments about how difficult SIEM deployments are, how complicated SIEM management is, and how companies frequently end up in a sea of alerts with no clue what to do about them. I guessed what the impact would be on an organization for a failed security investment. How after getting buy-in, making a sizable investment in technology, and allocating staff to only end up with something that doesn’t meet any goals would be devastating. Not only would this not provide the needed foundation for intelligence driven security but the failure would linger for a long time for the organization. Any other request for security resources would be even more difficult because it will be looked upon as another wasted investment since the investment in SIEM failed. Any other security initiatives may be looked at with doubt and wonder if they can even be successful since the security office failed with the SIEM initiative. Needless to say, failure was possible but it wasn’t an option in my opinion.

I put most of my personal time I allocate for research, writing, and reading on hold to allow me to focus on building the SOC. I spent my time instead researching and learning from others about how to build an effective security monitoring capability. A small portion of what I explored was mentioned in the posts:  Linkz for SIEM, Linkz for Detection and Response, Making Incident Response a Security Program Enabler, and Linkz for Intelligence Driven Security and Threat Intelligence. In essence, my time at the $dayJob was spent implementing SIEM capabilities, building processes, training staff, and managing security monitoring while my personal time was spent exploring all aspects related to security monitoring and intelligence driven security.

It was a personal sacrifice I made for my employer to ensure our SIEM project would be successful but in the past year my knowledge and skills have grown by leaps and bounds. There’s a personal sacrifice leading a SIEM implementation but making that sacrifice is worth it in the end.

Prepare, Prepare, and Prepare Some More

In John Boyd’s autobiography Boyd: The Fighter Pilot Who Changed the Art of War one lesson we all can learn is how he approached things. Take the following quote about someone who Boyd influenced and was a member on his team:

“His attitude was “Maybe so. But if not me, who?” He was the right man in the right place at the right time. He had done his homework and knew his briefing was rock solid.”

The lesson we can all learn is for us to do our homework; to prepare for each possibility and the things that can and will go wrong. This is probably the best advice I could give anyone traveling down this path. You might be the right man in the right place at the right time to lead an organization as they deploy SIEM technology but you need to do your homework. You need to research and explore the issues trying to be solved, developing a detailed project plan with phases, getting buy-in by explaining the issues and why SIEM is the technology to solve it,  identifying the exact logs required and what in those logs is needed, etc.. The list goes on but the point of this advice is to prepare. Prepare, and then prepare some more since the effort and time spent doing this will ensure a smooth deployment. I spent a considerable amount of time preparing upfront and during the deployment and this helped me avoid pitfalls that could had impacted the project.

Leverage Use Cases

As I started this journey one of the first things I did was to learn from others who took this journey before me. The main person who influenced my thoughts and thus my approach was Anton Chuvakin. In most articles he advocates to leverage use cases when deploying a SIEM and hands down this is the best advice for a successful SIEM project. He authored a lot of posts on the subject but the best one as it relates to a SIEM project is the one hyperlinked in my quote below:

“The best reference I found to help architect a SIEM solution is a slide deck by Anton Chuvakin. The presentation is Five Best and Five Worst Practices for SIEM and it outlines the major areas to include in your SIEM project (16 to be exact). It may not cover everything -such as building rules, alarms, and establishing triage processes - but it does an outstanding job outlining how to avoid the pitfalls others have fallen in. If anyone is considering a SIEM deployment or in the midst of a SIEM deployment then this is the one link they will want to read.”

His slide deck influenced my SIEM project plan from the point of getting buy-in to the actual implementation. To implement a use case there wasn’t a lot of information on it so I put together the SIEM Use Case Implementation Mind Map to have a repeatable process for each use case.

The beauty in leveraging use cases. Not only does it make building a SOC more manageable by focusing on detecting certain activity in smaller chunks and building the processes around those chucks but it makes it very easy to show others the value SIEM adds. If the SIEM deployment takes one year to complete then using multiple use case can show progress and what was accomplished throughout the year. The value added is shown throughout the year instead of waiting until the end. That is if the proper preparation was done in advanced.

Focus on Triage

Thinking back over the past year and what I found to be the most challenging with SIEM technology or any detection technology for that matter is how events/alerts need to be handled. I found bringing in logs, creating custom detection rules, and tuning rules to be easy compared to developing the triage processes surrounding each category of events/alerts. My previous thoughts on the subject still ring true today:

In my opinion establishing triage processes is the second most critical step (detection rules are the first.) Triage is what determines what is accepted as "good" behavior, what needs to be addressed, and what needs to be escalated. After the rules are implemented take some time reviewing the rules that fired. Evaluate the activity that triggered the rule and try out different triage techniques. This is repeated until there is a repeatable triage process for the rules. Continue testing the repeatable triage process to make it more efficient and faster. Look at the false positives and determine if there is a way to identify them sooner in the process? Look at the techniques that require more in-depth analysis and move them to later in the process? The triage process walks a fine line between being as fast as possible and using resources as efficient as possible. Remember, the more time spent on one alarm the less time is spent on others; the less time on one alarm increases malicious activity being missed.

The triage processes are critical and can help prevent encountering the paraphrased comment I opened this post with. We are overwhelmed with data and are not sure what to look at or collect? Unfortunately, when it comes to triage for the most part you are on your own. This was one area I found to be truly lacking in documentation and the few cheat sheets I found didn’t account for the majority of stuff one needs to consider. Determining what triage processes were needed followed by developing those processes and then training others was definitely a tough challenge. It was tougher since it also involved honing those triaging processes to improve their efficiency and speed. Despite the difficulty, focusing on triage enables you to see through the noise and know what to look for.

Metrics – These Are Needed

If someone would had told me two years ago I be reading about security metrics I would had laughed out loud. Two years ago I didn’t see the value metrics offer until my eyes were opened exploring intelligence driven security. Another comment I saw in the SIEM forum was the following quote: “what information should be provided to the management on the daily basis which can justify the purchase of  SIEM”. This is where metrics can come in to play. By recording certain information it makes it easier to communicate what one is detecting and responding to. Communicating metrics and trends shows value to management, highlights weak areas in the security program, or uncovers patterns in attacks. During the past year I explored various information security metrics. As it relates to the SIEM, the VERIS schema is probably the best one I found for recording the information from detecting and responding to security events. As soon as you have the documented information then the fun part is finding creative ways to communicate these to others.

You Are Alone Standing on the Shoulders of Others

As I embarked on this journey I learned from others who have either dealt with SIEM technology or security monitoring. Initially, I read online what they published and made available to the public. I reached out to a few people about the additional questions I had. I looked around locally and reached out to others who were managing their own security monitoring capabilities. Needless to say, my SIEM journey was successful since I was standing on the shoulders of those who came before me.

However, at the end of the day each organization is different. Others can provide you with advice and share their experience but what is actually encountered is unique. The environment is unique and certain aspects of the use cases are unique as well. Due to this uniqueness, most of the time throughout the year we (my team and myself) were on our own. I mention this as a word of caution to others who may expect a vendor or others to solve their SIEM challenges for them. Others and vendors can help to a certain extent but you and your team are the ones who need to solve the challenges you face. At that point you will find yourself alone standing on the shoulders of others.
 

Looking Forward to SIEM Year 2

Reflecting back over the past year it has been a demanding challenge. The journey hasn't ended since work still needs to be done. There will always be work to do and new challenges to work through. Hopefully, the experienced I shared will avert someone from stepping on a SIEM land mind resulting in them yelling out for help as they are drowning in the sea of logs and alerts.

You end up having to talk to a range of people when building out an internal incident response process. It's a natural consequence because the way people did things in the past is changing and these changes will impact the way they do things going forward. The people you need to communicate with is dependent upon the organization and what the changes actually are. In my case, I ended up discussing incident response with a cross section from the information technology department including: helpdesk, server groups, security units, and management. At some point during the discussions the "why" incident response is needed has to be addressed in order to get buy-in to implement the changes. Thinking about the "why" and the various audiences who need to hear it makes it more clear how concise your explanation needs to be. The message has to be clear and convey the reality we find ourselves in with the threats we face without adding any additional FUD (fear, uncertainty, and doubt) in attempts to influence people's decisions.

Prevention Will Eventually Fail

Despite our best efforts and everything we try to do to secure organizations the end result will be the same. The preventative controls we put in place to protect organizations from the threats we face will fail. The gravity of the situation is illustrated in a three year old FireEye quote:

"today’s cyber criminals are nearly 100% effective at breaking through traditional security defenses in every organization and industry, from the security savvy to security laggards"

The defense in-depth strategies of applying layers of security controls to protect data is incapable of preventing compromises and data breaches. The continuous stream of news about breached organizations from retail stores to universities to public sector organizations is our constant reminder that prevention will not prevent threats from accomplishing what they are trying to do.

Incident Response - The Last Line of Defense

Anton Chuvakin framed the conundrum we find ourselves in with his paper Security Incident Response in the Age of APT (behind a pay wall):

"First, prevention and preventative security controls will fail. Prevention fails on a daily basis at many organizations; it will suffice to look at antivirus tools and contrast their 99%-plus deployment rates with widespread ongoing malware infection rates."

"Second, detection also fails on a frequent basis. A copy of Verizon Data Breach Investigations Report reveals plentiful evidence of that."

"What remains of the entire realm of information security. Only incident response."

"Thus, IR simply has to be there because this is where the security of an organization will fall after all else fails - and it will."

In essence, after every other security control fails an organization's last line of defense is incident response. A line that needs to: investigate, contain, remediate, detect further compromises across the enterprise, and reduce future compromises. This last line of defense is solely dependent on having the right people (incident responders) to carry it out. As incident responders, our job is to hold this line against the threats our organizations face on a daily basis despite everything else failing around us.

To be a catalyst for change this is the message we must convey:

Prevention will fail and when it does, the last line of defense to thwart the threats we are up against is the incident response process and its staff.

"Look, if you had one shot, or one opportunity,
To seize everything you ever wanted. One moment
Would you capture it or just let it slip?"

~ Eminem


Everybody has a story. Everybody has a reason about why they ended up in the Digital Forensic and Incident Response (DFIR) field. Sharing these experiences is beneficial to those looking to follow in your footsteps; students fresh out of college, career changers, or people looking to do something different in DFIR. In this post I'm sharing my story, my story about how I became an incident responder. A path that has been very challenging while rewarding at the same time. A path that started with the mindset seen in the "Lose Yourself" lyrics below.

"You better lose yourself in the music, the moment
You own it, you better never let it go
You only get one shot, do not miss your chance to blow
The opportunity comes once in a lifetime"

Formulate Your Plot

At the time I was working in a security unit doing network penetration testing and digital forensics support for investigations. How I ended up in this unit in the first place was due to the same mindset I'm about to describe. I enjoyed the offensive side of the house but I knew it wasn't my passion. Digital forensics was at one point challenging but it became very repetitive mostly working fraud investigations. I wanted something more. I wanted something where you are constantly challenged; I wanted to do incident response. I set my sights on incident response being the end goal and knew everything I would do was to help me reach that goal. I didn't know where this path would lead but I thought about my preferences which were in this order: incident responder in my own organization, incident responder with a specific organization in the NYS public sector, or joining an established rock solid IR team.

Focus on the Process

In DFIR and information security in general, people have a tendency to focus on the tools one should use. The better approach and the one I take is to initially focus on the process one uses to leverage tools to accomplish something. Within incident response there are numerous processes that are dictated by an incident's classification. To make it more manageable as I started my journey into incident response I focused on one specific incident type (malicious code incidents). I set out to learn everything about what examination steps one uses to investigate a machine compromised with malicious code, what artifacts to parse, and the tools one uses.

My plan wasn't to only be skilled at malicious code incidents since my focus was on the larger incident response field. In addition to learning the technical skills and knowledge, I spent considerable time better understanding the larger incident response process. How the process should work, how to design the process, how to build and manage a CSIRT, and how to manage incidents. I even focused on incident response while I was going for my Masters of Science in Information Assurance. I took the incident response management track as well as made this my focus on assignments where we had flexibility with choosing our own topics.

Focus on the Skill Set

Learning the processes is only the first step; my next step was to develop my skill set carrying out those processes. I spent considerable time practicing the malicious code  investigation process by compromising test systems followed by examining them. In a future post I'll share how I did this so others can follow suit. I did this for months. In the beginning it was to learn the process then it was to be more efficient then it was to be faster.

As I was working towards my goal I kept my eyes open for the opportunities that come once in a lifetime. I knew I wasn't ready to approach my organization about doing IR work since I had to own it when I did. However, other opportunities presented themselves when family members and friends reached out to me as their "IT support guy" because their systems were infected. This opportunity allowed me to continue building my skill set while helping others. In addition to practicing on test systems, I began making it known to family and friends that I will fix their infected computers for free.

Search for the Opportunity

Opportunities have a tendency to just appear but sometimes you have to seek them out. At the time I was well prepared with my knowledge and skill set in incident response so I was confident I could own certain opportunities if I found them. I started to pursue my first preference for doing IR work, which was for my current organization. I didn't ask them to send me to training or to let me help them with their incident response process. Instead I wanted them to see the value in what IR can do for an organization besides putting out fires but I had to do it in a way to compliment my skills.

I got the word out to the other security units that I could assist them with any infected systems. I made two things clear. First, I would tell them what the root cause was so they can start to mitigate infections by strengthen their controls. I knew root cause analysis wasn't consistently being done and for the security units to have access to this new skill set was instant value for them. My second point was a calculated risk but I made it clear I would be faster than their current process as well as the IT shops who re-image infected systems. If I was going to be doing the work it had to be faster than their current processes. If it wasn't then why should they even bother with me. I knew being faster would add value to the organization by freeing up FTEs (full time employees) to do other work.

I occasionally kept putting out reminders to the security units about my offer as well as getting my supervisor to remain on board for me to do this work. I can't remember how long this selling went on for (maybe a month or two) but my opportunity finally presented itself. There was an infected machine and they wanted to know the root cause. This was my shot and I knew there were two outcomes. If I came back with nothing or if my response was I can't do this work without training then they probably wouldn't had come back to me for help again. If I nailed it and showed them the value in root cause analysis for minor malicious code events then maybe I would do this work more frequently. Needless to say, the preparation I did on my own enabled me to nail the examination and I came through on the two points I sold to them to get their buy-in. Nailing the first examination wasn't enough because I had to own this and lose myself in the DFIR music.

Own the Opportunity

I and my organization had a taste of using the IR skill set for security events that were not considered to be incidents. Now I had to own this opportunity. I continued working to improve my skill set through compromising test systems and helping anyone who asked. I continued buying and reading DFIR books as well as blogs, papers, articles, etc.. I continued to hone my process to make it faster. I sacrificed my free personal time to live and breathe DFIR. The request for malicious code assistances kept coming in and each time I was better than the last. I kept getting faster and I kept showing my organization more value in what IR can do.

As I said, opportunities have a tendency to present themselves. After some time building up this working relationship there was a priority security incident. A highly visible website was potentially compromised and a determination about what happened had to be done as soon as possible. The case was mine if I wanted it and I knew I was prepared due to the months I lost myself in the DFIR music. This opportunity was different and had more at stake. My organization leveraged a third party IR service for priority incidents. In this incident, my organization used this service in addition to my assistance. To make the stakes even higher, initially we (myself and the third party) were not allowed to communicate with each other. This was an opportunity for me to not only reassure myself my place in the IR field but for me to own my place in my organization's incident response process. I worked the case with my co-worker (who was a network penetration tester with zero DFIR experience) and we were able to come back with answers before the third party service. In the end, the server wasn't compromised and everyone can stand down.

I continued losing myself in the DFIR music and owned each new opportunity that presented itself. This journey has lead to where I am today. I'm building out my organization's enterprise-wide incident response capability, developing our CSIRT, and improving our response capability by making it faster. I'm improving our detection capability by architecturing and managing our SIEM deployment as well as combining our detection and response capabilities.

Lose Yourself in the DFIR Music

The path that lead me to become an incident responder has been very challenging but rewarding. It required sacrifices and a lot of work to be prepared for the opportunities that God put in my path. It requires constant motivation so I will be better tomorrow than I am today. It requires me to approach my career as if each opportunity may be the last. It requires me to have the mindset seen in the "Lose Yourself" lyrics.

"You better lose yourself in the music, the moment
You own it, you better never let it go
You only get one shot, do not miss your chance to blow
The opportunity comes once in a lifetime"

It was a little over two years ago when I started Journey Into Incident Response (aka jIIr). In these two years I learned a lot about blogging on technical topics so I wanted to share some tips and advice for those considering taking the plunge into the blogosphere. The following are some things to consider from planning your blog to setting it up to maintaining it:

      -  Why
      -  Motivation
      -  Post Frequency
      -  Content
      -  Publishing Method
      -  Blog Template
      -  Blog Configuration
      -  Blog Post Tips
      -  Advertising
      -  Gauging Interest
      -  Motivation

Why

My first question to any would be blogger is why. Why do you want to put yourself through the agony of sacrificing your personal time and resources so you can share your experiences and research. There are other avenues to take; write an article and submit it to a site offering free content (such as DFI News or Forensic Focus). Write a post for a blog that publishes guest posts (such as SANs Forensic blog or Girl, Unallocated). I’m not trying to talk anyone out of blogging since it has been a rewarding experience. On the contrary, I’m just trying to get my point across that blogging is a lot of work and in the end you are doing it for free. When I decided to start blogging I told my wife “it will be easy”. Sure I had a new born to go along with my two other sons, was pursing my Masters degree, and still had a full time security job but I seriously thought putting together a few posts a month would be a cake walk. My perspective changed pretty quick after I wrote my first few posts.

If you have any hesitation about not wanting to put in the work then consider the other options available. If you made up your mind and blogging is the route you want to take then the remaining tips may be helpful.

Motivation

People who start blogging have a reason why they are doing it. For some it’s to give back to the community, for others it’s for name recognition, for some it’s a platform they control to share their research, and for the rest it’s any number of reasons. Before you even start setting up a blog make sure you know what your true motivation is. There will be times when you have writers block or you just don’t feel like writing anything so whatever your reason is for blogging needs to be enough to motivate you to put the keyboard to word processor. Your motivation needs to come from within since most of the time others can’t do this for you.

Frequency

This was something that helped me so I wanted to pass this nugget along. Before you even start worrying about content, site design, or any other details take some time to consider how often you want to update your blog. If you look at other DFIR blogs you will notice a range in how often they are updated. From multiple updates in a week to monthly updates to yearly postings. As an author, I found it helpful to set myself a monthly goal of how many posts I wanted to put together. It helped to not only plan better on how to reach my monthly goal but it helped get me into the habit of writing regularly. As a blog reader, I tend to check the sites with a more regular update schedule more so I assume others readers do the same. If a blog gets updated at random times then it tends to fall off my radar until I see it in my RSS feeds. Whatever goal you set for yourself it’s not something that needs to be publicized. It’s a personal goal for yourself and only for you to know. I’m only mentioning my goal since I’m giving this advice. I always wanted to write three substantial posts per month so at the end of the year I would have 36 decent write-ups.

Content

I also found it helpful to at least pick a theme for the blog. There are times when I’m not sure what to write about so having a theme gives me something to fall back on. As you might have noticed my theme is in the title: Journey Into Incident Response. I wanted to blog about the different things I learn and experience as I work on improving my skills and knowledge to investigate security incidents. If you do decide to pick a theme it doesn’t mean you are locked into it. Write about whatever you want whether if it’s about a book, tool, article, or anything else you are working on in the moment. The theme is just a crutch for when you start to run out of ideas which brings me to my next point. Make sure you have a blog idea hopper. Keep a list of different ideas for future posts and always add to it when you think of something new. Some ideas may never go beyond the hopper while others may turn into great articles. One of the reasons why I don’t struggle with ideas for posts is because I constantly have between 5 to 10 ideas in my hopper. If I need something to write about then I just look over my hopper and pick the one that interests me the most. Case in point, the idea for this post has been in my hopper for months.

Publishing Method

At this point you know why you want to blog, what your motivation is, how often you will update it, and you have a general idea about what content you want. The biggest decision you will now make is how you want to host your blog. The two most frequent publishing applications I see DFIR bloggers use are Word Press and Blogger. If you aren’t sure about what publishing application to use then reach out to the blog authors you read. I bet the authors are more than willing to let you know why they choose what they did and how they feel about their decision. As for me I went with Blogger for two simple reasons. First was because most of the blogs I followed at the time used the service and second was because I didn’t want to have to worry about maintaining software updates. All I want to do is blog and Blogger enabled me to do that.

Blog Template

The second biggest decision will be what template to use for your blog. The template is how your blog looks and what people will stare at when reading your posts. Your publishing application may have a few templates available that meet your needs. If the built-in templates aren’t what you are looking for then there are free templates available online. What sites should you use to get a free template? Great question and it’s still one I can’t really answer today. The last thing I wanted was to get a free template with an embedded malicious link that would attack any visitor of my blog. So I took a closer look at a few DFIR blogs I followed to see where they got their templates. I went to each blog and viewed the site’s source code to find the template’s author website. The screenshot is from my blog’s current template but it’s also the website I saw in a ton of other DFIR blogs.

Blog Configuration

Remember growing up when people said first impressions matter? I think the statement is true even for blogging. When I was in the process of configuring my blog one setting I loved was the ability to restrict access to my blog. I pretty much prevented anyone from seeing the blog while I tried out different templates, options, and layouts. In Blogger I even used the setting preventing the site getting indexed by Google. I only removed the permissions, thus allowing anyone to see the blog, when I had everything set up the way I wanted. Configuring your blog is a matter of preference so my only advice is to don’t unveil it until it’s set up the way you want it.

Blog Post Tips

ITAuditSecurity has some great posts about blogging tips. As it relates to putting a post together one tip I wanted to echo was in his post Blogging: Choose Great Titles and Intro Sentences. His reasoning is not only do they grab the attention of readers but they also help in having better search engine results. I completely agree with his points and I wanted to build on it with another point. Picking good titles and intro sentences helps to let the reader know exactly what the post will be about. If the point of the post can’t be conveyed in a title or one sentence then make sure it is conveyed in the first paragraph. If the content of the post isn’t clear upfront then some readers will stop reading before they reach the part of the post where you make your point. In all of my posts I try very hard to make sure that the reader knows exactly what I’ll be talking about by the time they finish reading the first paragraph.

Advertising

I remember thinking very clearly when I was getting ready to launch the blog “how do you advertise it to others”. I thought there was some secret so I reached out to Harlan for advice. At the time I was just a name from the Win4n6 group who Harlan helped once before but I figured who else would be better to ask them someone who has been blogging for years. Harlan’s response to my question about the secret to advertising was:

“Add your blog to your email signature, and tell others about it. Pass it around. I, and others, constantly link to blogs and posts that are interesting and informative. There's no real secret”

Here I was thinking there was some secret; some involved advertising process only known to bloggers but in the end advertising has actually been the easy part. Two years later I can honestly say Harlan’s advice was spot on.

Gauging Interest

Don’t get me wrong about my next comment. I truly appreciate all the feedback I have gotten over the last two years. The conversations in person, comments offline, and comments posted to jIIr. When you start blogging treat any feedback you get like gold. Feedback is the best way to get an idea about your blog’s content so cherish it when someone gives it to you. The reason is because the majority of blog readers don’t provide feedback. They don’t leave comments, send emails, or contact you using other means. Thinking about it I fall in the same boat. I follow over 300 blogs and my comment to read ratio is pretty low. For the first year blogging it felt as if I was talking to myself. I would keep posting different content but I didn’t get a lot of feedback. I wasn’t sure what content went over well and which ones as Harlan says “went over like a fart”. In these situations Google Analytics will be your friend. Google Analytics keeps stats about your site such as pageviews for each post and referrals to your blog. For the times when I don’t get feedback I can get a rough idea about the content people like by looking at the page views. However, some of my posts where I got great feedback were the same ones with low pageviews. Leverage Google Analytics as a tool to guage interest on your site but remember it is not fool-proof.

Motivation

As I mentioned before blogging has been one of the most rewarding things I have done. It has required a lot of sacrifice but it has made me into a better DFIR practitioner. There are times when I felt as if I wasn’t adding value; times when I was flying high because my research and posts has helped others. Regardless of what happens when you blog, the most important advice I can give is to stay true to what motivated you to blog in the first place. If you are working towards accomplishing what you set out to do then the rest doesn’t matter. Enjoy the ride and remember to say thanks to those who give shout outs about your blog or provide feedback.

Certain events in life cause you to reflect on humility and put back into perspective the meaningful things in life. You remember that in time almost everything is replaceable. Another forensicator will fill your shoes at work and your organization will continue to go on. Another researcher will continue your research and the little that you did accomplish will eventually just be a footnote. Another person will step up to provide assistance to others in forensic forums and listservs. Your possessions and equipment will become someone else’s to enjoy and use. When looking at the big picture, the work we do and value will eventually fade away and life will go on as if we were never there. One of the only things remaining will be the impact we make on others in the little time we have available to us. One doesn’t need a lot of time or resources to make an impact; all that’s needed is having a certain perspective.

Everyone should look out not [only] for his own interests, but also for the interests of others. Philippians 2:4

Having an outlook that looks beyond one’s own self interests can positively impact others and I think the statement holds true regardless of religious beliefs. A perspective that takes into consideration others’ interests is displayed everyday in the Digital Forensic and Incident Response (DFIR) community. DFIR forums have thousands of members but there are only a few who regularly take the time to research and provide answers to others’ questions. DFIR listservs are very similar that despite their membership the minority are the ones who regularly try to help others. Look at the quality information (books, articles, blogs, white papers, etc) available throughout the community and their authorship is only a small fraction of the people in the community. These are just a few examples out of many how individuals within the DFIR community use their time and resources in an effort to not only better themselves but to educate others as well.

When I look at the overall DFIR community I think there’s only a minority who are looking beyond their own interests in an effort to help others. A few people have helped me over my career which contributed to where I am today. They never asked for anything in return and were genuinely interested in trying to help others (myself included). If the DFIR community is what it is because of a few people giving up their time and resources to make a positive impact on others, then I can only wonder what our community would look like if the majority of people looked beyond their own interests to look after the interests of others. In the meantime all I can do is to continue to try to remember to look beyond myself in every aspect of my life. To try to consider those around me so I can help whoever crosses my path needing assistance. When the day is over one of the only things remaining will be the impact I have on others.

There won’t be any links pointing to Dr. Phil, Dear Abby, or Aunt Cleo. Not that there’s anything wrong that… They just don’t provide advice on a career in DFIR.

Getting Started in DFIR

Harlan put together the post Getting Started which contains great advice for people looking to get into DF. I think his advice even applies to folks already working in the field. DF is huge with a lot of areas for specialization. Harlan’s first tip was to pick something and start there. How true is that advice for us since we aren’t Abby from NCIS (a forensic expert in everything)? People have their expertise: Windows, Macs, cell phones, Linux, etc. but there is always room to expand our knowledge and skills. The best way to expand into other DF areas is to “pick something and start there”.

Another tip is to have a passion for the work we do. In Harlan’s words “in this industry, you can't sit back and wait for stuff to come to you...you have to go after it”. I completely agree with this statement and DF is not the field to get complacent in. There needs to be a drive deep down inside to continuously want to improve your knowledge and skills. For example, it would be easy to be complacent to maintain knowledge only about the Windows XP operating system if it’s the technology normally faced. However, it would be ignoring the fact that at some point in the near future encounters with Windows 7 boxes and non-Windows system will be the norm. A passion for DF is needed to push yourself so you can learn and improve your skills on your own without someone (i.e. an employer) telling you what you should be doing.

I wanted to touch on those two tips but the entire post is well worth the read, regardless if you are looking to get into DF or already arrived.

Speaking about a Passion

Little Mac over at the Forensicaliente blog shared his thoughts about needing a drive to succeed in DF. I’m not musically inclined but he uses a good analogy to explain what it takes to be successful. Check out his post Is Scottish Fiddle like Digital Forensics?.

Breaking into the Field

Lenny Zelster discussed How to Get Into Digital Forensics or Security Incident Response on his blog last month. One issue facing people looking to break into the field is that organizations may not be willing to spend the time and resources to train a person new to the field. Lenny suggested people should leverage their current positions to acquire relevant DFIR skills.

Lenny’s advice doesn’t apply to how I broke into the field since DFIR was basically dropped into my lab when I was tasked with developing the DF capability for my organization. However, his advice is spot on for how I was able to land my first position in the information security field (which is what lead me into DFIR). I was first exposed to security during my undergraduate studies when I took a few courses on the topic. It was intriguing but the reality was there weren’t a lot of security jobs in my area which meant my destination was still IT operations. I continued down the track pushing me further into IT but I always kept my desire for security work in mind. After graduation I took a position in an IT shop where I had a range of responsibilities including networking and server administration. In this role, I wanted to learn how to secure the technology I was responsible for managing and what techniques to use to test security controls. This is due diligence as being a system admin but it also allowed me to get knowledge and some skills in the security field. In addition to operational security, I even tried to push an initiative to develop and establish an information security policy. Unfortunately, the initiative failed and it was my first lesson in nothing will be successful without management’s support. All was not lost because the experience and my research taught me a lot about security being a process that supports the business. This is a key concept about security and up until that point my focus was on security's technical aspects.

I leveraged the position I was in to acquire knowledge and skills about my chosen field (security). My actions weren’t completely self serving since my employer benefited from having someone to help secure their network. I didn’t realize how valuable it was to expand my knowledge and skills until my first security job interview. Going in I thought I lacked the skills and knowledge but over the course of the interview I realized I had a lot more to offer. I took the initiative to expand my skillset and it was an important factor in helping me land in the security field. My experience is very similar to the Lenny’s advice except his post is about getting into the DFIR field.

Get a plan before going into the weeds

Rounding out the links providing sound guidance, Bill over at the Unchained Forensics blog gave some good advice in his recent post Explosions Explosions. He shared his thoughts on how he approaches examinations. One comment he made that I wanted to highlight was “more and more of my most efficient time is being used at the case planning stage”. He mentions how he thinks about his plan to tackle the case, including identifying potential data of interest, before he even starts his examination. I think it’s a great point to keep reinforcing for people new and old to DFIR.

I remember when I was new to the field. I had a newly established process and skillset but I lacked certain wisdom in how to approach cases. As expected, I went above and beyond in examining my first few cases. I even thought I was able to do some “cool stuff” the person requesting DF assistance would be interested in. There was one small issue I overlooked. The person was only interested in specific data’s content while I went beyond that, way beyond that. I wasted time and the cool stuff I thought I did was never even used. I learned two things from the experience. First was to make sure I understand what I’m being asked to do; even if it means asking follow-up questions or educating the requestor about DF. The second lesson was to think about what I’m going to do before I do it. What data do I need? What steps in my procedures should I complete? What procedural steps can be omitted? What’s my measure for success telling me when the examination is complete? Taking the time beforehand to gather your thoughts and develop a plan helps to keep the examination focused on the customer’s needs while limiting the “cool stuff” that’s not even needed.

Books On demand

If someone were to ask me what is the best training I have every taken I know exactly what I would say. A book, computer, Google, and time. That’s it and the cost is pretty minimal since only a book needs to be purchased. I’m not knocking training courses but classes cannot compare to educating yourself through reading, researching, and testing. I never heard about Books24x7 until I started working for my current employer. Books24x7 is virtual library providing access to “in-class books, book summaries, research reports and best practices”. The books in my subscription include topics on: security, DFIR, certification, business, programming, operating systems, networking, and databases. I can find the information I’m looking for by searching numerous books whether I’m researching, testing, or working. A quick search for DFIR books located: Malware Forensics: Investigating and Analyzing Malicious Code, Windows Registry Forensics: Advanced Digital Forensic Analysis of the Windows Registry, Windows Forensic Analysis Toolkit Second Edition, Malware Analyst's Cookbook: Tools and Techniques for Fighting Malicious Code, EnCase Computer Forensics: The Official EnCE: EnCase Certified Examiner Study Guide, and UNIX and Linux Forensic Analysis Toolkit. That’s only a few books from the pages and pages of search results for DFIR. Talk about a wealth of information at your fingertips.

The cost may be a little steep for an individual but it might be more reasonable for an organization. If an organization’s employees have a passion for their work and take the initiative to acquire new skills then Books24x7 could be an option as a training expense. Plus, it could save money from not having to purchase technical books for staff. Please note, I don’t benefit in any way by mentioning this service on my blog. I wanted to share the site since it’s been a valuable resource when I’m doing my job or self training to learn more about DFIR and security.