Self-Inspections. Why do we do them? How can we do them better? How does a well performed self-inspection checklist to make life easier?
Two Questions
We’re going to use the following two basic questions over the course of our remaining time:
- Why do we do what we do?
- What benefit do you get out of meeting the requirement?
I cannot speak for my counterparts that you may encounter. Personally, I abhor busy work. Busy work is doubly harmful, because not only does it waste your time that is time that you could have spent doing something useful, or beneficial to you. So it may seem strange that I want you to conduct better self inspections.
This leads us to an important point. You, as the CyberSecurity practitioner, should get benefit out of every single requirement levied upon you. If you don’t, you need to have a conversation with your SCA or DAO.
Self-Inspections, Why?
In an effort to conduct better self-inspections let’s start with our first question. Why do we conduct Self-Inspections? The obvious answer is because we have to. The magic book says I have to do one. My PSO/CPSO insists that I do one prior to our inspection. That is “A” reason why, but not “the” reason why.
As outlined in the JAFAN, there was an idea behind self-inspections. The idea was that the industry locations would police themselves and the Government would come in on a re-occurring basis to check their homework if you will and validate just how good a job they are doing self-policing. This was supposed to “…with good stewardship, result in cost savings to both the Government and Contractors alike” JAFAN 6/0 1-206. It is not intended to be burdensome. “The SAP security compliance process represents a unified and streamlined approach to the SAP security compliance inspections.” DODM 5205.07 Volume 1.
Regardless of which manual is being followed (JAFAN Series or DoD Volumes) the intent is the same. After verifying the Self-Inspection, the Core Compliance Items are validated along with the identified Special Emphasis Items. Self-Inspections were always intended to show that sites were capable of policing themselves. That is the why behind self-inspections. Sites use the self-inspection process to show that they can successfully manage their CyberSecurity program.
Inspection Checklists are not just for Inspections
Since we are talking about “Why do we do what we do”. What other possible reason would you have to execute your self-inspection checklist? In what other way could you derive personal benefit from this requirement? If you are a new ISSM to a campus or a new ISSO to a program/effort, what is an easy way that you can quickly and repeatedly capture the status of the Cyber Security Program you just walked into? I’ll give you a couple of minutes.
How to Conduct a Self-Inspection
Now that we’ve gotten our heads wrapped around the why we do self-inspections; we understand the intent of the self-inspection program. Let’s tackle something really provocative and talk about the how to conduct a self-inspection.
Part of being in my job is determining what a “good” self-inspection looks like. I’ll be honest, there is art associated with this as well as science. When you are sitting down to conduct your self-inspection, keep our first question in mind, “Why do we do what we do”. You want to show your inspector that you do a good job of self-inspecting. Let’s tackle that first.
The first thing is to avoid pencil whipping or giving the appearance that you pencil whipped your checklist. While just checking the Yes/No/N/A block on the checklist is an approach, that’s not where you want go. If this is a new concept to you. Don’t worry about it. Your replacement will have a different look on self-inspections than you did.
Let’s take an easy question and look at it.
13. Are all personnel performing privileged user functions certified as required for the position they hold? [DoD 8570.1-M or replacement; AT-3]
Very easy to look at this item, check Yes and go on to question 14. Right?
Remember, this is the opportunity to inspect yourself. Look at each of your SysAdmins and get a copy of their certifications. See and document when they expire and capture that information. Yes, the comment block is small. If your response won’t fit, use a separate piece of paper or better yet, keep track of the information in a spreadsheet. In the comment block, reference the spreadsheet and where it is kept. Reference where the certificate copies are kept for each of your Priv Users, and be specific. The more specific you are in your comment lends credence to your inspector that you actually conducted a self-inspection and that you didn’t just pencil whip it. Be careful. If you keep track of your Privileged User certifications in a separate spreadsheet, when it comes time to prepare for your inspection you have to review the spreadsheet.
Here is something else to keep in mind and why this is called It Depends. The requirement says “all personnel performing privileged user functions”. So, that’s not just your SysAdmins. This is easily overlooked. Who has elevated privileges on your system? Who can do something more than a general user? Identify them and verify that their training is sufficient to the task. Obviously, as the ISSO/ISSM you are going to have privileges to review audit records. Are you trained and certified to use your audit reduction tool? Do you have your security certification as required? What about your Data Transfer Agents? Have they been trained in how to use the automated tools you have in place to support your Assured File Transfer process? Depending on your system, there can be more here than just your SysAdmins. Please take a honest look at how your system is setup. Depending on how your system is setup, your answer to this simple question may be different.
The other reason and I can’t stress this enough. This is our second lens that we talked about. You should get something out of every requirement. What can you as the ISSO/ISSM get out of a self-inspection? Preparation for your inspection for starters. Take the time and document where the training certificates are kept. You know I’m going to ask to at least spot check them. Inspections can be stressful and uncomfortable events. Make your life easier and capture where the evidence is so you don’t have to hunt for it on the network share in a conference room during an inspection. Don’t be that ISSM who is digging through a file share on the big screen in a conference room because you can’t remember if you have a copy of the certificates in your folder or if they are kept in the personnel files. Document where they are located, and find them with confidence.
Don’t get me wrong, inspections are important. But there is a more importanter reason to do this. This helps build and support your training budget. Look, PMs are stingy with their money. Identify early that Privileged Users have certifications that are going to expire and they need training. Give them some heads up. Help them to plan. Let them know what certifications are going to expire next year. Whatever the case may be. Your inspector is going to notice expired certifications and it will probably come up in the out brief or inspection report. I’m not saying that I would say “I told you so”. I’m not saying that I wouldn’t. Imagine the street cred you get with Management when you identify things six months ahead of your inspection. Then your inspector identifies it. Why do we do what we do. What benefit do we get out of meeting the requirement.
Imagine the situation where you identified an issue during your self-inspection and get to talk to your inspector about how you fixed it. Or the steps that you are in the middle of taking to fix it, because you already created your CAP and you are working issues to closure. Ideally, your inspector should start your inspection by asking if there was anything you want to talk about or discuss before you get started. Perfect opportunity to toot your own horn with something that you fixed. At the very least, something that you discovered and are working to fix.
I know that two of our SysAdmins have expired Security+ certifications. We identified that in October when we conducted our self-inspection. I talked with the Program Manager and they set aside money from the training budget for classes that were scheduled in April. The SysAdmins took the class and have their tests scheduled in May and June respectively.
Don’t just imagine yourself in that situation, make it happen. The crux of this bloviating is to tell you to take advantage of this opportunity to make your life better. Inspections are stressful. Stack the deck in your favor, or at least have the location of everything identified that your inspector may ask for.
Here’s something to note. If your inspector does not start off the inspection and ask if you have anything you want to discuss before you get started, take control and suggest it. “Hey Dave. Before we get started with our inspection. I wanted to go over a couple of things that we identified as deficiencies and corrected or have started to correct.” Be proactive. Do not make your inspector wait until they get to question 13 and see something that’s wrong. Be up front and tell them when you get started.
I would much rather start an inspection going over the things that you identified and closed than where the bathrooms are located and where I stand in front of the building in case of a fire. Let’s face it. I’m grabbing my cell phone and following the sea of humanity fleeing the building. I don’t care about your corporate structure, who the VP of whatever is that may or may not show up for the out brief. While I’m sure she’s nice; she doesn’t matter to me. Quit stalling and get the party started.
Be Malicious
You know that your anti-virus and your other network tools are going to get looked at. Make life easy on yourself. Depending on how your system is setup, put a copy of the EICAR Test File on a CD or other piece of media. Use that piece of media to test your End Point Security, Audit reduction tool and your AV solution at one time. What happens when a user inserts a CD. What is supposed to happen. Is something supposed to work or something else not supposed to work? Do positive and negative testing. Does your End Point Security application alert correctly. Is there an audit event. Does the event have all of the required information. Is your AV solution configured to block or delete malware on contact. Does it alert the user or keep the reporting in the background? Does it send alerts to the appropriate staff? Does the mail group that it was configured to alert still exist? The list of things that you can check with one simple test are incredible. The ability to see if your network tools are configured and working together. Priceless. I mean, you know I’m going to do something like this on an inspection, right. I hate myself for even admitting that I need to say this: Do not use live malware to test a system. Yes, it’s been done. Yes, it was stupid. Yes, you can ask Johnny about it sometime.
What do you do with it
Question 60 on the checklist is interesting from my standpoint.
60. Does the organization comply with the vulnerability scanning requirements outlined in JSIG 3.14.1.5? [RA-5]
Please tell me that during an inspection your answer is never just Yes. We run Nessus every month. There is a big old “And” just waiting to be answered. If all that’s being done is running Nessus, printing a report and dropping it in a safe; you are doing busy work and wasting everyone’s time. That’s OK. Your replacement will make better use of the resources made available to them. Run your report, but do something with it. Identify what vulnerabilities apply to your system and then fix them. Re-run your vulnerability scan to make sure your vulnerabilities are fixed.
Same thing with the SCAP Sanners or STIG tools. whatever they are and whichever you use. They are just tools. Use them to make your life better. Do not be a slave to the tool.
Something to keep in mind though. If you do not understand how long it takes to deploy a patch on your network, you are wrong. What good does it do to run a scan every month, when it takes three months to deploy patches? Figure that out first. If only you had a log of what changes you made to your system over a period of time. If only you had a process that you used to inflict change on your system. Something that you kept track of with approvals and dates and stuff. If only…
If you are continually scanning for vulnerabilities before you can get the initial patches deployed, you are going to be scanning inside your kill chain and piss everyone off. That’s not the point of this at all. Do not be a slave to the tool.
No. You’re a Tool
Everyone has a dashboard or a tool that they use in some way, shape, or form for network management. You will be surprised to know that your inspectors want to see these tools in action. Please for the love of NIST. Do not let the inspection be the first time that you or your SysAdmin has opened up that dashboard. Not knowing your logon and password to Splunk is not indicative of someone who reviews the audit logs every week, I’m just saying.
Do not let your inspector ask why is that red? What’s with that exclamation mark? Why hasn’t that client received AV updates in the last six months? It has happened to me before. Once upon an inspection, I was troubleshooting why a client hasn’t had an AV update in over six months. Neither the ISSO or SysAdmin noticed the problem, let alone even looked into it. Which was strange, because they both claimed that they used the tool every month. They just never bothered to look into it. In the old days, we called that a FTLC, Failure To Look Cool.
Minimally, and I mean absolutely minimally you should go through all of the tools that you use (Audit, Patch Management, Active Directory, anti-virus, etc) and at least count the number of servers and workstations each one knows about. Don’t play games and talk yourself into or out of the numbers that the client shows, take it at face value. Make a simple table. Columns for the different applications. Rows for the types of systems, Workstations, Servers, Linux systems, Switches/Routers, etc. Add a column that is a known good source of the number of systems that you have. Maybe that is Active Directory, LDAP, or even your hardware list. It doesn’t matter. The important part of this exercise is what do you notice as you look across the table that you’ve made? Are they even close?
I know what you’re thinking. We just migrated a bunch of clients to Win10, so these extra 50 clients are the old Win7 machines. Or we just instituted a new naming convention and these computer accounts are the dead ones that have not been removed yet. Great. Fix the tool. Do not allow your SysAdmin to talk you to the correct number. Cleaning out dead computer accounts is not sexy. No one becomes a MCSE so they can clean old computer accounts out of an AD Forest. Know what you should see in the dashboard and make it right. Know what truth should be and make sure that you see it. The idea here is not to create busy work for yourself. The idea is to know what your network, what your system looks like. Do not trust that an application or an agent is always installed. Your replacement sure won’t.
For those of you playing the home game. You are keeping track of what you should be scanning on your network, right? When you scan your subnet or IP range, you are tracking how many workstations, servers, switches and routers the vulnerability scanner picked up and bouncing that across what should be on your network, right?
Next Level
This exercise here, you should do periodically as part of your Continuous Monitoring to ensure that your network management tools are still effective and appropriate to maintain your systems. If you really want to show your peers up, export a list of clients from each of your tools. Import the client lists and your list of known systems into an Access database and run an Unmatched Query for each of your tools. This will tell you, by name, which systems are missing from the tool. Oh snap!
Sometimes Things Shouldn’t Make Sense
For those times and cases where the tool should be inaccurate know why. For example, if you have 5 Linux servers they may not be in Active Directory, because they are not supposed to be there. Know and understand that at inspection time. That is the story that you need to be able to tell. Look at the question from this stand point. If you do not periodically validate that all of your network clients are reporting to your Audit Reduction Tool, how do you know that you are seeing everything in your audit logs?
I’ll be honest. First Impressions matter. If you’re part of a big campus inspection, you may have less than an hour with your inspector before they have to visit the next area. The inspector is evaluating your Cyber Security program. What do you want them to see when you open up that dashboard? Do you want them to see that everything is squared away and accurate, meaning that you have a good understanding of your network? Do you want them to see a tap dance as you and your SysAdmin try to figure out what is on your network and what systems are not reporting in? Do you like long uncomfortable pauses during your inspection while you are trying to figure out how an application works. In a conference room. In front of people. If you do, you’re in a great position to make it happen. Let’s go for double in this round and make sure your boss is there as well.
Something else to consider. Your PM bought these tools to make your life easier. Use the tools to manage your network. Do not be a slave to the tool. Hopefully, your Program Manager included training with the new network management tools you picked up. If not, get some. Now that I think about it. That is a really good idea. We should make folks get training related to the system that they are responsible for maintaining. I bet we could capture that some place like the DoD 8570.01-M or the DoD 8140.01. But we don’t need to, because we already talked about that, right?
Some of these tools come with a license that can be quite expensive. Managing the clients in the tool correctly is part of your license management. The point that I’m trying to make is that there are a whole bunch of other reasons and benefits to managing your network management tools. Reasons that exceed your inspection. If you are the PM and your ISSO over spends license dollars because of workstations that have not been in service for a year or more; what is your impression of their Cyber Security program.
At the end of the day, your compliance inspection is going to evaluate how well you are managing your Cyber security program. Obviously, use your time to prepare for your inspection. While you are preparing, get some benefit out of the work that you are doing, besides the rating.
Hopefully, this gave you something to think about. If you liked it, let my boss know. If you didn’t, contact Human Resources and my name is Johnny Grant, like the dead President. If you have questions about something discussed, you can let me know. Hopefully, you will let your SCA or DAO know.
It Depends 