Between Security Assessment, Compliance Inspections, and Continuous Monitoring we do a lot of assessments. At first blush, it is easy to think that they are all the same and that if we do one of them we meet the requirements of the others and do not need to do them. I mean, we’re just assessing an IT system and how different could it be?
You fell victim to one of the classic blunders! The most famous is never get involved in a land war in Asia, but only slightly less well-known is this: never go in against a Sicilian when death is on the line!
Vizzini
A blunder that is even less well-known than that, is that all assessments are the same! Why should we take time out of our days to discuss why these assessments are different? Because if we understand why we conduct them, hopefully we will understand how to do them better. Speaking of which, why do we do what we do?
To keep everyone on the same sheet of music, we’re going to call each of these events “tests”. The reason why is that some of these events are formally called Assessments of some flavor. Using assessment and Assessment differently, is a bit of a pain. So, the events themselves will be called a test, generically speaking. Now. Let’s channel our inner Fezzik and go back to the beginning and talk about Security Assessments as our first type of test.
Back in the old days when the DITSCAP and DIACAP roamed the surface of the earth, we had two different but related tests. The CT&E and the ST&E. The biggest difference between the two was where they were conducted. A CT&E was typically conducted in a lab environment early in a system’s life. It was there to prove capability of the system. Sometimes you will even hear … let’s call them experienced … practitioners call these Lab Based Security Assessments. Same basic thought. The ST&E was typically conducted later in a system’s life and at the site. Either way, it was done to capture the configuration of the system at the beginning of the system’s operational life, or at the start of it’s ATO.
This is where the RMF Security Assessment comes into play. It has pulled these two related tests into one name. The same idea resides here. Demonstrate the configuration of the information system. Verify that policies and procedures exist. A Security Assessment is not a replacement for ConMon or Compliance Inspections. We know this for a very simple reason. Security Assessments can be conducted prior to a system reaching an operational state or being granted an ATO. How can you verify conformance to security policy if the system is not operating yet? How do you evaluate the effectiveness of a policy before you have even tried it?
Now, this is all well and good, but how do we know that? Follow the RMF. Step 4 is Assess. Since that is a distinct step that is before Monitor, know that these two activities are different. Specifically, Task 4-2: “The security assessment, frequently conducted by the SCA, or other AO designee, determines the extent to which the security controls are implemented correctly, operating as intended, and producing the desired outcome with respect to meeting the security requirements.”
So I can clearly not choose the wine in front of you.
Vizzini
We can also see that this test is not related to a Compliance Inspection because of Task 4.4: “Conduct initial remediation actions based on the findings and recommendations of the SAR and reassess remediated controls, as appropriate.” Key here is “Conduct initial remediation actions..” Again, the security assessment is a test that is completed prior to the ATO being granted.
So I can clearly not choose the wine in front of me.
Vizzini
This is where some of you will attempt to turn the JSIG against me. You’re quote the last part of Step 4 of the RMF where we see: “Security control assessments may be carried out during system development, system implementation, and system operation/maintenance (as part of continuous monitoring).” This is where you are exactly right. We conduct Security Assessments during system development (CT&E), when it’s been fielded (ST&E) and as part of ConMon. Because the JSIG says that a Security Assessment can be carried out as part of ConMon it does not have to be part of ConMon. That alone should tell us that a Security Assessment is not the same as ConMon.
Let me explain… No, there is too much. Let me sum up.
Inigo Montoya
We’ve talked about ConMon before. Leave me to say that a major part of continuous monitoring, that we don’t see in the other types of assessments is effectiveness. ConMon is not just about re-running a STIG or SCAP scan. It is asking yourself the question, “Is this the best way to do this?” If firmly believe that if your ConMon process doesn’t include this question, you may not be doing it wrong, but you are missing out on our two lenses. I’m not going to rehash my thoughts on ConMon. If you are that curious, then might I suggest, or even.
The last test that we have may seem like an odd duck, the Security Compliance Inspection. It is honestly not as odd as you may think. The rest of the world calls them an Audit, either internal or external. Whether you call it an Audit or Security Compliance Inspection, they’re both effectively the same. They both can be pretty painful. The DoD 5205.07 Vol 1 talks about four different types of compliance inspections. For the purpose of this discussion, we’re going to focus on the Core Compliance Inspection. First, they are the most common. Secondly, if you’re going through a Full Scope, a Reinspection, or a No Notice inspection; these are not indicative of a well running Security Program and you probably have better things to do with your time than to read blogs on the Internet.
The DoD 5205.07 vol 1 Enclosure 9 (type that 5 times fast) says that these inspections are to validate SAP security processes and procedures. In a nutshell, are you doing what you are supposed to be doing. We know that inherently. Look at the Self Inspection Checklist. Yes/No 100% is the expectation. Do you meet the requirements?
We are men of action. Lies do not become us.
Wesley
Self Inspections. Compliance Inspections. Audits. They take us away from what our “day job” is, but the perform an important function. Inspection prep is something that we have to build into our schedules and calendars. It’s a chance for us (self inspection) or someone else to double check our homework and see what we have missed. I do not say this to take away from their importance. These tests are an important part of the job. If we use them the right way. Remember our lenses. Why do we do what we do and what is the benefit to me. I would use a Compliance Inspection just like any other network tool at my disposal.
If I’m new to an organization or position; an Audit is a great way to find out what’s what and where I need to improve. Having another set of eyes perform the Compliance Inspection can be the same thing. Except you have someone who is not as close to the situation as you are. Maybe if I have a new ISSO come in, having them conduct a Self Inspection can have that combined effect. They find out what is going on and where some holes are and they are a bit farther removed from the situation than I am. Totally removed to be honest. The thing to keep in mind about this type of test; it is very compliance focused. Yes/No.
Have fun storming the castle!
Miracle Max, Wizard
Now that we’ve blathered on about the different types of assessments, let’s bring it all together. There should be a reason that we’ve gone through all of this, right? There should be something that we get out it, besides reading a blog and burning some time out of our day. To wrap things up, we are going to take a look at one specific control under the lens of each of these different types of assessments.
PS-6 ACCESS AGREEMENTS
Control: The organization:
a. Develops and documents access agreements for organizational information systems;
b. Reviews and updates the access agreements at least annually; and
c. Ensures that individuals requiring access to organizational information and information systems:
- Sign appropriate access agreements prior to being granted access; and
- Re-sign access agreements to maintain access to organizational information systems when access
agreements have been updated or at least annually
We could do this with a myriad of controls in the JSIG. Over 900 of them for that matter. (For the record, the formatting is straight from the JSIG. I’m not trying to make a statement or the like.) If we were conducting a security assessment of the system, CT&E or even ST&E, when we get to this control, we are looking more that the verbiage of the user agreement meets the requirements in the JSIG. We want to make sure that we have one for when the system goes live.
During a Self-Inspection we are looking at how many of them we have, should be one for every user account, that they have been signed within the last year, and that we have a copy of them two years after someone’s access has been removed. Meaning that if someone has been on the effort for a million years, we have a stack of a million user agreements that they’ve completed. There’s probably a list of users (current as of the inspection) and a stack/folder of agreements and we go one for one down the list. Maybe it’s just a simple look of I have 97 users and 97 user agreements. I spot check 25-30% of them and if those are right, I have a good feeling that it’s correct.
Under ConMon we are looking at effectiveness. Did the way we had everyone sign their user agreements; how well did that work? Let’s say that we had everyone sign their agreement at the all hands security refresher training. Did we hit 100%? If not, how far from 100% were we? How long did it take us to figure out we didn’t hit 100%? Is there a better way that we can have users review and re-sign their agreements? Is the way that we file and track user agreements the most efficient way to do it? Are they printed and kept in a file folder? Are they part of a person’s personnel file? When a user has their account deactivated, are the processes integrated to ensure that their user agreements are set aside and kept for 2 years? How about, has the required verbiage changed over the last year?
Here’s a fun one to consider as part of ConMon. If I did a spot check of user agreements as part of my self inspection, was that an effective way of checking if I had them all? Oh. See what we did there.
You rush a miracle man, you get rotten miracles.
Miracle Max, Wizard
The point of all of this is that we should be looking at each control differently, depending on what type of assessment we are conducting. Each type of assessment has a different focus. By this point, hopefully you agree with me that each of these assessments are different and that they have a different purpose and focus behind them. Hopefully, the next time you conduct or are subject to an assessment you use these differences to your advantage. Hopefully, if you don’t agree with what I have said, you let me know. As you wish.
It Depends 