When it comes to FISMA we have seen two things:
- Over the last 8 years the federal government has spent an incomprehensible amount of money on security (whatever that means).
- People have railed against it as a stupid waste of time and an impediment to real security (again, whatever that means).
One of the groups out there that has been both outspoken in its criticism of FISMA (and by extension, NIST) has also been one of the most respected names in the information security field, SANS. SANS seems to have stuck true to its technical beginnings, and decried both the FISMA legislation as well as the work NIST has done related to FISMA. Recently, however, they seem to have done an about-face on the topic, and I commend them for that- it was absolutely necessary.
In this post I'll take a quick look at the situation, explain why I think most of the criticism of FISMA and NIST is misguided, and talk about why the road to hell is paved with good intentions. I use SANS here because I think the situation illuminates some key points, but like I said above, I commend them for turning the discussion in a more positive direction, hopefully leading an industry in which they are generally extremely well-respected on the path to progress with them.
At one time you could not read an article that contained "SANS" and "NIST" that was not about SANS tearing down the work NIST was doing. Here are some of the comments to which I'm referring.
SANS states that "[t]he new version of 800-53 solves three fatal problems in the old version - calling for common controls (rather than system by system controls), continuous monitoring (rather than periodic certifications), and prioritizing controls (rather than asking IGs to test everything)." Talk about missing the mark. Guidance regarding common controls has been a part of either 800-53 or 800-37 at least since May of 2004 when the original SP 800-37 was released. In fact, Continuous Monitoring was explicitly defined in there as well.
The comment about asking IGs to test everything is a little more cryptic. Are they referring to the continuous monitoring phase once a baseline security assessment was done as part of the initial accreditation? Well, again the original 800-37 defines subtask 9.2 as Selected Security Control Assessment. It does not require all controls to be assessed, and any organization with a properly functioning IA group is targeting those controls which are subject to greater risks of change over time such as technical controls. If they are talking about not testing everything even during the initial certification, we can't go back 6+ years for definitive documentation from NIST, but SP 800-53A (which contained this guidance in draft form at least since 2006) has provided tiered levels of rigor for the assessment process so that less effort should be applied to a system which is less critical to the organization.
If that's not a strawman argument, then I don't know what is.
In his congressional testimony in 2010 Alan asks "why did NIST not develop standards that enabled cost-effective vulnerability and risk-reduction?" Are there problems with federal information security? Well, if you read Part 1 of this series, you can surmise my answer would be yes. We can take as a basic assumption, that NIST's guidance is not perfect. However, I think it's a pretty good start. The main problem is that the implementation of reasonable guidance has been bungled.
In his testimony Alan also states that "[t]he very same companies hired to write the guidance then turn around and charge agencies tens or hundreds of thousands of dollars for reports that comply with NIST guidance, but are out-of-date and not useful." This is one point where I can find some common ground with Alan although I think the situation is insidious than he makes it out to be. In reality, there are only a handful of companies who benefit from being able to market the fact that "we helped develop the guidance so we can do the best job helping you comply with it." The main problem, as I stated already, is due to some combination of incompetence, lack of an understanding of the various areas of security, and a general lack of creativity.
The first issue should not need explanation- there are simply some people in every profession who are not very good at their job. On the second point, however, there are people who understand policy very well, but who do not understand how an IDS works or what it's true purpose and value are. These are the policy wonks. Policy wonks have a tendency to make a lot of "thou shalt" and "thou musteth" statements, but don't understand that a large part of securing a system or an enterprise needs to be left to the architects and engineers to make cost-effective decisions about implementing security controls. In effect, this is what leads the technical, operational people like Alan to throw up their hands and start screaming about how NIST did not develop standards that enable cost-effective vulnerability and risk-reduction (see above), or that it's stupid to think that everything can be solved with a report.
So, Alan is right even though I think his frustration is a bit misdirected. As a result though, I think he shares a mis-perception that a lot of people have regarding FISMA and NIST- that these things need to be thrown in the trash so we can start over with something better. First of all, FISMA is so general as to not prescribe anything specific. It does not demand reports. It does not require wasting of money. What it requires is that agencies implement security programs which meet an acceptable level of rigor as defined by NIST. So, what about that would require total overhaul of the legislation?
Further, the NIST requirements and guidance as currently defined leave enough flexibility so as to be extremely useful. You can use scoping, tailoring, and common controls in determining the controls which are required for a system. You can accept risks or fix them based on an analysis of risks versus costs. There is no hard line one way or the other. So again, it doesn't sound so bad, or too rigid and inflexible.
Basically, Alan's idea of the To-Be state of security is "FISMA-compliant," and in effect the NIST guidance should steer you in that general direction. Again, the problem is not the guidance. It is found in the implementers who struggle to determine how to define and use common controls. It is found in the IG audit that focuses on the wrong things and turns up findings that are unhelpful.
Also, you've got agency CIOs, OMB, and OIGs all demanding some sort of reporting related to security. In fact, the primary FISMA reporting requirements which CISOs care about and which Alan is railing against come from OMB. If there is a problem with those, then fix the report format and content instead of calling for a complete overhaul of the FISMA legislation and NIST documentation. These are things not required within FISMA or within NIST guidance, but from the bureaucracy charged with overseeing and managing government. By blaming FISMA or NIST we misidentify the source(s) of the problem, and when we do that we guarantee that efforts to fix it will be misapplied. We also run the risk of throwing out a perfectly good and useful foundational library of documentation.
So, while the 20 Critical Controls effort out of SANS could have been a massively divisive undertaking, they have rightly modified their approach (and rhetoric about NIST) to create a more integrated set of guidance. They even went as far as to praise some of the work NIST has done in the latest revision of the 20 Critical Controls, and have provided mapping of these to the NIST SP 800-53 controls. The result is that they've helped drive federal security in the right direction instead of leading some sort of holy war.