Software Monitoring for NERC CIP – What, Why and How – Part 2

In Part 1 of this series, I walked through the background of the NERC CIP v5 controls and outlined what needs to be monitored for NERC CIP software requirements. In this final part of the series, we will take what we have learned and explore approaches for meeting the requirements, while considering security value. NERC CIP is supposed to be for security, after all!At a high level, Tripwire’s Whitelist Profiler (WLP), a Tripwire Enterprise (TE) product extension, has many of the features needed for meeting the software monitoring requirements but process is important, as well. Additionally, in some cases, there are multiple approaches to a requirement, so the entity gets to choose what fits best for their interpretation and process.OS and Firmware VersionOS version is generally tracked one of two ways, but both are easy. With a strict change management approach, TE can read the OS version and show if there are changes, which of course are very rare. A more scalable approach with policy is to also test the OS version, so the result flows into a unified view of compliance.Probably the slickest approach is to monitor the OS version with WLP’s so-called “additional software” feature. The OS version will be reported right along with other software, or the implementation can even be broken out to show OS version per its CIP part number.Firmware version is similarly monitored by reading it from the device. Add to that a test in TE and the control can be represented in a graphical summary red/green compliance view. The main variance customers ask to accommodate is during firmware updates across a fleet of devices where more than one version would be considered acceptable. This use case, however, is easily accommodated with a policy test.Firmware monitoring gets particularly interesting when considering endpoints Tripwire cannot or perhaps should not connect to. Some substation endpoints cannot robustly support building and/or closing a telnet or SSH (secure shell) connection. In such a case, a trusted intermediary can be used to gather firmware version and other configuration details, as well as relay the data back to Tripwire for unified reporting. (The options here could easily become another blog entry.)And finally, if a device is not reachable even by an intermediary tool, a person can paste the configuration into a designated location reachable by Tripwire. One customer even built a web frontend for this use case. Until all devices are somehow reachable in this case, the manual step unfortunately seems required but at least the device configuration is evaluated by standard controls and reported alongside all other devices.Intentionally Installed SoftwareThere is a two-pronged approach for this requirement. Most intentionally installed software registers itself (on Windows), or is installed by RPM. WLP will capture these as designed. PuTTY, OSI Monarch and Oracle on Linux are examples that would not be captured by WLP out of the box as they don’t register with the OS.The so-called “additional software” feature simply would need to be configured to identify these applications. Internet Explorer can also be included in this scope although most entities consider it as part of the Windows OS.Custom software: What was…Prior to WECC’s announcement (referenced in my first blog post) about how it was going to interpret the requirement, the effort across regions was to implement some method of searching key portions of the file system for executable files. Portions of the file system that typically house intentionally installed software would often be excluded from the search.While this smaller scope is more scalable to scan, it is not as secure. Still, the scanning usually uncovers anywhere from 50 to 200 files per system[1], which then inspires a healthy discussion about how to manage the files: put them in a standard location for instance, or consider eliminating some.For one customer, the scanning for custom software had been in place for a few weeks on some production systems. When we looked at the scan history of one system, we could see an admin had done some PowerShell scripting on the desktop, placed some .ps files on the recycle bin, and not emptied the recycle bin. Additionally, a major software package had been deployed in a non-standard location and also happened to include a few copies of PuTTY. Good to know!I encourage every entity to check with their region about how to interpret “custom” software. For non-NERC compliance purposes, I encourage customers to consider using the custom software monitoring control as it strikes a balance between scalable and informative.The population of executable files on a system can be “large,” but it’s also generally pretty stable. Thus, a monitoring approach based on change management tends to scale well.Custom software: What is…At least in western audit region of WECC, the guidance has been to not worry about the “may include scripts” language in the Guidelines and Technical Basis section of the requirement document and leave it to the entity to determine for themselves what scripts or applications should be listed as “custom.”In practice, the scope has amounted to a very short or zero-length list of in-house developed utilities. The cleanest approach I have seen is to use WLP software monitoring feature in such a way as to report just on these few, in-house developed utilities. This approach results in identical reporting as is produced for intentionally installed software.Security PatchesA staple part of the approach is to implement a rule in TE to list out the installed patches. This simple step documents system state with a listing of the patches and any day-to-day changes that work well for change management objectives. Patches can be listed in one block of text and tracked as such, or they can be tracked as individual items (elements), as per the preference of the entity. Customers sometimes also use third-party tools for patch reporting (e.g. WSUS for Windows or Satellite for Linux).Example Implementation of WLP Software Rules for Windows:Here is a summary of the approaches with comments about how widespread the approaches are:

Leave a Reply