Document Name: NCSA Network Security Policy
Accountable: Adam Slagell
Authors: Adam Slagell, Joerg Heintz, & Mike Dopheide
Approved: <insert date>
With the new network infrastructure being deployed in the NPCF, the Security Team has taken the opportunity to start fresh and divide systems into different security zones in an effort to better monitor and protect the Center's high-value assets. In the past, the Network Intrusion Detection System (NIDS) was limited to monitoring traffic only at the NCSA border routers (and even here with some holes). It was very difficult to detect suspicious traffic between internal systems and other Teragrid systems, in particular malware infections or multi-stage incidents. With this new model, we will be able to monitor traffic between security zones providing for previously unavailable visibility into the internal networks. Traffic between zones will not only be monitored by the NIDS, but full network flows will be generated. As a result, we will be better able to contain a security incident, profile the expected traffic within our own networks and more quickly identify and respond to security incidents. Some of these zones are physically separate in different buildings, but others are separated by network topology and virtual networks. The key distinguishing feature of a zone is that traffic between zones will be monitored, and hence all hosts within a zone must be trusted at the same level. The purpose of this document is to define the different security zones and outline requirements for vetting systems into each zone.
It is important to note, however, that placement into a particular zone is not necessarily driven by the role or function of a particular system. That is, these zones were not designed to represent "tiers" of systems, either by value or function, nor are they intended as a "map" of recommended system placement. While the security zones may serve as a guide for those tasked with assigning physical space for NCSA systems, the allocation of this space is not a security role and the Security Team does not dictate this placement. Security's primary role with systems that have been recently assigned, or reassigned, to a given network zone is simply to ensure that the system meets the security requirements for that zone.
Additionally, it is critical to understand that while in many cases particular physical locations will map to particular security zones, physical locations and security zones are independent concepts. As such, system owners/managers should likely use physical data center requirements to drive their requests for space for those systems, and not what security zone or zones are available in that space. If a system cannot meet the requirements for the security zone generally available in the physical space to which it has been assigned, it will either have to be reassigned or the networking infrastructure modified to place it within a different security zone not previously available in its physical location. Conversely, if a system's security requirements are not met by the security zone generally available in the physical space to which it has been assigned, the Security Team will work with the system owner/manager to ensure a proper level of security services are provided or that the system is reassigned to a more appropriate zone.
The Security Team will be responsible for verifying that regular audits are performed for compliance with this policy, however the actual auditing may be performed by full-time system administrators, particularly for major production resources. Furthermore, approval from the Security Team is required prior to installation of any systems in Zone 1 (described below). Where possible, this verification and auditing will be done in an automated manner. For example, some policy compliance can be checked by the NIDS, and other components can be checked by security tools on the systems being protected, e.g., file integrity checkers and change control software.
Violations of this policy may result in immediate disconnection of systems by the Security Team, especially in critical and sensitive zones. Failure to obtain prior approval for installations based on zone policies or attempts to circumvent these policies will be reported to senior management at the NCSA.
For the purposes of this document, production systems are defined as any system, to include allocated systems, intended to provide reliable computational and/or data services to a networked constituency. These systems include not only “customer facing” hosts, such as webservers, file servers, login nodes, etc., but also the infrastructure required to support these systems, such as backend database servers, backup and storage systems, authentication servers, etc.
For any rule or policy, there are likely to be rare exceptions. The NCSA Security Operations team will review and requests sent to them for an exception, and the NCSA CSO will decide whether or not an exception or alternate course of action will be made in response to a request. Appeals to decisions can be made to the Director's Office.
This zone hosts all of the hardware that still resides within ACB. This is a temporary zone, needed only until NCSA's external connectivity is switched over to the NPCF and all NCSA systems are moved out of ACB. The general deadline for this is June 30, 2011.
The variety of systems in ACB is extensive, but there should be no new systems added to this zone, the exception being replacement of failed hardware on systems still running production services out of ACB. It is requested that teams notify the Security Team when/if production systems are moved from ACB to another building.
The NCSA network border is currently at ACB, though ACB will become a stub before July 1, 2011. Traffic into Zone 0 from the outside world is monitored by security taps within ACB with the exception of the 3 dedicated TeraGrid links. Additionally, during the transition, the 10G links between ACB and NPCF will be monitored as an entry point into Zone 1.
This is the primary zone within the NPCF data center with direct access to significant external bandwidth and no external firewall. The High Perf DC Zone is not appropriate for building new systems and/or configurations until those systems have been fully hardened because most network filtering will be impractical due to sensitivities regarding performance expectations.
It is expected that systems within this zone will generally be large HPC systems and their associated data storage hardware, which either require data transfer rates greater than 10 Gb/s or have a sizable physical footprint. While not all systems selected for this zone may match this description, to beplaced in this zone, systems must be approved by the NPCF RAF Space Committee and vetted by the Security Team.
Auditing to ensure systems adhere to these policies will be performed regularly. Given the importance of ensuring each system in this zone is pre-approved to be there, the Security Team will maintain an approved list of systems for this zone and they will be periodically audited.
The Security Team will provide assistance hardening systems; examples include syslog/syslog_ng configurations, Samhain installation, and the building of an instrumented ssh daemon that will be required on most systems. Systems may be removed from the zone if they develop a history of security related issues. Project/administrator history and criticality will be considered with the possibility of a probationary period before systems are reconnected to this zone.
Systems will not be given external network access from this zone until vetted by the Security Team. (1. Group asks RAF committee to house a system in NPCF. 2) They ask neteng for connections and IPs. 3) Neteng gives them access to the test/install zone and refers them to us.) A vetting period of up to one week must be planned for. Systems in this zone and administrators of those systems must:
All external links into this zone will be monitored, including internal sub-zones and those links to/from other NCSA buildings. Due to financial and physical limitations, traffic completely within this zone cannot be monitored.
This defines a type of system-specific zone for systems that are physically located on the floor in 2020 NPCF, but which have not yet met all of the security requirements to be part of the High Performance DC Zone. In most cases, router ACLs would be sufficient for protecting these systems while they are brought online and secured before being moved into Zone 1. These connections will be limited to the extent than they can be monitored via temporary taps or SPAN ports.
Systems in this zone may be in the process of initial configuration, but have reached the point of requiring network access to download updates and additional software. As always, to be physically located in the NPCF, systems must be approved by the NPCF RAF Space Committee.
As the purpose of this zone is to provide network access to systems as they are being configured, it's unrealistic to expect them to have a full suite of security tools already configured and working. Packet filtering will allow minimal incoming services to help protect systems that may come with vulnerable software out of the box. These systems must:
NPCF room 2016 is for testing and installing new systems and will reside behind a 1G Firewalled network link. Systems requiring more than 1G bandwidth, but unable to meet a sufficient level of system security to be moved to the High Performance Datacenter zone must provide and connect to the network through a firewall, which must be vetted by security. (In some cases, router ACLs or a NAT could be deemed sufficient.) The firewall(s) will limit traffic both in and out of the test zone and traffic from this network will be monitored. Requests for changes to any firewall configurations should be directed to networking and coordinated with the Security Team.
Systems in this zone may be in the process of initial configuration or semi-permanent test systems.
Systems in this zone have the same requirements as those in Zone 1b.
The NCSA Building houses the majority of NCSA staff, but also has several small server rooms. With staff and guests moving systems in and out, the networks within this zone are not highly trusted.
It is the goal of the Security Team to eventually be able to separate the NCSA Building into several sub-zones based on logically or physically separate networks. At this time those changes are not feasible and are beyond the scope of this document.
A large variety of systems includes staff laptops, workstations, test systems, and may include machines providing production services that, for whatever reason, do not merit or require placement in the NPCF data center.
Traffic within this zone should be regarded with suspicion as these networks are only nominally more trusted than general Internet traffic. As a result, traffic between this zone and the others will be monitored. The Security Team provides assistance with hardening hosts and can provide additional monitoring if requested.
The NPCF building houses several offices whose traffic should be handled separately from that of the main datacenter zone. Command Center systems that are not dedicated CnC systems for Zone 1 machines, i.e., networked only to internal subnets, will be treated as members of Zone 3b.
NPCF Office systems share the same security requirements as NCSA Building systems (Zone 3a) but are a separate sub zone because they will be physically monitored at separate points.
NCSA Wireless connectivity is available in all of our buildings (ACB, NCSA & NPCF). For campus wireless networks, all traffic is tunneled outside the NCSA and treated as external traffic. For this reason, they do not fall under any NCSA zone and are not explicitly addressed.
There are both NCSA and University provided wireless networks in the NCSA and the NPCF buildings. These wireless networks are split into multiple logical zones. These networks all have different authentication mechanisms, rules about guest accounts and may or may not utilize link-level encryption. The security requirement for NCSA wireless networks (those that are managed by NCSA or provide NCSA IP addresses) is simply that an adversary gains no additional benefit by physically locating themselves within wireless range of NCSA facilities. For example, by using the strong encryption and authentication technologies of WPA2 or VPNs, it can be reasonably assumed that an attacker already needs a valid Kerberos credential which would give them access to this zone through the VPN anyway.
It is strongly suggested that the UIUC-Guest network be made available in NCSA and NPCF for the use of the majority of visitors and guests. It is also strongly desired that we deprecate the NCSA-Portal network as it potentially gives adversaries additional advantages of proximity.
Types of Systems:
Generally, these systems will be laptops, tablets, and/or smart phones owned by NCSA employees or guests.
Guest laptops are likely candidates for victimization by malicious network traffic, email/web client attacks or viruses. Therefore, the preferred way for guests to obtain Internet access is through the EDURoam, UIUC Public Wifi (where available) or by getting a guest account on UIUCNet, which has good coverage all over campus. These networks are run by CITES and traffic is tunneled outside the NCSA network. Therefore, guest systems would not logically be a part of the NCSA network and would be treated as external Internet traffic.
Requirements that apply to ALL wireless networks in NCSA buildings (whether operated by NCSA or CITES) are:
NCSA offers a VPN service for employees working remotely.
Systems connected to the NCSA VPN networks are treated as external systems with monitoring on the NCSA side of the VPN tunnel.
Sanitized from this copy
This is an island zone only for the NPCF physical security systems.
All NPCF physical security systems, and only those systems, are part of this zone. This includes the camera DVRs, the badge readers, the iris scanners, the ACMS workstations (for badging, control and enrollment), and the ACMS database server.
It should be noted that there will hopefully be a way for alerts to be passed out of this network in the future, but this connection should be unidirectional.
This defines a type of zone rather than a specific zone itself. An Island zone has its own, unmonitored external bandwidth either provided by a 3rd party or too large to be monitored by the border NIDS. The island may have an additional, smaller bandwidth connection directly into NPCF zones that will be monitored with the NIDS and treated as external traffic. This includes site-to-site VPN traffic traversing NCSA's external links. This traffic must not be routed inside the NCSA network without also being monitored by a NIDS, and it will be trusted no more than other Internet traffic.
These are systems (typically servers) for special projects or private sector partners that bring in their own systems to physically reside in NPCF, but that also require their own external connectivity. Other large systems that have their own pipes, but do not wish to pay for monitoring, can opt for an island zone.