NCSA logically divides its network into several different trust zones. Traffic between these zones is monitored by a Network Intrusion Detection System (NIDS), but traffic within a single zone may not be visible to the NIDS. Therefore, systems within a single zone must be trusted and hence hardened to the same level.
These zones can vary significantly in how they are trusted: from networks trusted little more than the general Internet to networks that require stringent vetting and auditing. Most networks are public, but some are very isolated and not even routed. The common requirements across all zones are only that systems follow University security policies and that the Security and Networking teams can quickly identify the location and responsible party for all hosts on our networks.
For the purposes of this document, production systems are defined as any system, to include allocated systems, intended to provide reliable computational and/or data services to a networked constituency. These systems include not only “customer facing” hosts, such as web servers, file servers, login nodes, etc., but also the infrastructure required to support these systems, such as backend database servers, backup and storage systems, authentication servers, etc.
NCSA IT Operations Board
The leaders of ADS (Advanced Digital Services), ITS (Information Technology Systems), and Security are responsible for application of this policy. These three groups are the service providers of infrastructure at NCSA and meet regularly to discuss security issues and strategy for providing better services.
The Security Team is responsible to ensure regular auditing of this policy and automates this when possible. However, responsible does not always mean executing every audit on their own. This is a group endeavor among all the NCSA service providers and requires coordination and cooperation between ADS, ITS and Security.
Violations of this policy may result in immediate disconnection of systems by the Security Team, especially in critical and sensitive zones. Failure to obtain prior approval for installations based on zone policies or attempts to circumvent these policies will be reported to senior management at the NCSA.
For any rule or policy, exceptions may be needed. The Security Office will review requests for exceptions. Decisions will be made by the Security Office after appropriate consultation with ADS and ITS leadership. Appeals to decisions can be made to the Director's Office.
The Security Office will review this policy annually with the leadership of ADS and ITS to see if changes are needed. It will also be updated as needed for new network environments that are needed.
NCSA Network Zones
The following zones and their accompanied policies are described logically as specific subnets are subject to change.
High Performance Datacenter (HPDC) Zone
This is the zone for production systems the data center and consists of most machines in 2020 NPCF. It includes both public and private networks.
Types of Systems:
Systems requiring high availability, physical security and high performance networking are hosted here. This includes not just HPCs, but core storage, security, networking equipment, and more. These systems are built in a firewalled subzone until fully vetted by the security team, which is responsible for regular auditing of systems against the security requirements below.
- Until vetted, these machines are firewalled to only accept connections from NCSA hosts or to port 22 (SSH).
- Have a vulnerability and patch management plan in place.
- Disable any unnecessary service and accounts, and enforce with host-based firewalls where possible.
- Inform the security team if the list of services changes.
- Utilize the security team's host-based IDS if possible.
- Forward system logs to the security team's collector.
- Utilize non-local accounts for remote access unless otherwise approved.
- Require two-factor bastions, jump-hosts or VPNs for access to administrative interfaces.
- Disable IP-forwarding and do not bridge networks without approval from Security & Networking.
- Maintain and enforce an up-to-date list of authorized administrators, and keep records up-to-date so that the Security team can quickly determine responsible parties for the system. At least one responsible party must be a full-time employee working at the NCSA.
- Provide the security team with accounts on the system or a way to quickly get access 24/7 for emergencies.
All external links into and out of this zone are monitored by the network IDS. New hosts that appear on this network but have not been vetted may be automatically or manually blocked at the border gateway until investigated and vetted. Network traffic entirely within this zone is unmonitored by the IDS, but netflows are collected.
Zone 1b: Installation
This defines a type of system-specific zone for systems that are physically located on the floor in 2020 NPCF, but which have not yet met all of the security requirements to be part of the High Performance DC Zone. In most cases, router ACLs would be sufficient for protecting these systems while they are brought online and secured before being moved into Zone 1. These connections will be limited to the extent than they can be monitored via temporary taps or SPAN ports.
Types of Systems:
Systems in this zone may be in the process of initial configuration, but have reached the point of requiring network access to download updates and additional software. As always, to be physically located in the NPCF, systems must be approved by the NPCF RAF Space Committee.
As the purpose of this zone is to provide network access to systems as they are being configured, it's unrealistic to expect them to have a full suite of security tools already configured and working. Packet filtering will allow minimal incoming services to help protect systems that may come with vulnerable software out of the box. These systems must:
- restrict network access until systems are vetted
- use secure, non-default passwords; and
- be hardened according to the requirements for Zone 1 as quickly as possible if the intent is to move the system to that network.
- Systems transitioning to Zone 1 must still be vetted by the Security Team
Zone 2: Testing
NPCF room 2016 is for testing and installing new systems and will reside behind a 1G Firewalled network link. Systems requiring more than 1G bandwidth, but unable to meet a sufficient level of system security to be moved to the High Performance Datacenter zone must provide and connect to the network through a firewall, which must be vetted by security. (In some cases, router ACLs or a NAT could be deemed sufficient.) The firewall(s) will limit traffic both in and out of the test zone and traffic from this network will be monitored. Requests for changes to any firewall configurations should be directed to networking and coordinated with the Security Team.
Types of Systems:
Systems in this zone may be in the process of initial configuration or semi-permanent test systems.
Systems in this zone have the same requirements as those in Zone 1b.
Zone 3a: NCSA Building
The NCSA Building houses the majority of NCSA staff, but also has several small server rooms. With staff and guests moving systems in and out, the networks within this zone are not highly trusted.
It is the goal of the Security Team to eventually be able to separate the NCSA Building into several sub-zones based on logically or physically separate networks. At this time those changes are not feasible and are beyond the scope of this document.
Types of Systems:
A large variety of systems includes staff laptops, workstations, test systems, and may include machines providing production services that, for whatever reason, do not merit or require placement in the NPCF data center.
Traffic within this zone should be regarded with suspicion as these networks are only nominally more trusted than general Internet traffic. As a result, traffic between this zone and the others will be monitored. The Security Team provides assistance with hardening hosts and can provide additional monitoring if requested.
- Production systems (See definition above) in this zone must:
- adhere to security best practices including maintaining security patches, running minimal services, and removing unnecessary setuid binaries;
- provide the Security Team a list of running/listening services;
- maintain a regularly audited list of authorized administrators and/or users;
- use secure, non-default passwords or stronger authentication methods;
- forward syslogs to the central syslog server;
- use a host-based IDS when applicable;
- provide the Security Team with interactive logins to investigate security incidents;
- be located in one of the designated, locked server rooms; and
- must not have wireless network adapters, which could bridge zones, activated; and
- the Security Team will maintain an up-to-date list of these systems
- Goal: For servers and small cluster that may be in the NCSA Building, we want to prevent the currently common occurrence of security finding out about systems after-the-fact.
- Administrators of non-production systems, including workstations and laptops, should:
- install security updates;
- use secure, non-default passwords;
- turn off unneeded services;
- be registered in DNSWorks;
- Goal: Individual workstations and laptops should be registered in order to prevent un-authenticated access to our networks.
- report any suspicious activity to the Security Team; and
- realize that the Security Team can revoke access of any user or system on which a compromise or malicious activity is detected.
Zone 3b: NPCF Offices
The NPCF building houses several offices whose traffic should be handled separately from that of the main datacenter zone. Command Center systems that are not dedicated CnC systems for Zone 1 machines, i.e., networked only to internal subnets, will be treated as members of Zone 3b.
NPCF Office systems share the same security requirements as NCSA Building systems (Zone 3a) but are a separate sub zone because they will be physically monitored at separate points.
Zone 4a: Wireless
NCSA Wireless connectivity is available in all of our buildings (ACB, NCSA & NPCF). For campus wireless networks, all traffic is tunneled outside the NCSA and treated as external traffic. For this reason, they do not fall under any NCSA zone and are not explicitly addressed.
There are both NCSA and University provided wireless networks in the NCSA and the NPCF buildings. These wireless networks are split into multiple logical zones. These networks all have different authentication mechanisms, rules about guest accounts and may or may not utilize link-level encryption. The security requirement for NCSA wireless networks (those that are managed by NCSA or provide NCSA IP addresses) is simply that an adversary gains no additional benefit by physically locating themselves within wireless range of NCSA facilities. For example, by using the strong encryption and authentication technologies of WPA2 or VPNs, it can be reasonably assumed that an attacker already needs a valid Kerberos credential which would give them access to this zone through the VPN anyway.
It is strongly suggested that the UIUC-Guest network be made available in NCSA and NPCF for the use of the majority of visitors and guests. It is also strongly desired that we deprecate the NCSA-Portal network as it potentially gives adversaries additional advantages of proximity.
Types of Systems:
Generally, these systems will be laptops, tablets, and/or smart phones owned by NCSA employees or guests.
Guest laptops are likely candidates for victimization by malicious network traffic, email/web client attacks or viruses. Therefore, the preferred way for guests to obtain Internet access is through the EDURoam, UIUC Public Wifi (where available) or by getting a guest account on UIUCNet, which has good coverage all over campus. These networks are run by CITES and traffic is tunneled outside the NCSA network. Therefore, guest systems would not logically be a part of the NCSA network and would be treated as external Internet traffic.
Requirements that apply to ALL wireless networks in NCSA buildings (whether operated by NCSA or CITES) are:
- wireless networks in NCSA buildings must not give an adversary without NCSA authentication credentials an advantage over simply attacking from the Internet;
- only the NCSA and/or CITES network teams can configure access points and networking hardware for the wireless network -- there will be no rogue or unapproved wireless networks;
- the Security Team will regularly audit/scan for rogue access points; and
- use and configuration of the wireless networks must be compliant with all relevant NCSA and University policies.
Zone 4b: VPN
NCSA offers a VPN service for employees working remotely.
Systems connected to the NCSA VPN networks are treated as external systems with monitoring on the NCSA side of the VPN tunnel.
Zone 5: Blue Waters Management VLANs
Sanitized from this copy
Zone 6: Blue Waters Service VLANs
Sanitized from this copy
Zone 7: Physical Security Systems
This is an island zone only for the NPCF physical security systems.
Types of Systems:
All NPCF physical security systems, and only those systems, are part of this zone. This includes the camera DVRs, the badge readers, the iris scanners, the ACMS workstations (for badging, control and enrollment), and the ACMS database server.
- Devices on this network are on their own physical network, but they are bridged-together private VLANs on shared routers and switches;
- This VLAN is completely private and does not allow for access to or from the outside world, except for occasional VPN access into this zone that is provided to allow the physical security contractors (ITG) to perform maintenance. This VPN access is turned off unless maintenance is being performed; and
- VPN access is also restricted to a small number of remote IP addresses and requires a strong authentication for access.
It should be noted that there will hopefully be a way for alerts to be passed out of this network in the future, but this connection should be unidirectional.
Zone: Island / PSP (Private Sector Program)
This defines a type of zone rather than a specific zone itself. An Island zone has its own, unmonitored external bandwidth either provided by a 3rd party or too large to be monitored by the border NIDS. The island may have an additional, smaller bandwidth connection directly into NPCF zones that will be monitored with the NIDS and treated as external traffic. This includes site-to-site VPN traffic traversing NCSA's external links. This traffic must not be routed inside the NCSA network without also being monitored by a NIDS, and it will be trusted no more than other Internet traffic.
Types of Systems:
These are systems (typically servers) for special projects or private sector partners that bring in their own systems to physically reside in NPCF, but that also require their own external connectivity. Other large systems that have their own pipes, but do not wish to pay for monitoring, can opt for an island zone.
- These zones have their own outside connectivity (preferably non-NCSA IPs), and no direct connection to our network. For the purposes of security monitoring they are treated as external systems;
- If the owners/administrators of systems on this network require security monitoring beyond that provided if/when traffic from these systems crosses into an NCSA network, that monitoring must be coordinated with the Security Team and any additional costs of that monitoring borne by the owners/administrators of the systems.