Document Name: NCSA Incident Response Policy
Approved: June 22, 2020
This document represents the policies and procedures in place for handling information security incidents impacting services and infrastructure of the National Center for Supercomputing Applications (NCSA) and associated projects. This includes publicly available services, supporting information and information systems, and infrastructure used byNCSA personnel in support of NCSA and associated projects.
For the purpose of this document, an information security incident (henceforth an "incident") includes any known or suspected event that compromises or has the potential to compromise any NCSA information asset, including computing infrastructure, confidential data, or computing service, as well as flagrant violations of NCSA policy by project personnel, staff, or users of externally-facing services. Incident that do not involve information assets fall outside of the scope of this document.
In order to guide time-sensitive tactical decisions we use the following goals in decreasing order:
These priorities may be adjusted as warranted.
|Initial Responder||IRL/TM||Initial Responder assesses an incident and coordinates response. For non-critical issues most IRST members will follow standard procedures. For unique or critical issues the IRL will determine the appropriate course of action.|
|Communications||TL||The TL or designate will coordinate timely communication with the DO and CISO as situations develop for issues that warrant such notification. The TL or designate will also update the all appropriate communications paths for notifying NCSA users.|
Signs of an incident primarily come from IRST monitoring. Other sources of notification could come from:
The IRST will make a determination as to whether these events and indicators constitute an incident. The IRST will trigger this incident response policy if it determines that the events and indicators do indeed constitute an incident. The severity classifcation and corresponding actions are discussed below.
It is a required that any significant NCSA projects have a point of contact for coordinating security incidents and that this information, including appropriate methods of communication and expected availability, be maintained by both the project and IRST. In cases where a project contact is not specified the assumed contact is the project's Principle Investigator (PI).
Any incidents that need to be reported to the IRST should be done initially through the ticketing system either directly through the ticketing system interface or via email at the following email address: email@example.com. This is to ensure that progress on an incident is tracked through a ticket. If the ticketing system is unavailable the NCSA Help Desk should be contacted to relay information to the appropriate personnel.
Response procedures can not be considered reliable unless they are regularly tested. The first application of an incident response procedure should not be in the heat of an actual incident, however, it will likely occur if new procedures must be developed on the fly.
Procedures dealing with common events should be tested on a regular bases especially when the infrastructure for which this tool either relies on or affects changes. The IRST should have test accounts in all appropriate systems to evaluate the effectiveness of these procedures.
On most occasions these tests should be unannounced and the response times should be recorded for review.
The following non-exhaustive list of procedures should be tested:
NCSA IRST will maintain the following on an ongoing basis in order to facilitate response to an incident. Some of these must be maintained by other parties within NCSA.
The IRST team maintains procedures for incident handling: IRST - Incident Response Procedures. This document provides specific instructions and procedures for IRST.
All security alerts should be tracked through the NCSA Ticketing system, JIRA, specifically in the HELP+SECURITY queue. This can be done through logging in directly to JIRA or by sending an email to firstname.lastname@example.org.
For incidents that require IMMEDIATE attentions contact an IRST TM directly or contact the NCSA Help Desk.
Issues that affect individual projects but not NCSA directly should be reported to the PI of that project.
Incidents may include, but are not limited to:
The IRST team will use its judgement to determine whether events necessitate a triggering of this incident response policy.
Information security incidents are classified based on their perceived impact. A classification should be determined as quickly and carefully as possible and classification may change as understanding of the incident improves. The first person to respond to the incident should attempt to give a first-estimate categorization in order to guide response. The following classifications are based on NIST 800-61 section 3.2.6.
High: an incident is considered High Severity if it involves:
Medium: an incident is considered Medium Severity if it involves:
Low: an incident is considered Low Severity if it involves:
Based on this initial categorization, the following actions should be taken:
High Severity: Project and/or NCSA CISO and primary maintainer of the affected system(s) should be notified immediately. Notification should not be sent blind and attempts to contact a responsible party should continue until a response is confirmed.
Medium Severity: The project and/or NCSA CISO and primary maintainer of the affected system(s) should be notified by voice during business hours and email notification during off-hours.
Low Severity: Email notification should be sent to the project and/or NCSA CISO and primary maintainer of the affected system(s).
In all cases any compromised accounts should be disabled.
Incidents involving electronic Protected Health Information (ePHI) are a special case, which requires extra steps to be followed to guard against the flow of ePHI outside the NCSA Health care Component, i.e. the workforce members who are a part of the covered entity at NCSA.
Any incidents that expose ePHI shall be reported to the NCSA CISO and HIPAA Liaison as soon as confirmation or reasonable inference of exposure is made. Subsequently they will contact University of Illinois and UIUC HIPAA officers.
Discussions regarding an incident involving ePHI will be limited to the parties mentioned above, those in the NCSA Health Care Component and other parts of the University of Illinois covered entity as necessary and authorized.
This applies to any systems with ePHI which are involved in an incident investigation or at anytime a breach is suspected.
It is assumed that any such incident happens on server in the Advanced Computational Healthcare Enclave. If PHI was removed from this system be workforce members outside of the breach workflow above for an incident investigation, then workforce members broke multiple University and NCSA policies and disciplinary action will be in order. We will work with the UIUC HIPAA liaison, NCSA human resource, and NCSa senior management to decide how best to address such a serious policy violation.
In any incident, it is important to act quickly in order to keep damage from spreading. Before or in parallel with the formation of an incident response team (when required), the person doing initial triage should take steps to prevent the problem from spreading to other accounts or resources. This includes isolating systems by blackhole routing the host, switch port disable, or physical disconnection from the network.
The IRL may form an incident response team. The composition of the team will depend on the nature of the incident and may evolve over time. IRT members may include individuals outside of the security group that have specialized skills and/or knowledge of the affected system(s).
Information capture during an incident is essential to fast and appropriate resolution of the incident, as well as to understanding the incident’s cause, working with law enforcement regarding the incident, and doing an effective postmortem. Members of the incident response team should log their actions, on paper if needed to isolate record-keeping from potentially compromised assets, along with times and observations made. Additionally, team members must, whenever possible, keep copies of malicious software found, and other signs of compromise for later analysis. Efforts should be taken to keep affected systems powered on should volatile forensic evidence need to be collected.
Once an incident is contained and all evidence is collected, the IRL will provide the all-clear to restore services of the affected host.
A compromised host ideally should be rebuilt from scratch ensuring that all security updates and applied. Any outstanding vulnerabilities that are not patched, especially those that were used to compromise the host initially, should be mitigated. If mitigation of a current vulnerability is not possible the host should not be returned to service without the expressed risk-acceptance of the system/service administrator, the CISO of the project and the CISO of NCSA.
If a host can not be rebuilt from scratch, the host will need to be patched and go through a vetting by the security team to determine if there are any backdoors or other potential vulnerabilities. The host will not be allowed onto the network until there is risk-acceptance by the system/service administrator, the CISO of the project and the CISO of NCSA.
Once a system/service is restored, including restoration of data from backups, an assessment of the host should be performed to ensure that the initial threat vector used to compromise the system/service is patched/mitigated.
The NCSA CISO is responsible for reporting any incidents to external parties outside of NCSA. The IRL is responsible for reporting incidents to the CISO and the Director's Office.
Please see the contact procedures for external partners: NCSA Security Contact Process.
Following an incident, any issues identified during the incident response process should be reviewed. Existing processes should be checked for accuracy and should be updated as necessary. It is recommended that a formal meeting occur to verbally review the incident to ensure that nothing was missed and all relevant information is recorded.