...
Previous Outages or Maintenance
2021-11-30 0900 | 2021-11-30 1100 | TechServices connectivity at NPCF (wireless, facilities, IRIS, Prox scanners). | Tech Services will be replacing several network devices at NPCF that will impact a variety of services at NPCF. | The Tech Services will be replacing 3 devices at NPCF. Along with sporadic wireless outages, some facilities networks (such as IRIS and card readers) will be offline while some equipment is replaced. The main router replacement should only take 5 mins or so. The wireless switches will take 15-20 mins each. | help+neteng@ncsa.illinois.edu |
| ||||||||
2021-11-30 1800 | 2021-12-01 00:15 | NCSA File & Print Servers | Scheduled Windows Server Maintenance | File & Print Shares were unavailable during maintenance. Users were not able to access shares on Fileserver (e.g. home, busnoff, hr, etc.), and printing was unavailable. | help+service@ncsa.illinois.edu |
| ||||||||
2021-11-19 12:52 | 2021-11-19 13:22 | lsst-esx08 | server crashed | The following VMs rebooted: Idap-Isst-ncsa3 | lsst-admin@ncsa.illinois.edu |
| ||||||||
2021-11-18 1400 | 2021-11-18 1750 | ICI Metrics & Alerts | Migration to RHEL 8, ASD Puppet control, & CILogon authentication | The viewing of ICI dashboards and the firing of ICI alerts was unavailable during this migration | malone12@illinois.edu, bglick@illinois.edu |
| ||||||||
2021-11-11 0925 | 2021-11-11 0940 | NCSA website | Communications launched the newly redesigned NCSA site. | During launch, you may experience some down-time between while NCSA’s technical team re-points the URL to the new site. | communications@lists.ncsa.illinois.edu |
| ||||||||
2021-11-09 0700 | 2021-11-09 1545 | iForge | Quarterly Maintenance | All systems unavailable | iforge-admin@lists.ncsa.illinois.edu |
| ||||||||
2021-11-03 0000 | 2021-11-04 | Netdot SSL Certificate | The SSL certificate for Netdot expired and network engineering replaced it with a new one. | SSL certificate expired. Service remained available throughout the period | help+neteng@ncsa.illinois.edu |
| ||||||||
2021-11-03 1100 | 2021-11-03 1400 | ESnet 100G link migration. | ESnet engineers will be migrating NCSA's 100G link to the new ESnet6 infrastructure. | The link will be down during the migration. Traffic will fall back to alternative paths. | help+neteng@ncsa.illinois.edu |
| ||||||||
2021-11-03 1100 | 2021-11-03 1120 | NCSA GitLab | GitLab was updated to latest version. | All GitLab services were be unavailable | help+service@ncsa.illinois.edu |
| ||||||||
2021-11-03 1000 | 2021-11-03 1020 | Core Router Linecard Replacement | Neteng replaced a linecard in one of the core routers | All connections to this linecard are redundant and no outage has been reported. | neteng@ncsa.illinois.edu |
| ||||||||
2021-11-02 15:20 | 2021-11-02 16:37 | Production version of DCIM for CMDB (https://ncsa-cmdb.ncsa.illinois.edu) | Invalid certificate issue | (Fixed) The production version of CMDB will be unavailable until new certificate is received and applied. In the interim, the test server (https://ncsa-cmdb-test.ncsa.illinois.edu) has been made available for use, with all current data. | Kimber Blum (kimber7@illinois.edu) |
| ||||||||
2021-11-02 0800 | 2021-11-02 0900 | cilogon.org | Update to OA4MP v5.2.3 | Address several small issues in the back-end service | help@cilogon.org |
| ||||||||
0600 | 0710 | Jira | Jira Upgrade | Jira | help+service@illinois.edu |
| ||||||||
2021-10-25 1800 | 2021-10-26 0018 | NCSA File & Print Servers | Scheduled Windows Server Maintenance | File & Print Shares will be unavailable during maintenance. Users will not be able to access shares on Fileserver (e.g. home, busnoff, hr, etc.), and printing will be unavailable. | help+service@ncsa.illinois.edu |
| ||||||||
2021-10-20 0800 | 2021-10-20 1800 | ICCP | ICCP Quarterly Maintenance
| ICCP Cluster nodes only | help@campuscluster.illinois.edu |
| ||||||||
2021-10-20 0700 | 2021-10-20 0715 | IDDS | IDDS maintenance (puppet changes) | All IDDS services | idds-admin@ncsa.illinois.edu |
| ||||||||
2021-10-15 1230 | 2021-10-15 0713 | NCSA GitLab | Server ran out of disk space | All GitLab services were unavailable | help+service@ncsa.illinois.edu |
| ||||||||
2021-10-11 0800 | 2021-10-11 1900 | Nightingale, ACHE | Planned maintenance on the Nightingale cluster and the ache-dist switch | There was an outage for the following services during the maintenance:
| help+service@ncsa.illinois.edu |
| ||||||||
2021-10-04 1000 | 2021-10-04 1005 | www.ncsa.illinois.edu per-user web directories | Per-user web directories on the main NCSA website are being redirected to a new website dedicated to per-user web directories. | URLs like www.ncsa.ncsa.illinois.edu/People/* are redirected to their new home at https://users.ncsa.illinois.edu/*. | help+service@ncsa.illinois.edu |
| ||||||||
2021-09-30 0800 | 2021-09-30 1200 | LSST | LSST Quarterly Maintenance
| All LSST services hosted at NCSA | lsst-admin@ncsa.illinois.edu |
| ||||||||
2021-09-29 0800 | 2021-09-29 0900 | cilogon.org | Update to OA4MP v5.2.2 | Update Java database libraries, and address several small issues | help@cilogon.org |
| ||||||||
2021-09-29 0800 | 2021-09-29 0813 | CMDB / openDCIM | Installing/upgrading to CMDB release Sep2021 | The openDCIM front end of CMDB will be down for 15-30 minutes |
| |||||||||
2021-09-28 0700 | 2021-09-28 1554 | NPCF work on facility power | Deenergizing power to transformer TX-4C-1020, pulling and terminating busduct cabling from transformer to room 2020. | One third of Sonexion racks will lose source 1 power (Feed C) and will continue to operate on source2 degrading reliability by losing power redundancy. |
| |||||||||
2021-09-28 0700 | 2021-09-28 0900 | Blue Waters | A rack of scratch lost power during the power outage. | Scratch was partially unavailable due to TOR power resiliency issue. |
| |||||||||
2021-09-28 0800 | 2021-09-28 0900 | idp.ncsa.illinois.edu | Assert eduPersonAssurance Cappuccino profile for NCSA Staff | NCSA Staff logging in with the NCSA Identity Provider will be able to get Silver CA certificates from cilogon.org | help+idp@ncsa.illinois.edu |
| ||||||||
2021-09-21-14:50 | 2021-09-21-15:02 | vcenter appliance controlling ASD vsphere | vcenter appliance was upgraded | vsphere.ncsa.illinois.edu was off-line for 12 minutes. | help+service@ncsa.illinois.edu |
| ||||||||
2021-09-21 0700 | 2021-09-20 1115 | Blue Waters | Power Work caused non redundant switches and misconfigured servers to shutoff | Blue Waters Compute, Login and Scheduler | bw-admin@ncsa.illinois.edu |
| ||||||||
2021-09-20 1800 | 2021-09-20 2130 | NCSA File & Print Servers | Scheduled Windows Server Maintenance | File & Print Shares were unavailable during maintenance. Users were not be able to access shares on Fileserver (e.g. home, busnoff, hr, etc.), and printing was unavailable. | help+service@ncsa.illinois.edu |
| ||||||||
2021-09-14 0000 | 2021-09-14 0600 | Internet2 WAN circuit | Internet2 will be migrating our WAN circuit to new hardware. | Traffic over that path will reroute while the change happens. We anticipate the migration to take less than 30 mins. | help+neteng@ncsa.illinois.edu |
| ||||||||
0600 | 0900 | Wiki | Upgrade to next version | Wiki will be unavailable |
| |||||||||
2021-09-09 0600 | 2021-09-09 0700 | NCSA VPN | Software Upgrades | The appliances hosting the NCSA VPN will be patched. Users will experience a brief disconnect as load is failed over between the appliances. | help+neteng@ncsa.illinois.edu |
| ||||||||
2021-09-08 1300 | 2021-09-08 1400 | Group prod_b Bastion hosts | Out of cycle patching | Bastion hosts in group prod_b will be patched and rebooted. (see MOTD for group assignment) | help+security@ncsa.illinois.edu |
| ||||||||
2021-09-08 0900 | 2021-09-08 1000 | Group prod_a Bastion hosts | Out of cycle patching | Bastion hosts in group prod_a will be patched and rebooted. (see MOTD for group assignment) | help+security@ncsa.illinois.edu |
| ||||||||
2021-09-02 9:30 AM | 2021-09-02 1PM | PDU in rack AA81 | We are replacing a PDU in NPCF rack AA81 | All systems in the rack have redundant power connections. No service outages are expected from this work | help+service@ncsa.illinois.edu |
| ||||||||
2021-09-01 0700 | 2021-09-01 0800 | cilogon.org | Update to OA4MP v5.2.1 | Device Authorization Grant Flow transactions will be stored in database rather than in memory | help@cilogon.org |
| ||||||||
1200 | 1205 | Wiki | Security patch is being applied | Wiki will be down | help+service@ncsa.illinois.edu |
| ||||||||
2021-08-25 9:00am | 2021-08-25 6:45pm | Blue Waters | System reboot due to blade fallout coinciding with HSN reroute and SMW not recovering. | All jobs interrupted | jenos@illinois.edu |
| ||||||||
2021-08-19 0538 | 2021-08-19 0700 | IRST systems hosted on IRST Node 2 | Storage controller failure, all VMs taken offline | some prod_b systems, and non-redundant services. | eyrich@illinois.edu |
| ||||||||
2021-08-19 5:34 | 2021-08-19 6:20 | cilogon.org | Storage controller failure in IRST VM farm | cilogon.org was unreachable until we initiated fail-over to our backup servers at NICS. | help@cilogon.org |
| ||||||||
2021-08-18 1136 | 2021-08-18 1156 | NCSA Wiki | Test instance caused interference. | NCSA Wiki | help+service@ncsa.illinois.edu |
| ||||||||
2021-08-17 0500 | 2021-08-17 0700 | NCSA/NPCF Wide Area Network | Between 5:00AM and 7:00 AM CDT on 08/17/2021, Campus ICCN Engineers will be upgrading firmware on the ICCN router 710rtr at the Starlight facility in Chicago. | Our peerings with MREN and OmniPoP will go down. All traffic destined for those peerings will reroute via other peerings, so no production impact is expected. | help+neteng@ncsa.illinois.edu |
| ||||||||
2021-08-16 1800 | 2021-08-17 0000 | NCSA File & Print Servers | Scheduled Windows Server Maintenance | File & Print Shares will be unavailable during maintenance. Users will not able to access shares on Fileserver (e.g. home, busnoff, hr, etc.), and printing will be unavailable. | help+service@ncsa.illinois.edu |
| ||||||||
2021-08-12 9:54 | 2021-08-12 1012 | Jira | Attempted snapshot of Jira in vSphere was too intensive for the system | Jira | help+service@illinois.edu |
| ||||||||
2021-08-10 2000 | 2021-08-011 0000 | Radiant API and Web access | Radiant cluster name change. | During this time access to the API endpoints and the Horizon web dashboard will be intermittently unavailable. Instances will continue to run and be available over the network with no interruptions. | radiant-admin@ncsa.illinois.edu |
| ||||||||
2021-08-10 07:00 | 2021-08-10 17:10 | iForge | Quarterly Maintenance | All systems unavailable | iforge-admin@lists.ncsa.illinois.edu |
| ||||||||
2021-08-09 1421 | 2021-08-09 1440 | NCSA Wiki | DB conflict configuration with Wiki & Wiki-Test | NCSA Wiki was unaccessible | help+service@ncsa.illinois.edu |
| ||||||||
2021-08-05 1000 | 2021-08-05 1030 | NPCF Core Router - Linecard Reboot | A problem was identified on one of the line cards in our core router requiring a reboot of the linecard. The linecard was successfully rebooted and we will continue monitoring the hardware for further issues. | All connections to this linecard are redundant and there was no impact to users. | neteng@ncsa.illinois.edu |
| ||||||||
2021-08-05 0800 | 2021-08-05 1000 | LSST | LSST Emergency OS Patching | LSST services hosted at NCSA except:
| lsst-admin@ncsa.illinois.edu |
| ||||||||
2021-08-04 0800 | 2021-08-04 1700 | Radiant API and Web access | Installation of new Radiant cluster Cluster name changes are starting at 1100; This will make the horizon dashboard unreachable. | During this time access to the API endpoints and the Horizon web dashboard will be intermittently unavailable. Instances will continue to run and be available over the network with no interruptions. | radiant-admin@ncsa.illinois.edu |
| ||||||||
2021-08-04 0700 | 2021-08-04 0800 | cilogon.org | Update to OA4MP v5.2.0 | Added support for Device Authorization Grant Flow (RFC 8628) | help@cilogon.org |
| ||||||||
2021-08-03 0800 | 2021-08-03 1700 | Radiant API and Web access | Installation of new Radiant cluster | During this time access to the API endpoints and the Horizon web dashboard will be intermittently unavailable. Instances will continue to run and be available over the network with no interruptions. | radiant-admin@ncsa.illinois.edu |
| ||||||||
2021-08-03 9:00 am | 2021-08-03 11:30 am | Radiant Cluster | A change was made to the firewall that unintentionally restricted access for instances and other internal cluster communication. | Access to instances and workload | radiant-admin@ncsa.illinois.edu |
| ||||||||
2021-07-31 0600 | 2021-07-31 0630 | CILogon hosted services | Infrastructure maintenance | During this time each service hosted by CILogon including COmanage Registry, LDAP, Grouper, SAML proxy, and MDQ will become unavailable for a short time. Each individual service outage will last less than 5 minutes. Services that will not be impacted include: * OIDC clients that do not query LDAP for resolving attributes * X.509 certificate issuance and certificate revocation lists * LIGO and GW-Astronomy services | help@cilogon.org |
| ||||||||
2021-07-29 1300 | 2021-07-29 1400 | IRST-run bastion hosts (pool B) | Security patching | Hosts managed by IRST will be patched and rebooted. Only hosts in pool B will be patched at this time | help+security@ncsa.illinois.edu |
| ||||||||
2021-07-29 0900 | 2021-07-29 1000 | IRST-run bastion hosts (pool A) | Security patching | Hosts managed by IRST will be patched and rebooted. Only hosts in pool A will be patched at this time | help+security@ncsa.illinois.edu |
| ||||||||
2021-07-28 1000 | 2021-07-28 1050 | LSST | OS Updates on only NCSA Test Stand (NTS) | Only the LSST NCSA Test Stand (NTS) services hosted at NCSA | lsst-admin@ncsa.illinois.edu |
| ||||||||
2021-07-27 0600 | 2021-07-27 0900 | Jira | Upgrade | Jira will be unavailable |
| |||||||||
2021-07-26 1800 | 2021-07-27 0000 | NCSA File & Print Servers | Scheduled Windows Server Maintenance | File & Print Shares were unavailable during maintenance. Users were not able to access shares on Fileserver (e.g. home, busnoff, hr, etc.), and printing was unavailable. | help+service@ncsa.illinois.edu |
| ||||||||
2021-07-21 0800 | 2021-07-21 2900 | ICCP | ICCP Quarterly Maintenance
| All ICCP services | help@campuscluster.illinois.edu |
| ||||||||
2021-07-21 15:24 | 2021-07-21 21:50 | ASD Vshpere cluster in 3003 | One of the 4 hypervisors in the cluster paniced. Unscheduled preventative maintenance is being preformed on it and the other 3 nodes in the cluster. | after the initial outage at 15:24, there should be no additional outages. | help+service@ncsa.illinois.edu |
| ||||||||
2021-07-13 0700 | 2021-07-13 0800 | cilogon.org | Update to OA4MP v5.1.4. | The OAuth2/OIDC backend of the CILogon Service will be updated to OA4MP v5.1.4. | help@cilogon.org |
| ||||||||
2021-07-08 0800 | 2121-07-08 1000 | OpenAFS | The remaining OpenAFS database servers were upgraded. | No service impacts were seen | help+service@ncsa.illinois.efu |
| ||||||||
2021-07-07 0600 | 2021-07-07 0800 | CILogon AWS Hosted Services | Upgrading AWS RDS Aurora MySQL v5.6 to v5.7 | COmanage Registry and Grouper services hosted by CILogon will be unavailable | help@cilogon.org |
| ||||||||
2021-07-01 2140 | 2021-07-01 1430 | Horizon dashboard access was down for the entire period. Cluster networking was down from 1200 to1430. | Investigations into Horizon dashboard accessibility issues resulted in the application of an incorrect default network gateway for the cluster around noon. This was corrected and networking functionality restored around 1400. Instances began recovering soon thereafter. | Radiant admins believe running instances have recovered on their own but we advise everyone to check their systems and report any issues they see to the help desk. | help@ncsa.illinois.edu |
| ||||||||
2021-07-01 0247 | 2021-07-01 1300 | Various systems in NPCF, ACB, NCSA | There was a power event in the Champaign-Urbana area at around 2:47AM today. Details about the cause are currently unknown. This event caused disruptions to systems at the NCSA building, NPCF and ACB. Known issues have generally been resolved but there may be unidentified issues lingering. If you encounter any problems, please notify NCSA help desk staff (help@ncsa.illinois.edu). | Multiple systems/services were impacted. All have been recovered and return to normal operations is complete. | NCSA help desk |
|
2021-06-29 22:00 | 2021-06-29 23:59 | NCSA 4th Floor Office network | Rebooting one or more of the office switches on the NCSA Building 4th floor to resolve a phone issue. | Office port connectivity will be intermittent during the maintenance window. | Matt Kollross | RESOLVED |
2021-06-24 0800 | 2021-06-24 1345 | LSST |
| Prod/Stable K8S | lsst-admin@ncsa.illinois.edu | RESOLVED |
2021-06-24 0800 | 2021-06-24 1200 | LSST | LSST Quarterly Maintenance
| All LSST services hosted at NCSA EXCEPT Prod/Stable K8S | lsst-admin@ncsa.illinois.edu | COMPLETE |
2021-06-22 0000 | 2021-06-22 0400 | Internet2 WAN link | Internet2 will be migrating NCSA's physical port to their new next generation infrastructure. | During the maintenance, our I2 connection will be down. Traffic will reroute to other connections. Some point to point connections maybe unavailable for period of time. The maintenance window is not expected to take all 4 hours. | Matt Kollross | COMPLETE |
2021-06-21 1800 | 2021-06-22 0000 | NCSA File & Print Servers | Scheduled Windows Server Maintenance | File & Print Shares were unavailable during maintenance. Users were not able to access shares on Fileserver (e.g. home, busnoff, hr, etc.), and printing was unavailable. | help+service@ncsa.illinois.edu | COMPLETE |
2021-06-17-0700 | 2021-06-17-0820 | OpenAFS | The OpenAFS database server kaskaskia was upgraded | No service outages were observed or reported. | help+service@ncsa.illinois.exdu | COMPLETE |
2021-06-12 2200 | 2021-06-15 1500 | LSST Firewall | The NPCF secondary firewall was offline due to a hard drive failure. | No impact occurred to production services as the primary firewall stayed online. | RESOLVED | |
2021-06-14 1700 | 2021-06-15 0958 | NCSA GitLab | Attempt to fix an authentication bug for a particular user accidentally broke all authentication through the web interface, | Authentication through the web interface did not work. | help+service@ncsa.illinois.edu | RESOLVED |
2021-06-11 | 2021-06-11 0905 | NCSA Jira | Jira email problem | Jira is not accepting issues via email, you can still create issue directly via Jira GUI | RESOLVED | |
2021-06-10 0700 | 2021-06-10 0800 | cilogon.org | Update to OA4MP v5.1.3. | The OAuth2/OIDC backend of the CILogon Service will be updated to OA4MP v5.1.3. | help@cilogon.org | COMPLETE |
1000 | 1030 | Jira.ncsa.illinois.edu | Configuration change to address a vulnerability | There should not be any service interruption, but as with all things, it is possible | help+service@ncsa.illinois.edu | RESOLVED |
2021-06-02 | 2021-06-02 | Netdot | Netdot web access now requires 2FA via SSL VPN, or Cerberus proxy. | Security requested that Netdot require 2FA, in order to access the web interface. To accommodate that request, the Netdot firewall has limited web access to the VPN subnet or via proxy from the Cerberus jump hosts. | Matt Kollross | RESOLVED |
2021-05-25 | 2021-05-26 | vcenters for ache and ASD | emergency security updates were applied. | the administrative interface was off-line for about 20 minutes as the updates were installed. | help+service@ncsa.illinois.edu | RESOLVED |
2021-05-26 1000 | 2021-05-26 1030 | VoIP phones at NPCF | Migrating the VoIP networks to a campus IP to enable future migrations by tech services. | After the networks are migrated, a reboot all phones at the NPCF building will be performed. | Matt Kollross | RESOLVED |
2021-05-21 1800 | 2021-05-21 1900 | VoIP phones at the NCSA building | Migrating the VoIP networks to a campus IP to enable future migrations by tech services. | After the networks are migrated, a reboot all phones at the NCSA building will be performed. | Matt Kollross | RESOLVED |
2021-05-20 05:40 | 2021-05-20 08:45 | LSST | ESXi host outage causing degradation of select services. | Degradation of select services:
Also loss of redundancy for some underlying services, including auth/access & k8s head nodes. | lsst-admin@ncsa.illinois.edu | RESOLVED |
2021-05-15 0600 | 2021-05-15 0800 | CILogon hosted services including COmanage Registry, LDAP, SAML proxy, SAML AA, MDQ | Maintenance | All CILogon hosted services were temporarily unavailable. | help@cilogon.org | COMPLETE |
2021-05-12 07:00 | 2021-05-12 08:00 | NCSA Internal Web Server Upgrade (aka Savannah or MIS Tools) | Updates were made that will affect the availability of the NCSA internal website and Savannah system. The system was be unavailable during this time. | COMPLETE | ||
2021-05-11 07:00 | 2021-05-11 19:00 | iForge | Quarterly Maintenance | All systems unavailable | COMPLETE | |
2021-05-06 0900 | 2021-05-06 0945 | WAN Link Migration | NCSA Neteng migrated the WAN link to Internet 2 to new hardware. | Traffic was automatically re-routed to redundant paths during the link outage. Any connections relying on layer-2 connections over AL2S saw a brief blip as the connection is cut over. Affected parties were contacted in advance. | help+neteng@ncsa.illinois.edu | COMPLETE |
2021-05-03 0600 | 2021-05-03 0630 | CILogon Multi-tenant COmanage Registry | Upgrade to version 3.3.2 | The service at https://registry.cilogon.org was unavailable | help@cilogon.org | COMPLETE |
2021-04-29 1600 | 2021-04-29 1700 |
| Add new nodes into Condor service pools |
| lsst-admin@ncsa.illinois.edu | COMPLETE |
2021-04-21 08:00 | 2021-04-21 20:00 | ICCP | ICCP Quarterly Maintenance | The scheduler will be down. All compute nodes will be converted to rhel7.9 with RedHat IB. | COMPLETE | |
2021-04-15 1600 | 2021-04-15 1700 | NCSA Opensource | Upgrade of OS on all machines related to opensource | jira, wiki, git etc hosted at https://opensource.ncsa.illinois.edu/ | kooper@illinois.edu | COMPLETE |
2021-04-15 12:25 | 2021-04-15 14:45 | ICI vmware | Several hosts on the vmware service were experiencing timeouts
| no or intermittent connectivity to these hosts | help+service@ncsa.illinois.edu | RESOLVED Root cause is still being investigated. |
2021-04-15 0900 | 2021-04-15 0942 | CMDB | Applying new certificates and restarting services | CMDB, including web interface, will be down briefly during the update. | ncsagroup+org_itsm@ncsa.illinois.edu | RESOLVED |
2021-04-15 0900 | 2021-04-15 0920 | WAN Link Migration | NCSA Neteng will migrated the WAN link to ESnet to new hardware. | Traffic was automatically re-routed to redundant paths during the link outage. | help+neteng@ncsa.illinois.edu | RESOLVED |
2021-04-14 15:00 | 2021-04-14 15:00 | git.ncsa.illinois.edu | Users can no longer access repositories from git clients over HTTPS using their NCSA password. | NCSA passwords can not access repositories with Git clients. Instead use ssh keys over SSH or personal access tokens over HTTPS. We thought this went into effect during git changes on Nov 2, 2020 but discovered it was still working until we made changes to GitLab to fully remove LDAP functionality. | help+service@ncsa.illinois.edu | COMPLETE |
2021-04-13 1415 | 2021-04-13 1845 | git.ncsa.illinois.edu | The GitLab website at git.ncsa.illinois.edu was having issues with authentication. The LDAP server that it uses was timing out. |
| help+service@ncsa.illinois.edu | RESOLVED |
2021-04-13 0800 | 2021-04-13 0830 | cilogon.org | Update to OA4MP v5.1.1. | The OAuth2/OIDC backend of the CILogon Service will be updated to OA4MP v5.1.1. | help@cilogon.org | COMPLETE |
2021-04-12 1800 | 2021-04-12 2245 | File & Print Servers | Monthly Windows File & Print Server Maintenance | Windows File Shares such as HR, Business Office, Home, etc. and printing in the NCSA & NPCF buildings were unavailable. | help+service@ncsa.illinois.edu | COMPLETE |
2021-04-10 0600 | 2021-04-10 0800 | CILogon hosted COmanage, Grouper, SATOSA, LDAP | On Saturday, April 10, the CILogon team will perform maintenance on the infrastructure used for hosted services. | As part of the maintenance all COmanage Registry, LDAP, Grouper, SAML proxy, SAML attribute authority, and MDQ services hosted by CILogon may experience brief outages. We do not expect that any specific service outage will last for more than a minute. | help@cilogon.org | COMPLETE |
2021-04-08 0900 | 2021-04-08 1045 | WAN Link Migration | NCSA Neteng migrated the WAN link to ICCN Node-1 to new hardware. | Traffic was automatically re-routed to redundant paths during the link outage. Issues were noticed by users during the outage and are currently being investigated in cooperation with our upstream provider. | help+neteng@ncsa.illinois.edu | COMPLETE |
2021-04-08 0730 | 2021-04-08 0734 | NCSA Wiki | NCSA's Wiki service was restarted | NCSA's Wiki service was restarted to apply a new SSL certificate and renewed Confluence license. The wiki was not available for 4 minutes while it reloaded. | help+service@ncsa.illinois.edu | COMPLETE |
2021-04-07 1610 | 2021-04-07 1733 | Internal Savannah/MIS website | The Savannah/MIS website would not load due to a corrupted MySQL database table referenced across all of the Savannah tools. | Internal/Savannah | help+service@ncsa.illinois.edu | RESOLVED |
1st report 7:30am Monday | 8:19am Monday | NCSA LDAP2 | ldap2 is not responsive to authentication requests | NCSA Jira, any systems using LDAP2 as its only source. | help+service@ncsa.illinois.edu | RESOLVED |
2021-03-30 0800 | 2021-03-30 0845 | DNS1 | A software issue was causing BIND to fail. | DNS was not able to resolve during the period of time. DNS2 remained operational. | neteng+help@ncsa.illinois.edu | RESOLVED |
2021-03-23 2000 | 2021-03-23 2025 | NCSA VPN | The standby VPN hardware was replaced and transitioned into the current VPN cluster. Failover went as expected and firmware was upgraded on the primary after load was shifted to the new standby VPN. | Failover between the appliances occurred without issue and there was no impact to users. | neteng@ncsa.illinois.edu | RESOLVED |
2021-03-18 1230 | 1255 | Jira | Some functionality will be limited due to user limit being reached | Jira | help@service@ncsa.illinois.edu | RESOLVED |
~16:40 | 17:58 | AnyConnect VPN Service | An issue with SSL on the VPN service has caused an issue that has disconnected all users. Network engineering is looking into the issue. Due to a hardware failure and the VPN not failing over properly to the standby users were unable to connect to the VPN. This was due to an issue with syncing certificates. | During the outage, expect that you won't be able to connect/maintain a connection to the VPN | help+neteng@ncsa.illinois.edu | RESOLVED |
2021-03-16 0950 | 2021-03-16 1000 | CMDB | Will be applying updates per security vetting | CMDB, including web interface, will be down briefly during the update. | ncsagroup+org_itsm@ncsa.illinois.edu | RESOLVED |
2021-03-11 | 2021-03-11 | WAN Link Migration | NCSA Neteng migrated the link to ICCN to new hardware. | Traffic was automatically re-routed to redundant paths during the link outage. | help+neteng@ncsa.illinois.edu | RESOLVED |
2021-03-04 | 2021-03-04 | WAN Link Migration | NCSA Neteng migrated the 100G link to MREN to new hardware. | Traffic was automatically re-routed to redundant paths during the link outage. | help+neteng@ncsa.illinois.edu | RESOLVED |
2021-03-01 22:11 | 2021-03-01 22:47 | NCSA vSphere | About 40 VMs lost connection to their NFS storage. | Several VM-based services were timing out during the issue, including: vSphere management, a kerberos replica, a ldap replica, httpproxy, license servers, NCSA fileserver, Identity message queuing, monitoring. That triggered some of those VMs to switch to use read-only disk, needing to be rebooted later. | service@ncsa.illinois.edu | RESOLVED |
2021-04-29 1600 | 2021-04-29 1700 |
| Add new nodes into Condor service pools |
| lsst-admin@ncsa.illinois.edu | COMPLETE |
2021-04-21 08:00 | 2021-04-21 20:00 | ICCP | ICCP Quarterly Maintenance | The scheduler will be down. All compute nodes will be converted to rhel7.9 with RedHat IB. | COMPLETE | |
2021-04-15 1600 | 2021-04-15 1700 | NCSA Opensource | Upgrade of OS on all machines related to opensource | jira, wiki, git etc hosted at https://opensource.ncsa.illinois.edu/ | kooper@illinois.edu | COMPLETE |
2021-04-15 12:25 | 2021-04-15 14:45 | ICI vmware | Several hosts on the vmware service were experiencing timeouts
| no or intermittent connectivity to these hosts | help+service@ncsa.illinois.edu | RESOLVED Root cause is still being investigated. |
2021-04-15 0900 | 2021-04-15 0942 | CMDB | Applying new certificates and restarting services | CMDB, including web interface, will be down briefly during the update. | ncsagroup+org_itsm@ncsa.illinois.edu | RESOLVED |
2021-04-15 0900 | 2021-04-15 0920 | WAN Link Migration | NCSA Neteng will migrated the WAN link to ESnet to new hardware. | Traffic was automatically re-routed to redundant paths during the link outage. | help+neteng@ncsa.illinois.edu | RESOLVED |
2021-04-14 15:00 | 2021-04-14 15:00 | git.ncsa.illinois.edu | Users can no longer access repositories from git clients over HTTPS using their NCSA password. | NCSA passwords can not access repositories with Git clients. Instead use ssh keys over SSH or personal access tokens over HTTPS. We thought this went into effect during git changes on Nov 2, 2020 but discovered it was still working until we made changes to GitLab to fully remove LDAP functionality. | help+service@ncsa.illinois.edu | COMPLETE |
2021-04-13 1415 | 2021-04-13 1845 | git.ncsa.illinois.edu | The GitLab website at git.ncsa.illinois.edu was having issues with authentication. The LDAP server that it uses was timing out. |
| help+service@ncsa.illinois.edu | RESOLVED |
2021-04-13 0800 | 2021-04-13 0830 | cilogon.org | Update to OA4MP v5.1.1. | The OAuth2/OIDC backend of the CILogon Service will be updated to OA4MP v5.1.1. | help@cilogon.org | COMPLETE |
2021-04-12 1800 | 2021-04-12 2245 | File & Print Servers | Monthly Windows File & Print Server Maintenance | Windows File Shares such as HR, Business Office, Home, etc. and printing in the NCSA & NPCF buildings were unavailable. | help+service@ncsa.illinois.edu | COMPLETE |
2021-04-10 0600 | 2021-04-10 0800 | CILogon hosted COmanage, Grouper, SATOSA, LDAP | On Saturday, April 10, the CILogon team will perform maintenance on the infrastructure used for hosted services. | As part of the maintenance all COmanage Registry, LDAP, Grouper, SAML proxy, SAML attribute authority, and MDQ services hosted by CILogon may experience brief outages. We do not expect that any specific service outage will last for more than a minute. | help@cilogon.org | COMPLETE |
2021-04-08 0900 | 2021-04-08 1045 | WAN Link Migration | NCSA Neteng migrated the WAN link to ICCN Node-1 to new hardware. | Traffic was automatically re-routed to redundant paths during the link outage. Issues were noticed by users during the outage and are currently being investigated in cooperation with our upstream provider. | help+neteng@ncsa.illinois.edu | COMPLETE |
2021-04-08 0730 | 2021-04-08 0734 | NCSA Wiki | NCSA's Wiki service was restarted | NCSA's Wiki service was restarted to apply a new SSL certificate and renewed Confluence license. The wiki was not available for 4 minutes while it reloaded. | help+service@ncsa.illinois.edu | COMPLETE |
2021-04-07 1610 | 2021-04-07 1733 | Internal Savannah/MIS website | The Savannah/MIS website would not load due to a corrupted MySQL database table referenced across all of the Savannah tools. | Internal/Savannah | help+service@ncsa.illinois.edu | RESOLVED |
1st report 7:30am Monday | 8:19am Monday | NCSA LDAP2 | ldap2 is not responsive to authentication requests | NCSA Jira, any systems using LDAP2 as its only source. | help+service@ncsa.illinois.edu | RESOLVED |
2021-03-30 0800 | 2021-03-30 0845 | DNS1 | A software issue was causing BIND to fail. | DNS was not able to resolve during the period of time. DNS2 remained operational. | neteng+help@ncsa.illinois.edu | RESOLVED |
2021-03-23 2000 | 2021-03-23 2025 | NCSA VPN | The standby VPN hardware was replaced and transitioned into the current VPN cluster. Failover went as expected and firmware was upgraded on the primary after load was shifted to the new standby VPN. | Failover between the appliances occurred without issue and there was no impact to users. | neteng@ncsa.illinois.edu | RESOLVED |
2021-03-18 1230 | 1255 | Jira | Some functionality will be limited due to user limit being reached | Jira | help@service@ncsa.illinois.edu | RESOLVED |
~16:40 | 17:58 | AnyConnect VPN Service | An issue with SSL on the VPN service has caused an issue that has disconnected all users. Network engineering is looking into the issue. Due to a hardware failure and the VPN not failing over properly to the standby users were unable to connect to the VPN. This was due to an issue with syncing certificates. | During the outage, expect that you won't be able to connect/maintain a connection to the VPN | help+neteng@ncsa.illinois.edu | RESOLVED |
2021-03-16 0950 | 2021-03-16 1000 | CMDB | Will be applying updates per security vetting | CMDB, including web interface, will be down briefly during the update. | ncsagroup+org_itsm@ncsa.illinois.edu | RESOLVED |
2021-03-11 | 2021-03-11 | WAN Link Migration | NCSA Neteng migrated the link to ICCN to new hardware. | Traffic was automatically re-routed to redundant paths during the link outage. | help+neteng@ncsa.illinois.edu | RESOLVED |
2021-03-04 | 2021-03-04 | WAN Link Migration | NCSA Neteng migrated the 100G link to MREN to new hardware. | Traffic was automatically re-routed to redundant paths during the link outage. | help+neteng@ncsa.illinois.edu | RESOLVED |
2021-03-01 22:11 | 2021-03-01 22:47 | NCSA vSphere | About 40 VMs lost connection to their NFS storage. | Several VM-based services were timing out during the issue, including: vSphere management, a kerberos replica, a ldap replica, httpproxy, license servers, NCSA fileserver, Identity message queuing, monitoring. That triggered some of those VMs to switch to use read-only disk, needing to be rebooted later. | service@ncsa.illinois.edu | RESOLVED |
2021-02-25 0800 | 2021-02-25 1200 | LSST | LSST Quarterly Maintenance
| All LSST services hosted at NCSA | lsst-admin@ncsa.illinois.edu | COMPLETE |
2021-02- | 2021-02-25 | WAN Link Migration | NCSA Neteng migrated the 100G link to CARNE to new hardware. | The link to campus through the CARNE router was migrated to new hardware. Traffic was automatically re-routed to redundant paths during each link outage. | help+neteng@ncsa.illinois.edu | COMPLETE |
2021-2-18 2:30 pm | 2021-2-18 6pm | vsphere.ncsa.illinois.edu | logins were broken due to a cert caching issue on vsphere. | login to the administrative interface is available | service@ncsa.illinois.edu | COMPLETE |
2021-02-18 2pm | 2021-02-18 2:30pm | ldap | The certs on several ldap servers were set to expire tomorrow and next week, they were refreshed. | ldap server certs were refreshed prior to their expirations. | help+service@ncsa.illinois.edu | COMPLETE |
2021-02-17. 10:00 a.m. | 2021-02-17. 10:30 a.m. | Netdot maint | patching | Netdot may be unavailable during this time. | help+neteng@ncsa.illinois.edu | COMPLETE |
2021-02-16 0800 | 2021-02-16 1000 | cilogon.org | Update to OA4MP v5.1 had problems. | Several clients reported issues with OA4MP 5.1, so we reverted to OA4MP 4.4.5 at noon. | help@cilogon.org | CANCELED |
2021-02-15 1800 | 2021-02-17 00:40 | File & Print Servers | Monthly Windows File & Print Server Maintenance | Windows File Shares such as HR, Business Office, Home, etc. and printing in the NCSA & NPCF buildings will be unavailable. | help+service@ncsa.illinois.edu | COMPLETE |
2021-02-11 00:00 | 2021-02-11 04:00 | ICCP - MWT2 | “OmniPoP is doing maintenance for their hardware refresh on February 11 (600 W Chicago) between midnight and 4 a.m.CST. This will mean that the CARNE 100G OmniPoP connection will go down for a time during the Feb 11 window. Most of the traffic that would take this link will reroute to using other links. The only ICCP user that may be impacted is MWT2 because their primary path to UChicago is over this circuit. However, we do have a backup UChicago peering over the CARNE Internet2 100G circuit, so that path will be taken assuming that UChicago's backup path doesn't go through the 6WC OmniPoP switch. The tertiary path to UChicago would be through the ESnet LHCONE peering which goes over the CARNE I2 AL2S 100G. MWT2 shouldn’t need to do anything on their side to prepare for this work.” | All traffic should reroute during this maintenance, but MWT2 may experience brief connectivity issues to UChicago | neteng@ncsa.illinois.edu | COMPLETE |
2021-02-09 07:00 | 2021-02-09 15:35 | iforgeftp1 | public interface is down | S3 connection | iforge-admin@lists.ncsa.illinois.edu | COMPLETE |
2021-02-09 07:00 | 2021-02-09 17:00 | iForge cluster | Quarterly Maintenance | All systems unavailable | iforge-admin@lists.ncsa.illinois.edu | COMPLETE |
2021-01-28 10:00am | 2021-01-28 12:00pm | Radiant | Security updates and API endpoint hardening | The web interface to Radiant and the API interfaces will be unavailable during the maintenance period. | radiant-admin@lists.ncsa.illinois.edu | COMPLETE |
2021-01-27 8:00 am | 2021-01-27 8:00 pm | Open Storage Network POD | Updating Ceph to containerized implementation | See Previous Column | bdickin2@illinois.edu | COMPLETE |
2021-01-25 | 2021-01-26 | NCSA LDAP | Users with /bin/csh had their shells changed to /bin/bash | Users logging into systems that don't override the /bin/csh data already will find they are using /bin/bash when they login. | help+service@ncsa.illinois.edu | COMPLETE |
2021-01-26 13:45 | 2021-01-26 14:05 | administrative interface to vsphere.ncsa.illinosi.edu | administrative interface to vsphere.ncsa.illinosi.edu was upgraded to current patch level | Administrative interfaces to vm's were unavailable for about 20 minutes. | help+service@ncsa.illinois.edu | COMPLETE |
2021-01-20 08:00 | 2021-01-20 20:00 | ICCP | ICCP Quarterly Maintenance
| Cluster-wide outage | help@campuscluster.illinois.edu | COMPLETE |
2021-01-12 07:00 | 2021-01-12 8:00 | JIRA | JIRA Upgrade to 8.13.2 | All JIRA users | help+its@ncsa.illinois.edu | COMPLETE |
2021-01-04 09:14 | 2021-01-04 12:10 | Slack | Slack service issues | All Slack systems | https://status.slack.com/ | OUTAGE |
2020-12-21 18:00 | 2020-12-22 0700 | File & Print Servers | Monthly Windows File & Print Server Maintenance | Windows File Shares such as HR, Business Office, Home, etc. and printing in the NCSA & NPCF buildings were unavailable. | help+its@ncsa.illinois.edu | COMPLETE |
2020-12-18 | 2020-12-18 | NCSA LDAP | New user accounts will have their shell in ldap set to /bin/bash | New users will have /bin/bash as their default shell | help+service@ncsa.illinois.edu | COMPLETE |
2020-12-10 0800 | 2020-12-10 1400 | LSST | LSST Quarterly Maintenance
| All LSST services hosted at NCSA | lsst-admin@ncsa.illinois.edu | COMPLETE |
2020-12-09 0800 | 2020-12-10 1000 | HAL | HAL Quarterly Maintenance | HAL cluster | help+isl@ncsa.illinois.edu | COMPLETED |
2020-11-24 10:30 | 2020-11-24 17:35 | Blue Waters compute | Boot RAID lost fiber channel connection for reasons not understood | Full system outage | jenos@illinois.edu | COMPLETED |
2020-11-19 11:00 a.m. | 2020-11-19 11:30 a.m. | DNS1 Migration | Neteng will be migrating DNS1 to a new switch. | We need to physical move the cable to DNS1 which will cause momentary outage for dns queries to DNS1. DNS2 will not be affected by the migration. | help+neteng@ncsa.illinois.edu | COMPLETED |
2020-11-18 09:00 | 2020-11-17 17:00 | UPS battery Monitor | Program and configure the new BMS | All UPS connected loads | rantissi@illinois.edu | COMPLETED |
2020-11-16 18:00 | 2020-11-16 21:30 | NCSA File & Print Servers | File & Print servers were offline for scheduled maintenance. Windows File Shares and printing were unavailable. | Windows File Shares such as HR, Business Office, Home, etc. and printing in the NCSA & NPCF buildings were unavailable. | help+its@ncsa.illinois.edu | COMPLETED |
2020-10-14 13:45 | 2020-10-14 14:04 | NCSA Wiki & Jira | NCSA's Wiki & Jira servers were restarted. | Wiki & Jira were offline while their servers reboot. | help+its@ncsa.illinois.edu | COMPLETED |
2020-11-13 12:00 | 2020-11-13 18:00 | Software Directorate VM Farm | Failing power supply of switch will be replaced, will use this to upgrade OS as well. | NCSA OpenSource, INCORE, etc (all machines running on 141.142.277.X). | kooper@illinois.edu | COMPLETED |
2020-11-12 9:55 | 2020-11-12 10:15 | cilogon.org | Update to OA4MP v5.0.2 was unsuccessful. | CILogon Service has been reverted to OA4MP 4.4.5. | help@cilogon.org | DELAYED |
2020-11-10 07:00 | 2020-11-10 21:05 | iForge cluster | Quarterly Maintenance Switching to GPFS 5 formatted filesystem. | All iForge nodes. | COMPLETE | |
2020-11-05 10:00 | 2020-11-05 12:24 | NPCF enterprise UPS | UPS maintenance, replace defective communication cards | Any rack (system) that is UPS power fed | rantissi@illinoise.edu |
|
2020-11-04 19:00 | 2020-110-4 21:00 | NCSA Building Router (2 of 2) | Software Updates | Software Updates will be applied to one of the NCSA building routers. Traffic will fall back to the seconds router. No network traffic should be affected. | help+neteng@ncsa.illinois.edu | COMPLETED |
2020-11-02 08:00 | 2020-11-02 10:20 | NCSA GitLab git.ncsa.illinois.edu | LDAP authentication was disabled for NCSA GitLab. Users of the GitLab web interface are required to authenticate to NCSA through CILogon. | NCSA passwords can no longer access repositories. Use GitLab personal access tokens to authenticate against Git over HTTPS. | help+its@ncsa.illinois.edu | COMPLETED |
2020-11-02 09:00 | 2020-11-02 09:05 | NCSA Duo | The icon shown in the Duo app for NCSA will be updated to match the icon used in NCSA Slack. | NCSA Duo App pushes will show updated icon to match NCSA Slack. May need to restart phone/app to see updated icon. | help+duo@ncsa.illinois.edu | COMPLETED |
2020-10-30 08:00 | 2020-10-30 08:45 | SVN at subversion.ncsa.illinois.edu | Retired SVN Service at subversion.ncsa.illinois.edu | SVN is no longer be available. NCSA users are recommended to use one of our various Git repository options. | help+its@ncsa.illinois.edu | COMPLETED |
2020-10-29 9:00 | 2020-10-29 11:15 | iForge | maintenance to switch to GPFS version 5. | All nodes. | iforge-admin@ncsa.illinois.edu | COMPLETED |
2020-10-28 19:00 | 2020-10-28 21:00 | NCSA Building Router (1 of 2) | Software Updates | Software Updates will be applied to one of the NCSA building routers. Traffic will fall back to the seconds router. No network traffic should be affected. | help+neteng@ncsa.illinois.edu | COMPLETED |
2020-10-27 22:00 | 2020-10-27 22:10 | NCSA VPN | Firmware Updates | Firmware updates were applied to the NCSA VPN. Any AnyConnect VPN sessions were reset during the maintenance and users may need to reconnect. Any IPSEC sessions failed over to the standby unit and were not affected. | help+neteng@ncsa.illinois.edu | COMPLETED |
2020-10-27 09:00 | 2020-10-27 09:15 |
| The DNS CNAME entries for idp.ncsa.illinios.edu and crl.ncsa.illinois.edu will be changed from cilogon-web.ncsa.illinois.edu to shib-docker.security.ncsa.illinois.edu (141.142.149.33). NCSA Shib IdP v4.0.1 is currently up and running at 141.142.149.33. | help+idp@ncsa.illinois.edu | COMPLETED | |
2020-10-22 11am | 2020-10-22 1pm | security.ncsa.illinois.edu and grid.ncsa.illinois.edu | Cert replacement issue | Sites were down | cpitcel | RESOLVED |
2020 10-21 0900 | 2020 10-21 1400 | WAN Link Migration | ICCN Engineers will be migrating NCSA's 100G WAN links over to new optical cards. Below is the timetable for the moves: 10:00am CARNE (Node 1) to I2 (710 N Lakeshore Dr) 10:30am NCSA (Node 2) to MREN (710 N Lakeshore Dr) 11:30am CARNE (Node 2) to OmniPop (600 West Chicago) 12:00pm NCSA (Node 1) to I2 (600 West Chicago) 12:15pm NCSA (Node 1) to ESNet (600 West Chicago) | Individual links will be migrated one at a time, each taking roughly 15-20 minutes to complete, leaving redundant paths operational. Traffic will automatically be re-routed to these redundant paths during each link outage. There are exceptions where certain services won't failover in this way. In these cases, individual notifications have been sent out to affected parties. | help+neteng@ncsa.illinois.edu | COMPLETED |
2020-10-13 08:00 | 2020-10-13 09:00 | CILogon | Update to OA4MP v5.0 OAuth2/OIDC Libraries encountered issue with Syngenta IdP. Reverted to OA4MP v4.4.5. Will be addressed in future OA4MP update. | https://cilogon.org | help@cilogon.org | DELAYED |
2020 10-13 06:00 | 2020 10-13 06:15 | CILogon COmanage Registry at https://registry.cilogon.org | Service stack restart. | COmanage Registry and LDAP directory for the multi-tenant services. | help@cilogon.org | COMPLETED |
2020-10-12 18:00 | 2020-10-12 21:30 | NCSA File & Print Servers | Monthly Maintenance for Updates / Backup Checks | File & Print Servers were unavailable. Printing was offline, and fileserver shares were unavailable. | help+its@ncsa.illinois.edu | COMPLETED |
2020-10-01 06:00 | 2020-10-01 06:15 | NCSA VPN | The certificate for sslvpn.ncsa.illinois.edu was updated. | The SSL certificate has been updated. | neteng@ncsa.illinois.edu | COMPLETED |
2020-09-28 21:00 | 2020-09-29 16:45 | Nebula | Network card failed in network node, was replaced and network settings reconfigured | All non-virtual networking services for Nebula instances (north/south traffic) | nebula@ncsa.illinois.edu | COMPLETED |
2020-09-23 | 2020-09-24 | LSST | Monthly Maintenance:
| ALL LSST systems | lsst-admin@ncsa.illinois.edu | COMPLETED |
2020-09-22 | 2020-09-22 | CILogon multi-tenant COmanage Registry | Upgrade to version 3.3.0 | COmanage Registry service at https://registry.cilogon.org | help@cilogon.org | COMPLETED |
2020-09-14 18:00 | 2020-09-15 21:30 | NCSA File & Print Servers | Windows file and print servers were patched and unavailable during maintenance. | Access to Fileserver (Business Office, HR, Home, and Swap shared drives) was unavailable, printing was unavailable. | help+its@ncsa.illinois.edu | COMPLETED |
2020-09-08 08:00 | 2020-09-08 09:00 | CILogon | Update CILogon OIDC Client Admin API | https://cilogon.org | help@cilogon.org | COMPLETED |
2020-09-02 0800 | 2020-09-02 1600 | Core-East | Software Upgrades | No user impact expected | help+neteng@ncsa.illinois.edu | COMPLETED |
2020-09-01 | 2020-09-01 | ldaps://ldap.cilogon.org | Restart of LDAP gateway service containers. | All LDAP services operated by CILogon. | help@cilogon.org | COMPLETED |
...