13:22
status.ncsa.illinois.edu
...
Note |
---|
Watch this page in the wiki to subscribe to automatic updates to this status page. |
NPCF Issues
Date | Time | Duration /msec | Event Type | What |
---|
Happened? |
---|
Effects |
---|
07/ |
05/ |
2024 |
08: |
22: |
19 |
33.99 |
Sag |
Vab 456
Vbc 457
Vca 456
Vab 455
Vbc 456
Vca 455
Vab 461
Vbc 460
Vca 462
Vab 440
Vbc 454
Vca 460
Vab 426
Vbc 403
Vca 424
Lost 2 XE6 cabinets due to isolation XDP 4.8
Cause: Water supply line to XDP 4.8 sprang a leak.
Called Blue Waters on call HonWai Leong
Cary were notified 4:58 PM
Unit will be off line waiting on hose replacement ETA is 2 week
Voltage Sag Vab 415
Voltage Sag Vca 527
Voltage Sag Vab 408
Voltage sag Vca 423
Voltage sag Vbc 460
Voltage Sag Vab 420
Voltage Sag Vca 432
574
574
657
Van 266
Vab 461
Vca 463
500
517
583
Van 266
Vab 461
Vca 464
562
562
544
529
680
680
Vbc 465
Vab 459
Vca 460
780
798
963
Vbc 464
Vca 453
Vab 456
Feed D | Voltage level Vbn 228.44v, 82.47of full scale. | None reported, UPD discharged for 36 seconds | |||||||||||
07/05/2024 | 08:22:24 | 49.99 | Sag Feed C | Voltage level Vbn 181.41v, 65.49 of full scale. | None reported, UPS discharged for 36 seconds | ||||||||
07/05/2024 | 08:22:17 | 33.00 | Sag Feed B | Voltage level Vbn 224.99v, 81.22 of full scale. | None reported, UPS discharged for 36 seconds | ||||||||
07/05/2024 | 08:22:23 | 49.99 | Sag Feed A | Voltage level Vbn 215.93v, 77.95 of full scale. | None reported, UPS discharged for 36 seconds | ||||||||
03/23/2024 | 12:42:28 | 33 | Sag Feed D | Voltage level Vbn 230.31v, 83.15% of full scale. | None reported | ||||||||
03/23/2024 | 12:42:23 | 16.99 | Sag Feed D | Voltage level Vbn 233.23v, 84.20% of full scale. | None reported | ||||||||
03/23/2024 | 12:42:33 | 33 | Sag Feed C | Voltage level Vbn 216.13v, 78.02% of full scale. | None reported | ||||||||
03/23/2024 | 12:42:28 | 16 | Sag Feed C | Voltage level Vbn 217.57v, 78.55% of full scale. | None reported | ||||||||
03/23/2024 | 12:42:26 | 33 | Sag Feed B | Voltage level Vbn 219.17v, 79.12% of full scale. | None reported | ||||||||
03/23/2024 | 12:42:21 | 16.994 | Sag Feed B | Voltage level Vbn 219.37v, 79.20% of full scale. | None reported | ||||||||
03/23/2024 | 12:42:32 | 33.99 | Sag Feed A | Voltage level Vbn 215v, 77.65% of full scale. | None reported | ||||||||
03/23/2024 | 12:42:27 | 33.99 | Sag Feed A | Voltage level Vbn 224.21v, 89.97% of full scale. | None reported | ||||||||
01/27/2024 | 21:40:19 | 66 | Sag Feed D | Voltage level Vcn 172.12v, 62.14% of full scale. | None reported | ||||||||
01/27/2024 | 21:40:19 | 49.99 | Sag Feed C | Voltage level Vcn 171.71v, 61.63% of full scale. | mForge reported servers reboot. | ||||||||
01/27/2024 | 21:40:12 | 66 | Sag Feed B | Voltage level Vcn 132.83v, 47.95% of full scale. | None reported | ||||||||
01/27/2024 | 21:40:18 | 66.99 | Sang Feed A | Voltage level Vcn 181.26v, 65.44% of full scale. | mForge reported servers reboot. | ||||||||
08/19/2023 | 03:19:07 | 49.99 | Sang Feed A | Voltage level Vbn 230.42v, 83.18 % of full scale. | No issues were reported | ||||||||
08/19/2023 | 03:18:22 | 49.99 | Sang Feed A | Voltage level Vbn 221v, 80.12 % of full scale. | No issues were reported | ||||||||
08/19/2023 | 03:18:12 | 15.99 | Sang Feed A | Voltage level Vbn 234.15v, 84.53 % of full scale. | No issues were reported | ||||||||
07/17/2023 | 20:25:21 | 17.003 | Sag Feed A | Voltage level Vbn 234.98v, 84.83 % of full scale. | No issues were reported | ||||||||
07/17/2023 | 20:25:22 | 16.000 | Sag Feed C | Voltage level Vbn 235.14v, 84.89 % of full scale. | No issues were reported | ||||||||
07/08/2023 | 04:31:34 | 34.66 | Feed C | Sag events on Feed C phase a, b, c. Lowest voltage level Vbn 200.35v, 72.33 % of full scale. | Some systems dropped off due to low voltage. | ||||||||
07/08/2023 | 04:31:27 | 67.00 | Feed B | Sag events on Feed B phase a, b, c. Lowest voltage level Van 184.10v, 66.46 % of full scale. | Some systems dropped off due to low voltage. | ||||||||
07/08/2023 | 04:31:33 | 83.99 | Feed A | Sag events on Feed A phase a, b, c. Lowest voltage level Vbn 190.5v, 68.61 % of full scale. | Some systems dropped off due to low voltage. | ||||||||
07/08/2023 | 04:31:29 | 66.00 | Feed D | Sag events on Feed C phase a, b, c. Lowest voltage level Van 197.17v, 71.18 % of full scale. | Some systems dropped off due to low voltage. | ||||||||
06/29/2023 | 13:16:37 | 382.99 | Voltage Sag Feed D | Total 15 events on Feed D. Vbn (all phases) voltage level 171.23, 61.81% of full scale | Most equipment at the data center went offline | ||||||||
06/29/2023 | 13:16:43 | 17.00 | Voltage Surge Feed C | Total 3 events on Feed C. Vbn (all phases) voltage level 12,827.52V, 427.58% of full scale | Serious power quality issue check severs power supplies for possible damage . Most equipment at the data center went offline | ||||||||
06/29/2023 | 13:16:49 | 82.995 | Voltage Sag Feed C | Total 23 events on Feed C. Vbn (all phases) voltage level 163.02, 58.85% of full scale | Most equipment at the data center went offline | ||||||||
06/29/2023 | 13:16:35 | 383.997 | Feed B Sag (brown out) | Total 14 events on Feed B. Vbn (all phases) voltage level 173.47, 62.62% of full scale | Most equipment at the data center went offline | ||||||||
06/29/2023 | 13:16:42 | 383.995 | Feed A Sag (brown out) | Total 15 events on Feed A. Vbn (all phases) voltage level 158.58, 57.25% of full scale | Most equipment at the data center went offline | ||||||||
06/29/2023 | 13:18:26 | 18.00 | All Power feeds | Sag and swell on all facility feeds | Please check you equipment | ||||||||
05/27/2023 | 02:59:28 | 34.00 | E20, Voltage Sag Feed D | Vbn voltage level 216.17, 78.04% of full scale | No impact on production reported | ||||||||
05/27/2023 | 02:59:23 | 33.99 | E20, Voltage Sag Feed D | Van voltage level 200.91, 72.53% of full scale | No impact on production reported | ||||||||
05/27/2023 | 02:59:33 | 33.99 | E19, Voltage Sag Feed C | Vbn voltage level 226.04, 81.60% of full scale | No impact on production reported | ||||||||
05/27/2023 | 02:59:28 | 33.00 | E19, Voltage Sag Feed C | Van voltage level 223.01, 80.51% of full scale | No impact on production reported | ||||||||
05/27/2023 | 02:59.21 | 16.99 | E18, Voltage Sag Feed B | Van voltage level 233.19, 84.18% of full scale | No impact on production reported | ||||||||
05/27/2023 | 02:59:32 | 16.99 | E17, Voltage Sag Feed A | Vbn voltage level 232.70, 84.01% of full scale | No impact on production reported | ||||||||
05/27/2023 | 02:59:27 | 32.99 | E17, Voltage Sag Feed A | Van voltage level 223.37, 86.64% of full scale | No impact on production reported | ||||||||
03/31/2023 | 10:25:45 | 533 | E20, Voltage Sag Feed D | Va RMS =234.397 (84.62%), Vb=233.849 (84.42%) of full scall | No impact on production reported | ||||||||
09/24/2022 | 03:40:44PM | 16.99 | Voltage Sag Feed B | Vbn voltage level 2231.01, 83.40% of full scale | No impact on production reported | ||||||||
09/24/2022 | 03:40:46PM | 32.99 | Voltage Sag Feed D | Vbn voltage level 219.70, 79.31% of full scale | No impact on production reported | ||||||||
09/24/2022 | 03:40:50PM | 32.99 | Voltage Sag Feed A | Vbn voltage level 214.69, 77.5% of full scale | No impact on production reported | ||||||||
09/24/2022 | 03:40:51PM | 33 | Voltage Sag Feed C | Vbn voltage level 218.81, 98.99% of full scale | No impact on production reported | ||||||||
08/03/022 | 05:58:49 | 16.00 | Voltage Sag Feed C | Van voltage level 235.21, 84.91% of full scale | No impact on production reported | ||||||||
08/03/022 | 05:58:48 | 17.00 | Voltage Sag Feed A | Van voltage level 243.72, 84.73% of full scale | No impact on production reported | ||||||||
3/6/2022 | 03:21:08 | 17.00 | Voltage Sag Feed D | Vcn voltage level 231.80, 86.68% of full scale | No impact on production reported | ||||||||
3/6/2022 | 03:21:22 | 33.99 | Voltage Sag Feed C | Vcn voltage level 218.45, 78.86% of full scale | No impact on production reported | ||||||||
3/6/2022 | 04:20:56 | 33.00 | Voltage Sag Feed B | Vcn voltage level 231.59, 83.61% of full scale | No impact on production reported | ||||||||
3/6/2022 | 03:21:11 | 16.99 | Voltage Sag Feed A | Vcn voltage level 232.85, 84.6% of full scale | No impact on production but there was a delay of 3 hours to send notfication out | ||||||||
02/09/2022 | 05:55:03 | 33 | Voltage Sag Feed D | Vbn voltage level 2632.76, 83.04% of full scale |
| ||||||||
02/09/2022 | 05:55:12 | 34 | Voltage Sag Feed C | Vbn voltage level 206.88, 74.69% of full scale |
| ||||||||
02/09/2022 | 05:55:01 | 34 | Voltage Sag Feed A | Vbn voltage level 231.44, 83.55% of full scale |
| ||||||||
12/01/2021 | 01:21:55 | 17 | Voltage Sag Feed C | Van voltage level 232.83, 84.05% of full scale | No issues reported
| ||||||||
10/11/2021 | 15:42:45 | 49.99 | Voltage Sag Feed A | Vcn voltage level 216.06, 78.00% of full scale | No issues reported | ||||||||
10/11/2021 | 16:42:35 | 16 | Voltage Sag Feed B | Van voltage level 230.96, 83.38% of full scale | No issues reported | ||||||||
10/11/2021 | 15:42:56 | 49.999 | Voltage Sag Feed C | Vcn voltage level 227.16, 82.01% of full scale | No issues reported | ||||||||
10/11/2021 | 15:42:47 | 49.999 | Voltage Sag Feed D | Vcn voltage level 218.35, 78.83% of full scale | No issues reported | ||||||||
10/07/2021 | 14:42:23 | 67 | Voltage Sag Feed A | Van voltage level 195.23, 70.48% of full scale | Some equipment rebooted | ||||||||
10/07/2021 | 13:42:13 | 49.999 | Voltage Sag Feed B | Van voltage level 220.37, 79.56% of full scale | Some equipment rebooted | ||||||||
10/07/2021 | 14:42:34 | 49.999 | Voltage Sag Feed C | Vcn voltage level 211.66, 76.41% of full scale | Some equipment rebooted | ||||||||
10/07/2021 | 12:42:25 | 66.000 | Voltage Sag Feed D | Van voltage level 196.45, 70.92% of full scale | Some equipment rebooted | ||||||||
10/01/2021 | 18:38:37 | 66 | Voltage Sag Feed D | Vbn voltage level 208v, 75.03% of full scale | Multiple systems impact | ||||||||
10/01/2021 | 17:38:46 | 49.99 | Voltage Sag Feed C | Vbn voltage level 218.32, 81.32% of full scale | Multiple systems impact | ||||||||
10/01/2021 | 17:38:35 | 50 | Voltage Sag Feed B | Vbn voltage level 225.24, 75.29% of full scale | Multiple systems impact | ||||||||
10/01/2021 | 17:38:35 | 49.99 | Voltage Sag Feed A | Vbn voltage level 208.54, 78.82% of full scale | Multiple systems impact | ||||||||
09/07/2021 | 06:34:04 | 33 | Voltage Sag Feed C | Vcn voltage level 218.49, 79.24% of full scale | No problems reported | ||||||||
09/07/2021 | 06:33:55 | 16 | Voltage Sag Feed D | Vcn voltage level 228.69, 82.56% of full scale | No problems reported | ||||||||
09/07/2021 | 06:33:53 | 33 | Voltage Sag Feed A | Vcn voltage level 220.20, 79.50% of full scale | No problems reported | ||||||||
8/12/2021 | 05:58:06 | 32.99 | Voltage Sag Feed C | Vcn voltage level 229.85, 82.97% of full scale | No impact to production equipment (just a couple of servers seeing low voltage at one point)
| ||||||||
8/12/2021 | 05:57:55 | 17.00 | Voltage Sag Feed C | Vcn voltage level 229.85, 82.97% of full scale |
| ||||||||
8/12/2021 | 05:48:33 | 17.00 | Voltage Sag Feed C | Vcn voltage level 235.03, 84.85% of full scale |
| ||||||||
07/01/2021 | 02:47:28 | 83.99 | Voltage Sag Feed A | Vbn voltage level dropped to 73.18% and Vcn voltage level dropped to 74.48% | Many production equipment reboot reported
| ||||||||
07/01/2021 | 02:47:31 | 83 | Voltage Sag Feed D | Vbn voltage level dropped to 76.46% and Vcn voltage level dropped to 72.10% | Many production equipment reboot reported
| ||||||||
07/01/2021 | 02:47:19 | 83 ms | Voltage Sag Feed B | Voltage level dropped to 73.49% of nominal Vcn 204 | TBD (BW on feed)
| ||||||||
07/01/2021 | 02:47:39 | 83 ms | Voltage SagFeed C | Voltage level dropped to 67.12% of nominal Vcn 187 | TBD (BW Storage, iForge, LSST on feed)
| ||||||||
05/11/2021 | 06:44:40 | 1244-E1 CBEMA CURVE THRESHOLD POSSIBLY ENCROACHED from email at 3:55am 5/12/2021 | |||||||||||
05/11/2021 | 06:44:40 | 33 ms | Voltage Sag Feed A | Voltage level dropped to 79.44% of nominal Van 220 | |||||||||
05/11/2021 | 06:44:51 | 17 ms | Voltage Sag Feed C | Voltage level dropped to 83.75% of nominal Van 232 | |||||||||
4/02/2021 | 11:18:18 | 34 ms | Voltage Sag Feed A | Vcn 73.66% of full scale voltage | mForge severs dropped out; also LSST (in NCSA 3003) and iForge —
| ||||||||
10/02/2020 | 12:33:53 | 533 ms | Voltage Swell Feed A | Voltage above nominal value by 110.8% Van 307 | |||||||||
10/2/2020 | 06:51:26 | 533ms | Voltage Swell; Feed A | Voltage above nominal value by 110.0% Vcn 304 | |||||||||
10/02/2020 | 06:46:33 | 533 ms | Voltage Sag Feed B | Voltage Level dropped to 76.5% of nominalVcn 211 | |||||||||
10/02/2020 | 06:42:21 | 533 ms | Voltage Swell Feed C | Voltage level above nominal value by 110.6% Vcn 306 | |||||||||
10/02/2020 | 05:46:54 | 533 ms | Voltage Sag Feed C | Voltage level dropped to 88.8% of nominal Vcn 246 | |||||||||
10/02/2020 | 05:46:46 | 533ms | Voltage Sag Feed D | Voltage level dropped to 80.2% of nominal Vcn 222 | |||||||||
10/02/2020 | 05:46:54 | 533ms | Voltage Sag Feed A | Voltage Level dropped to 85.2% of nominal Vcn 235.963 | Effects: BW, iForge, mForge, LSST | ||||||||
8/25/2020 | 12:22:22 | 897 | Voltage Swell Feed D | Voltage above nominal value by 219.1% Van 607 | |||||||||
08/24/2020 | 04:52:36 | 636 | Voltage Swell Feed D | Voltage above nominal value by 177.6% Van 492 | |||||||||
08/23/2020 | 00:18:07 | 237 | Voltage Swell Feed D | Voltage above nominal value by 115.8% Van 321 | |||||||||
08/20/2020 | 05:05:57 | 141 | Voltage Swell Feed B | Voltage above nominal value by 174.0% Van 482 | |||||||||
08/19/2020 | 07:21:00 | 794 | Voltage Swell Feed B | Voltage above nominal value by 149.8% Van 415 | |||||||||
8/17/2020 | 03:43:06 | 676 | Voltage Swell Feed B | Voltage above nominal value by 162.22% Van 438 | |||||||||
08/16/2020 | 06:36:16 | 353 | Voltage Swell Feed B | Voltage above nominal value by 174.0% Vbn 482 | |||||||||
08/09/2020 | 14:45:47 | 468 | Voltage Swell Feed D | Voltage above nominal value by 142.2% Van 394 | |||||||||
08/05/2020 | 02:07:37 | 610 | Voltage Swell Feed B | Voltage above nominal value by 194.2% Van 538 | |||||||||
08/03/2020 | 12:17:00 | 244 | Voltage Swell Feed D | Voltage above nominal value by 198% Van 550 | |||||||||
08/02/2020 | 08:13:30 | 95 | Voltage Sag Feed D | Voltage level dropped to 89.5% of nominal value Vcn 248 | |||||||||
08/02/2020 | 08:10:55 | 219 | Voltage Sag Feed B | Voltage level dropped to 89.9% of nominal value Vcn 249 | |||||||||
08/01/2020 | 23:31:54 | 490 | Voltage Swell Feed D | Voltage above nominal value by 129% Vab 618 | |||||||||
07/31/2020 | 02:46:21 | 11 | Voltage Swell Feed B | Voltage above nominal value by 174.4% Vbn 483 | |||||||||
07/28/2020 | 06:56:50 | 401 | Voltage Swell Feed D | Voltage above nominal value by 170% Van 473 | |||||||||
07/13/2020 | 05:05:05 | 717 | Voltage Swell Feed B | Voltage above nominal value by 183% Vcn 508 | |||||||||
07/12/2020 | 09:58:45 | 562 | Voltage Swell Feed B | Voltage above nominal value by 180.1% Vbn 499 | |||||||||
07/11/2020 | 21:50:42 | 188 | Voltage Sag Feed D | Voltage level dropped to 82.3% of nominal value Vcn 228 | |||||||||
07/11/2020 | 21:48:11 | 513 | Voltage Sag Feed B | Voltage level dropped to 83.0% of nominal value Vcn 230 | |||||||||
07/11/2020 | 21:51:10 | 962 | Voltage Sag Feed A | Voltage level dropped to 83.0% of nominal value Vcn 230 | mForge nodes down per Jim Long | ||||||||
06/28/2020 | 07:17:07 | 923 | Voltage Swell Feed B | Voltage above nominal value by 175.1% Vbn 485 | |||||||||
06/22/2020 | 16:08:01 | 72 | Voltage Sag Feed A | Voltage level dropped to 88.1% of nominal value Vbn 244 | |||||||||
06/22/2020 | 16:07:41 | 903 | Voltage Sag Feed D | Voltage level dropped to 76.2% of nominal value Vbn 211 | |||||||||
06/22/2020 | 16:05:16 | 738 | Voltage Sag Feed B | Voltage level dropped to 82.7% of nominal value Vbn 229 | |||||||||
06/14/2020 | 16:09:46 | 632 | Voltage Swell Feed B | Voltage above nominal value by 117.7% Van 326 | |||||||||
06/13/2020 | 17:19:25 | 989 | Voltage Swell Feed D | Voltage above nominal value by 246.6% VAN 683 | |||||||||
06/05/2020 | 00:28:40 | 435 | Voltage Swell Feed D | Voltage above nominal value by 331.8% Van 919 | |||||||||
05/30/2020 | 05:22:27 | 184 | Voltage Swell Feed B | Voltage above nominal value by 175.1% Vbn 485 | |||||||||
05/21/2020 | 17:12:43 | 265 | Voltage Swell Feed D | Voltage above nominal value by 214.1% | |||||||||
05/21/2020 | 08:43:07 | 501 | Voltage Swell Feed D | Voltage above nominal value by 307.2% of nominal value Van 851 | |||||||||
05/19/2020 | 15:00:20 | 3 | Voltage Sag Feed A | Voltage Level dropped to 88.3% of nominal value | |||||||||
05/18/2020 | 21:09:04 | 219 | Voltage | Voltage above nominal value by 175.5% | |||||||||
05/15/2020 | 18:24:33 | 261 | Voltage Sag Feed B | Voltage Level dropped to 54.4% of nominal value Vca 435 | |||||||||
5/11/2020 | 19:29:01 | 486 | Voltage Swell Feed B | Voltage above nominal value by 183.8% Vbn 509 | |||||||||
5/11/2020 | 01:26:30 | 886 | Voltage Swell Feed B | Voltage above nominal value by 175% Van 484 | |||||||||
05/06/2020 | 09:17:40 | UNK | UNK | "Insight" alert email on Feed A, no accompanying "Onboard" detail email | |||||||||
05/05/2020 | 23:09:11 | 79 | Voltage Swell Feed B | Voltage above nominal value by 241.2% Van 668 | |||||||||
05/05/2020 | 15:13:02 | 446 | Voltage Swell Feed B | Voltage above nominal value by 174.7% Vcn 484 | |||||||||
04/29/2020 | 11:28:11 | 16 | Voltage Swell Feed B | Voltage above nominal value by 136.5% Van 378 | |||||||||
04/25/2020 | 01:09:59 | 282 | Voltage Swell Feed B | Voltage above nominal value by 277.6% Van 769 | |||||||||
04/22/2020 | 11:26:23 | 522 | Voltage Swell Feed B | Voltage above nominal value by 150.9% Van 418 | |||||||||
04/19/2020 | 19:51:47 | 416 | Voltage Swell Feed B | Voltage above nominal value by Van 626 | |||||||||
04/18/2020 | 10:58:10 | 736 | Voltage Swell Feed B | Voltage above nominal value by 179.4% Vcn 497 | |||||||||
04/15/2020 | 17:09:11 | 993 | Voltage Swell Feed B | Voltage above nominal value by 171.1% Van 474 | |||||||||
04/12/2020 | 17:37:10 | 368 | Voltage Swell Feed B | Voltage above nominal value by 171.8% Vcn 476 | |||||||||
04/12/2020 | 09:50:09 | 948 | Voltage Swell Feed B | Voltage above nominal value by 259.2% Van 700 | |||||||||
04/11/2020 | 08:44:25 | 411 | Voltage Swell Feed B | Voltage above nominal value by 268.2% Van 743 | |||||||||
04/10/2020 | 11:07:25 | 645 | Voltage Sag Feed B | Voltage Level dropped to 88.9% of nominal value Vcn 246 | |||||||||
04/08/2020 | 17:12:50 | 626 | Voltage Sag Feed B | Voltage Level dropped to 91.3% of nominal value Vbn 253 | |||||||||
04/06/2020 | 18:19:45 | 300 | Voltage Swell Feed B | Voltage above nominal value by 257.4% Van 713 | |||||||||
04/06/2020 | 10:38:40 | 541 | Voltage Swell Feed B | Voltage above nominal value by Van 821 | |||||||||
04/05/2020 | 09:48:46 | 787 | Voltage Swell Feed D | Voltage above nominal value by Van 568 | |||||||||
04/04/2020 | 03:36:51 | 273 | Voltage Swell Feed B | Voltage above nominal value by 191.0% Van 529 | |||||||||
03/28/2020 | 18:49:10 | 699 | Voltage Sag Feed B | Voltage level dropped Van 249 | |||||||||
03/28/2020 | 00:35:48 | 904 | Voltage Swell Feed B | Voltage above nominal value by 249.5% Van 691 | |||||||||
03/25/2020 | 10:16:28 | 513 | Voltage Swell Feed B | Voltage above nominal value by 174.3% Vcn 483 | |||||||||
03/23/2020 | 06:54:20 | 916 | Voltage Sag Feed D | Voltage level dropped to 89.5% of nominal value Vcn 248 | |||||||||
03/23/2020 | 06:52:24 | 893 | Voltage Sag Feed B | Voltage level dropped to 87.7% of nominal value Vcn 243 | |||||||||
03/23/2020 | 06:54:42 | 629 | Voltage Sag Feed A | Voltage level dropped to 85.9% of nominal value Vcn 238 | |||||||||
03/22/2020 | 00:47:33 | 956 | Voltage Swell Feed B | Voltage above nominal value by 165.6% Van 477 | |||||||||
03/21/2020 | 03:30:27 | 724 | Voltage Swell Feed B | Voltage above nominal value by 182.7% Vcn 506 | |||||||||
03/20/2020 | 13:22:01 | 205 | Voltage Swell Feed B | Voltage Vbn 480 | |||||||||
03/18/2020 | 17:41:16 | 320 | Voltage Swell Feed B | Voltage Van 479 | |||||||||
03/17/2020 | 23:03:54 | 11 | Voltage Swell Feed B | Voltage above nominal value by 229.6% Van 636 | |||||||||
03/16/2020 | 04:13:51 | 435 | Voltage Swell Feed B | Voltage above nominal value by 173% Vbn 478 | |||||||||
03/11/2020 | 21:06:50 | 876 | Voltage Swell Feed B | Voltage above nominal value by 265.7% Van 736 | |||||||||
03/09/2020 | 11:58:43 | 398 | Voltage Sag Feed A | Voltage level dropped to 84.5% of nominal value Vbn 234 | |||||||||
03/09/2020 | 11:56:37 | 938 | Voltage Sag Feed B | Voltage level dropped to 82.3% of nominal value Vbn 225 | |||||||||
03/09/2020 | 11:58:27 | 30 | Voltage Sag Feed D | Voltage level dropped to 82.7% of nominal value Vbn 229 | |||||||||
03/08/2020 | 13:19:53 | 274 | Voltage Swell Feed B | Voltage above nominal value by 174.7% Van 484 | |||||||||
03/01/2020 | 19:46:46 | 71 | Voltage Swell Feed B | Voltage above nominal value by 135.38% Van 375 | |||||||||
02/28/2020 | 04:50:25 | 773 | Voltage Swell Feed B | Voltage above nominal value by 265.3% Van 735 | |||||||||
02/27/2020 | 22:08:22 | 2 | Voltage Swell Feed B | Voltage above nominal value by 104.38 % Van 501 | |||||||||
02/24/2020 | 22:38:17 | 221 | Voltage Swell Feed B | Voltage above nominal value by 238.9% Van 662 | |||||||||
02/24/2020 | 04:13:13 | 652 | Voltage Swell Feed B | Voltage above nominal value by 119% Vab 569 | |||||||||
02/18/2020 | 01:15:26 | 266 | Voltage Swell Feed B | Voltage above nominal value by 256.3% Van 710 | |||||||||
02/13/2020 | 18:06:13 | 739 | Voltage Swell Feed B | Voltage above nominal value by 172.20% Vbn 477 | |||||||||
02/02/2020 | 17:01:54 | 58 | Voltage Swell Feed B | Voltage above nominal value by 152.0% Van 421 | |||||||||
01/29/2020 | 03:08:43 | 211 | Voltage Swell Feed D | Voltage above nominal value by 205.4% Van 569 | |||||||||
01/28/2020 | 05:19:04 | 241 | Voltage Swell Feed B | Voltage above nominal value by 174.7% Van 484 | |||||||||
01/27/2020 | 18:47:35 | 222 | Voltage Swell Feed D | Voltage above nominal value by 147.6%, Van 409 | |||||||||
01/24/2020 | 02:30:05 | 998 | Voltage Swell Feed B | Voltage above nominal value by 184.1% Vbn 510 | |||||||||
01/23/2020 | 10:57:41 | 509 | Voltage Swell Feed C | Voltage above nominal value by 134.7% Van 373 | |||||||||
01/23/2020 | 09:49:29 | 141 | Voltage Swell Feed C | Voltage above nominal value by 268.2% Van 743 | |||||||||
01/19/2020 | 08:06:18 | 863 | Voltage Swell Feed C | Voltage above nominal value by 325.3% Van 901 | |||||||||
03/23/2020 | 16:54:25 | 484 | Voltage Swell Feed C | Voltage above nominal value by 113.72% Van 315 | |||||||||
01/16/2020 | 21:14:33 | 940 | Voltage Swell Feed B | Voltage above nominal value by 182.6% Vbn 506 | |||||||||
01/12/2020 | 19:05:04 | 646 | Voltage Swell Feed B | Voltage above nominal value by 172.5% Van 478 | |||||||||
01/11/2020 | 07:39:20 | 863 | Voltage Swell Feed B | Voltage above nominal value by 252.3% Van 699 | |||||||||
01/10/2020 | 20:21:29 | 395 | Voltage Swell Feed B | Voltage above nominal value by 201.8% Van 559 | |||||||||
01/10/2020 | 01:58:32 | 692 | Voltage Swell Feed B | Voltage above nominal value by 199.6% Van 553 | |||||||||
01/09/2020 | 07:48:15 | 509 | Voltage Swell Feed B | Voltage above nominal value by 175.5% Vbn 486 | |||||||||
01/07/2020 | 19:09:06 | 940 | Voltage Swell Feed B | Voltage above nominal value by
| |||||||||
01/06/2020 | 17:26:12 | 102 | Voltage Swell Feed B | Voltage above nominal value by 196.7% Van 545 | |||||||||
01/05/2020 | 22:41:00 | 222 | Voltage Swell Feed B | Voltage above nominal value by 119.9% Van 332 | |||||||||
12/26/2019 | 15:16:44 | 862 | Voltage Swell Feed B | Voltage level above nominal value by Vcn 485 | |||||||||
12/23/2019 | 15:32:21 | 576 | Voltage Swell Feed B | Voltage level above nominal value by Van 659 | |||||||||
12/19/2019 | 13:00:07 | 855 | Voltage Swell Feed B | Voltage above nominal value by 119.9% Van 332 | |||||||||
12/17/2019 | 16:29:46 | 180 | Voltage Swell Feed B | Voltage above nominal value by 172.5% Vcn 478 | |||||||||
11/16/2019 | 8:00:00 | 2 hrs | Transformer Work | Replace defective transformer temperature controller TX-5C | No impact on any production being feed from transformer TX-5C - Complete | ||||||||
11/28/2019 | 02:57:32 | 757 | Voltage Swell Feed D | Voltage above nominal value by 199.3% Van 552 | |||||||||
11/21/2019 | 14:02:08 | 317 | Voltage Swell Feed B | Voltage above nominal value by 172.6% Vcn 478 | |||||||||
11/17/2019 | 03:05:44 | 15 | Voltage Swell Feed B | Voltage above nominal value by 184% Vbn 484 | |||||||||
11/16/2019 | 18:31:10 | 331 | Voltage swell Feed B | Voltage above nominal value by 174.0% Van 482 | |||||||||
11/13/2019 | 21:43:53 | 688 | Voltage Swell Feed B | Voltage above nominal value by 188.1% Van 521 | |||||||||
11/10/2019 | 14:39:20 | 560 | Voltage Swell Feed B | Voltage above nominal value by 170.7% Vbn 473 | |||||||||
11/07/2019 | 12:25:59 | 649 | Voltage Sag Feed A | Voltage level dropped to 89.9% of nominal value. Van 249 | |||||||||
11/07/2019 | 03:18:21 | 891 | Voltage Swell Feed B | Voltage above nominal value by 173.2% Vbn 480 | |||||||||
11/04/2019 | 16:56:57 | 522 | Voltage Swell Feed D | Voltage level above nominal value by 326.3% Van 904 | |||||||||
11/03/2019 | 11:29:06 | 568 | Voltage Swell Feed D | Voltage level above nominal value by 133.2% Van 369 | |||||||||
11/03/2019 | 05:35 | 541 | Voltage Swell Feed B | Voltage above nominal value by 173.6% Van 481 | |||||||||
11/01/2019 | 16:12:33 | 883 | Voltage Swell Feed B | Voltage level above nominal value by
| |||||||||
11/01/2019 | 03:29:46 | 12 | Voltage Swell Feed B | Voltage level above nominal value by 278.7% Van 772 | |||||||||
10/31/2019 | 21:27:37 | 979 | Voltage Swell Feed B | Voltage level above nominal value by 173.2% Vbn 480 | |||||||||
10/31/2019 | 04:55:49 | 971 | Voltage Swell Feed B | Voltage level above nominal value by 179.8% Vbn 498 | |||||||||
10/26/2019 | 22:08:20 | 144 | Voltage Swell Feed B | Voltage level above nominal value by 172.2% Vcn 477 | |||||||||
10/21/2019 | 17:04 | 80 | Voltage Swell Feed B | Voltage level dropped to 90.6% of nominal value. Van 435 | |||||||||
10/21/2019 | 14:41:13 33 | 335 | Voltage Sag Feed D | Voltage level dropped to 88.5% of nominal value. Vca 425 | |||||||||
10/21/2019 | 14:42:29 | 335 | Voltage Sag Feed D | Voltage level dropped to 88.5% of nominal value. Vcn 245 | |||||||||
10/13/2019 | 03:52:29 | 296 | Voltage Swell Feed B | Voltage level above nominal value by 170.1% Vbn 473 | |||||||||
10/12/2019 | 04:41:18 | 660 | Voltage Swell Feed D | Voltage level above nominal value by 111.6% Van 309 | |||||||||
10/11/2019 | 22:24:55 | 264 | Voltage Swell Feed B | Voltage level above nominal value by 171.5% Vcn 475 | |||||||||
10/11/2019 | 10:31:17 | 645 | Voltage Sag Feed B | Voltage level dropped to 82% of nominal value. Vcn 229 | |||||||||
10/11/2019 | 10:32:21 | 509 | Voltage Sag Feed A | Voltage level dropped to 76% of nominal value. Vcn 212 | |||||||||
10/11/2019 | 10:32:23 | 427 | Voltage Sag Feed D | Voltage level dropped to 90% of nominal value. Vcn 250 | |||||||||
10/08/2019 | 09:57:59 | 942 | Voltage Swell Feed B | Voltage level above nominal value by 170% Van 428 | |||||||||
10/05/2019 | 04:39:39 | 697 | Voltage Swell Feed B | Voltage level above nominal value by 174.7% Vbn 484 | |||||||||
10/03/2019 | 08:04:14 | 584 | Voltage Swell Feed C | Voltage level above nominal value by 135%. Vcn 375 | |||||||||
9/24/2019 | 2:14:46 | 673 | Voltage Swell Feed B | Voltage level above nominal value by 173%. Van 479 | |||||||||
9/23/2019 | 23:41:55 | 346 | Voltage Swell Feed B | Voltage level above nominal value by 150% Vca 722 | |||||||||
9/20/2019 | 21:17:05 | 464 | Voltage Swell Feed D | Voltage level above nominal value by 194.375% Vab 933 | |||||||||
9/15/2019 | 11:46:41 | 13 | Voltage Swell Feed C | Voltage level above nominal value by 135.74% Van 376 | |||||||||
9/15/2019 | 10:26:28 | 108 | Voltage Sag Feed A | Voltage level dropped to 89.53% of nominal value Vcn 248 | |||||||||
9/15/2019 | 10:25:24 | 850 | Voltage Sag Feed B | Voltage level dropped to 90.25% of nominal value Vcn 250 | |||||||||
9/15/2019 | 10:26:24 | 339 | Voltage Sag Feed D | Voltage level dropped to 85.20% of nominal value Vcn 236 | |||||||||
9/15/2019 | 10:28:11 | 145 | Voltage Sag Feed C | Voltage level dropped to 87.73% of nominal value Vcn 243 | |||||||||
09/14/2019 | 05:00:34 | 759 | Voltage Swell Feed C | Voltage level above nominal value by 136.1% Vbn 377 | |||||||||
09/07/2019 | 02:13:07 | 968 | Voltage Swell Feed B | Voltage level above nominal value by 183.4% Van 508 | |||||||||
08/20/2019 | 08:30:40 | 326 | Voltage Swell Feed B | Voltage level above nominal value by 124% | |||||||||
08/20/2019 | 20:17:31 | 245 | Voltage Swell Feed B | Voltage level above nominal value by 264.6% Vca 733 | |||||||||
08/20/2019 | 11:43:05 | 75 | Voltage Swell Feed D | Voltage level above nominal value by 161% Van 773 | |||||||||
08/20/2019 | 11:10:39 | 19 | Voltage Sag Feed D | Voltage level dropped to 90% of nominal value Vbn 250 | |||||||||
08/20/2019 | 11:09:45 | 949 | Voltage Sag Feed B | Voltage level dropped to 87% of nominal value Vbn 242 | |||||||||
08/20/2019 | 11:10:35 | 106 | Voltage Sag Feed A | Voltage level dropped to 77% of nominal value Vbn 213 | |||||||||
08/14/2019 | 04:55:47 | 464 | Voltage Swell Feed B | Voltage Level above nominal value by Vbn 486 | |||||||||
08/13/2019 | 20:29:15 | 901 | Voltage Swell Feed B | Voltage Level above nominal value by 173% Vbn 480 | |||||||||
08/07/2019 | 09:32:11 | 360 | Voltage Swell Feed B | Voltage Level above nominal value by Van 344 | |||||||||
08/03/2019 | 07:39:58 | 890 | Voltage Sag Feed D | Voltage Level Van 247 | |||||||||
07/20/2019 | 759 | Voltage Swell Feed B | Voltage Level above nominal value by Van 624 | ||||||||||
07/05/2019 | 12:48:40 | 464 | Voltage Sag Feed B | Voltage level dropped to 75.0% of nominal value Vcn 205 | BW cabinets EPO'd | ||||||||
07/05/2019 | 12:49:12 | 386 | Voltage Sag Feed D | Voltage level dropped to 72.6% of nominal value Vcn 201 | BW cabinets EPO'd | ||||||||
07/05/2019 | 12:49:12 | 65 | Voltage Sag Feed A | Voltage level dropped to 89.2% of nominal value Vcn 247 | BW cabinets EPO'd, mForge lost GPFS | ||||||||
07/02/2019 | 15:19:59 | 986 | Voltage Swell Feed C | Voltage level above nominal value by 133% Vbn 370 | |||||||||
6/30/2019 | 16:22:22 | 14 | Voltage Sag Feed C | Voltage level dropped to 76.2 % of nominal value | |||||||||
6/30/2019 | 16:11:19 | 114 | Voltage Sag Feed A | Voltage level dropped to 90.3% of nominal value Van 250 | some mForge jobs dropped, VM Farm issues. | ||||||||
6/30/2019 | 16:11:16 | 303 | Voltage Sag Feed D | Voltage level dropped to 75.5 % of nominal value Vbn 209 | |||||||||
6/30/2019 | 16:10:42 | 6 | Voltage Sag Feed B | Voltage level dropped to 77.6% of nominal value Vbn 215 | |||||||||
6/29/2019 | 14:28:00 | 625 | Voltage Sag | Voltage level dropped to 85.56% of nominal value Vcn 237 | |||||||||
6/22/2019 | 12:22:17 | 884 | Voltage Swell Feed C | Voltage level above nominal level by 189% Vca 911 | |||||||||
6/19/2019 | 03:03:08 | 28 | Voltage Swell Feed B | Voltage level above nominal level by 172.6% Van 478 | |||||||||
6/15/2019 | 06:13:08 | 549 | Voltage Swell Feed B | Voltage level above nominal level by 175% Vcn 485 | |||||||||
05/27/2019 | 06:13:59 | 716 | Voltage Sag Feed D | Voltage level dropped to 76.9% of nominal value Vbn 213 | |||||||||
05/27/2019 | 06:13:59 | 124 | Voltage Sag Feed A | Voltage level dropped to 76.2% of nominal value Vbn 211 | |||||||||
05/27/2019 | 06:13:37 | 381 | Voltage Sag Feed B | Voltage level dropped to 82.3% of nominal value Vbn 229 | |||||||||
05/26/2019 | 10:31:25 | 838 | Voltage Swell Feed B | Voltage level above nominal value by 197% Van 545 | |||||||||
05/26/2019 | 01:48:24 | 137 | Voltage Sag Feed A | Voltage level dropped to 90.4% of nominal value Vab 434 | |||||||||
05/26/2019 | 01:48:23 | 970 | Voltage Sag Feed D | Voltage level dropped to 89.8% of nominal value Vab 431 | |||||||||
05/23/2019 | 05:28:39 | 191 | Voltage Sag Feed D | Voltage level dropped to 89.5% of nominal value Van 248 | Non-reported | ||||||||
05/23/2019 | 05:28:41 | 366 | Voltage Sag Feed A | Voltage level dropped to 89.5% of nominal value Van 248 | Non-reported | ||||||||
05/21/2019 | 22:01:36 | 734 | Voltage Swell Feed C | Voltage level above nominal value by 297% Van 822 | Non-reported | ||||||||
05/19/2019 | 07:50:21 | 33 | Voltage Swell Feed C | Voltage level above nominal value by 129% Van 358 | Non-reported | ||||||||
05/17/2019 | 8:32:17 | 334 | Voltage Swell Feed C | Voltage level above nominal value by 120% Van 334 | Non-reported | ||||||||
05/14/2019 | 06:48:37 | 413 | Voltage Swell Feed B | Voltage level above nominal value by 167% Van 465 | Non-reported | ||||||||
05/13/2019 | 12:58:28 | 303 | Voltage Swell Feed C | Voltage level above nominal value by 218.4% Van 605 | Non-reported | ||||||||
05/11/2019 | 06:28:46 | 676 | Voltage Swell Feed B | Voltage level above nominal value by 312% Van 867 | This is a very large voltage swell I reached out to S&F for possible cause. | ||||||||
05/09/2019 | 05:27:27 | 664 | Voltage Swell Feed C | Voltage level above nominal value by 153.9% Van 413 | Non-reported | ||||||||
05/09/2019 | 02:000:09 | 593 | Voltage Swell Feed B | Voltage level above nominal value by 177 % Van 479 | Non-reported | ||||||||
04/11/2019 | 23:04:48 | 391 | Voltage Sag Feed C | Voltage level dropped to 88.8% of nominal value Vcn 246 | Non-reported | ||||||||
04/11/2019 | 23:04:31 | 205 | Voltage Sag Feed C | Voltage level dropped to 88.8% of nominal value Vcn 246 | Non-reported | ||||||||
04/11/2019 | 24:05:32 | 584 | Voltage Sag Feed D | Voltage level dropped to 88.8% of nominal value Vcn 246 | Non-reported | ||||||||
04/11/2019 | 24:05:15 | 399 | Voltage Sag Feed D | Voltage level dropped to 89% of nominal value Vcn 247 | Non-reported | ||||||||
04/11/2019 | 13:03:54 | 782 | Voltage Swell Feed C | Voltage level 119% of nominal value | Non-reported | ||||||||
03/19/2019 | 05:00:23 (03:39:43) | 791 | Voltage Sag Feed D | Voltage level dropped to 90.6% of nominal value Vbn 251 Vab 434 | Non-reported | ||||||||
03/10/2019 | 07:16:44 | 2 | Voltage Sag Feed A | Voltage level dropped to 86.2% of nominal value Vbn 23 | |||||||||
03/10/2019 | 07:16:43 | 985 | Voltage Sag Feed A | Voltage level dropped to 89.5% of nominal value Vbc 430 | |||||||||
03/10/2019 | 07:16:43 | 967 | Voltage Sag Feed A | Voltage level dropped to 87.7% of nominal value Vcn 243 Vca 430 | |||||||||
03/10/2019 | 7:21:23 | 811 | Voltage Sag Feed C | Voltage level dropped to 81.5% of nominal value Vbn 226 Vab 418 | |||||||||
03/10/2019 | 7:21:23 | 777 | Voltage Sag Feed C | Voltage level dropped to 86.8% of nominal value Vbc 421 Vca 417 | |||||||||
03/10/2019 | 7:21:23 | 760 | Voltage Sag Feed C | Voltage level dropped to 89.9% of nominal value Vcn 249 | |||||||||
03/10/2019 | 7:16:50 | 0 | Voltage Sag Feed D | Voltage level dropped to 81.2% of nominal value Vbn 225 Vab 421 | |||||||||
03/10/2019 | 7:16:49 | 966 | Voltage Sag Feed D | Voltage level dropped to 87.2% of nominal value Vbc 419 | |||||||||
03/10/2019 | 7:16:49 | 948 | Voltage Sag Feed D | Voltage level dropped to 89.5% of nominal value Vcn 248 V ca 430 | |||||||||
03/10/2019 | 7:14:08 | 723 | Voltage Sag Feed B | Voltage level dropped to 86% of nominal value Vbn 238 Vab 433 | |||||||||
03/10/2019 | 7:14:08 | 688 | Voltage Sag Feed B | Voltage level dropped to 86.3% of nominal value Vcn 239 Vbc 433 Vca 425 | |||||||||
03/09/2019 | 22:19:43 | 537 | Voltage Sag Feed C | Voltage level dropped to 67.87% of nominal value Vcn 188 Vbc 368 Vca 370 | Lsst and Forge reported servers rebooting Facility fire alarm panel power supply failure | ||||||||
03/09/2019 | 22:19:37 | 942 | Voltage Sag Feed C | Voltage level dropped to 89% of nominal value Vcn 247 Vca 433 Vbc 362 | |||||||||
03/09/2019 | 22:19:37 | 523 | Voltage Sag Feed C | Voltage level dropped to 90.2% of nominal value Vcn 250 | |||||||||
03/09/2019 | 22:19:37 | 505 | Voltage Sag Feed C | Voltage level dropped to 73.3% of nominal value Van 203 Vab 390 Vca 388 | |||||||||
03/09/2019 | 22:14:56 | 346 | Voltage Sag Feed A | Voltage level dropped to 75.8% of nominal value Vcn 210 Vbc 398 Vca 391 | |||||||||
03/09/2019 | 22:14:55 | 746 | Voltage Sag Feed A | Voltage level dropped to 88.5% of nominal value Vcn 234 Vbc 427 Vca 425 | |||||||||
03/09/2019 | 22:14:50 | 313 | Voltage Sag Feed A | Voltage level dropped to 76.5% of nominal value Vcn 212 Vab 395 Vca 402 | Lsst and Forge reported servers rebooting Facility fire alarm panel power supply failure | ||||||||
03/09/2019 | 22:12:21 | 206 | Voltage Sag Feed B | Voltage level dropped to 83% of nominal value Vcn 230 Vbc 421 Vca 417 | Lsst and Forge reported servers rebooting Facility fire alarm panel power supply failure | ||||||||
03/09/2019 | 22:12:20 | 623 | Voltage Sag Feed B | Voltage level dropped to 74% of nominal value Vcn 205 Vbc 390 Vca 389 | Lsst and Forge reported servers rebooting Facility fire alarm panel power supply failure | ||||||||
03/09/2019 | 22:12:15 | 189 | Voltage Sag Feed B | Voltage level dropped to 71.8% of nominal value Van 199 Vab 382 Vca 379 | Lsst and Forge reported servers rebooting Facility fire alarm panel power supply failure | ||||||||
03/09/2019 | 22:15:03 | 264 281 864 | Voltage Sag Feed D | Voltage level dropped to 67.14% of nominal value Vcn 246 Vca 432 Vbc 361 Vcn 186 Vbc 366 Vca 367 | Lsst and Forge reported servers rebooting Facility fire alarm panel power supply failure | ||||||||
03/09/2019 | 22:14:57 | 831 | Voltage Sag Feed D | Voltage level dropped to 74% of nominal value Van 205 Vab 395 Vca 388 Vcn 250 | Lsst and Forge reported servers rebooting Facility fire alarm panel power supply failure | ||||||||
01/20/2019 | 22:10:55 | 586 586 604 817 817 | Voltage Sag Feed B | Voltage level dropped to 87.7% of nominal value Van 243 Vca 432 Vab 435 Van 243 Vab 429 | Non-reported | ||||||||
01/20/2019 | 22:13:16 | 154 172 189 385 403 420 | Voltage Sag Feed A | Voltage level dropped to 88% of nominal value Van 244 Vca 422 Vab 430 Van 248 Vab 424 Vca 431 | Non-reported | ||||||||
01/20/2019 | 22:14:02 | 645 | Voltage Sag Feed D | Voltage level dropped to 80.1% of nominal value Van 22 Vab 409 Vca 413 | Non-reported | ||||||||
01/20/2019 | 22:13:22 | 414 414 414 | Voltage Sag Feed D | Voltage level dropped to 80.8% of nominal value Van 224 Vab 428 Vca 399 | Non-reported | ||||||||
01/20/2019 | 22:17:28 | 517 517 517 | Voltage Sag Feed C | Voltage level dropped to 80.8% of nominal value Van 224 Vab 412 Vca 414 | Non-reported | ||||||||
01/20/2019 | 22:17:28 | 287 287 287 | Voltage Sag Feed C | Voltage level dropped to 81.2% of nominal value Van 225 Vab 428 Vca 398 | Non-reported | ||||||||
01/09/2019 | 10:42:17 | 244 | Voltage sag Feed C | Voltage level dropped to 88% of nominal Vbn 244 | Facility power wide impact | ||||||||
01/09/2019 | 10:38:18 | 456 | Voltage sag Feed D | Voltage level dropped to 88% of nominal Vbn 244 Vbc 428 | Facility power wide impact | ||||||||
01/09/2019 | 10:00:35 | 351 | Voltage sag Feed A | Voltage level dropped to 89.5% of nominal Vbn 248 Vab 430 | Blue waters mForge and racks lost power and caused production interruptions. | ||||||||
01/09/2019 | 10:17:59 | N/A | Low voltage | mForge rack 5 UPS 01 switched to battery | UPS on battery, power outage on feed C | ||||||||
01/09/2019 | 09:58:12 | 890 | Voltage sag Feed B | Voltage level dropped to 89.5% of nominal Vbn 248 Vab 430 | Blue waters mForge and racks lost power and caused production interruptions. | ||||||||
12/31/2018 | 00:35:17 | 816 | Voltage sag Feed C | Voltage level dropped to 88.4% of nominal Vcn 246 | |||||||||
12/31/2018 | 03:31:26 | 110 | Voltage Sag Feed A | Voltage level dropped to 88.4% of nominal Vcn 245 | |||||||||
12/27/2018 | 13:04:48 | 527 545 977 | Voltage Sag Feed A | Voltage level dropped to 88.4% of nominal Vab 423 Vbn 245 Van 246 | This incident was caused by human error during main substation protective relays upgrade which accidentally tripped the feeder to NPCF | ||||||||
12/27/208 | 13:00:14 | 382 | Voltage sag Feed C | Voltage level dropped to 79% of nominal Vca 382 | This incident was caused by human error during main substation protective relays upgrade which accidentally tripped the feeder to NPCF | ||||||||
12/21/2018 | 10:22:17 | 157 157 157 | Voltage sage Feed D | Voltage level dropped to 85.6% of nominal Vbn 237 Vab 431 Vbc 424 | Non-reported | ||||||||
12/21/2018 | 10:22:09 | 806 806 806 857 | Voltage sage Feed A | Voltage level dropped to 70.7% of nominal Vbn 196 Vab 370 Vbc 382 Van 248 | Non-reported | ||||||||
12/21/2018 | 10:26:00 | 629 629 629 664 | Voltage sage Feed C | Voltage level dropped to 71.5% of nominal Vbn 198 Vab 373 Vbc 384 Van 249 | Non-reported | ||||||||
12/21/2018 | 10:20:02 | 244 | Voltage sage Feed B | Voltage level dropped to 81% of nominal Vbn 225 Vab 413 Vbc 410 | Non-reported | ||||||||
12/21/2018 | 10:19:07 | 730 | Voltage sage Feed D | Voltage level dropped to 85% of nominal Vbn 236 Vab 428 Vbc 425 | Non-reported | ||||||||
12/21/2018 | 10:16:52 | 802 819 819 | Voltage sage Feed B | Voltage level dropped to 91% of nominal Vbn 250 Vab 413 Vbc 411 | Non-reported | ||||||||
12/21/2018 | 10:19:00 | 382 | Voltage sage Feed A | Voltage level dropped to 71% of nominal Vbn 197 Vab 366 Vbc 388 | LSST reported power problem in there system | ||||||||
12/21/2018 | 10:22:51 | 188 | Voltage sage Feed C | Voltage level dropped to 89% of nominal Vbn 247 Vab 367 Vbc 391 | LSST reported power problem in there system | ||||||||
12/21/2018 | 10:36 | N/A | UPS 01 under voltage | Under voltage condition was detected by mforge rack 4 ups | Non-reported | ||||||||
12/18/2018 | 11:44:02 | 568 | Voltage Sag on Feed B | Voltage level dropped to 83.3% of nominal Vbn 231 Vab 417 Vbc 425 | Non-reported | ||||||||
12/18/2018 | 11:46:10 | 451 | Voltage sag feed D | Voltage level dropped to 86.6% of nominal Vbn 240 Vbc 429 Vab 425 | Non-reported | ||||||||
12/18/2018 | 11:46:03 | 756 | Voltage sag Feed A | Vbn level dropped to 72 % of nominal Vbn 199 Vab 375 Vbc 388 | LSST reported production equipment down due to the power event; Industry systems were not affected | ||||||||
12/18/2018 | 11:49:59 | 954 | Voltage sage feed C | Voltage level dropped to 72% of nominal Vbn 200 Vab 375 Vbc 389 Van 249 | LSST reported production equipment down due to the power event; Industry systems were not affected | ||||||||
12/18/2018 | 12:02:40 | N/A | Power interruption | Siemens BMS report power interruption and transfer to backup generator | Non-Reported | ||||||||
12/18/2018 | 12:02 | N/A | Low Voltage and low frequency | mForge rack UPS 1 reported low input voltage condition | Non-reported | ||||||||
12/11/2018 | 18:14:28 | 823 | Voltage Sag Feed B | Van 236 Vbn 225 Voltage at 81% of nominal Vcn 246 Vab 390 Vbc 408 Vca 427 | Non-reported | ||||||||
12/11/2018 | 12:27:01 | 747 | Voltage Sag event Feed C | Van 228 Volage at 82% of nominal Vbn 248 Vab 403 Vca 417 | Trouble was report by Condo rack T86, please check your production equipment at NPCF. | ||||||||
12/11/2018 | 12:39:27 | N/A | mForge UPS 1 Rack 5 UPS Alarm | Power waveform distorted (under voltage) | Please check mforge for possible trouble Feed C | ||||||||
12/01/2018 | 16:55:46 | 451 | Voltage Sag Feed B | Voltage sag to 88.9% of nominal voltage for 207 msec Van 246 Vcn 248 Vca 417 | Non-reported | ||||||||
12/01/2018 | 17:01:36 | 52 | Voltage Sag Feed C | Voltage sag to 88% of nominal voltage for 52 msec Van 244 Vcn 248 Vca 415 | Non-reported | ||||||||
12/01/2018 | 16:57:55 | 207 | Voltage Sag Feed A | Voltage sag to 89.2% of nominal voltage for 207 msec Van 247 Vca 425 Vcn 248 | Non-reported | ||||||||
11/18/2018 | 13:47:34 | 558 | Voltage Swell Feed D | Voltage swell 32% above nominal Van 484 Vab 634 Vca 635 | Non-reported | ||||||||
11/10/2018 | 4:27:42 | 493 493 493 511 511 511 | Voltage sag Van 249 Vbn 249 Vab 414 Vbc 389 Vca 378 | Voltage sag on all feeder feeder A, B, C and D to 62.3% of normal | Reported equipment rebooting | ||||||||
11/05/2018 | 4:49:42 | 45 | Voltage swell Van 526 Vab 639 Vca 677 | Voltage swell as high as 90% on feed B | Non reported | ||||||||
11/05/2018 | 04:54:08 | 714 | Voltage Swell Van 377 Vbn 347 Vcn 343 | Voltage swell as high as 36% on feeder C | Non reported | ||||||||
11/04/2018 | 02:08:18 | 907 | Voltage sag Vca 433 | 9.7% voltage sag on Feed A | Non reported | ||||||||
10/15/2018 | 06:23:00 (no day light saving) | 801 | voltage Sag Vcb 229 Van 224 Vbc 414 Vca 400 | Power tranformer servering the Atkins building shorted causing power outage on Ameren Blue power line to the campus substation witch affected most of campus buildings power supply. | At 07:33:26 a transformer failed on campus caused one Ameren power line feed to campus substation to open resulting in a single power feed to the substation, the event was not detected by NPCF feeder D meter and substation switchgear MP02 circuit breaker (D feed) feeder relay, below is the sequence of events:
| ||||||||
09/28/2018 | 15:52:54 15:52:59 | 684 797 | Voltage sag Feed C | Voltage Level Vca 432 | Non-reported | ||||||||
09/19/2018 | 02:31:46 | 401 | Voltage Swell Feeder D | Van 322 | Non-Reported | ||||||||
8/29/2018 | 14:27:58 | 524 524 | Voltage Swell D | Van 345 Vab 538 | Non-Reported | ||||||||
8/17/2018 | 22:17:09 | 443 443 443 | Voltage Swell Feeder D | Van 625 Vab 746 Vca 713 | Non-reported | ||||||||
08/17/20018 | 00:29:27 | 532 532 549 | Voltage Sag Feeder A | Van 243 Vab 436 Vca 431 | Non-Reported | ||||||||
08/17/2018 | 00:28:03 | 654 672 | Voltage Sag Feeder B | Van 244 Vca 436 | Non-Reported | ||||||||
08/16/2018 | 00:31:59 | 446 | Voltage Sag Feeder c | Van 249 | Non-Reported | ||||||||
08/16/2018 | 23:59:05 | 730 | Voltage Sag Feeder C | Van 250 | Non-Reported | ||||||||
08/16/2018 | 23:55:09 | 245 431 434 | Voltage Sag Feeder B | Van 245 Vca 431 Vab 434 | Non-Reported | ||||||||
08/07/2017 | 0:30:55 | 796 796 796 | Voltage Swell Feeder D | Van 618 Vab 713 Vca 728 | Non-reported | ||||||||
08/06/2018 | 13:53:28 | 861 861 861 | Voltage Sag Feeder A | Van 246 Vcn 229 Vca 386 | Non-reported | ||||||||
08/03/2018 | 17:14:56 | 787 | Voltage Swell Feeder D | Van 370 Vab 534 | Non-reported | ||||||||
07/28/2018 | 02:20:55 | 305 | Voltage Sag Feeder A | Pickup Van 249 | Non-reported | ||||||||
07/11/2018 | 12:28:31 | 66 | Voltage swell Feeder D | Van 545 Vab 650 Vca 690 | Non-Reported | ||||||||
07/03/2018 | 4:35:11 | 289 | Voltage swell Feeder B | Van 451 Vbn 490 Vcn 498 | Non-Reported The email notification received via email 11 minutes after event occurrence. | ||||||||
06/22/20018 | 13:20:02 | 222 410 411 | Voltage Sag Feeder A | Voltage level Vbn 222 Vab 410 Vbc 411 | Non-reported | ||||||||
06/22/20018 | 13:20:05 | 237 430 424 | Voltage Sag Feeder D | Voltage level Vbn 237 Vab 476 Vbc 424 | Non-reported | ||||||||
06/22/20018 | 13:21:56 | 243 431 424 | Voltage Sag Feeder C | Voltage level Vbn 243 Vbc 431 Vab 424 | Non-reported | ||||||||
06/22/20018 | 13:18:58 | 136 136 187 | Voltage Sag Feeder B | Voltage level Vbn 246 Vbc 436 Vab 437 | Non-reported | ||||||||
6/25/2018 | 8 am | one week | non critical equipment maintenance | VFD preventive maintenance | NPCF office area may experience temperature rise for as short period of time. | ||||||||
06/12/2018 | 03:41:46 | 65 65 83 | Power loss Feeder A |
| Blue Waters: 72 cabinet lost power all on feeder A and the entire system was rebooted. iForge: Lost about 90 nodes. mForge: reported trouble with some nodes ( 50% of the nodes were lost) mForge: gpfs was impacted and reported an outage. The UPS and the PDUs on feeder A did not detect any problems. Confirmed by F&S, the voltage dropped by 37.37% for 65 milli second which is considered an outage. The cause: Utility transmission lines effected by the weather/storms F&S are investigating why meters alerts are not being sent out, possibly filtered by spam network filters. | ||||||||
06/04/2018 | 16:00:00 | 17 hours | power loss to rack EE133 mForge | Circuit breaker tripped on closing panel PP1A dead cover | Loss of power to feed A to mForge rack EE133, no production loss and it is a save due to power diversity from a different feed, it took a long time to discover to lack of power monitoring. | ||||||||
3/09/2018 | 12:41:51 | 214 400 400 | Voltage Sag Feeder C | Van 214 Vab 400 Vca 400 | Non-reported 22 74% below nominal | ||||||||
3/09/2018 | 12:41:00 | 3 3 21 | Voltage Sag Feeder D | Van 237 Vab 423 Vca 432 | Non-reported 11.8 % below nominal | ||||||||
3/09/2018 | 12:40:56 | 171 | Voltage Sag Feeder A | Van 211 Vab 389 Vca 402 | Non-reported 18.9 % below nominal | ||||||||
3/09/2018 | 12:40:29 | 667 667 677 | Voltage Sag Feeder B | Van 236 Vab 418 Vca 434 | Non-reported 12.9 % below nominal | ||||||||
02/19/2018 | 12:08:09 | 775 775 792 | Voltage Sag Feeder C | Vcn 246 Vca 411 Van 247 | Non-reported 14.3% below nominal | ||||||||
02/19/2018 | 12:08:04 | 178 178 178 | Voltage Sag Feeder C | Van 247 Vcn 244 Vca 408 | Non-reported 15% below nominal | ||||||||
02/19/2018 | 12:07:20 | 564 | Voltage Sag Feeder A | Vca 431 | Non-reported 10.5% below nominal | ||||||||
02/19/2018 | 12:07:14 | 964 | Voltage Sag Feeder A | Vca 430 | Non-reported 10.4% below nominal | ||||||||
02/19/2018 | 12:06:54 12:06:59 | 434 430 | Voltage Sag Feerer A | Vca 434 Vca 430 | Non-reported 10.4% below nominal | ||||||||
02/19/2018 | 11:45:42 | 384 384 384 | Voltage Sag Feeder C | Van 246 Vcn 244 Vca 408 | Non-reported 15 % below nominal | ||||||||
02/19/2018 | 11:44:53 | 189 | Voltage Sag Feeder A | Vca 429 | Non-reported 9.79 % below nominal | ||||||||
02/19/2018 | 11:44:32 | 352 | Voltage Sag Feeder B | Vca | Non-reported 9.79 % below nominal | ||||||||
02/19/2018 | 10:16:21 | 896 896 896 | Voltage Sag Feeder C | Van 246 Vcn 243 Vca 407 | Non-reported 15.2 % below nominal | ||||||||
02/19/2018 | 10:15:32 | 802 | Voltage Sag Feeder A | Vca 430 | Non-reported 10.41% below nominal | ||||||||
02/19/2018 | 10:15:11 | 963 | Voltage Sag Feeder B | Vca 430 | Non-reported 10.41% below nominal | ||||||||
02/04/2018 | 13:07:35 | 476 | Voltage Sag Feeder B | Vca 428 | Non-reported 10.8% below nominal | ||||||||
02/04/2018 | 13:008:24 | 185 | Voltage sag Feed C | Van 245 Vcn 242 Vca 403 | Non-reported 16% blow nominal | ||||||||
02/04/2018 | 13:07:52 | 564 | Voltage sag Feed A | Vca 247 | Non-reported 11 % below nominal | ||||||||
02/01/2018 | 08:40:16 | 894 894 894 | Voltage Swell Feed C | 361 331 332 | 30% above 277 Volts Non-reported | ||||||||
02/01/2018 | 4:39:27 | 716 716 716 | Voltage Swell Feed C | 360 331 327 | 29.9 % above 277 Volts | ||||||||
01/24/2018 | 17:16:59 | 508 925 975 108 175 342 | Voltage Swell Feeder A | Vbn 289 Vbc 500 Vca 500 Vab 501 Van 289 Vcn 289 | Swell level is 4.3% above nominal | ||||||||
01/21/2018 | 20:44:20 | 543 543 543 | Voltage Sag Feed A | Vcn 254 Vbc 454 Vca 447 | Sag level at 8.3% Non-reported | ||||||||
01/21/2018 | 20:44:49 | 691 691 | Voltage Sag Feed C | Vcn 259 Vca 452 | Sag level at 6.5% Non-reported | ||||||||
01/04/2018 | 15:24:49 | 192 192 192 210 | Voltage Sag Feed A | Vcn 219 Vbc 408 Vca 404 Vbn 251 | Sag level at 15.8% Non-reported | ||||||||
01/04/2018 | 15:24:49 | 789 789 789 | Voltage Sag Feed D | Vcn 237 Vbc 428 Vca 429 | Sag level at 10.8% Non-reported | ||||||||
01/04/2018 | 15:25:07 | 266 226 226 244 | Voltage Sag Feed C | Vcn 252 Vbc 448 Vca 450 Vbn 249 | Sag level at 6.67% Non-reported | ||||||||
01/01/2018 | 15:42:50 | 627 627 627 644 | Voltage Sag Feed B | Vcn 246 Vbc 438 Vca 445 Vbn 258 | Sag level at 8.75% Non-reported | ||||||||
12/19/2017 | 8:31:01 | 852 852 852 852 | Voltage sag feed C | Vbn 248 Vab 447 Vbc 434 Vcn 261 | Sag level at 9.58% Non-reported | ||||||||
12/19/2017 | 8:30:57 | 449 499 517 | Voltage sag feed D | Vbn 258 Vbc 447 Vba 451 | Sag level at 6.87% Non-reported | ||||||||
12/19/2017 | 8:30:49 | 990 990 990 | Voltage sag feed A | Vab 439 Vbc 427 Vbn 242 Vcn262 | Sag level at 11.04% Non-reported | ||||||||
12/19/2017 | 8:30:55 | 193 193 | Voltage sag feed B | Vbn 257 Vbc 447 | Sag level at 6.87% Non-reported | ||||||||
12/16/2017 | 9:53:37 | 31 31 31 | Voltage sag feed A | Van 255 Vcn 257 Vca 437 | Sag level at 8.95% Non-reported | ||||||||
12/14/2017 | 5:27:25 | 825 91 91 | Voltage sag feed A | Vab 445 Vbc 451 Vca 449 | Sag level at 7.29% Non-reported | ||||||||
12/14/2017 | 5:27:27 | 429 695 695 | Voltage sag Feed C | Vab 450 Vbc 452 Vca 450 | Sag level at 6.25% Non-reported | ||||||||
12/12/2017 | 12:00 | Change meters threshold to 94% | Sag level at 6% | ||||||||||
12/10/2017 | 14:32:09 | 851 851 868 851 | Voltage sag Feed A | Vbc 455 Vca 451 Van 264 Vcn 256 | Non-reported | ||||||||
12/10/2017 | 14:29:10 | 960 960 960 960 | Voltage sag Feed B | Van 266 Vcn 254 Vbc 456 Vca 444 | Non-reported | ||||||||
12/10/2017 | 14:35:57 | 231 231 231 249 | Voltage Sag Feed C | Vcn 250 Vbc 448 Vca 445 Van 265 | Non-reported | ||||||||
12/10/2017 | 14:32:57 | 981 981 981 998 | Voltage Sag Feed D | Vcn 256 Vbc 458 Vca 447 Van 266 | Non-reported | ||||||||
12/08/2017 | 8:59:06 | 416 | Voltage sag Feed C | Vab 459 Vbc 459 Vca 464 | Non-reported | ||||||||
12/07/2017 | 09:02:11 | 812 | Voltage sag Feed C | Vab 458 Vbc 458 Vca 457 | Non-reported | ||||||||
12/07/2017 | 08:06:44 | 33 | Voltage Sag @ the 277 level Feed A | Vbn 268 Phase to neutral measurement | Non-reported | ||||||||
12/06/2017 | 08:54:50 | 940 | Voltage Sag feed B | Vab 461 | Non-reported | ||||||||
12/05/2017 | 00:56:50 | 181 | Volt Sag Feed D | Vab 453 Vca 458 | Tower 1 and 2 controller failure | ||||||||
12/05/2017 | 00:56:50 | 928 | Volt Sag Feed A | Vab 446 Vca 449 | Tower 1 and 2 controller failure | ||||||||
12/05/2017 | 00:59:07 | 833 | Volt Sag Feed C | Vab 463 Vca 452 | Tower 1 and 2 controller failure | ||||||||
12/05/2017 | 00:53:03 | 650 | Volt Sag Feed B | Vab 460 | Tower 1 and 2 controller failure |
598
598
598
12/02/2017 | 11:48:38 | 354 354 354 | Voltage Sag Feeder C | Vab 457 Vbc 456 Vca 454 | Non-reported |
11/ |
28/2017 |
Volt Sag
Feed B
Volt Sag
Feed C
Vab 463
Vca 452
Volt Sag
Feed A
Vab 446
Vca 449
Volt Sag
Feed D
Vab 453
Vca 458
Voltage Sag @ the 277 level
Feed A
Vbn 268
Phase to neutral measurement
Voltage sag
Feed C
Vab 458
Vbc 458
Vca 457
Voltage sag
Feed C
Vab 459
Vbc 459
Vca 464
981
981
981
998
Voltage Sag
Feed D
Vcn 256
Vbc 458
Vca 447
Van 266
231
231
231
249
Voltage Sag
Feed C
Vcn 250
Vbc 448
Vca 445
Van 265
960
960
960
960
Voltage sag
Feed B
Van 266
Vcn 254
Vbc 456
Vca 444
851
851
868
851
Voltage sag
Feed A
Vbc 455
Vca 451
Van 264
Vcn 256
429
695
695
Voltage sag
Feed C
Vab 450
Vbc 452
Vca 450
Sag level at 6.25%
Non-reported
825
91
91
Voltage sag
feed A
Vab 445
Vbc 451
Vca 449
20:36.27 | 598 598 598 | Voltage Sag | Vab 460 Vbc 460 Vca 458 | Non-reported | |
11/23/2017 | 22:17:21 | 780 798 963 | Voltage sag Feeder C | Vbc 464 Vca 453 Vab 456 | Non-reported |
11/18/2017 | 06:16:59 | 529 680 680 | Voltage Sag Feeder C | Vbc 465 Vab 459 Vca 460 | Non-reported |
11/15/2017 | 14:14:46 | 562 562 544 | Voltage Sag Feeder C | Vbc 457 Vca 457 Vab 462 | Non-reported |
11/14/2017 | 07:08:38 | 500 517 583 | Voltage Sag Feeder C | Van 266 Vab 461 Vca 464 | Non-reported |
11/14/2017 | 07:05:41 | 574 574 657 | Voltage Sag Feeder A | Van 266 Vab 461 Vca 463 | Non-reported |
11/13/2017 | 08:42:32 | 646 | Voltage Sag Feeder D | Voltage Sag Vab 420 Voltage Sag Vca 432 | None |
11/13/2017 | 08:44:36 | 472 | Voltage Sag Feeder C | Voltage Sag Vab 408 Voltage sag Vca 423 Voltage sag Vbc 460 | None |
11/13/2017 | 08:38:51 | 877 | Voltage Sag Feeder B | Voltage Sag Vab 415 Voltage Sag Vca 527 | None |
11/10/2017 | 15:50 | off line | Cooling | Damaged chilled water supply hose | Lost 2 XE6 cabinets due to isolation XDP 4.8 Cause: Water supply line to XDP 4.8 sprang a leak. Called Blue Waters on call HonWai Leong Cary were notified 4:58 PM Unit will be off line waiting on hose replacement ETA is 2 week |
11/04/2017 | 20:00:58 | 998 | Voltage Sag Feeder C | Vab 456 Vbc 457 Vca 456 | None |
10/31/2017 | 20:08:31 | 309 | Voltage Sag Feeder | Vab 455 Vbc 456 Vca 455 | None |
10/30/2017 | 6:37:18 | Not reported | Voltage Sag Feeder C
| Vab 461 Vbc 460 Vca 462 | None |
10/29/2017 | 10:07:51 | 224 | Voltage Sag Feeder B | Vab 440 Vbc 454 Vca 460 | None |
10/26/2017 | 7:57:23 | 690 | Voltage Sag | Vab 426 Vbc 403 Vca 424 | LSST: Servers rebooted at NCSA and NPCF |
Sag level at 6.45%
Non-reported