Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

DateTimeDuration /msecEvent TypeWhat happened?Impact?
07/11/201812:28:31

66

 

Voltage swell

Feeder D

Van 545

Vab 650

Vca 690

Non-Reported
07/03/20184:35:11289

Voltage swell

Feeder B

Van 451

Vbn 490

Vcn 498

Non-Reported

The email notification received via email 11 minutes after event occurrence.

06/22/2001813:20:02

222

410

411

Voltage Sag Feeder A

Voltage level

Vbn 222

Vab 410

Vbc 411

Non-reported
06/22/2001813:20:05

237

430

424

Voltage Sag Feeder D

Voltage level

Vbn 237

Vab 476

Vbc 424

Non-reported
06/22/2001813:21:56

243

431

424

Voltage Sag Feeder C

 

 

Voltage level

Vbn 243

Vbc 431

Vab 424

Non-reported
06/22/20018

13:18:58

 

136

136

187

Voltage Sag Feeder B

Voltage level

Vbn 246

Vbc 436

Vab 437

Non-reported
6/25/20188 amone weeknon critical equipment maintenanceVFD preventive maintenanceNPCF office area may experience temperature rise for as short period of time.
06/12/201803:41:46

65

65

83

 

Power loss

Feeder A

  • Site Feeder A experienced Unknown event ( still investigating)
  • Power monitoring meters are unable to send alarms
  • Van 173.48 V
  • Vcn 175.70 V
  • Vbn 189.36 V

Blue Waters: 72 cabinet lost power all on feeder A and the entire system was rebooted.

iForge: Lost about 90 nodes.

mForge: reported trouble with some nodes ( 50% of the nodes were lost)

mForge: gpfs was impacted and reported an outage.

The UPS and the PDUs on feeder A did not detect any problems.

Confirmed by F&S, the voltage dropped by 37.37% for 65 milli second which is considered an outage.

The cause: Utility transmission lines effected by the weather/storms

F&S are investigating why meters alerts are not being sent out, possibly filtered by spam network filters.

06/04/201816:00:0017 hourspower loss to rack EE133 mForgeCircuit breaker tripped on closing panel PP1A dead coverLoss of power to feed A to mForge rack EE133, no production loss and it is a save due to power diversity from a different feed, it took a long time to discover to lack of power monitoring.
3/09/201812:41:51

214

400

400

Voltage Sag

Feeder C

Van 214

Vab 400

Vca 400

Non-reported

22

74% below nominal

3/09/201812:41:00

3

3

21

 

Voltage Sag

Feeder D

Van 237

Vab 423

Vca 432

Non-reported

11.8 % below nominal

3/09/201812:40:56171

Voltage Sag

Feeder A

Van 211

Vab 389

Vca 402

Non-reported

18.9 % below nominal

3/09/201812:40:29

667

667

677

Voltage Sag

Feeder B

Van 236

Vab 418

Vca 434

Non-reported

12.9 % below nominal

02/19/201812:08:09

775

775

792

Voltage Sag

Vcn 246

Vca 411

Van 247

Non-reported

14.3% below nominal

02/19/201812:08:04

178

178

178

Voltage Sag

Van 247

Vcn 244

Vca 408

Non-reported

15% below nominal

02/19/201812:07:20564Voltage SagVca 431

Non-reported

10.5% below nominal

02/19/201812:07:14964Voltage SagVca 430

Non-reported

10.4% below nominal

02/19/2018

12:06:54

12:06:59

434

430

Voltage Sag

Vca 434

Vca 430

Non-reported

10.4% below nominal

02/19/201811:45:42

384

384

384
Voltage Sag

Van 246

Vcn 244

Vca 408

Non-reported

15 % below nominal

02/19/201811:44:53189Voltage SagVca 429

Non-reported

9.79 % below nominal

02/19/201811:44:32352Voltage SagVca

Non-reported

9.79 % below nominal

02/19/201810:16:21

896

896

896

 

Voltage Sag

Van 246

Vcn 243

Vca 407

Non-reported

15.2 % below nominal

02/19/201810:15:32802Voltage SagVca 430

Non-reported

10.41% below nominal

02/19/201810:15:11963Voltage SagVca 430

Non-reported

10.41% below nominal

02/04/201813:07:35476Voltage SagVca 428

Non-reported

10.8% below nominal

02/04/201813:008:24185

Voltage sag

Feed C

Van 245

Vcn 242

Vca 403

Non-reported

16% blow nominal

02/04/201813:07:52564

Voltage sag

Feed A

Vca 247

Non-reported

11 % below  nominal

02/01/201808:40:16

894

894

894

Voltage Swell

Feed C

361

331

332

30% above 277 Volts

Non-reported

02/01/20184:39:27

716

716

716

 

Voltage Swell

Feed C

360

331

327

 

29.9 % above 277 Volts

 

01/24/201817:16:59

508

925

975

108

175

342

 

 

Voltage Swell

Feeder A

Vbn 289

Vbc 500

Vca 500

Vab 501

Van 289

Vcn 289

Swell level is 4.3% above nominal
01/21/201820:44:20

543

543

543

 

Voltage Sag

Feed A

Vcn 254

Vbc 454

Vca 447

Sag level at 8.3%

Non-reported

01/21/201820:44:49

691

691

 

Voltage Sag

Feed C

Vcn 259

Vca 452

Sag level at 6.5%

Non-reported

01/04/201815:24:49

192

192

192

210

Voltage Sag

Feed A

Vcn 219

Vbc 408

Vca 404

Vbn 251

Sag level at 15.8%

Non-reported

01/04/201815:24:49

789

789

789

Voltage Sag

Feed D

Vcn 237

Vbc 428

Vca 429

Sag level at 10.8%

Non-reported

01/04/201815:25:07

266

226

226

244

Voltage Sag

Feed C

Vcn 252

Vbc 448

Vca 450

Vbn 249

Sag level at 6.67%

Non-reported 

01/01/201815:42:50

627

627

627

644

 

Voltage Sag

Feed B

Vcn 246

Vbc 438

Vca 445

Vbn 258

Sag level at 8.75%

Non-reported

12/19/20178:31:01

852

852

852

852

Voltage sag

feed C

Vbn 248

Vab 447

Vbc 434

Vcn 261

Sag level at 9.58%

Non-reported

12/19/20178:30:57

449

499

517

Voltage sag

feed D

Vbn 258

Vbc 447

Vba 451

Sag level at 6.87%

Non-reported

12/19/20178:30:49

990

990

990

Voltage sag

feed A

Vab 439

Vbc 427

Vbn 242

Vcn262

Sag level at 11.04%

Non-reported

12/19/20178:30:55

193

193

Voltage sag

feed B

Vbn 257

Vbc 447

Sag level at 6.87%

Non-reported

12/16/20179:53:37

31

31

31

Voltage sag

feed A

Van 255

Vcn 257

Vca 437

Sag level at 8.95%

Non-reported

12/14/20175:27:25

825

91

91

Voltage sag

feed A

Vab 445

Vbc 451

Vca 449

Sag level at 7.29%

Non-reported

12/14/20175:27:27

429

695

695

 

Voltage sag

Feed C

Vab 450

Vbc 452

Vca 450

Sag level at 6.25%

Non-reported

 

12/12/201712:00Change meters threshold to 94%  Sag level at 6%
12/10/201714:32:09

851

851

868

851

Voltage sag

Feed A

 

Vbc 455

Vca 451

Van 264

Vcn 256

Non-reported
12/10/201714:29:10

960

960

960

960

Voltage sag

Feed B

Van 266

Vcn 254

Vbc 456

Vca 444

Non-reported
12/10/201714:35:57

231

231

231

249

Voltage Sag

Feed C

Vcn 250

Vbc 448

Vca 445

Van 265

 

Non-reported
12/10/201714:32:57

981

981

981

998

 

Voltage Sag

Feed D

Vcn 256

Vbc 458

Vca 447

Van 266

 

Non-reported
12/08/20178:59:06416

Voltage sag

Feed C

Vab 459

Vbc 459

Vca 464

Non-reported
12/07/201709:02:11812

Voltage sag

Feed C

Vab 458

Vbc 458

Vca 457

Non-reported
12/07/201708:06:4433

Voltage Sag @ the 277 level

Feed A

Vbn 268

Phase to neutral measurement 

Non-reported
12/06/201708:54:50940Voltage Sag feed BVab 461Non-reported
12/05/201700:56:50181

Volt Sag

Feed D

Vab 453

Vca 458

Tower 1 and 2 controller failure
12/05/201700:56:50928

Volt Sag

Feed  A

Vab 446

Vca 449

Tower 1 and 2 controller failure
12/05/201700:59:07833

Volt Sag

Feed C

Vab 463

Vca 452

Tower 1 and 2 controller failure
12/05/201700:53:03650

Volt Sag

Feed B

Vab 460Tower 1 and 2 controller failure
12/02/201711:48:38

354

354

354

Voltage Sag
Vab 457                    
Vbc 456                        
Vca 454
Non-reported
11/28/201720:36.27

598

598

598

Voltage Sag
Vab 460                      
Vbc 460                          
Vca 458
Non-reported
11/23/201722:17:21

780

798

963

Voltage sag

Vbc 464

Vca 453

Vab 456

Non-reported
11/18/201706:16:59

529

680

680

Voltage Sag

Vbc 465

Vab 459

Vca 460

Non-reported
11/15/201714:14:46

562

562

544

Voltage Sag
Vbc  457                                
Vca  457                    
Vab 462                
   
Non-reported
11/14/201707:08:38

500

517

583

Voltage Sag

Van 266

Vab 461

Vca 464

Non-reported
11/14/201707:05:41

574

574

657

Voltage Sag

Van 266

Vab 461

Vca 463

 

Non-reported
11/13/201708:42:32646Voltage Sag

Voltage Sag Vab 420

Voltage Sag Vca 432

 

None     
11/13/2017 08:44:36 472Voltage Sag

Voltage Sag Vab 408

Voltage sag Vca 423

Voltage sag Vbc 460

None
11/13/201708:38:51 877Voltage Sag

Voltage Sag Vab 415

Voltage Sag Vca 527

           None      
11/10/201715:50off lineCoolingDamaged chilled water supply hose

Lost 2 XE6 cabinets due to isolation XDP 4.8

Cause: Water supply line to  XDP 4.8 sprang a leak.

Called  Blue Waters on call HonWai Leong

Cary were notified 4:58 PM

Unit will be off line waiting on hose replacement ETA is 2 week

11/04/201720:00:58998Voltage Sag

Vab 456

Vbc 457

Vca 456

None
10/31/201720:08:31309Voltage Sag

Vab 455

Vbc 456

Vca 455

None
10/30/20176:37:18Not reportedVoltage Sag

Vab 461

Vbc 460

Vca 462

None
10/29/201710:07:51224Voltage Sag

Vab 440

Vbc 454

Vca 460

None
10/26/20177:57:23690Voltage Sag

Vab 426

Vbc 403

Vca 424

LSST: Servers rebooted at NCSA and NPCF