# Generator testing causing surges?



## ElectricArmy (May 19, 2015)

Good day. This is an issue I am facing that has several parts:

1. I am the maintenance electrician for a large senior living facility that is only about five years old, in fact I was the foreman who closed out the nearly two-year long job in 2018.

2. It was built with a 4000A, 480V service, and a 900KW Cummins Diesel generator - real nice beast, but the generator only served two ATS' with limited loads, instead of the whole facility.

3. They live in an area with a notoriously unreliable power supply and have been subject to an above average number of interrupted power events such as random power losses, storm damage, brownouts, loss of one or two phases, and also phenomena that caused disturbances in some of their electronic equipment and lighting systems (Legrand Wattstopper digital systems), which I always thought were probably due to surges of some nature.

4. Due to the repeated problems listed above, they hired an engineering firm to see if they could get their generator to serve the whole facility, instead of the limited loads.

5. It was discovered that their facility was only ever using a maximum of approximately 500KW at any given time, even during peak demand, and therefore the firm gave the green light and they commenced a project to obtain and install a new, service rated ATS and have the generator serve the entire facility.

6. Well this project was completed by the EC this past May, and shortly afterwords, I received a communication from the IT director at the facility concerning the day after the shutdown, on the day when they were testing the new ATS, opening and closing the MCB under load multiple times.

He said on that day, there were two separate incidents wherein 11 of his security cameras suddenly and permanently ceased operating, 3 at exactly the same time in one incident, and 8 at exactly the same time in another incident, both on the same day. He sent me the last recorded screen shot from all 11 cameras, and indeed one can see that the time for the two batches is exactly the same.

Naturally, he had figured it had to do with the EC's work onsite and reached out to them saying that their work had destroyed 11 cameras. Naturally, they returned with a denial, stating that there are surge suppressors installed in the service equipment, and that couldn't possibly happen.

Note - there are indeed surge suppressors, one in the main lineup at the main breaker, SQD TVS4IMA32B1 with 9 on its display, and one in the first distribution panel, SQD HL4IMA12C1, on the load side of the now-decommissioned ATS that serves the power for the security systems, and with the number 991613 on it's display.

So he asked for my opinion, whether or not I thought these camera's were fried because of the ATS testing that, and the breakers being opened and closed under load.

So my questions are:

A. Could the ATS testing have killed the 11 cameras somehow?
B. What could have been the culprit for the death of these cameras?
C. If a surge killed these cameras, how come the surge suppressors didn't mitigate the damage?
D. Did the EC's work cause these cameras to fail, and should they be held liable?



Thank you all. Looking forward to hearing your thoughts.


----------



## Almost Retired (Sep 14, 2021)

@paulengr 
@gpop 
@JRaef 
and who ever else is qualified to answer more than me


----------



## Almost Retired (Sep 14, 2021)

ElectricArmy said:


> Good day. This is an issue I am facing that has several parts:
> 
> 1. I am the maintenance electrician for a large senior living facility that is only about five years old, in fact I was the foreman who closed out the nearly two-year long job in 2018.
> 
> ...


if your boss is considering legal recourse, he needs to hire a forensic engineer who will testify in court
our opinions on this site would be considered hearsay in court


----------



## backstay (Feb 3, 2011)

You may not have proof, but I don’t believe in coincidence.


----------



## ElectricArmy (May 19, 2015)

I recognize the opinions here could not be used as legal evidence, but as far as the electrical issue goes about the ATS testing and the failing of the cameras, and surges, etc., that question I am keen to answer.


----------



## splatz (May 23, 2015)

Most surge protectors don't start working until considerably over the normal operating voltage. For example a surge protector with a clamping voltage of 330V starts shunting the surge at 330V. 

With surges from lighting, the voltage is rising quickly and the duration between normal operating and clamping voltage is usually very brief; in this incident, it might not have been. 

A lot of IT equipment is powered by switching power supplies that tolerate a wide range of voltages, usually well over operating normal operating voltage. Newer cameras are mostly power over ethernet. Older cameras are usually coax for the video and 18-2 for the low voltage power, usually 12VDC, 24VDC, or 24VAC. A surge has to get past the low voltage power supply to get to the camera by power. However, if there are issues with the grounding, voltage can wind up on the shield of the coax. That's where I'd look with this incident. 

There are probably lots of other sensitive electronics in the building with and without surge protection that were not damaged. If I read you right, only the cameras were damaged, not all sensitive electronics got hit. So it's something about the cameras. 

That coax shield could be it. Of course the cable boxes and HDMI cords on the TVs and other things could also be affected current on the ground, but it's totally possible due to some condition - the construction of the cameras, the grounding of the low voltage power, whether the low voltage power is grounded, etc. - it would affect only the security cameras. 

Figuring this out will be difficult but the odds are astronomical that the cameras just coincidentally died when the switch was exercised. It's very likely that more stuff will get ruined in the future if you don't figure it out.


----------



## gpop (May 14, 2018)

A. Could the ATS testing have killed the 11 cameras somehow?

Yes

B. What could have been the culprit for the death of these cameras?

You supplied no useful info other than a high event count on the suppressor


C. If a surge killed these cameras, how come the surge suppressors didn't mitigate the damage?

define surge then define rich voltage. As you gave no real life numbers on voltage or type of power supply on camera's its not worth guessing


D. Did the EC's work cause these cameras to fail, and should they be held liable?

Bet the contract said what to do not what the expected results would be. 

Transfer switch will monitor the generator based on the settings it was programmed too. Once the generator is running with in these settings the transfer switch will go ahead and transfer. 
If you tell it that its ok to transfer 10% rich it will. Now your 120v is 10% rich and anything dc is even richer. 

Most security camera's would have a UPS unit that would act as a buffer.

Get some real life data like voltage and hertz and we can offer better advice.


----------



## MHElectric (Oct 14, 2011)

Here’s my 2cents.

Earlier this year someone ran into a power pole outside our neighborhood after a bad rain storm and we lost power for about 24hrs. I cranked my portable generator up and powered us through it.

Almost immediately afterwards the ice maker and the control panel on my wife’s new refrigerator goes out and I cant reset it despite several attempts.

Fast forward to recently, we finally got an appliance company out there and after some minor work he got everything running again. He told my wife that the spikes & surges from a generator will wreck havoc on appliances and any kind of sensitive equipment.

Do the camera systems have a surge protector between them and their power supply? Like a layered surge protector? Or is it just that big surge protector at the MDP?

Just thought I’d throw that out there.


----------



## MHElectric (Oct 14, 2011)

gpop said:


> Most security camera's would have a UPS unit that would act as a buffer.


Do you know how well the UPS unit handles surges? Just curious.

I know the basics behind them but I dont know all the specifics. Like how many surges or brownout’s they can handle?

I thought they mostly regulated the incoming power kinda like a scrubber or something. Similar to how a water filtration system cleans up the incoming water source.


----------



## Wirenuting (Sep 12, 2010)

Is it an open transition or closed transition?
What switch and what options were ordered?
Did it transfer out of phase?

In regards to electronics on the system, get an ups for everything critical.

Since your so lightly loaded, test it as required, I always did it during working hours for more load and to keep everyone used to power blinks. 
Load bank that unit yearly to keep that diesel clean.


----------



## gpop (May 14, 2018)

MHElectric said:


> Do you know how well the UPS unit handles surges? Just curious.
> 
> I know the basics behind them but I dont know all the specifics. Like how many surges or brownout’s they can handle?
> 
> I thought they mostly regulated the incoming power kinda like a scrubber or something. Similar to how a water filtration system cleans up the incoming water source.


Different ups do different things. All of them have a very narrow band for what they consider normal power and will quickly switch to battery to protect the load. 

If the OP's info on the suppressor is correct then 991613 event's would be a indication that the generator is well above 480v compared to the 9 events recorded on the mains power side. UPS would have said not exceptable and switched to battery mode.


----------



## Wirenuting (Sep 12, 2010)

MHElectric said:


> Do you know how well the UPS unit handles surges? Just curious.
> 
> I know the basics behind them but I dont know all the specifics. Like how many surges or brownout’s they can handle?
> 
> I thought they mostly regulated the incoming power kinda like a scrubber or something. Similar to how a water filtration system cleans up the incoming water source.


Quality ones will work great and last as long as the battery life.
I used the smaller ones on the building automation panels in the hospital I worked at. Kept the OR’s air handlers and chillers from resetting the program.


----------



## joe-nwt (Mar 28, 2019)

gpop said:


> Get some real life data like voltage and hertz and we can offer better advice.


This.


----------



## MikeFL (Apr 16, 2016)

ElectricArmy said:


> ....
> 3. They live *in an area with a notoriously unreliable power supply and have been subject to an above average number of interrupted power events such as random power losses, storm damage, brownouts, loss of one or two phases, and also phenomena that caused disturbances in some of their electronic equipment and lighting systems*
> ...
> A. Could the ATS testing have killed the 11 cameras somehow?
> ...


If the power is that bad read up on surge at www.nemasurge.org.
There are 3 types (layers) of surge arrestors:
1) at the service
2) at the MDP
3) at the device(s) being protected.

If the power quality is that bad, the surge arrestors you have may be worn out. If they have MOV's and they have been repeatedly taking surges, they're toast. 

It would be wise to do a power quality study while cycling the system from utility power to genset power and back to utility power and look for any anomalies. @brian john is probably our best qualified expert on this.


----------



## MikeFL (Apr 16, 2016)

Another consideration is to see if the utility sells surge arrestors. Some do and they charge a small monthly fee. They have a connected equipment warranty and they do pay out on it when things get smoked.


----------



## micromind (Aug 11, 2007)

I find that often it's not that the gen produces lousy power, the problem is that a lot of switching power supplies cannot handle the re-transfer back to utility. It's quick but not quick enough or too quick. If it were a whole second or so, the power supply would start at zero, no problems but a flying start is too much for it to handle. 

I've seen VFDs blow up this way too.


----------



## MHElectric (Oct 14, 2011)

Lot of smart dudes in this place. I realize that once certain questions get posted…… 

And I am not one of them.


----------



## nrp3 (Jan 24, 2009)

I know when I look at ordering higher end transfer switches for commercial applications there are different types of mechanisms. I imagine they have their place. I wouldn’t have given security cameras much thought. I guess I’d be looking at gen output voltages and what kind of ups if any is protecting the security camera gear.


----------



## splatz (May 23, 2015)

A line interactive UPS, which is the cheaper ones, normally passes power straight through, then switches to battery power when power is disrupted. There is always a time to switch. Most of them have MOV surge protection built in, but surprisingly crappy surge protection. As usual MOV surge protection depends on low impedance on the EGC to function. Some of these are really small and cheap, they look like a bulked up surge protector and cost $50 or so at Staples. 

An online UPS, aka a double conversion UPS, uses utility power and a rectifier to charge the batteries, and an inverter to power the load - the load is always on battery. There is usually some filtering and conditioning on the load side. One benefit with these is that small fluctuations in input power don't matter, the load is always getting just about exactly the nominal voltage. Browns over time can reduce lifespan of electronics. 

I have had great luck with these for 30 years now. You do pay extra though. You can get anything from shoebox size cord and plug 700VA units to big systems that will run an entire data center.


----------



## Wardenclyffe (Jan 11, 2019)

Also, when the generator starts up, the voltage may not yet be stabilized at the instant the ATS makes the transfer. This initial surge of not yet regulated power can damage electronic systems.







Protection of Automatic Transfer Switches


Protect Automatic Transfer Switches - surge protective devices with sine wave tracking.




www.surgesuppression.com


----------



## SWDweller (Dec 9, 2020)

It has been mentioned but I am going to mention it again. The operation of the EM system has to be suspect. Some allow the voltage to sag to a set point and then the Gen will start and transfer. Others allow the voltage to go almost to zero before engaging. Something not mentioned, are the transfer switches 3 pole or 4 pole? IE switching the neutral? Specifically what loads are on the Gen? Do the transfers work together or are they staged by a few seconds. 

Beings this is a care facility where are the records of the last EM system test? When was the last time the Gen was load banked? Something my state requires for the health care certification.

You state TVSS's but I did not get a clear picture that there are any TVSS's on the Gen.
Specs on the TVSS's, specifically the let through. 
A 900 kw machine should have voltage and Hz regulation controls where as most everything made as a portable below 20kw does not. Pretty much below 20kw rpm is how out put is controlled..


----------



## kb1jb1 (Nov 11, 2017)

MHElectric said:


> Here’s my 2cents.
> 
> Earlier this year someone ran into a power pole outside our neighborhood after a bad rain storm and we lost power for about 24hrs. I cranked my portable generator up and powered us through it.
> 
> ...


Whenever I install a portable generator interlock and power inlet or the Reliance 6 circuit transfer switch I make sure the homeowner knows not to use their good TV or other electronic equipment. Unplug the stuff and have a sacrificial refrigerator. The small cheap generators from Costco do not have clean, filtered power.


----------



## kb1jb1 (Nov 11, 2017)

Just wondering if these are PoE cameras or do they get their power from a more local source? If the cameras get power from multiple sources, could they be out of sync? If they are PoE, how long are the runs? Some Cat6 cable is 24 AWG and some is 23AWG. My experience with camera systems is limited to the cabling and installation. I sub out the technical and programming stuff.


----------



## Slay301 (Apr 23, 2018)

So I’m assuming this happened while exercising? How far out of phase angle does your ats let the transfer switch complete the switch ? If it’s to far out you have increased current when the switching is completing . On the flip side if you try to get it to close it will take forever to switch. I’ve seen it before when people don’t get the ats commissioned


----------



## ElectricArmy (May 19, 2015)

Hello everyone,

THANK you all tremendously for your replies. In an age of insanity and all sorts of cheap, cotton-candy information, this stuff is excellent and, I can clearly see, coming from reliable, experienced sources. I'm quite grateful.

That being said, a few things:

First, sorry it's taken a while to get back on here.

Second, I think I have received enough direction to help the client out from here on.

Third, as for the requests for all the various pieces of real-time data and information, unfortunately I don't really have any of it. I get that I didn't have much to begin with either.

I said I am their maintenance electrician, but what that means is only that they call me to do certain small work, and fix broken things here and there. Of the contractors they have, I am one of a select few who helped build the building and have any intimate knowledge of its bowels. However, I don't work for them full time, nor do I spend alot of time there lately, though I did several years ago, so the collection of that information isn't really feasible for me, though I wish I could.

Their first manager who I dealt with was great and he was really taking big issues head on, in fact he spear-headed the ATS project that generated this issue, but they got rid of him about a year and a half ago, in favor of a more agreeable fellow, because he was too head strong for them. The new guy is nice, but doesn't call me near as much. I miss the old guy.

I can say that I know that the cameras are PoE, with the cabling being 23 AWG Cat6. This is a young facility, with construction being completed back in September, 2018 (I think it was a two year project).

Also, the ATS is brand new, just installed a few months ago, and now services the whole house, whereas there are two former ATS' that once served only certain loads, the smaller one was for life safety only, the larger for optional equipment. But those are now bypassed and not in use, as far as I know.

Anyway, I do appreciate all the wisdom shared, I wish I had more relevant fodder to keep the fire burning.

Thank you all and if I have more questions concerning this stuff, I'll continue asking.


----------



## hornetd (Oct 30, 2014)

Thread is too old and I didn't notice that. Sorry!


----------



## Nomadic (5 mo ago)

What brand and type of ATS. Is it a isolation ATS? Are the nutrals bonded between line and genset. I believe they should not be. I see monthly transfer tests done at surgical centers and senior care facilities at my place and no complants for cameras and sensitive electronics. We do a full out power failure test and let the ATS do its thing. It run for the set time for it to see steady power and then phase in and switch back over.

Maybe check setting on the ATS as well. Could be bringing in the emergency source as the genset is still comming online. To soon its undervolt to the electronics. 

Was the timestamp at the power failure point or the return from Emergency to Normal?

It will work till it don't.


----------

