# Ethernet control network wiring



## mburtis (Sep 1, 2018)

Just wondering what opinions may be out there regarding wiring industrial ethernet networks, between PLCs or HMIs or the like. Does wire type matter Cat 5 vs 5e vs 6 vs something else? Solid vs stranded, factory made vs self terminated. Any other details to pay attention to?


----------



## Jlarson (Jun 28, 2009)

We usually run ubiquity tough cable to everything since I always have a spool or two around, land on shielded jacks/patch panels, then use premade patch cables. I absolutly despise making up rj-45 plugs.


----------



## Navyguy (Mar 15, 2010)

Most modern places are using fibre so they don't have to deal with the 600 volt issues. You still need to match insulation ratings and such.

If you go copper, shielded is a must and keep it away from everything as possible. Remember to keep the industrial network physically separated from the administration network and if the two shall meet it should be through a DMZ with limited access to either.

Remember the Target hack happen because they got into the administrative network because the HVAC controllers were connected to the internet.

If you call any of the big players (Belden, Hubbell, Panduit, Leviton) they will do a site visit and design a proposal for you.

Cheers
John


----------



## splatz (May 23, 2015)

At a minimum I'd make it a rule to terminate cables in jacks rather than plugs (patch panels at the switch locations, surface mount boxes at the outlets) and reterminate any plugs in jacks. Use factory made patch cords, avoid making your own plugs if at all possible. The rules for bend radius and low pulling tension are good but the most important rule is cable length, be strict with the 100m / 328' limit. You can't really even find Cat 5 for a long time, but and everyone acts like it's insane to use cat 5, but the truth is it isn't that big of a deal, it's far more important to install what you use correctly.


----------



## mburtis (Sep 1, 2018)

So your telling me this isn't what you would call an ideal install ? 









The cable that was installed with our last update several years ago appears to be pretty run of the mill cat 6 23 gauge solid. No shielding or even exotic insulation and it appears they made up the ends on site to connect to the PLCs, switches, etc. 

What sort of things are likely to create the most interference? Is multiple ethernet cables in a single metal conduit likely to cause any issues? What about running into or through enclosures containing radio equipment such as 900 mhz radios, or what about being in the same conduit as a radio cable? All the ethernet cables run to other control panels with lots of 120v in them, but none of them are in immediate exposure to 480v equipment or close to any vfds. 

Are there patch panels and other components designed to go in a control enclosures or mount on din rails for these sorts of applications ?


----------



## mburtis (Sep 1, 2018)

Forgot to ask is the 328 ft max cable length just due to line resistance and signal strength etc.? Is that between components and can say an ordinary switch extend that or are there special boosters? We have one run that is probably close to that length, oddly enough it's the only one that doesn't have errors constantly.


----------



## Navyguy (Mar 15, 2010)

I would not say "ideal" but more like "classic" LOL

Everything causes interference at some level, but the issue is more about packet stability. As @splatz said, most industrial equipment will run perfectly on a 10Mbit system (his comment about Cat5 cable) and I have yet to find any that are use 10Gbit or even 1Gbit.

I think the only thing you really would have to deal with in you plant would be proximity to motors (and their equipment) and any sort of HID lighting. This is why the shielding is important, because you will have to enter the enclosure with the VFD or softstart and there is a lot of "noise" in those sorts of things. Pretty much if you are keeping the minimum distance away from everything else you are good to go. Another classic is the guy tie wrapping the cable to the electrical conduit; but if it is shielded cable it probably does not matter too much.

From a purist perspective, some will say that Cat cables will create interference with each other, but I don't think there is an issue there. The issue I believe is the "bundling" of Cat cables and the physical weight being placed on the bottom cable and further how it sits in the tray, especially if it is basket wire tray.

To me the tie wrap verses velcro is like the ground pin up or down argument; but I will agree that if you improperly use the tie wrap you will damage the cable but that is true about any material.

I am not into radios anymore, but I know we generally separated the cables from the 400 cycle stuff on the ships / rigs as much as possible, but again shielding and insulation is your friend here. Radios are only burst transmitting in digital anyway; it is not like the old analogue prolonged transmissions, thus my comment about packet stability previously. In my experience I don't think I have ever seen an antenna cable in conduit, now in you plant it might be required if you are passing through some different zones / divisions, but since it just passing through as long as it was intrinsically safe I am not seeing the requirement for conduit.

So from a TIA standard the maximum recommended length is 100 m (328 feet) which consists of 90 m (295 feet) of permanent link and 5 m (16 feet) of connecting cables (patch cords) at each end.

Cheers

John


----------



## splatz (May 23, 2015)

mburtis said:


> What sort of things are likely to create the most interference? Is multiple ethernet cables in a single metal conduit likely to cause any issues? What about running into or through enclosures containing radio equipment such as 900 mhz radios, or what about being in the same conduit as a radio cable? All the ethernet cables run to other control panels with lots of 120v in them, but none of them are in immediate exposure to 480v equipment or close to any vfds.
> *The radios and bundles of ethernet in conduit are not likely to be a problem. The radio cable between the radio and antenna is going to be coax and won't emit any EFI. Ethernet cables are OK to bundle. *
> 
> 
> ...


*remarks above*


----------



## mburtis (Sep 1, 2018)

Thanks for all the information on this. I guess surface mount or din rail mount shallow boxes with jacks would be sufficient out in the control panels. May very well investigate fitting a mini rack in the main panel to hold router, switches, and patch panel, or just a surface mount patch panel. 

The very first experience I ever had with ethernet control was not a good one due to vfds. This was back when I was an engineer building mine parts, we put in a 250 ton hydraulic press and updated the controls with vfds on the main pump motors and a new plc. Our first attempt was to control the vfd speed via ethernet from the plc. Communications would be fine until about a second after the drive started and then nothing. We tried various things and never did get it to work. We ended up rush shipping an analogue card and hooking up a 4-20 signal that worked like a charm. Now I realize a proper shielded cable probably would have made it work, oh well at the time I was just a mechanical engineer doing control wiring.....

Of course now you guys had to go and mention fiber and send my easily distracted brain down another rabbit hole of electrical voodoo. I never realized fiber was starting to be used for inside building and plant floor control type applications. I always thought fiber was just for cross country lines or applications that absolutely had to have blazing speeds. Makes sense now why a lot of industrial switches have a port or two for fiber. Really seems like the support and price have made it a viable competitor to copper.


----------



## SWDweller (Dec 9, 2020)

The one issue I have had crop up is when the signal source, is off a different transformer than the signal receive. Sometimes grounded Cat cable can be a problem in that situation. I start with the power side and see what is up. Make a simple drawing then start connecting the signal equipment. Not a fan of wireless in industrial controls but I have used it.


----------



## mburtis (Sep 1, 2018)

Thanks for that info, I'll have to look into that a little. We have stuff spread all over so different sources of power could very well be a reality.


----------



## Jlarson (Jun 28, 2009)

That's a dumpster fire.

I try and keep the data crap out of the control panels, we'll put a little punch down block mount patch panel in to catch the PLC and whatever but that's the most I like to do. Network stuff belongs in a 19" rack, put one in it's own enclosure or back in the MCC building.


----------



## gpop (May 14, 2018)

designers expect network interference in the 60 hertz range so they design everything around that frequency. Add a radio or a vfd and you may experience problems.

I have been lucky and after running miles of cat 5, cat 6, icky, etc over the years while ignoring most of the basic rules i have never run into a network problem that could not be traced to a bad install.

Coil of extra network cable probably 50 loops hidden under a mcc and someone ran a vfd feed through the center of the coil. 
Ends installed in the wrong pattern (signal using one wire on 2 different twisted pairs) kinda works but its un-stable.
Damaged install tool so one of the pins would not hit correctly.


----------



## gpop (May 14, 2018)

mburtis said:


> Of course now you guys had to go and mention fiber and send my easily distracted brain down another rabbit hole of electrical voodoo. I never realized fiber was starting to be used for inside building and plant floor control type applications. I always thought fiber was just for cross country lines or applications that absolutely had to have blazing speeds. Makes sense now why a lot of industrial switches have a port or two for fiber. Really seems like the support and price have made it a viable competitor to copper.


We use a lot of fiber but it expensive and hard to repair unless you have the tool onsite. Redundancy like everything else is the best bet so when the primary goes down it automatically switch's to the backup and sends a alarm. Your processor and network card probably already has that feature.


----------



## paulengr (Oct 8, 2017)

mburtis said:


> Just wondering what opinions may be out there regarding wiring industrial ethernet networks, between PLCs or HMIs or the like. Does wire type matter Cat 5 vs 5e vs 6 vs something else? Solid vs stranded, factory made vs self terminated. Any other details to pay attention to?


Stranded for vibration and corrosion resistance.

CAT 6 is NOT future proofing. Look at the standards. 10BASE1000 works just fine on CAT 5E with no problems. The next jump up to 10 GBPs barely works for short distances on 5E. It doesn’t work any better on 6. In fact part if the problem is the RJ45 connector no longer has enough bandwidth. So CAT 7 was born (with multiple incompatible tricked out incompatible RJ45 connectors) hiding coax inside the cable. So there’s your future proofing…they skipped from CAT 5E to 7. No need for CAT 6 at all.

CAT 5 is dead. There are serious issues with the spec so 5E fixed all that.

Overall I’d say some kind of Ethernet is here to stay. Siemens and Ether at (who uses that?) both bank on fake Ethernet. It’s the same cable standards but they cheat and use goofy nonstandard switches. AB for their part created an incompatible 4 pin M12 style Ethernet connector but uses mostly stick switches (multi casting is an issue). But as far as using it in general it’s great. Although we generally have two dominant protocols (Modbus/TCP and Ethernet/IP) it’s easy to make networking work anywhere, any place. The biggest hassle is just recognizing that industrial Ethernet is a little different than say web browsing traffic. Prior to this we had to deal with all kinds of screwy obscure protocols. Just crack open say Automation Direct manuals and start looking at how complicated it is. “Serial” comes in hundreds if variations.


----------



## mburtis (Sep 1, 2018)

Jlarson said:


> That's a dumpster fire.
> 
> I try and keep the data crap out of the control panels, we'll put a little punch down block mount patch panel in to catch the PLC and whatever but that's the most I like to do. Network stuff belongs in a 19" rack, put one in it's own enclosure or back in the MCC building.


I've been thinking on where I could maybe add a rack to centralize all the network stuff and just run cables out to all the equipment. I would like to upgrade/consolidate some of the switches as well. Have a couple of nice switches instead of 14 Walmart consumer grade specials stacked on top of each other.


----------



## Navyguy (Mar 15, 2010)

paulengr said:


> Stranded for vibration and corrosion resistance.
> 
> CAT 6 is NOT future proofing. Look at the standards. 10BASE1000 works just fine on CAT 5E with no problems. The next jump up to 10 GBPs barely works for short distances on 5E. It doesn’t work any better on 6. In fact part if the problem is the RJ45 connector no longer has enough bandwidth. So CAT 7 was born (with multiple incompatible tricked out incompatible RJ45 connectors) hiding coax inside the cable. So there’s your future proofing…they skipped from CAT 5E to 7. No need for CAT 6 at all.
> 
> CAT 5 is dead. There are serious issues with the spec so 5E fixed all that.


I don't agree 100% with this. The Cat 7 or Cat 8 is just not a practical install unless you are in a data centre; you either need Cat 6 or jump to fibre. The Cat 7 and Cat 8 are about the size of my thumb and are not practical in any high density assembly or in a standard conduit and box arrangement. The only place they are good is open racks and tray.

Even the first generation of fibre OM1 or OS1 is still better then the best copper out there and we are now working on OM4...

Cheers
John


----------



## Jlarson (Jun 28, 2009)

All the fiber stuff is cool and great for long distance outside plant stuff. I love killing IT dreams telling them the PLC doens't need fiber run to it just to send the occasional alarm to the poor on-call bastard or to get polled every 5-10 by the control room PC lol.


----------



## splatz (May 23, 2015)

Jlarson said:


> I try and keep the data crap out of the control panels, we'll put a little punch down block mount patch panel in to catch the PLC and whatever but that's the most I like to do. Network stuff belongs in a 19" rack, put one in it's own enclosure or back in the MCC building.


I agree. I put small DIN rail mounted switches that are rated for the environment in the control cabinets for the equipment in the cabinet, that works out well. Sometimes feeding some of the automation network from there makes sense, but it's not as a rule. You don't have to locate your network's distribution points where the automation control enclosures sit. If you do, if it's more than a few network cables, you're probably better off putting a separate enclosure with rack mount rails next to it to house a patch panel and switch. 

In your case I bet it would be best to put that router right at the demarc, right next to the T1, with a switch and a patch panel and a wall mount rack. From there, branch out to distribution spots with rack mount enclosures housing a switch and a patch panel. If you branch out with fiber, you can go long distances to those distribution spots. Locate those distribution spots with the 328' distance limit from the switch to the endpoints over copper in mind. If possible, put them in a climate controlled clean area adjacent to the plant rather than on the plant floor, so you don't have to worry about using switches rated for the temperature etc.


----------



## splatz (May 23, 2015)

paulengr said:


> Stranded for vibration and corrosion resistance.


You want solid category cable. 

Stranded category cable is for patch cords and plugs and short distances, solid wire is for punch down patch panels and jacks. 

You'll find stranded category 5e and cat 6 cable isn't really stocked and sold, it's hard to find stranded CMR / CMP rated cat 5e/6. You have to install cat 5e/6 with very little pulling tension, even solid - stranded wouldn't hold up well at all. But even if it did, you don't want to have an all-plug network, that would be painful.


----------



## paulengr (Oct 8, 2017)

Navyguy said:


> I don't agree 100% with this. The Cat 7 or Cat 8 is just not a practical install unless you are in a data centre; you either need Cat 6 or jump to fibre. The Cat 7 and Cat 8 are about the size of my thumb and are not practical in any high density assembly or in a standard conduit and box arrangement. The only place they are good is open racks and tray.
> 
> Even the first generation of fibre OM1 or OS1 is still better then the best copper out there and we are now working on OM4...
> 
> ...


Ok so you proved that CAT 5E is the practical limit.

In terms of fiber you have multimode, single mode, and I’ll mention plastic fiber. If I’m future proofing hands down single mode is the way to go. You can go miles depending on termination quality. And if you need more bandwidth you can switch to cheap CWDM or if you need more DWDM. With multi mode 1 GBPs is the limit and it is pretty severely distance limited. I’ll also mention plastic fiber because it is used in controls. It’s cheap. Anyone can terminate it with basic tools. But it’s not long range or high speed. That’s fine for say IO networks.


----------



## just the cowboy (Sep 4, 2013)

splatz said:


> At a minimum I'd make it a rule to terminate cables in jacks rather than plugs (patch panels at the switch locations, surface mount boxes at the outlets) and reterminate any plugs in jacks.


We are actually going in the other direction. We are getting rid of our jacks and installing plugs. We are doing it for security reasons, we can run a cable with a plug right to a computer an lock it in a cage. This way they can't just unplug a computer and plug in the jack. Same thing with patch panels all cables get a plug and go right in the switch.


----------



## mburtis (Sep 1, 2018)

just the cowboy said:


> We are actually going in the other direction. We are getting rid of our jacks and installing plugs. We are doing it for security reasons, we can run a cable with a plug right to a computer an lock it in a cage. This way they can't just unplug a computer and plug in the jack. Same thing with patch panels all cables get a plug and go right in the switch.


Is this mostly to protect against internal threats from operators etc, knowingly or unknowingly? I'm assuming you don't have stuff out were the general public could be plugging in.


----------



## just the cowboy (Sep 4, 2013)

mburtis said:


> Is this mostly to protect against internal threats from operators etc, knowingly or unknowingly? I'm assuming you don't have stuff out were the general public could be plugging in.


Contractors and guests. Since we are 100% air gapped from the outside world the next step is to secure the rest of the system from physical threats. We are also relocating any shared space with IT so they can not accidently cross on to our network. It never stops, just the risk reduction changes.


----------



## mburtis (Sep 1, 2018)

Your approach to security seems very thorough. Doesn't seem like many places are willing to put that much effort into it


----------



## splatz (May 23, 2015)

just the cowboy said:


> We are actually going in the other direction. We are getting rid of our jacks and installing plugs. We are doing it for security reasons, we can run a cable with a plug right to a computer an lock it in a cage. This way they can't just unplug a computer and plug in the jack. Same thing with patch panels all cables get a plug and go right in the switch.


I can see how that would prevent people accidentally or casually screwing around with connections. (It would not stop someone who's willing to cut the cable...) I am pretty adamant about solid wire and jacks though. I think I'd either put a lockable in-use cover on the wall plate or terminate the cable in a little one-port surface mount jack within the cage.


----------



## Navyguy (Mar 15, 2010)

just the cowboy said:


> We are actually going in the other direction. We are getting rid of our jacks and installing plugs. We are doing it for security reasons, we can run a cable with a plug right to a computer an lock it in a cage. This way they can't just unplug a computer and plug in the jack. Same thing with patch panels all cables get a plug and go right in the switch.


There is a reason I carry these in my magic kit of stuff...








Cheers
John


----------



## paulengr (Oct 8, 2017)

splatz said:


> I can see how that would prevent people accidentally or casually screwing around with connections. (It would not stop someone who's willing to cut the cable...) I am pretty adamant about solid wire and jacks though. I think I'd either put a lockable in-use cover on the wall plate or terminate the cable in a little one-port surface mount jack within the cage.


It stops nothing. I carry an adapter (female to female) in my bag for some troubleshooting. I could just as easily plug in a small 
switch. TP Link sells a 4 port managed switch for $30 so I can Wireshark the laptop. I’ll bet the lock is easily removed.

If on the other hand the switch was programmed to say look for MAC addresses that doesn’t stop MAC spoofing but certainly ups the difficulty. And with ports and patch panels nothing to stop you from blocking open ports. Cisco even outright powers down blocked ports.


----------



## splatz (May 23, 2015)

paulengr said:


> It stops nothing. I carry an adapter (female to female) in my bag for some troubleshooting.


I think @just the cowboy is saying the cable comes unbroken out of the wall / ceiling into the cage. So they can't unplug the plug from the back of the computer without opening the cage, or cutting the cable. It wouldn't stop a Russian agent that broke in mission impossible style, cut the cord between where it comes out of the wall and the cage, reterminated it (in a jack if I have my way) and connected their device. 

Let's say a contractor comes in and is looking for an open port for their laptop. If they can simply unplug something, and plug in their laptop, they may feel it's OK to "borrow" that port, but they wouldn't feel like it's OK to pick the lock or cut the cord. 

I see this one all the time, employees want wifi for their phones so they can screw around while they're at work. So one that fancies themself the tech guy brings in a wifi router from home and plugs it in an available jack. (Sometimes they'll plug into one of the LAN ports on the router rather than the internet port, which is even worse...) They kind of know they shouldn't be doing this but figure it's no big deal. I think breaking into a locked cabinet would be enough of a speedbump to prevent that kind of stuff. 

And of course the employee that doesn't want to call IT and just starts plugging and unplugging things and doing their own troubleshooting ... again, enough of a speedbump they wouldn't do that.


----------

