# Learning industrial communications



## Peewee0413 (Oct 18, 2012)

mburtis said:


> I'm to the point now where I need to start learning more about setting up and maintaining industrial communication networks. Ethernet/IP between plcs and stuff like that. I know very little about it and have only a very basic understanding of how it works. We have extensive and complicated networks setup interconnecting plcs and the SCADA system and radio data from remote sites etc. Some of which aren't entirely reliable. It would be incredibly valuable if I could troubleshoot, maintain, otherwise work on these networks.
> 
> Any one have any advice or resources on how to learn this stuff. Should I start with learning certain foundational information first, ie learn serial communication first then move on to the newer protocols like ethernet? Are there any good websites or books or classes to take on this stuff?


Depending on where you live, there may be online courses at a local college. You usually don't have to sign up for a degree, and pay just for a single course. I paid for a plc course when transitioning from 500 to 5000. Was pretty cool actually, the instructor gave me credit for the work if I just walked around assisting the students.

Sent from my SM-G998U using Tapatalk


----------



## Wardenclyffe (Jan 11, 2019)

See if your tech school has courses leading to,









Network+ (Plus) Certification | CompTIA IT Certifications


Network+ certifies IT infrastructure skills for troubleshooting, configuring and managing networks. Learn about the certification, available training and the exam.



www.comptia.org


----------



## mburtis (Sep 1, 2018)

Thanks for that link. Appears they offer a selection of online training as well. Slippery slope of knowledge... I dont want to get roped into being an IT guy but I would like to be able to work on our IO networks.


----------



## emtnut (Mar 1, 2015)

mburtis said:


> I'm to the point now where I need to start learning more about setting up and maintaining industrial communication networks. Ethernet/IP between plcs and stuff like that. I know very little about it and have only a very basic understanding of how it works. We have extensive and complicated networks setup interconnecting plcs and the SCADA system and radio data from remote sites etc. Some of which aren't entirely reliable. It would be incredibly valuable if I could troubleshoot, maintain, otherwise work on these networks.
> 
> Any one have any advice or resources on how to learn this stuff. Should I start with learning certain foundational information first, ie learn serial communication first then move on to the newer protocols like ethernet? Are there any good websites or books or classes to take on this stuff?


I did quite a bit of networking with the SCADA stuff I've done, and with Traffic systems too.
By far, the best book I've ever read was this one. It explains the OSI layer, hubs/switches/routers, subnetting etc.
(Now you know I'm old, I actually read books lol )

For the PLC comms, it depends what you are using. There is tons of stuff on RS232/RS485 on the net.
Comms between PLCs was DH+ in my day (which was RS485).
I'd imagine most are Ethernet now, so the book above is a must.

For your Radios, depends what you are using. Spread Spectrum is used alot, and lots of stuff on the net.
If you are using licenced Radios, then reading up on Amateur Ham radio would be really helpful.

The sites that are not all that reliable, on Radio links ? That's common with Radios. Knowing ERP, fade marging, antenna gain, S/N ratios, and receiver sensitivity is a must (all googleable, search for Spread Spectrum basics or something like that)

Reading stuff is good, but just playing with the equipment and reading the user/installation manuals will really bring you up the curve.


----------



## paulengr (Oct 8, 2017)

Ethernet/IP is actually one of the toughest to learn using the Allen Bradley default settings. Plus the protocol as simple as it is, is not free (as in can’t just download the specs).

First starting with the physical signal formats and the wiring one thing about industrial networks is that they start with basic communication systems designed for computers then tweak it in some screwy way to make it proprietary and incompatible with the original standard. For instance Siemens Profinet is standard Ethernet but with nonstandard Ethernet switches and adapters.

Start with learning RS-485, RS-422, and RS-232. Those are used a lot even though they are called different things and the protocols are widely available as public information. Also try to learn as much about CAN as you can. This is another popular signal format.

Now the tough ones are Arcnet and Ethernet. These are iEEE standards and unless you download pirated versions off sci-lib you will just have to research them indirectly or pay a lot for documents you will never use. At the electrical level you will have to either use an oscilloscope or have a PC with adapter cards as your troubleshooting tools. Even scope troubleshooting is a crap shoot.

The second hardware problem with Communication in general is that generally speaking everything is “electrically long” which means it’s a transmission line. With controls we can usually assume everything is an electrical short circuit. Wire has almost zero resistance. With power we throw in a little impedance for voltage drop. With communications cables have capacitance and inductance. How you terminate cables is important, especially Ethernet. Unwrapping even inches of twisted pair causes communication failures. With RS-485/232 shielding and EMI matter a lot. This is where standard electrical wiring practices are very different.

Beyond this everything else is just data formatting on top of it. RS-485 and RS-232 use 10 bit symbols (start bit, 7 or 8 Dara bits, parity bit if it’s 7 Dara bits, stop bit). Then the data format on top of this varies. This is the foundation of Modbus and Siemens Profibus and even AB Bluehose (although that one requires nonstandard hardware).

With Ethernet we get into switches and routers. In some ways there is a movie out there that does a great job explaining this but it’s not hard. This is just how we get data from one physical device to another.

Finally we get to the IP packet which is a standard freely available published in an RFC. It’s VERY simple. On top of that we have ARP, UDP, and TCP. If you have a basic knowledge and you download and use Wireshark (free) you can easily peek at data packets and see what’s inside. Almost everything internet related is built on these 3 protocols. For instance with Modbus they took the original Modbus RS-485 packet and simply transmit them inside TCP packers. When it gets to the PLC the software unwraps the IP packet coming from the hardware then it unwraps TCP and the data passes to the Modbus software. In straight Modbus it just goes from hardware (UART chip) to the Modbus software.

Ethernet/IP has a couple twists and turns. First we start with Controlnet. Controlnet is standard Arcnet physically except is uses 10 Mbps instead of 5 Mbps and uses 50 ohm coaxial cable with BNC connectors. The data packet format is universal. But the group that publishes the standard wants several thousand dollars for the details. You have to hunt around to find it. In terms of reading it unlike say tge RFCs the Controlnet stuff is very, very hard to read. It also includes something called a device profile. This means they recognize that from a PLC point of view VFDs are all pretty similar so they have a standard data format for those so that you can in theory swap say an Allen Bradley for a Schneider drive. With AB ZpLCs in particular the VFDs do for instance implement a standard VFD profile but Allen Bradley uses proprietary nonstandard data formats for everything in their PLCs. So even though they write the protocol, they don’t use it! Moron and others do.

So taking the same data packet format if we stick it on CAN bus this us called Devicenet.

So that brings us to Ethernet. This is where things break down. With CAN and even Arcnet it’s a broadcast system. Within reasonable limits every device on a cable sees every other device and we can have as many devices as we need. The original “thick” and “thin” Ethernet worked the same but nobody has used those cables in decades. We use twisted pair cables with RJ-45 connectors between just two devices and with current speeds multiple device broadcast even with hubs is simply impossible. As I mentioned Modbus recognizes that all we need is communication between ONE PLC and ONE say IO card. But Controlnet is not that way. You are allowed to have multiple PLCs reading data from one IO card. And there is another twist because of its Arcnet upbringing. With Modbus as an example communications is always request/response. So a PLC will send a command to say a VFD asking “send me the RPM of the motor” and the VFD responds “1000”. In Arcnet you have defined time slots for every device to talk. If it doesn’t have anything to send the slot is just empty. So in Controlnet it makes a lot more sense for a PLC to send a command to a VFD like “send me the RPM every 100 milliseconds”.. The VFD responds with the time slot location. Another PLC can ask and get the same response.

Ethernet doesn’t work like this but it is so much faster we can just do the same thing without time slots. But IP Ethernet is inherently point to point, not broadcast. It can do it and UDP is the way to do it but we need a way to make switch touting work without flooding all devices with control packet traffic. So in Ethernet/IP we use request-response like Modbus/TCP. And we can set up Arcnet style “send me X even Y milliseconds” over UDP. And to duplicate the broadcast function we use multi casting. This is the one tricky part because 99.99% of Ethernet traffic never uses mukticasting. So although it uses standards setting up and getting this part right is not easy. Also flow control is necessary with the inherently low power communication cards used industrially which is poorly implemented by Cisco among others so again, lots of details here. Now this only matters because of two things. The first is if you use multiple PLCs reading the same IO, which is obviously pretty rare. The second reason is that Aljen Bradley PLCs default to using multicasting. If you turn this stuff off then you never need to worry about it but few AB programmers do or recognize how bad of a decision it is.

Anyways just to give you a taste. Networking is almost an entire area of its own. It is often a source if all kinds of problems. Most of the issues are either bad connections/terminations, bad adapters/ports, or bad setups and configuration.


----------



## SWDweller (Dec 9, 2020)

I was fortunate to be able to take the classes for an MCSE, when I was working. I immediately identified lots of holes in Microsoft's operating system. Wait that was back on 98SE. I loved NT and its stead fast security requirements. None of this I forgot my password so crack the log in. Ethernet is a protocal that is used a lot nowadays. It is full of holes by design. Makes keeping the processes secure harder.

485 has 2 protocols, 2 wire and 4 wire. Better add Modbus, Impacc, Blue Tooth and what ever else is common to your area to the list. Impacc has two flavors, the one that came from Westinghouse and the other one that ABB used on medium voltage circuit breakers. Sorry I do not know that history. No they did not talk to one another. All of the protocols have a distance maximum as well as a type of cable to be used. 
Impacc was almost bullet proof, you could loose one wire on the communication device and the network would still work under certain conditions. One of those stranger than fiction situations.

There are a ton of protocols and programs to get those bits and bites to the other machine. Some more complicated than other solutions. Since a lot of this stuff has aged gracefully you may find fixing/repairing is out of the question and replacement is the solution. When I was doing this, I had a dual boot computer, with admin privlages,To hang the laptop as a trouble shooting tool I had to change the IP address. IT departments would go wild.

I suggest Black Box as a place to start reading. They have case studies and the people are tops in their field. Always willing to help you with a solution. Maybe not the cheapest but it worked. Damned fast shipping as well. 

I always liked the variety of being a field electrician. I was offered several jobs in IT, they paid less. Might want to check on that today. Your going to need the support and help from a manager or two to head down this road. Jobs will pay more but will be a lot harder to find.
Planning on this as a career, better get your life in order. Security is a bitch in today's world. I had credentials from the DOE, I know for a fact that the FBI went to my high school and talked to my principal. They do and will tear you life open. Native born USA with parents that were natives made a big difference when my sweep was done. 

Congrats and get ready for the adventure as you acquire the skills.


----------



## mburtis (Sep 1, 2018)

Thanks for the responses, dont understand a big portion of it but gives me lots to research. Guess I'll start with trying to understand 232/485 since that's the basic building blocks. We will see how far down the rabbit hole I end up going. Honestly I hate computers and would rather live in a world of relays and 4-20 mA signals but I was born decades late for that. Definatley dont want to become a full time IT or low voltage guy, but when we lose comms it would be nice to know more than is the cable plugged in and is the little light blinking.


----------



## just the cowboy (Sep 4, 2013)

Focus only on what you use, and think black box. You do not need to know how to write the code on how to pack it you only need to know did it get there and how did it get there. ( for now )
Start with a flow chart showing your networks and subnetworks. Show IP address and station address with each subnetwork in a different color and learn what gets the data from one to the other ( router, switch, PLC or protocol convertor). Automation direct has some free classes and good free support on network setup and troubleshooting. Buy a few of their cheap productivity PLC's and network devices and start playing.


----------



## mburtis (Sep 1, 2018)

I actually unboxed a BRX plc and a headless c more hmi this morning so that we can play with plc and hmi programming and communications a little.


----------



## SWDweller (Dec 9, 2020)

I was working at a university and the boss went out and bought serial number 2 RPM Harmonics Analyzer and a 486-16 laptop with a color screen. First cabin back then. (Now owned by Danaher) No one volunteered to learn it. Then the voice in the back give it to the new guy. Want to guess my title? Lots of support from the bosses and fairly quickly the graphs made sense. Solved a lot of issues. Learning new things in today's world is one way to keep yourself employed over the long haul. It is hard to go home for the evening knowing some dirt just kicked your ass. 

One thing I did not bring up. Monitoring is heavy data, depending on the number of devices and the amount of information collected you can be talking about gigs a day, even terabytes. 
The reason I bring this up do not do what I did, crash the firewall at a nuclear weapons depository. You get to meet a LOT of armed people with helmets and shoes that you can see yourself in. Been there done that and got the t shirt. Thank god I wrote everything down including names, telco numbers and titles. People did go to the guardhouse over it.


----------



## mburtis (Sep 1, 2018)

So this question is going to betray my utter lack of knowledge in this area but here goes nothing. The RS 232 or RS 485 is basically just the physical/electrical side of things? Basically send pulses of voltage ( 232 referenced to ground 485 referenced between + and -) down the wire. You still need a protocol that determines what those binary codes of voltage mean to the devices on either end? This is where say ASCII or modbus comes in or any host of proprietary BS so that automation companies can ensure your stuck with their brand?


----------



## splatz (May 23, 2015)

mburtis said:


> So this question is going to betray my utter lack of knowledge in this area but here goes nothing. The RS 232 or RS 485 is basically just the physical/electrical side of things? Basically send pulses of voltage ( 232 referenced to ground 485 referenced between + and -) down the wire. You still need a protocol that determines what those binary codes of voltage mean to the devices on either end? This is where say ASCII or modbus comes in or any host of proprietary BS so that automation companies can ensure your stuck with their brand?


You have the right idea, in the networking classes they teach the OSI model to help understand this. To be honest I don't think it's that great of a model but the idea is it's layered. You can search for "OSI Model" and find about a zillion articles even though nobody really thinks about this outside of taking certification tests. 

The lower level protocols handle how you take data, an abstract thing, into a physical thing, and back. So for example RS-232 is a plan for turning data into voltages on a set of wires, along with a scheme for the two devices taking turns, checking for errors in transmission, etc. Basically RS232 is a way of sending and receiving a bunch of ones and zeros (bits) between two devices, RS485 is a way of sending and receiving bits between multiple devices on only two wires. 

These lower level protocols may hand off the bits directly to the application or there may be intermediate protocols in between but ultimately the stream goes to the software in the device and that software uses the stream to communicate, send and receive information. Modbus for example is a higher level protocol that can send and receive the stream over a variety of lower level protocols. So in very some cases the lowest level communications are 1000baseT (gigabit ethernet on twisted pair wire) and at intermediate levels TCP/IP, at a higher level ModbusTCP. That's a bit of an oversimplification but how far do you want to dive down this rabbit hole. 

Breaking things up into layers and protocols based on standards makes it so that the makers don't have to reinvent the wheel every time they build a device, build a communications method from the ground up, and protocols and standards allow things to interoperate, so different products from different makers can communicate right out of the box without the makers needing to work together.


----------



## STEM (Jan 16, 2015)

Practically speaking, move forward with this and ignore the past and the old stuff for now, just to get a decent start. In North America the two most prolific industrial networks are MODBUS/TCP and ETHERNET/IP. European fieldbus networks are not uncommon but so many protocols and different media and you only have so much time.

But wait... these are just two of the thousands of protocols available to be used over a TCP/IP network. You need to learn basic networking first. Protocols that the modern internet depends on. If you don't know the difference between a bridge, router, switch and a hub, if you can't describe what NAT means, if you don't know what a DHCP or DNS server does, then try this basic primer on-line or search for your own. Basics of Computer Networking - GeeksforGeeks

I would not worry too much about serial media - note media, not protocols. RS232 and RS485 are definitely still around but thankfully, not many people are installing it on a new installation in most industries.

MODBUS/TCP is the same old MODBUS from the serial days just encapsulated into Ethernet frames. Essentially MODBUS in any flavor on any media has a number of numerical functions to choose from for data transfer functions.

Same goes basically for Ethernet/IP although there are important additions since the days of DeviceNet and DF1 protocols. ODVA is the organization that administers DeviceNet, CIP and Ethernet/IP. The protocol manuals are available to download from the ODVA site. EtherNet/IP™ | ODVA Technologies | Industrial Automation

The most important thing to note about these two protocols is that they can both be the same, but Ethernet/IP can be different. Much different. When they are the same, you are using messaging program functions to send data back and forth usually point to point to point. Where Ethernet/IP gets different is in it's second "mode" (implicit messaging) where it adopts a produced/consumed model and maps data into your PLC IO tree directly, eliminating the need for messaging instructions and allowing for multiple listeners or mapping other PLCs IO to yours.

Be aware, the more you think you know, the more likely you are to be humiliated.


----------



## STEM (Jan 16, 2015)

mburtis said:


> I actually unboxed a BRX plc and a headless c more hmi this morning so that we can play with plc and hmi programming and communications a little.


Nice. I love that little BRX and CMore line of products. Done quite a few projects with them and they're big bang for the buck. If you need any pointers, don't be shy.


----------



## gpop (May 14, 2018)

If you really want to go down the rabbit hole then buy a couple of arduino's. You will learn how to program baud rate, hand shakes and all sorts of stuff that is normally done for you in a back ground of a plug and play network. You can learn to work with serial, can-bus, 232, 485, ethernet and a bunch of other networks. 

If you do not want to go to far down the hole then really all you need to learn is how to set up your laptop. Simple things like port assignment, fixed ip addressing, baud rate, cmd prompt instructions for ping and dns flushing are more the bread and butter side of this job. Hart is also a useful network to learn but its a royal pain in the arse. 
Half the fight is setting up the parts (hardware and software) you need for your laptop to talk to hardware no matter what network you are using.


----------



## splatz (May 23, 2015)

gpop said:


> If you really want to go down the rabbit hole then buy a couple of arduino's. You will learn how to program baud rate, hand shakes and all sorts of stuff that is normally done for you in a back ground of a plug and play network. You can learn to work with serial, can-bus, 232, 485, ethernet and a bunch of other networks.
> 
> If you do not want to go to far down the hole then really all you need to learn is how to set up your laptop. Simple things like port assignment, fixed ip addressing, baud rate, cmd prompt instructions for ping and dns flushing are more the bread and butter side of this job. Hart is also a useful network to learn but its a royal pain in the arse.
> Half the fight is setting up the parts (hardware and software) you need for your laptop to talk to hardware no matter what network you are using.


It's easy enough to do all that without the arduino's and the coding though, you can do all that with two computers, say a desktop and a laptop, but you'll probably have to buy a usb-serial adapter for the laptop but most business desktops still have a serial port. There are free terminal programs you can use to make the two talk.


----------



## gpop (May 14, 2018)

splatz said:


> It's easy enough to do all that without the arduino's and the coding though, you can do all that with two computers, say a desktop and a laptop, but you'll probably have to buy a usb-serial adapter for the laptop but most business desktops still have a serial port. There are free terminal programs you can use to make the two talk.


Cheaper to screw up $10 worth of arduinos plus you get the fun of wiring tx and rx.


----------



## mburtis (Sep 1, 2018)

Thanks for all the replies. I'll admit I have no clue what 3/4s of it means but I'll be reading on it as I have time. Like I said we bought a plc and a hmi which are both serial and ethernet capable so I'll start with just trying to Google my way to getting those to talk to each other. I probably wont understand what I'm doing but I can mash keys. We have some chemical feed pumps that are serial capable too so I can try to hook that up. If I can get comfortable with connecting to the network and hooking the legos together that will go a long ways. Hopefully by playing with the components everything else will slowly start making sense. Due to security and comfort I think I'll stick to working on little stand alone stuff for secondary equipment and work into troubleshooting our existing stuff. Meanwhile I have lots of avenues and acrynoms to google.


----------



## gpop (May 14, 2018)

mburtis said:


> Thanks for all the replies. I'll admit I have no clue what 3/4s of it means but I'll be reading on it as I have time. Like I said we bought a plc and a hmi which are both serial and ethernet capable so I'll start with just trying to Google my way to getting those to talk to each other. I probably wont understand what I'm doing but I can mash keys. We have some chemical feed pumps that are serial capable too so I can try to hook that up. If I can get comfortable with connecting to the network and hooking the legos together that will go a long ways. Hopefully by playing with the components everything else will slowly start making sense. Due to security and comfort I think I'll stick to working on little stand alone stuff for secondary equipment and work into troubleshooting our existing stuff. Meanwhile I have lots of avenues and acrynoms to google.



Sounds like a good plan. Honestly you will learn more in few days getting your butt kicked with a real life problem then you will learn in months reading a book. 
I still use you-tube as a lot of manufacturers have training/user videos that fill in the blanks that seem to be missing in the manual. (hell i even had to watch a video on rockwell last week just to see where the down load button was hidden on there website. The fact there are 20 videos online showing how to do something so simple suggests that there website really sucks)


----------



## STEM (Jan 16, 2015)

mburtis said:


> Thanks for all the replies. I'll admit I have no clue what 3/4s of it means but I'll be reading on it as I have time. Like I said we bought a plc and a hmi which are both serial and ethernet capable so I'll start with just trying to Google my way to getting those to talk to each other. I probably wont understand what I'm doing but I can mash keys. We have some chemical feed pumps that are serial capable too so I can try to hook that up. If I can get comfortable with connecting to the network and hooking the legos together that will go a long ways. Hopefully by playing with the components everything else will slowly start making sense. Due to security and comfort I think I'll stick to working on little stand alone stuff for secondary equipment and work into troubleshooting our existing stuff. Meanwhile I have lots of avenues and acrynoms to google.


If you want to talk to both the HMI and PLC over Ethernet, order the bargain switch so you can plug in and see both at the same time. SE2-SW5U 
Both the Cmore and the BRX software will help you address both on the same subnet when you try to connect so you can talk to them and they can talk to each other.
Like gpop said, nothing like the real thing to actually learn it!


----------



## just the cowboy (Sep 4, 2013)

STEM said:


> Practically speaking, move forward with this and ignore the past and the old stuff for now, just to get a decent start. In North America the two most prolific industrial networks are MODBUS/TCP and ETHERNET/IP. European fieldbus networks are not uncommon but so many protocols and different media and you only have so much time.


He is in water like me. Between serial radios and existing old stuff it never goes away. We actually just built a new plant and Modbus 485 was used by two manufactures one for generator and the other was power monitor, that's all they offered.
We also use Modbus 485 as network segregation to pass data off our network to other towns water treatment plant's PLC's. One of my guys is putting a Modbus Panelview in now so the only thing leaving the building to an unsecured area is 485 no way to get to a switch/network.

Cowboy


----------



## STEM (Jan 16, 2015)

just the cowboy said:


> He is in water like me. Between serial radios and existing old stuff it never goes away. We actually just built a new plant and Modbus 485 was used by two manufactures one for generator and the other was power monitor, that's all they offered.
> We also use Modbus 485 as network segregation to pass data off our network to other towns water treatment plant's PLC's. One of my guys is putting a Modbus Panelview in now so the only thing leaving the building to an unsecured area is 485 no way to get to a switch/network.
> 
> Cowboy


You would love this then. I use this on any old Modbus serial networks.


https://download.schneider-electric.com/files?p_enDocType=User+guide&p_File_Name=63230-319-211A1EN-ES-FR.pdf&p_Doc_Ref=63230-319-211A1EN-ES-FR


----------



## mburtis (Sep 1, 2018)

just the cowboy said:


> He is in water like me. Between serial radios and existing old stuff it never goes away. We actually just built a new plant and Modbus 485 was used by two manufactures one for generator and the other was power monitor, that's all they offered.
> We also use Modbus 485 as network segregation to pass data off our network to other towns water treatment plant's PLC's. One of my guys is putting a Modbus Panelview in now so the only thing leaving the building to an unsecured area is 485 no way to get to a switch/network.
> 
> Cowboy


I wish someone put that much thought into our system. Our Scada system has been hacked twice in the last year. Just robots and didn't affect anything operationally but still caused us to be without our remote laptop for several weeks and compromised minor amounts of data, plus screwing our alarm dialer up. Dont worry though the city hall IT guy is going to swoop in a save the day..... one reason I want to learn basics so I can have redundant stand alone systems set up, in no way connected to anything outside the plant. That way if the computer goes down it goes in the garbage can and we can still make water.


----------



## STEM (Jan 16, 2015)

mburtis said:


> I wish someone put that much thought into our system. Our Scada system has been hacked twice in the last year. Just robots and didn't affect anything operationally but still caused us to be without our remote laptop for several weeks and compromised minor amounts of data, plus screwing our alarm dialer up. Dont worry though the city hall IT guy is going to swoop in a save the day..... one reason I want to learn basics so I can have redundant stand alone systems set up, in no way connected to anything outside the plant. That way if the computer goes down it goes in the garbage can and we can still make water.


Hopefully IT has at least given you administrative access to your own laptop. If they have, you should learn about and start using Virtual machines. I have over a dozen with different operating systems and software for different sites. For a single site for maintenance the bonus is, you can back up the VM files often and just restore if something ruins your VM, you can start the VM on another computer if your laptop vanishes or if IT takes it to upgrade you to another laptop, the time it takes to copy the VM onto your new machine is the time it takes to set up you new laptop instead of days of installation and fiddling. Here's a screen shot of two VMs running at the same time on my Windows 10 laptop.


----------



## mburtis (Sep 1, 2018)

So there is actually a virtual machine running on our scada computers. I dont understand any of it but i think that all of our programming is running on the VM and then there is just a client running that the operators actually see. All that stuff is over my head and no way I'm touching that card house program


----------



## emtnut (Mar 1, 2015)

mburtis said:


> I wish someone put that much thought into our system. Our Scada system has been hacked twice in the last year. Just robots and didn't affect anything operationally but still caused us to be without our remote laptop for several weeks and compromised minor amounts of data, plus screwing our alarm dialer up. Dont worry though the city hall IT guy is going to swoop in a save the day..... one reason I want to learn basics so I can have redundant stand alone systems set up, in no way connected to anything outside the plant. That way if the computer goes down it goes in the garbage can and we can still make water.


Both the water plant and later at Traffic Ops, we had our own network ... separate from corporate IT.

It's VERY common for many utilities to do this. You CAN NOT have IT run an essential network. Just.doesn't.work !


----------



## STEM (Jan 16, 2015)

mburtis said:


> So there is actually a virtual machine running on our scada computers. I dont understand any of it but i think that all of our programming is running on the VM and then there is just a client running that the operators actually see. All that stuff is over my head and no way I'm touching that card house program


You should copy the VM files then on planned downtime. It most likely is a VM on Microsoft Hyper-V but it could also be VMWare, which I use. Never the less, you'd be wise to have a vackup copy of the VM files themselves. Once you've done that, you could take yet another copy and play with it an learn something about it


mburtis said:


> So there is actually a virtual machine running on our scada computers. I dont understand any of it but i think that all of our programming is running on the VM and then there is just a client running that the operators actually see. All that stuff is over my head and no way I'm touching that card house program


You should copy the VM files on planned downtime. It most likely is a VM on Microsoft Hyper-V but it could also be VMWare, which I use. Never the less, you'd be wise to have a backup copy of the VM files themselves. Once you've done that, you could take yet another copy of it and play with it an learn something about it on your own laptop.

I also agree that most IT departments DO NOT KNOW anything about industrial networks or industrial software. Point in case: Ask them if they can explain what Ethernet/IP or MODBUS/TCP is.


----------



## mburtis (Sep 1, 2018)

I think it may be vmware actually. All I know is that it seems one wrong look away crashing. The condensending little mole rat programmer that set it up 3 years ago tried to walk the plant supervisor through backing it up and it was so confusing the plant supervisor gave up. The local programmer we hire all the time doesn't even want to touch it. As much as I didn't get along with him it's not entirely the programmers fault. It's essentially the same plc and scada program from 1993 that started out on plc 5, migrated to slc500 and now is on controllogix, the computer side of the program is just band aided and migrated as well so it's just great.

I always used to say that I just herded the angry pixies and once the angry pixies became imaginary it was not my problem. Dipping my toe into the imaginary bits and we will see where it goes. My attitude towards computers has always been kinda like Quigley down under and pistols... never had much use for one but will use one if I have to.


----------



## STEM (Jan 16, 2015)

mburtis said:


> I think it may be vmware actually. All I know is that it seems one wrong look away crashing. The condensending little mole rat programmer that set it up 3 years ago tried to walk the plant supervisor through backing it up and it was so confusing the plant supervisor gave up. The local programmer we hire all the time doesn't even want to touch it. As much as I didn't get along with him it's not entirely the programmers fault. It's essentially the same plc and scada program from 1993 that started out on plc 5, migrated to slc500 and now is on controllogix, the computer side of the program is just band aided and migrated as well so it's just great.
> 
> I always used to say that I just herded the angry pixies and once the angry pixies became imaginary it was not my problem. Dipping my toe into the imaginary bits and we will see where it goes. My attitude towards computers has always been kinda like Quigley down under and pistols... never had much use for one but will use one if I have to.


I hate to hear of people making technology seem harder than it is just to make themselves look important. All you have to do is copy the darn folder where the VMWare machine is to another location WHEN the VM is shutdown. It's that simple. Then make a copy of the copy to play with BUT don't start the copy up on the same network as the original because they'll both have the same IP address and you also don't want it talking to anything either that the real one is talking to. You must have a PLC or some kind of logical processor it's getting it's info from.

OK now there's a caveat to all of this. I assumed that the base (host) computer actually has an operating system like Windows 10. If that's correct then for your laptop to play with the VM "offline" you can download (after you register for free) the VMWare player here:





VMware NOW


VMware now



my.vmware.com





One last thing. The first time you start up a copied VM it's going to ask you if you copied or moved it. Say you moved it! Depending on what VMWare feels like that day if you say you copied it, you may have to enter a new license number in.

This isn't hard and copying it as a backup is absolutely necessary even if you never intend on playing with it. Here's all there is to a VM. Note the .VMX file is the file that the player will use to execute the VM but it's just a text file you can look at if you drop in in notepad. It describes the computer to the VMWare engine. You an see the biggest file is the one that is your VM's hard drive and the next biggest is your VM's memory.


----------



## just the cowboy (Sep 4, 2013)

mburtis said:


> I wish someone put that much thought into our system. Our Scada system has been hacked twice in the last year. That way if the computer goes down it goes in the garbage can and we can still make water.


I am completely air-gapped using a Datadiode to push out reporting, no VPN for operators or SCADA staff.


emtnut said:


> Both the water plant and later at Traffic Ops, we had our own network ... separate from corporate IT.
> It's VERY common for many utilities to do this. You CAN NOT have IT run an essential network. Just.doesn't.work !


It took me a year to win this battle with IT. They now don't have ANY access or information on my network.


STEM said:


> I also agree that most IT departments DO NOT KNOW anything about industrial networks or industrial software. Point in case: Ask them if they can explain what Ethernet/IP or MODBUS/TCP is.


They were always saying " you need this NEW update installed now ". They have no concept of proofing updates across many different software platforms. We always asked " is it on the Rockwell approved list" and they would say I don't know but you should be good we tested it on our stuff with no problems.

It's good when I see we all deal with this, but I had great support to break away from IT .


----------



## LARMGUY (Aug 22, 2010)

All of the above is ok but I have found 90% or more of the troubleshooting I have done on non communicating systems all boiled down to the basics of the system. Cabling, installation, terminations, environmental changes, other trades, upgrades to systems causing noncompatibility, and updates to software. Your system is only as good as its weakest link.

I also remember an issue with communication that involved devices from different companies and every one of them was pointing to the other company and their equipment as the problem. When you start asking tech support when their flight will arrive you get the highest level of engineering support. Turns out after their flight arrived and they pulled out an analyzer that actually showed the strings of programming it was only one company and their equipment was configured wrong at the factory.


----------



## just the cowboy (Sep 4, 2013)

LARMGUY said:


> All of the above is ok but I have found 90% or more of the troubleshooting I have done on non communicating systems all boiled down to the basics of the system. Cabling, installation, terminations, environmental changes, other trades, upgrades to systems causing noncompatibility, and updates to software. Your system is only as good as its weakest link.
> 
> I also remember an issue with communication that involved devices from different companies and every one of them was pointing to the other company and their equipment as the problem. When you start asking tech support when their flight will arrive you get the highest level of engineering support. Turns out after their flight arrived and they pulled out an analyzer that actually showed the strings of programming it was only one company and their equipment was configured wrong at the factory.


I am having a project done right now involving a high speed ring, end point radios, polling, monitoring and network routing. I specifically specked that a single company be responsible to avoiding finger pointing. I only want to point at the one company and say " Well what is the fix "


----------



## GrayHair (Jan 14, 2013)

I hated being on somebody else's network. You get IT to open ports for you, a few months later an IT guy closes them and who gets the blame.
IT readdresses interfaces one weekend and gets one of yours because somebody left it unprotected.
They do corporate operating system updates that break something you use and you have to kludge a temporary fix. Heaven forbid that you rollback one of their updates!


----------



## mburtis (Sep 1, 2018)

just the cowboy said:


> I am completely air-gapped using a Datadiode to push out reporting, no VPN for operators or SCADA staff.
> 
> It took me a year to win this battle with IT. They now don't have ANY access or information on my network.
> 
> ...


So do you have any sort of remote access to the SCADA system? We have a laptop the on call person takes home with a VPN setup so they can make simple adjustments at night rather than having to come into the plant. We run unmanned 16 hrs a day. Nobody in the plant want the IT guy in our stuff but we dont have the knowledge to do it by ourselves and city hall doesn't understand the kind of money this sort of network takes to maintain.


----------



## just the cowboy (Sep 4, 2013)

mburtis said:


> So do you have any sort of remote access to the SCADA system? We have a laptop the on call person takes home with a VPN setup so they can make simple adjustments at night rather than having to come into the plant. We run unmanned 16 hrs a day. Nobody in the plant want the IT guy in our stuff but we dont have the knowledge to do it by ourselves and city hall doesn't understand the kind of money this sort of network takes to maintain.


We are now 24/7 operator staffed. We gave up the VPN for SCADA to remote in so we have to come in if needed. We are going to install a screen view only on the " Safe " side for supervisors to log in and look at screens, but they need an operator on the inside to change settings or screens.


----------



## mburtis (Sep 1, 2018)

I always joke that we just need to go back to shifts and be staffed 24/7 but it would never fly.


----------



## just the cowboy (Sep 4, 2013)

mburtis said:


> I always joke that we just need to go back to shifts and be staffed 24/7 but it would never fly.


Our operators work 4-10 hour days wed -sat and sun - wed. We run three shifts with overlap for turnover and paperwork


----------



## DragnUp (Jun 18, 2021)

i got thrown into the fire in a tiny company that was running 45+ serial point-to-point Freewave radios to a single master. Emerson/Fisher ROC 407's for the most part. I read and I asked questions and I experimented and I cursed the old boards that couldn't be relied upon but also couldn't be easily or cheaply replaced. I got my little COM port adapters and virtual COM port dongle and multimeter etc....

I wasn't smart enough to do any deep troubleshooting or clever configs... but i did begin to understand how those things _might_ be done... 

I really prefer being out in the field. I set this up myself, all from used parts I scrounged up. Got it talking and hooked up a couple meters to it and got it on the SCADA system.


----------



## emtnut (Mar 1, 2015)

DragnUp said:


>


So, 900Mhz spread rectum, 11 dbi antenna, horizontal polarization .... am I close ?


----------



## gpop (May 14, 2018)

If i vpn in it will take at least 10 minutes to pass all the security barriers and if the laptop or phone looses signal i get kicked off and have to redo all the security barriers again. When it works its great when it wants to be a pain its a real pain.


----------



## mburtis (Sep 1, 2018)

gpop said:


> If i vpn in it will take at least 10 minutes to pass all the security barriers and if the laptop or phone looses signal i get kicked off and have to redo all the security barriers again. When it works its great when it wants to be a pain its a real pain.


Wow that's way more secure than any of our stuff. Probably why our stuff keeps getting messed up.


----------



## paulengr (Oct 8, 2017)

mburtis said:


> Thanks for all the replies. I'll admit I have no clue what 3/4s of it means but I'll be reading on it as I have time. Like I said we bought a plc and a hmi which are both serial and ethernet capable so I'll start with just trying to Google my way to getting those to talk to each other. I probably wont understand what I'm doing but I can mash keys. We have some chemical feed pumps that are serial capable too so I can try to hook that up. If I can get comfortable with connecting to the network and hooking the legos together that will go a long ways. Hopefully by playing with the components everything else will slowly start making sense. Due to security and comfort I think I'll stick to working on little stand alone stuff for secondary equipment and work into troubleshooting our existing stuff. Meanwhile I have lots of avenues and acrynoms to google.


The thing with RS-232 is that one signal is ideally +15 V to ground and the other is -15 V to ground. I forget which is which as far as a zero or one. RS-485 is the same way except you either use the same DC voltage between two wires or you invert it, and the voltages are much lower, like 2-3 V. This does two things. It avoids problems with ground loops and grounding. Since there is a third choice, no signal, you can have multiple connections. And with lower voltages it can use lower capacitance cable and go much longer distances, like several miles, To make bytes we transmit a one (start but), then 8 bits of data, and one or two additional one bits (stop bits) or something like that. With parity there are only 7 data bits. For the 8th but we count “1s” in the data. If it’s odd we make the last bit a 1 so the total 1s are even. For odd parity we do the opposite but everyone uses even. With CAN to increase the number of bits per symbol they transmit a longer data pulse to encode several bits at once.

With 10 Mbps Ethernet a 1 is a positive voltage for a half bit and a negative voltage for a half bit, and a 0 is just the opposite. This is done so the signal looks roughly like a square wave centered around 20 MHz instead of 0 Hz so it avoids interference with 60 Hz power. With 100 Mbps and gigabit Ethernet the signal pattern uses multiple patterns and voltages to send several bits at once. Very hard to explain in words but it’s based on 5/7 coding.

In reality mist of the tine we watch the status bits and check cable problems. I crack out Wireshark for actual network problems which is rare but very powerful for catching things like when accounting bombards the system with huge data dumps at the end of the month because flow control was never done by IT.


----------



## gpop (May 14, 2018)

mburtis said:


> Wow that's way more secure than any of our stuff. Probably why our stuff keeps getting messed up.


 99% of security issues are the ID 10 T's sitting behind the keyboard handing out there passwords.


----------



## paulengr (Oct 8, 2017)

gpop said:


> 99% of security issues are the ID 10 T's sitting behind the keyboard handing out there passwords.


I like how IT people will just arbitrarily change settings in the middle of the day on the switches, then leave for lunch and take down a plant for hours.

Seriously here is the magic trick for plants and contractors. You run the cable and terminate it for the IT crowd. Install servers and racks too. They are scared of anything more complicated than a USB keyboard. Once you get them dependent on you, you have the keys to the kingdom.


----------



## mburtis (Sep 1, 2018)

paulengr said:


> I like how IT people will just arbitrarily change settings in the middle of the day on the switches, then leave for lunch and take down a plant for hours.
> 
> Seriously here is the magic trick for plants and contractors. You run the cable and terminate it for the IT crowd. Install servers and racks too. They are scared of anything more complicated than a USB keyboard. Once you get them dependent on you, you have the keys to the kingdom.


This is my number one concern with letting city hall it into our system. I just know at some point they are going to up date the security software or something else and it's not going to recognize the rockwell licenses or something and block everything. Of course IT likes to do this stuff about 7 PM on a friday, where as we touch nothing past lunch on friday. 

It was pretty funny the other day, the IT guy was at the plant and we got talking about moving some switches around and demo a bunch of old equipment. The IT guy is one of those poeple who always has to be the smartest and most important in the room and he was going on and on about how he wanted to do all this stuff but didn't have the time. I told him if he told me where he wanted the equipment I could run the wires for him. He got all puffed up and replied well you cant do that without a low voltage license. I just looked at and told him I'm the plant electrician I think I can handle a 20 ft ethernet cord.


----------



## mburtis (Sep 1, 2018)

gpop said:


> 99% of security issues are the ID 10 T's sitting behind the keyboard handing out there passwords.


This last hack was caused by something even better. Apparently the plant nor IT got an invoice for the firewall software or some thing. So it never got paid, so we were completely open for 3 days. Dont worry though IT is going to handle all the software licensing and billing for us now including rockwell. That wont be an issue at all....


----------

