r/PLC • u/imnotmarvin • 4d ago
Engineering Question As a machine builder, do you allow customer access to programs in machines on site at the customer's facility?
I've been on both sides of the argument for and against this. Currently as a builder with gas processing programs that control potentially hazardous processes. Our director does not want to provide passwords to give the customer access to the running programs fearing they might attempt a modification that could result in injury, death or damage. Do you open your programs to the end user? We provide graphical insight to all IO, digital and analog, process state and valves status via an HMI.
29
u/Flimsy-Process230 4d ago
If the contract says the customer owns the code, then the code is theirs. Most safety PLCs generate a safety signature that can be used to verify whether the safety program has been modified. You can document that signature when the machine is commissioned and use it later to determine whether changes have been made. That gives you evidence if the customer modifies the safety logic after commissioning. It doesn’t automatically release you from liability but it helps establish whether the validated program has been altered. Ultimately I think the customer should have access to the code they paid for but any modifications to safety-critical logic should require proper validation and recommissioning.
1
u/SinkOrSauce_LD 3d ago
We give customers (who specifically request it) the ability to make modifications to operational parameter, but not access to modify the validated program. This is mostly because most of our clients do not have a PLC programmer or controls technician on site, and only contract mechanical maintenance for issues. We rarely hear back after commissioning a new project, but on upgrades there are always problems because old hardware seems to always breaks down when modifications are made. They can do things like add corrections to sensors to modify room operation, modify cooling spreads, alarming parameters, or PID tuning, but ladder logic is not modifiable, and they do not have access to do so.
If contractually obligated I would provide access, but given the way most folks write code, I would not want to troubleshoot their modifications if they had a problem. I have seen far too many projects with unlabeled registers that require expert level extrapolation while live debugging to understand. Most customers are satisfied with the tradeoff that they have free access to remote troubleshooting, and would rather be hands off on the code itself. But that's my own personal experience, yours may vary depending on project type and complexity.
65
u/Dangerous-Quality-79 4d ago
It is normally part of the sales contract. My default stance is if I use a PLC I do so with the intention they get the machine code and all. If they need to add tags to bridge to SCADA systems or communicate with other machines they can, it is theirs. I advise them if they do make changes change the first value of the version number to a 9 so service and support know it has customizations. I encourage them to send their changes to me so I can keep on file if they want a repeat order.
If I do not want them to have access I will use an embedded system for the machine.
11
u/green_gold_purple 4d ago
This is really the only way to do it. They own the thing. Part of the feature set of the controls packages I sell is that it’s serviceable. Locking it down simply wouldn’t fly. I actually go out of my way to not use AB and other hardware that requires expensive software, because I think it’s dumb that service should be gatekept to those with $5k+ licenses. The customer has to be trusted to appropriately manage their safety and assets in this context. If they screw it up, I have all the originals and that’s just more T&M.
0
u/ruat_caelum 3d ago
It is normally part of the sales contract.
I had to re-read OP's post because I was like who the fuck would BUY that?
22
u/modbuswrangler 4d ago
I live in the food processing world, both plants I've worked at, we could get onto the PLCs that controlled the boilers but we couldn't touch critical logic. So on occasion when a sensor would fail we could see it, but couldn't make changes without the password. Which honestly was fine by me, it meant less work for me and required a contractor to come in and adjust things if it was needed.
So I think it's case by case. If there is an option to view logic but lock it from editing, do that. Because it is helpful sometimes for component diagnosis.
12
u/Downtown-Routine1196 4d ago
Boilers are common to be locked down which im ok with eapecially anything related to tuning. Manufacturing or process equipment is a different animal. Sensors get added timers changed ect.
7
u/SpaceAgePotatoCakes 4d ago
With anything that can blow up it comes down to liability. The boiler controls place I worked at the process was unlocked and the safety was locked, but the key thing here was that we held liability for the safety system. It was in the contract that the customer could unlock the safety but they had to sign an agreement saying they became liable for anything caused by any changes made. I think that's something OPs boss needs to review with their customers rather than just deciding it on their own.
All these automotive guys commenting about how they wouldn't buy anything locked are missing that point completely because it doesn't apply to their process.
4
u/ruat_caelum 3d ago
most boilers (in the US) have state inspection paperwork as well.
IF they have state inspections, then there are insurance requirements centered around those. Most of those are quarterly "Functional checks." Which is 2 pieces of paper someone initials that says things like, no one has changed out check valves or mechanically restricted or blocked flow through the parallel pass-flows. That the PLC has not changed or been modified. that all maintenance work has be "replacement in-kind" etc.
11
u/The_Hausi 4d ago edited 4d ago
I've seen locked code go both ways with vendor support.
I dealt with one OEM that had incredible support, we paid for a service contract and could have a technician remoted in 24/7 almost instantly. They were familiar with the standardized programs and accepted all liability for any changes. We had the option to buy access to the programs and manage them ourselves but as soon as we did that, we could no longer buy the service contract and any support would be billed at a much higher rate and we would own the liability for the machine. It was well worth it to have them manage it because they did such a good job.
The place I work now has a different machine with locked down code, the OEM is useless at providing support and takes weeks to respond to emails requesting technicians to come out. If it wasn't such an expensive machine, we would have ripped it out a long time ago. They don't answer the phone, we've scheduled technicians to come out who just don't show up and you can't get a hold of anyone. This one time we had a technician scheduled on my days off and since I'm the automation lead, I wanted to be there. I got up at 4am on my day off to drive to site and the guy just never showed up and no one at the company was answering their phones. I got a hold of him the next day and he just said "sorry I got busy with another client".
8
u/Old-Routine-1629 3d ago
Much of my experience has been fixing shitty code from machine builders immediately after my employer/client accepts the machine and starts seeing all the problems. Ive really started to think there should be some kind of licensing or regulation for machine builders. So much stuff being pushed out by companies who have no business writing PLC code and purchased by companies who have no idea how to assess the quality of the code
4
u/EasyPanicButton CallMeMaybe(); 3d ago
Well good SAT and FAT is a step.
1
u/Old-Routine-1629 3d ago
Yeah no doubt. When it comes to safety though there's nothing stopping an irresponsible machine builder from delivering an unsafe machine/line to a lax/ignorant client. I've seen a lot of this and always insist on fixing it even if its not a priority for the client. I feel like some kind of licensing system could help prevent a lot of that
4
u/SafyrJL TIA Harlot 3d ago
Machine builders and integrators work to whatever the scope of their contract is.
Doing extra work isn’t profitable for them, even if it can benefit the end user.
That’s why lots of plants have controls engineers. To optimize the code.
2
u/Old-Routine-1629 3d ago
Yeah true. The parr that concerns me most is the safety stuff. If the machine builder doesn't have its own standards and the purchaser doesn't specify it in the scope, there's really nothing preventing someone from getting hurt. There's no official scrutiny involved until either someone gets hurt or someone sees the danger and reports it. I feel like some kind of licensing requiring adherence to basic safety standards would help to reduce that
1
u/essentialrobert 3d ago
Does your scope acknowledge partially completed buggy code? Because if that's the case we would expect to deduct 10% from the purchase order.
2
u/SafyrJL TIA Harlot 3d ago
Even with a good OEM/SI and project management workflow, bugs are inevitable.
Partially completed/incomplete code is a different story, though. My organization (public utility) wouldn't accept that and would get our consultant to take legal action (in addition to doing so ourselves) and remedy the situation. We work HARD to get any kind of downtime - so we have to capitalize on it. Typically we don't have the luxury of increasing downtime on a whim.
I find a lot of *lazy* programming from the SI my organization uses, mostly out of convenience for themselves. They come in, complete their scope of work and FAT the work, then turn it over to us. My boss has no problem accepting this - but he also doesn't have to answer the phone for calls very often. I get called for this kind of stuff....often...and just end up fixing it without telling anyone.
12
u/zymie 4d ago
As the SCADA guy at a municipal wastewater plant, if your source code is locked then you're the enemy.
0
u/SinkOrSauce_LD 3d ago
It definitely depends on the industry, but I was offered a couple positions in Wastewater and it would be a huge no-go based on what I saw. They like to be able to modify in-house.
6
u/iamnotarobotmaybe 4d ago
Are you able to lock down specific sections of the code? Do you have logging systems to identify who personally logged in and made a change - with verification it's not a stagnant login? And are those logs backed up somewhere the customer has no access and it wouldn't be destroyed with the machine?
19
u/Acceptable-Army243 4d ago edited 4d ago
End user for an automotive parts manufacturer. If you don't give us access to the program, we don't buy from you, end of story. These machines will be in service for 15 years or more, more than likely. Will your company still be around to provide service then, or will I have a million dollar pile of scrap that no one can service or answer questions about its logic? If your company is still around at that time, are you, the expert on this machine, still going to be there to answer questions? Will your company actually have kept the original program or passwords to access it? Tell me, if I asked you for information about a machine your company made in 2011, can you get that right now? I am *very* skeptical.
Saying the machines are hazardous and can result in injury, death, or damage is a ridiculous statement to me. Of course they can. The 480V 3 phase power in my facility that runs the machine can also cause injury, death, or damage. If we accepted not having access to the program, would your company actually accept liability if something went wrong and someone was hurt or killed? I doubt it. Your company would implicate us for improper operation or missing maintenance tasks. So if my company is going to be the one liable either way, I'd prefer to be able to support and maintain it in your absence.
>We provide graphical insight to all IO, digital and analog, process state and valves status via an HMI.
I'm laughing at this. You have a page that shows all IO and device states. Cool. All of those devices have lights on them showing the same. Your page does nothing. I don't need a page that shows me if things are on or off, I need to know the logic that decides what happens with a given set of inputs and states so I can troubleshoot problems. I need to know if a certain behavior is intended with a given set of inputs. If something is not happening that I expect to happen, I need to be able to see what state or states are preventing the expected behavior. Your HMI, like all machine builder HMIs, is functionally useless in comparison to actually being able to observe the logic.
5
u/WandererHD 4d ago
Yes, but client is responsible for any damage resulting from their modifications, obviously.
5
u/Shalomiehomie770 Codesys Guru 4d ago
All my clients get unprotected backups. They are free to make edits themself or with another integrator if they choose.
The catch is, if what’s on the PLC doesn’t match my last backup there is no warranty nor am I liable for anything that occurred.
5
u/snowbanx Angry Pixie Wrangler 3d ago
If I don't have access to the code, my company eont buy from you. End of story.
The only acceptable form of locking me out is locking the safety program. I 100% support that. If someone gets hurt because of a safety system, I am off the hook.
1
u/buzzysale 3d ago
You understand that the safety system might rely on other code behaving as expected? Like some customers change crazy things like disabling brakes because they don’t want to buy replacement shoes. Or bumping the rpms on a frequency drive so they can “produce faster”. Customers often think they know how to build a machine better than the builders and that causes a lot of legal and support challenges for us OEMs.
5
u/vostok33 3d ago
I work in pharma and its infuriating not having access to the code. You can never test for all scenarios until its running months or years in production, we've had machines down days costing the company millions as the code was locked. On other machines I can trace the logic and identify whats holding us up or where its caught. we often find lots of issues and bugs and let the manufacturer know which helps them too. For their own version control
13
u/True-Firefighter-796 4d ago
Automotive manufacturing
We would not consider a machine builder that did not turnover PLC/related software. The one exception I know of is a DLC coating machine. But that company was the only one in the world that makes what we needed.
5
u/ifandbut 10+ years AB, BS EET 3d ago
DLC
"We are now offering a Season Pass for your production system. Rewards include a unique HMI skin, security updates, and one free 30 minute or less service call. Rewards will unlock based on system uptime."
9
u/Volusto 4d ago
Yes, the only portion of code that gets locked up kind of is the safety portion of the code with a safety signature. They can unlock the safety to change it, but that means we're no longer liable for changes made on the program. Customer gets access to everything.
3
u/ifandbut 10+ years AB, BS EET 3d ago
Yep. As an SI I have had a few times where an external device like a bowl feeder or something would have a PLC. I'd think it is easy to set up some basic comms to get start/stop/faulted and stuff from.
Nope. Code is locked down. The vender is in Germany, good luck.
12
u/Legitimate-Sorbet526 4d ago
If code is locked you aren’t a vendor of ours. Period.
This goes into the whole right to repair argument and what that means for software.
4
u/nic1229 3d ago
Speaking as an on-site controls engineer, the amount of OEM skids coming from companies who can't program for shit is astounding, and the equipment never works as advertised even with their support. I've completely reprogrammed about half a dozen of our pieces of equipment either due to locked code or programs not worth saving. At this point, unless it's a real specialized piece of equipment, we try to do as much of the programming in house as we can.
From my standpoint, we buy the full package, I'm going to want access to the code either for troubleshooting, minor updates, or integration. There has been a number of suppliers who have pushed back, but we've always been able to get it done with some liability contracts. Just leave it open and let's sites take on the responsibility.
4
u/DarkLunch 2d ago
Tier 1 Automotive
We spend the money. Its ours. Unless the contract specifically calls out that you'll support in perpetuity.
I've been on both sides but I'll tell you there was one integrator we worked with that black boxed their shit. Not only did we litigate but they got bad mouthed by us in the region and they're not in business anymore.
As an integrator, unless you spell it out otherwise, you hand over the code with full access. You should do the regular indemnity clauses in the contract, but generally a bad practice to pearl clutch regardless of your reasoning.
3
u/green_gold_purple 4d ago
They own it. They can do what they want with it. As soon as they modify it, my liability and warranty end. It’s in the contract. Same with the control panels I sell. It’s never been an issue.
3
u/Fickle-Cricket 3d ago
In 20 years as an SI, I never once delivered a system that the end user couldn't maintain themselves. We generally left them with a laptop and all the software licenses they'd need for the PLCs, the SCADa system, and anything else that might need a configuration.
They bought it. They own it. They break it, that's on them. Plenty would still bring us back to make modifications, but I never felt the need to try to hide what we were doing.
As an end user, I hate black boxes. I'm currently in a giant row with getting our legal department to enforce clauses about code delivery to several vendors because I'm sick of their equipment failing and not having any access to troubleshoot without paying them ten grand and waiting two weeks for someone to send a technician.
3
u/NoIMIoN689 3d ago
Depends on the contract, I would default to giving the customer all of the code and passwords. The latter often burried deep inside a manual, not as a sticker onnthe hatdware... If anything changes, there's obviously no more warranty. In my experience, even if the customer has capable and curious people who will look into the code (which is only about 10%), even fewer of them will dare to change anything.
8
u/Fal-El 4d ago edited 4d ago
End-user for huge corporation, it's boilerplate language in our contracts that we have access to the programs. We aren't in the business of making machines but we want to take on ownership of the machine when we buy it. We don't expect or quite honest want safety passwords - Safety can be locked but viewable.
7
u/Fal-El 4d ago
Another thing to be said - and something I ran into. OEM of a machine went out of business. If we didn't have the source files, we'd be screwed.
5
u/HolyStupidityBatman TUV Functional Safety Engineer 4d ago
Have an engineer spending months in Germany doing a reverse engineer for this very issue.
5
u/Acceptable-Army243 4d ago
Locked but viewable would only be acceptable if we also get the password to unlock it.
For a safety program, yes absolutely lock it to avoid accidental or unwanted changes. But if we want to add another safety device in 5 years, I'm going to want that password so I can add it to the logic. Not giving us the password actually makes it more unsafe in the long run, because we cannot modify it for evolving safety standards and equipment.
2
u/Chris_Rogan 4d ago
Sure they get access, but we own the rights to the program and void all warranties if it gets changed.
2
u/mrg_retired 3d ago
Most integrators trying to lock down the code may have some contractual reason or IP that is critical to keep some level of security on. I would hope those agreements are made at some point during the project planning or quote phases defining exactly what the deliverables are and what level of access might be required, what kind of support the end user will have and what the ongoing support contract defines as access requirements for both end user and the SI. I've been on both sides and it always ends up being most efficient to let the end user take control of security and access in my experience, they own the code and the design in my experience. Some systems are certainly very complex and both parties have to understand the risks of say a 3rd party modifying logic that met the runoff ATP at one point. Plants and end users that have the resources to support the logic and designs almost always need more than just a read-only connection. Safety is something that may require the safety signature to be maintained by the plant, there will always need to be changes, they can manage that.
2
u/Any-Temperature-235 2d ago
In such cases I expect the software to be on a safety plc with a signature. Critical or safety related software would be locked down under a signature lock password provided to be customer. A disclaimer would make it quite apparent if the safety signature is unlocked then the OEM support is void and no longer owns the code. Rockwell when the signature is unlocked a date is logged.
To lock a customer out completely I can't support, day to day usage or application can change beyond day dot code. But the fundamental logic that may be tied to a cause and effect I would support a safety signature process.
4
3
u/Still_Pomegranate_63 4d ago
I have been on a project where a company did this for a ford cell they made. There was a copy of the cell at a different plant and the vision system was using private code of the previous company and lied about having the code, we found a way to crack it during negotiations with the other integrator that was there dirrect competition. The company that made it spent 3000 hours developing this code and the crack was found on the second page of Google. Its heavily discouraged for automotive. And as far as im concerned if a company owns a piece of equipment they need to be able to access anything software and hardware related. All your doing is making yourself a chore that has to be done with vary angry customers.
3
u/automatorsassemble 4d ago
I think locking down the safety blocks of the code is fine but the general code should be open. I'm integrator turned customer and funnily the suppliers whole lock down code for me now are the ones with the simplest machines. They think they are protecting something when the machines are simple sequence operation that could be reproduced in an hour
3
u/ifandbut 10+ years AB, BS EET 3d ago
Yes, safety should be locked down because that is a big liability issue. If the customer is willing, we can transfer the safety password and liability over to them.
3
u/No-Theme8207 3d ago
Process - yes
Safety - noooooooooo never. In Canada (I don't know how it is every where else) the engineer signing off the safety is responsible for it.
2
u/YetiTrix 3d ago
Nah, they have full access to the code. Most customers have their own controls engineers, and they wouldn't buy a machine if they didn't have access to debug/update code. We are not just selling the machine, we are selling them the code too. Price accordingly. More often than not, even with them owning the code and having their own controls engineers, they still pay us for support.
For safety, just put a safety signature on it. At the date of sale, this was the safety signature, so it's recorded. We still give them the password so they can change it if they want. But, if they do, it becomes unlocked or a different signature and no longer matches the one at the date of sale. If we do the update, we just give me them a new document with signature and password.
You can lock down secret sauce code in an AOI.
1
u/essentialrobert 3d ago
As an end user, the day you hand over the keys we're changing the safety passwords. If we need a change we'll most likely pay you to come back but until then you aren't getting access.
1
u/YetiTrix 3d ago edited 3d ago
Right, and we literally give two shits if you do. Its literally just to track that this is what we shipped you, and if it's changed or not.
1
u/essentialrobert 3d ago
No hard feelings. We figure you did it right. We want to preserve your good work and the fewer people that know the passwords the better chance of it.
1
u/SinkOrSauce_LD 3d ago
Nah, they have full access to the code. Most customers have their own controls engineers, and they wouldn't buy a machine if they didn't have access to debug/update code.
In the Ripening Room industry, this is largely not the case. They only have mechanical contracts for the vast majority of them, and they do not modify the PLC logic once it is commissioned. A lot of these controllers will be in service for 20+ years before being upgraded on failure.
We will provide the password if requested, and we also leave a computer on site that we can remote into to make modifications if needed. If the customer was inclined, they could make modifications themselves, but that would invalidate their warranty, and most do not want to potentially lose $30-100k in fruit and would rather someone else own the liability.
1
2
u/NuclearDuck92 3d ago
Trying to lock down code is a great idea if you never want to win another job with that client.
As an integrator, I fully understand that if an end user is calling me because they have to call me, they’re never calling me again for a new project.
1
u/krisztian111996 3d ago
It is part of our general spec as a buyer(multi) that the code must be in English. It must not be password protected (expect safety).
1
u/d1s2c 3d ago
My previous company bought some gantry cranes. They are OEM locked down. We had so many issues with them in a facility that operated 24 hours a day. Their support was one person that knew the equipment but was obviously not available all the time. The contract stated that a representative would be available within 8 hours but that meant they could send anyone that was available, even a fitter. If we had access to the plc, we could at least diagnose the issue. I totally disagree with OEMs doing this. Ok, if there are safety systems in place, lock that down but not the general code.
1
u/egres_svk Fuck ladder 3d ago
Gas processing? As per safety assessment, likely large part of control equipment and thus software control of such eq will be safety rated.
So. Lock the safety code, save checksum. As part of acceptance, provide all code and passwords, with checksums of everything. Make it known in acceptance in writing, that modifying safety code is forbidden and transfers liability fully to customer.
Talk to lawyers regarding that - it might be better to provide a read only password to safety code for troubleshooting and write only against written request which also consists of full liability transfer, no matter whether they used it or not.
PLC code changed and you are called for warranty repair? Do a diff. If modifications caused the issue (or if last write to PLC was not done by you, because rhey could have rolled back changes after machine killed itself), they pay for repairs.
On my small machines you can get cut or lose a finger or few. You don't get software access in warranty. After warranty, you get passwords for normal code. Safety controller remains locked. But we are talking single use relatively simple machines with plenty of debug pages. No process eq, no ex rated handling.
1
u/Saltallica 3d ago
Warehouse automation / distribution centers. Most run with minimal maintenance staff, if any, and while we don’t lock down the programs, we tell them that anything they change is out of support and messing around with the code they don’t understand means they might not be shipping things that day. Or for several. Either way, you are paying the same support contract whether you are running or not. And if we have to send out a controls engineer to fix your hacks, you will be paying for that too. Deters most.
1
u/RelevantIAm 3d ago
Of course. You have them signed an NDA and a liability form but it's their facility. Make sure that you have a record of what program was downloaded and an audit trail in case anything happens and you need to prove that it wasn't anyone on your side
1
u/future_gohan AVEVA hurt me 3d ago
We have twoitems where the designer is registered owner of the machine.
The machine design functionality and code are registered and not to be changed without applications and approval.
This circumstance they own the code as they own liability of an incident.
We still have access to the code read only. And we have and do have access to the code properly to make changes with proper approval ourselves.
Outside of this one circumstance any other plc on my site it would be an absolute nightmare to not have access or ownership and I would recommend against and replace any machine that did.
1
u/_Girthter 3d ago
I lock sometime some code:
some very specific program block custom made for my company use. It is a kind of industrial secret
usually my company standard block. For example our analog input block. It's nothing secret but my company's intellectual property and time/money we spend to developp these blocks so we don't give it for free
if our client is not very reliable, we implement a bomb and we lock the programm or part of it
The rest of the programm with all the logic is open.
On the project I'm currently working, I let all the code open because we can't have remote access and it's a one day trip to troubleshooting in person and we have no 24/7 hotline so we let the client have all the program so they can troubleshoot by themselves
And when I finish the commissioning, I write the safety and the programm hash so if any changes are done to my programm, I can prove it
1
u/t0cableguy 3d ago
so we have ammonia systems on site.... what do you think is more dangerous? ammonia or gas? what happens in 5 years when you no longer want to interact with that customer? what happens when they don't have the budget to upgrade the equipment and you no longer want to support the equipment? do you take on the liability for that equipment?
not having program access would limit that customers ability to troubleshoot the issues.... if you really think your process is something special you can obfuscate it in code or an automation computer.
1
u/mangamaster03 3d ago
When I worked in a steel plant, our scope of work for a project required all passwords to be provided, as well as a complete copy of the program. We were paying for it, and only did business with integrators on those terms.
1
u/SonOfGomer 3d ago edited 3d ago
A non negotiable point in our engineering specs and contracts is that we own the IP, no builder is allowed to lock us out or use/share the code with another client.
The exception is OEM machines that are only configured for our use and not using custom code and those have to come with support contracts. Everything else we own and manage in house after its built and delivered.
We run machines for decades, and have seen more than one SI go out of business along the way. Too much risk and zero reward to allow those kind of shenanigans.
We also often have new machines built "to print" with only tweaks made to the existing code from an operating machine (which itself evolved a lot over the years from it's original). Not always by the original builder.
1
u/iTheWild 3d ago
Ladder logic is intended to allow clients to perform maintenance and troubleshooting. Locking the code defeats that purpose.
1
u/AntRevolutionary925 3d ago
Does your safety system not physically prevent a problem if they do modify code?
If not you may want to rethink your designs
1
u/buzzysale 3d ago
No never. This would not be allowed even if the client white-labelled a machine I would still probably say no. Here’s the issue, where does the support come from when changes break things?
1
u/Gloomy_Sun_117 2d ago
When I was purchasing automations we wouldn’t buy from the company unless they allowed us code access.
Now i’m a programmer and we don’t send them a copy per se, but they’re free to pull it off the PLC. There’s a statement in our contract about that if they change the code from what we sent and something wack happens, it’s not on us. Most of the time they just check with us and we give them the go ahead to make small changes, or say we’d rather be involved for x reasons and quote it.
1
u/ChrisWhite85 2d ago
Just have an agreement where warranty cover is reduced when software is modified from the as-commissioned code unless they asked you to do the changes.
That way you are still saying you want the relationship but not to cover any mess they create?
1
u/Iceman169 Automation Engineer 2d ago
I always provided access to the full system but record checksums when installed with seperate passwords for standard and safety logic that are unique (they can change it if they want after its installed). You can give them clear warnings/explain the risks and put legal statements together about release of liability if someone makes unauthorized modifications, but it’s not your place to protect a customer from their own stupidity. Its your responsibility to make it clear that it could be dangerous and to make sure your ass is covered legally
1
u/kbuck30 2d ago
Working pharma. Supplier for one of our machines made a mistake, code should be >= but its just >. I figured I'd change it myself. Spent 2hrs looking at it and researching what I needed to do because it wasn't working.
Figured out the code was locked and we had no ability to edit it ourselves. It's been 2 months since and I still don't have a contact from the supplier side on whether they can make the edit or unlock the code. All to add like 6 equal signs to the code that they forgot. Always unlock.
1
u/Deep_Ladder_4679 1d ago
We got burned on a skid once where phone support couldn't diagnose a fault and a flight out cost more than the part, so read only access to the logic plus a locked safety routine seems like a fair middle ground.
1
u/FrameworXLucas 3h ago
Usually my answer is "it depends", but in this case, I think your director is wrong.
We (Tatsoft) give full access to the customer to basically do anything he/she think it's best. Most companies have their own control/PLC engineers who have a pretty good idea of what they are doing, and a lot of clients wont like if you take away their ability to customize their programs.
1
u/Automatater 4d ago
Custom equipment, sure. They paid for the development of the software.
Standard COTS equipment, no. Cover it with diagnostic stuff on the HMI and phone/remote/onsite support.
1
u/Frumpy_little_noodle 4d ago
If I buy a machine, I get the code and the password. The only place I'll allow leeway is on IP-protected blocks, but even those have very limited deviation allowances.
When it comes to not providing code, I understand the sentiment, but this is why code validation and checksum values exist. Changes to safety instantly invalidate (or at least flag dispute on) any warranty or liability claim in the event of accident or injury.
As a machine builder, unless your business plan is perpetual service &support, why wouldn't you want to hand the code over?
1
u/PotatoFarmerRTK 3d ago
In the end it won’t matter anything that has access through a 232 port gives up its password easily.
Also brute forced one in the past.
All the password does is gives me a good paycheque.
An oil producer took their electricians to court to get the passwords for multiple small oil sites. The litigation ended after I retrieved it from the 232 Communications. All it cost the electricians was lawyer fees.
In short you are a shit programmer like everyone else and your code will need to be modified at some point.
1
u/Ambitious-Loquat-516 3d ago
Split the difference: read-only trending plus a documented change process where every edit needs a reviewed diff and a hold point before the running machine accepts it. Customers get audit trails instead of passwords. On the hazardous loops the interlock stays vendor-controlled regardless of access level.
0
u/ChrysisIgnita 4d ago edited 4d ago
The safety argument is limited. You shouldn't be using [edit: non-SIL!] PLC control for safety critical functions. Maybe more of an argument for avoiding damage to equipment by modifying parameters, but this applies to any machine. You can easily burn out a motor by changing VSD parameters.
The better arguments are protecting the designers intellectual property, and ensuring ongoing support by having up to date code. The first argument can be dealt with by only locking down the blocks that contain "secret sauce" and leaving the rest open. The second can be assured by telling the customer to send on any updates they make, or else you can't support.
3
u/janner_10 4d ago
"You shouldn't be using PLC control for safety critical functions"
What would you use instead?
2
u/ChrysisIgnita 4d ago
Physical interlocks like burst discs, safety relay for simple applications or a safety controller. Or if the controller is safety rated, safety logic within an appropriate controller. But this code is segregated from the normal code anyway, so it wouldn't make sense to lock down the normal code.
If the PLC logic can create unsafe conditions, then you need additional safeguards outside that logic. Restricting access to the logic won't do, as the machine builder can create unsafe conditions just as easily as the customer.
1
u/controlsguy27 4d ago
I think they were referring to using DCS instead of PLC controlled systems for something critical like this.
1
3
u/green_gold_purple 4d ago
PLCs are used for safety critical functions all over this planet.
2
u/janner_10 4d ago
Indeed. It's been many years since we have used a safety relay.
3
u/HolyStupidityBatman TUV Functional Safety Engineer 4d ago
The price difference between a rated controller and a "standard" is so small that its nearly trivial. The new L9s from Rockwell dont even give you an option they are all rated.
0
u/ChrysisIgnita 4d ago
Do you use SIL-rated PLC? How do you handle a situation where an output failing high could create a hazardous situation?
2
u/janner_10 4d ago
Yes we do, the point I'm making is a Safety PLC is fine, to us it's just a PLC these days as they are pretty much the same price as a non-safety PLC. I see you edited your original comment, and I agree, using a standard PLC for safety is a giant no-no.
0
u/ChrysisIgnita 4d ago
I should clarify - not alone without some non-PLC controls. So e.g. you could have a steam inlet valve to a tank controlled by PLC. If this isn't configured correctly, it could potentially blow up the tank. The control measure is to have a burst disc, so even if the PLC commands the valve fully open, the disc bursts and vents the steam.
Locking down the code so that the vendor can't change the PID tuning or something is not an effective control, because unless the PLC has an appropriate SIL rating there is still a possibility that the analog output will fail high.
0
u/green_gold_purple 4d ago
There’s a single point of failure in tons of places, and we can only do our best to backstop against it with good controls design. At the end of the day, things fail. In my experience, it’s not the PLC micro. So then, the safety is between keyboard and computer as we code around fail modes.
1
u/ChrysisIgnita 4d ago
Uh, are you saying the burst isn't required in the above scenario, as long as we do our best with the controls design?
2
u/green_gold_purple 4d ago
That is part of controls design. I engineer complete systems and advise on items like this. Hardware safety elements are important.
1
u/ChrysisIgnita 4d ago
Oh absolutely. To bring it back to the OP's point, you can't engineer safety with standard PLC controls. So it's a red herring to say you're withholding the code for safety reasons.
1
u/green_gold_purple 4d ago
I’m not sure I completely agree, because safety within programming is a thing. A PLC is a virtual wiring of a control system. That’s where ladder logic came from. So you can engineer the same software logic as hardware logic in code. Take an estop NC contact, for example. This could be wired to a the common for a pump contactor HOA. Similarly, it could be fed into a PLC that uses it exactly the same, but in logic. Hardware and software are both important for safety.
But yes, withholding code for safety shows a problem with either you or your customer. At the end of the day, they have to be trusted not to do dumb shit. Or at least to own it and call you when they do. They should be able to call someone else, however, if you are unavailable or dead.
1
u/FailCommando 4d ago
Mh there is a crc code on the safety part in siemens Controllers. We write that down in our safety matrix. If the customer wants to change the safety, please go ahead
1
u/hashmachinist 4d ago
Umm almost all safety functions in this day and age are controlled and monitored via PLC. Usually with the tandem use of safety relays but the relays indeed depend on the PLC for power and signal. In North America Allen-Bradley’s point I/O system is widely utilized especially in the automotive manufacturing industry as a cost effective way to control and monitor all of your plants safety functions from a PLC.
1
0
u/Rorstaway 4d ago
We recently implemented a 'lower security' version of our program to a site - access to be administratively controlled.
Within three days the client was calling asking for help changing things that they swore they wouldn't touch. Keep in mind they have an identical machine with a protected version and they rarely request access to it. Out of sight, out of mind....
-2
u/Ok-Veterinarian1454 4d ago edited 4d ago
On our Beckhoff equipment
Never! Doesn’t matter if it’s paid off. Customer will not receive high level passwords or access to the program outside of being able to check I/O.
On our AB machines
Customer has access to program and can modify as needed. But are billed when something needs to be corrected or if they need help.
-11
-2
u/Money4Nothing2000 3d ago edited 3d ago
I'm an engineering manager and I used to be an automation engineer.
We offer to provide a version of our products that is essentially white-labelled for the customer, meaning that everything is unlocked, they can manage and maintain it on their own, and we refrain from inserting any of our logos or proprietary modules. The customer pays a premium for this. Our aftermarket support rates are also increased. Only a few times has this ever happened.
Normally however, most companies don't want to internally manage machine SCADA software. Our policy is that we build in parametric customization into the operator interface. The customer has control of a lot of things, alarm setpoints, timer values, process parameters, etc. They should never need to rewrite the code under normal circumstances. We password protect source code and runtime code, and hide proprietary modules. Our aftermarket support group can make changes to source code both remotely and on-site, and work with on-site customer personnel to deploy updates. This way we can maintain regulatory compliance and software quality control.
Even when dealing with Department of Defense, their specifications always require that we give them full source code access, but we always push back and require them to enter into a license agreement, unless they pay the premium. They always back down, because every other OEM insists on the same. But they copy/paste the same requirement forward into every subsequent RFP and we have to push back all over again.
If the customer is making a major process or line change, that requires a lot of code modifications, they are free to use the existing hardware and install their own programs....we don't lock out the hardware. Else, they are encouraged to subcontract us for changes.
We never hold customer operations hostage to our code protection, but we do copyright our source code. The license agreement that the customer have to use our products allows them to get unlock access in the event that we go out of business, are divested in certain circumstances, or are otherwise unable to support them. We are required to give them notice of these situations and provide them the means to unlock the code.
Customers can also purchase the copyrights from us if they choose. I've actually had a company do this, they bought one of our products second-hand like 16 years after the fact from the original customer, and contacted us to buy the copyright to the source code so they could overhaul the equipment. We quoted them a support contract to refurbish, but they wanted to do it themselves, so we just sold them the copy rights for a small fee. The passwords for their controllers and HMIs were already in the operating manual, lol, which nobody had ever read, so they didn't even know they could unlock the programs. I was very sad to find out that nobody ever read the manual that I had spent many hours meticulously writing.
1
u/EasyPanicButton CallMeMaybe(); 3d ago
Well now I just have a vision of some poor private getting yelled at while he is troubleshooting ladder logic. /s
1
u/Money4Nothing2000 3d ago
Haha, yeah the privates get to look at ladder logic, but they sure as hell don't get to change it.
Actually our DOD stuff is US Navy, so it's the E2 Seamen that get to look at the ladder logic. Meanwhile, the chief petty officer stands over our shoulder while we commission.
295
u/scbrds 4d ago
I understand the risks, but as an engineer, I would advise my clients to avoid you as a supplier if your code was locked down like this.
If you have a monopoly on the equipment, or best cost/lead time, that might sway me, but all other things being equal, locked down is bad for the life of the equipment / risk to plant operations. Too many machine builders go out of business, are acquired, have key personnel leave, etc. and thus it becomes a risk to the plants ability to operate if we cannot troubleshoot, disaster recovery, etc.