How large language models can boost industrial automation

Have you already tested ChatGPT and got excited about how you could use it in your job? ChatGPT demonstrates in mind-blowing ways the potential impact that large language models could have on the way people interact with machines. According to Boston Consulting Group, generative AI (which is what ChatGPT is categorized as) will surely bring about a boost in productivity from people and machines working together; and there is no place where people and machines interact more than in industrial automation segments.

Large language models are created using deep learning algorithms and neural networks and then trained in massive amounts of text data. In recent years, large language models have started to spread into various industries and now we are seeing them increasingly applied in industrial automation. The ability of these models to understand and generate human language can greatly improve the efficiency of industrial processes. Our own conservative estimations show the potential of large language models to eliminate 20% of the effort required by an OEM to build a machine PLC application which, in turn, can mean a potential savings of between approximately €40,000–800,000 per year¹.

As technology matures and becomes more widely adopted, we expect to see an increasing number of real-world applications for large language models in industrial automation. And it is maturing very quickly – the ability of more mature models, from GPT-2 to GPT-4 for example, to handle much more content as inputs is exponentially increasing the applications of large language models in industrial automation.

One of the most important points is the ability to customize large language models for specific use cases or tasks that are relevant to industrial automation. In industries, three of these use cases/tasks can include:

1. Code generation, documentation, refactoring, and testing
2. Natural language interfaces
3. Automation system design and development

However, it’s also important to consider the limitations that may be prevalent in large language models and how these limitations need to be accounted for before the large language models are industrialized.

These opportunities, limitations, and how to industrialize large language models are explored further below.

1. Code generation, documentation, refactoring, and testing

Large language models can be used to generate code for industrial control systems such as PLCs or to generate HMI screens using natural language inputs. This can ease the application engineering process, reducing the time and effort required to develop control applications. Furthermore, large language models have the potential to improve the quality and consistency of generated code, leading to fewer errors and faster commissioning times. Our own quite conservative estimates show that on average 70% of the code generated by large language models is useable to code machine PLCs – representing potentially significant time savings. Another application can be the automatic generation of recipe code which would save time when changing parameters, suppliers, or ingredients. The time to create the recipe and then trigger the changes to the line directly impacts production time so any savings that could be made would increase efficiency. Taking it a step further, large language models can also then be used to automatically generate the documentation related to the code they generated – like automatic test scripts, which are always time-consuming tasks for the automation engineer.

For example, at Schneider Electric we have been testing the use of private access versions of large language AI models to train those models on our own specific EcoStruxure Machine Expert applications. The results are promising – with code able to be generated quickly and somewhat accurately, albeit while still needing the human eye cast over it. We can see that the same kind of model can be applied to other software applications.

2. Natural language interfaces

Large language models can also be used to create natural language interfaces for industrial automation systems, allowing operators to interact with these systems using human language rather than specialized programming languages. One of the key applications for this could be to access existing documentation, like technical product information, using natural language commands. In many industries, operators and maintenance personnel have to manually search for information contained within product documentation. Some even still do this in paper formats! However, as documentation becomes digitalized and fed into secure and specific large language models, these tools can be used to simply ask questions, and answers can be found very quickly. For example, “What does error code 8975 mean and how can I resolve it?” is one such question that could be asked. The key here is to use secure and specific large language models. The model should only draw answers from approved and officially provided manuals, technical descriptions, and source code.

One of the often-reported challenges faced by industrial businesses has been the fact that domain knowledge can reside with specific people. While this is evolving with the increased digital transformation of assets, digitalized work practices, and changing workforce demographics, many factories still encounter instances where a machine or asset breaks down but the only person who knows how to fix it isn’t on shift. Imagine how much time could be saved and productivity increased if the maintenance worker who is on that shift had the ability to use voice commands to troubleshoot, access documentation, and (with firm safety and security practices clearly in place) take corrective actions using AI.

Virtual assistants for technical support are another application where natural language interfaces with large language models could be useful to industries. Industrial companies that provide highly specialized and customized support to their customers over the phone – like machine builders working with industrial end users – can capture that very tacit knowledge to feed into a specific large language model that could be used by virtual assistants to enhance the customer experience and efficiently reduce resolution times.

The same can also be applied to the creation of user manuals and documentation for machines, automation products, and systems. Using large language models to create these essential documents can free up time for application engineers to apply their knowledge and skills to the development of better machines, products, and systems, increasing quality and improving time-to-market.

3. Automation system design and development

Whether it’s a brownfield expansion or a greenfield development, the design and development of automation systems require significant coordination between numerous vendor departments, customer departments, and often third parties like regulatory authorities. Using large language models to standardize on known parameters can reduce the time taken and thus increase the competitive advantage for all involved.

Starting with customer and partner inputs, several requirements like solution architectures, BOMs, safety plans, risk management plans etc. are required for system development. This currently takes several months, or even years, of rigorous work, clarification, and the involvement of experts from various functions to ensure quality and viability. Using large language models can speed up the whole process while ensuring adherence to compliance at all levels as required.

Ethical considerations and limitations of large language models

The use of large language models in industrial automation raises several ethical considerations and risks which must be carefully considered to ensure that the technology is used in a responsible and ethical manner.

• Safety: When talking about the possibility of AI models to perform actions in industrial automation, setting down clear safety practices must be the first consideration.
• Data privacy: Large language models require large amounts of data to be trained. This data can include sensitive information about individuals or confidential information about a given process so it is important to ensure that this data is protected and that confidentiality and individuals’ privacy rights are respected.
• Bias: Large language models can perpetuate and amplify societal biases present in data used for training. This could lead to unfair and discriminatory outcomes. It is important to identify and mitigate these biases to ensure fair and equitable outcomes.
• Security: Large language models can be vulnerable to malicious attacks such as model stealing or adversarial attacks. It is important to ensure the security of these models to protect against these threats.
• Explanation: Large language models can be difficult to interpret and understand, making it difficult to interpret and explain the responses. This can become problematic when these models are used in decision-making processes as it can be difficult to understand the basis for these decisions and to ensure that they are fair and reasonable.
• People focus: Through this article, we have seen some promising outcomes from the use of large language models in industrial automation, however it is essential that any output still has a human eye cast over it. These models must be used in a way that complements human capabilities. For example, a machine can perform a task which would then be validated by a human, or a machine can refine or challenge a human’s creative thinking.
• Reliability: The outcome of large language models may not necessarily be factual and, as of today, should always be checked by human experts before it is used.

A word on the industrialization of large language models

As mentioned above, to leverage the benefits of large language models, they must be deployed appropriately while taking into account all the ethical considerations and limitations. However, yet another important aspect is the potential base size of the large language models and where those significantly large language models could be/need to be deployed. For example, the GPT-3 model has 175 billion parameters while GPT-4 has a considerably higher efficacy with 1.8 trillion parameters. It is therefore evident that model execution would require an enormous amount of compute power and space.

A focus on pre-training large language models with domain-specific data is one way to improve performance and ease deployment. Different industries should evaluate how these base models are created and when such large models are needed versus smaller, specific solutions. For example, domain-specific large language models – like Manufacturing GPT, Healthcare GPT, Travel GPT etc. – would mean that smaller and more specific models could be created and then built on to include the necessary proprietary additions. This would also help to address the problem of hallucinations where non-related contexts are brought together. It could also be a way for organizations in the same industry to come together and jointly create a GPT for the most common uses in their domain while still retaining their specific IP.

Other uses of AI technology under investigation for industrial automation

Generative AI, especially large language models, is just one of the exciting applications of artificial intelligence in industrial automation. We are investigating other uses of AI technology for industries in categories like:

• AI-enhanced devices – where AI models for specific tasks are fully integrated into an embedded device, like an intelligent sensor or actuator, or in connected products where AI is used for auto-tuning or predictive maintenance
• AI-based control – this starts with AI-generated control applications which can be achieved through the large language model technologies discussed in this paper or with Deep Reinforcement Learning (DRL) technology which we are also investigating. AI-based control is also about the real-time integration of AI-based computer applications with control applications to give machines and processes the highest levels of autonomy
• Virtual sensors – an alternative to physical sensors that, given the available information, produce similar outputs
• AI-based insights – using data from the field to train predictive models to detect anomalies or give recommendations on maintenance practices

Conclusion

The use of large language models is becoming increasingly common across a wide range of industries. McKinsey notes that "generative AI has the potential to change the anatomy of work, augmenting the capabilities of individual workers by automating some of their individual activities.”

Large language models like GPT-4 are changing work practices in industrial automation by using them for code generation, with natural language interfaces and, further into the future, for automation system design and development.

Large language models use advanced machine learning techniques to generate high-quality code and documentation quickly and easily which significantly increases efficiency and reduces errors. However, it is very important to consider viability for code generation, to be mindful of the ethical considerations and risks associated with these models, and to consider how specific domains, like industrial automation, can work together to harness the potential power of large language models to change the way work is done.

Finally, large language models are only one exciting way that AI is being put to work in industries. There are many more offer categories in development that would see AI deployed to all layers of the technology stack – from connected products to apps, analytics, and services – and across the complete industrial automation lifecycle – from design and build to operate and maintain.