The Internet of Things (IoT) and Generative AI (GenAI) are two fascinating areas which are rapidly emerging as key technologies solving complex problems based on data in different domains. Both are powerful tools in their own space, but they have an even larger potential when combined. The power of IoT enables billions of devices to connect and push live data from the field, which enables us to bring out insights about their performance and real-time status. These IoT devices can range from small smart electrical lights at your home to a complex industrial equipment. Connected devices are rapidly bringing information from the field to the cloud, enabling their manufacturers to not only improve the product but also provide a number of digital services around the same for their users. On other hand, the power of GenAI is its ability to use artificial intelligence algorithms to generate new content, such as images, text, or music, that is similar to existing data. Large language models (LLMs) which power GenAI are trained on billions of data points. They are able to answer almost all your questions, and also interpret and generate new data presentable in a natural language for communication. This brings a significant leap in AI and AI-based application developments. Limitations of IoT and GenAI technologies 1. IoT – Data complexity IoT solutions deal with devices and data — rather, a large volume of devices and data. To process such large volumes of data, Machine Learning (ML) models are used to identify the patterns in different data points, but this is not enough. We need this data to be contextualized and interpreted along with other data points coming from different systems. It requires complex interpretation of data to help us understand the different behaviors or changes across assets and their performances. Digital Twins bridge this gap by combining different data sources and creating a virtual replica of the asset. Yet, from an end user perspective it is a complex environment. Sometimes, IoT-enabled assets may have technical interfaces to interact with its human operators. The availability of a natural language interface could improve this interaction and make it easy to use, specifically in cases where the end user may not be technical users, for example a user at home interacting with his smart home appliances. 2. GenAI – Lack of contextual and real-time data At the heart of GenAI lies the trained Large Language Model (LLM). The generated text responses are easy to read and provide detailed responses for a broad range of topics. Even though it looks like magic, it has limitations. For instance, its knowledge is limited to data points used to train them and the data available up to a certain date and time when the model was trained. It lacks real-time data updates to its knowledge and in some cases, it completely lacks knowledge if it is not part of its training set. This leads to situations in which models may be unable to give any answer or may give an incorrect answer, a phenomenon called as AI hallucinations. Assume a scenario where a user asks questions related to corporate policy of his organization. This data would not be available in generative models as it is private data. Another scenario is when a user asks about current events occurring in an IoT device; this information is real-time information and will not be available at the model training time. In such cases, the contextual information related the real things like an organization or assets are not known to the model as models are not trained for a particular objects’ information. The possibilities of interfacing Generative AI with real-time data from IoT are huge and can be taken to different industries. IoT and GenAI: A winning combination Now, let us combine IoT and GenAI to see how they complement each other and address the limitations. Simply put, IoT and Digital Twins can supply/deliver complex real-time and contextual data to GenAI models. The models can then leverage this up-to-date information, preventing it from hallucinations. In turn, generative models can simplify and interpret the complex data for ease of use and consumption by end-users in a natural language interface. In this way both technologies can complement each other and provide a comprehensive, beneficial solution. Let’s look at these benefits in detail. 1. Easy and natural language interaction with smart devices All smart devices from home, field or factories are connected to humans via some digital devices like mobiles and laptops or native displays. With natural language, these interactions can be much more immersive with voice, augmented- and virtual reality. Users can interface with devices with natural language, thus bringing Digital Twins to life. Home automation tools like Alexa and Google home already provide voice activated bots. With advancements in GenAI, this interaction will become more natural and will improve the product user experience immensely. This natural interfacing with smart devices will not be limited to only consumer devices, but can be applied in other domains like in Manufacturing, such as on the shop floor for engineers fulfilling tasks, or in Retail, helping shoppers with navigation, product enquires and billing. Even in healthcare, smart chatbots can help patients deal with their medical needs. The possibilities are immense. 2. Faster model developments with data generation IoT and Digital Twins rely on ML models to identify data patterns and solve various predictive and prescriptive use cases. For better results, you need these models to be trained on a variety of data from assets and processes. Data collection (specifically with a variety of scenarios in the data) can take time, leading to delays in model developments. GenAI can use its core capability of content generation to build any data gaps required to train the models. This method can also complement synthetic data generations using other methods like numeric simulations. This improves overall model quality and time to deploy. 3. Complex data interpretations and contextual awareness Digital Twins are virtual companions of physical assets. These systems carry multiple data views of the asset like product details, maintenance information, inventory data or even technical specifications and designs. IoT data collected by Digital Twins provides current live state of assets. This real-time data about assets or processes provides crucial insights and context to ground scenarios. A GenAI-based application, which is contextually aware of real-time information, can provide access to information for business owners from various data sources. These integrations can be achieved using various ecosystem frameworks that assist in Retrieval Augmented Generation (RAG). RAG provides a mechanism to optimize the output of LLMs with targeted information, to be more up to date than the LLM training data or to provide answers related to a particular organization or industry. In this way the GenAI system can provide more contextually appropriate answers and help avoid or minimize hallucinations. A use case: Wind farm management Let us take an example of an IoT solution managing a wind farm and connected wind turbine assets. Our target user groups here are service engineers and business owners, each with different expectations and different user journeys. The wind farm business owner can prompt the system with a question like “what is the reason for low energy generation?” or “what are the current trends projective our power output?”. Such prompts can help get a better understanding of insights from production information and interpret various data points that may be causing low power generation. In the absence of a simple interface, the task would be more complex. The business owner may need to look for specific reports and data, then corelate the information from enterprise systems like ERPs and collect information from IoT events and maintenance records. In another user case, imagine, a service engineer fulfilling a servicing task for a wind turbine, but unable to prompt the turbine Digital Twin system for insights and trending events to get better understanding of the situation without navigating through multiple applications pages. The service engineer can now prompt, “what are the current events and what was temperature trend before the failure?” or “are there any events that require my immediate attention?” With the summarization of data in such cases, service engineers can quickly be familiarized with the situation, potentially improving response times. Another important aspect of GenAI is to understand the context provided to it and bring contextual answers to user questions. Continuing the same example, a GenAI-enabled knowledge repository could enable the engineer with access to companies’ knowledge bases with a few voice commands. GenAI can quickly look at the relevant documents, manuals, or past maintenance reports. This can reduce the time to explore the knowledge repositories and bring relevant information to complete the servicing. Combine for full speed ahead With the above examples we can see that the possibilities of interfacing GenAI with real-time data from IoT are huge and can be applied to different industries. GenAI will improve interface IoT devices, train models better, enhance the management of historical information, and help users make better sense of IoT data. In becoming a knowledge bridge, the applications of GenAI and IoT are numerous. Learn more about how we at Eviden are already exploring various possibilities with IoT and GenAI. Stay tuned to this space for more.