The Intelligence of Things

 

Combining Artificial Intelligence with IoT to deliver Information and Autonomy.

The Internet of Things has been around for quite a while now.  The term was first coined in 1999, although in many ways it wasn’t until the 2010s that IoT became part of people’s lives with the widespread adoption of Smartphones.

But we are now seeing another trend growing at pace, the combination of Artificial Intelligence and the Internet of Things (AIoT).  The rapid development of AI recently has brought new possibilities for IoT which we are only just coming to understand and exploit.  AIoT is hot right now!

What is IoT?

Let’s establish some principles first.  The Internet of Things refers to a class of physical objects that are connected to the Internet in some way.  Your laptop and mobile phone are part of it.  Your car might well be.  Your Smart Speaker or Smart TV is, and many other consumer devices too.  The Internet of Things extends beyond the domestic too.  Many parts of the urban landscape are connected to the internet, and IoT is widespread in industrial processes, transport infrastructure, and equipment and in Healthcare.

What is AI?

Artificial Intelligence covers a wide range of techniques that allow computer software to go beyond straightforward deterministic rules.  Traditional computer software is basically a set of rules:  If this happens, do this.  If you click this button, open this page, or if this condition is detected, perform that action.  AI essentially extends this capability to allow pattern recognition and adaptation.  “If this sort of thing happens, do this”.  Or “If this sort of thing happens, do whichever of these set of things are most likely to be correct”.  AI is often thought of as ‘an expert in a box’.  If an expert is able to perform a given task, it’s often possible to teach a computer to do that same task, depending on some limitations which we’ll come to later.

 

The 5-Layer IOT Model

We think of IoT in terms of a 5 layer model: Sense: Acquiring raw data from sensors, or any input mechanism. Process: Processing the raw data, often to smooth it or compress it. Transfer: Taking the processed data and moving it from the device, either directly or via a local hub, which might be a mobile app.  But the ultimate end is most likely to get the data to a cloud-hosted backend Analyse:  Combining and analysing the processed data, potentially with additional product or logistical data, to produce useful insight Respond: This describes the effect the device will have.  If it does not induce some kind of response, what purpose does it have?   That response might be at a central location, for example, a call centre, or it might be to deliver information back to the user. Let’s consider an example. Imagine you run a fleet of delivery vehicles.  Each truck is fitted with various sensors and telematics, allowing you to pull data from each truck to a central hub to help you manage the operation. One thing you might want to know is exactly where each vehicle is located at any point in time.  You might fix a tracker device, basically a cell phone, which will read location and transmit it back to base.  Maybe the trucks are also fitted with temperature sensors to ensure that the cargo is kept at the correct temperature for the cargo. The thermometers sense temperature every 1/10th of a second.  They transmit this data to the tracker via Bluetooth Low Energy.  We don’t need to know the temperature that frequently, every minute is fine, so the tracker calculates a mean temperature each minute (process), and transfers that. The cell phone detects location by GPS and Cell Tower triangulation (sense).  It converts that information into coordinates (process).  It transmits this data back to base every second (transfer). Back at base, the location data is converted into an estimated time of arrival at location, which is fed into the logistics planning for the warehouse (Process).  The temperature data is monitored and if it’s seen to go outside of range, an alert is sent back to the driver who can pull off the road and investigate (respond). An alert might also go to the maintenance team to check the refrigeration units when the delivery is complete.

Where does Artificial Intelligence fit in the 5-Layer Model?

When we consider how AI might enhance this model, we think about the Process and Analyse phases of the model.  In our example, calculating the arrival time from the location involves some AI.  We’ve become so used to satellite navigation that we can forget what a marvel it is.  It’s able to take your current location, consider current road conditions, identify the best route (depending on various factors we ask it to consider), and calculate the time it will take us to complete this.  It does this using algorithms, essentially identifying and comparing all the possible routes to identify the best one.  If you use a tool like Google Maps or Apple Maps, the calculation is run on a server in the cloud and the results are transmitted back to the device to draw the map and route.  This is an example of Analyse Intelligence. Let’s consider another example.  The truck will be fitted with many sensors.  The engine will have numerous sensors detecting temperature, pressure, vibration, voltage, etc.  These will be fed into the engine management system and mechanics can use that information to diagnose issues and undertake preventative maintenance. Vibration data, for example, can be extremely useful in understanding driver performance, but vibration data is very noisy.  Collecting three axes (up, down, and sideways) 100 or 1,000 times a second generates a lot of data.  Sending that back to a server continuously would cause a lot of issues.  Instead, vibration data should be processed on the vehicle and compressed into a more succinct data set.  This can then be transmitted back to a server where it can be combined with other data and used to compile a driver performance profile, for example. Compressing vibration data into a vibration index is a very good example of using AI.  In this case, the AI will be looking at what is ‘normal’ for this vehicle and determining how far away from normal the given sensor is at a given moment.  It would want to distinguish between ‘driver is on a poorly maintained road’ from ‘driver driving too fast for the road conditions’ or ‘driver is using too low a gear’, for instance.  

Four Key Challenges

Our examples illustrate the four key challenges that IoT architectures need to address.

The Data Challenge

IoT devices collect lots of data.  The more data they collect, the more expensive (in terms of processing power, battery life, storage space, and cash cost) it is to transmit that data back to a server.  This pushes us to consider increasing the amount of processing that takes place on the device, or a local hub.  Then we only transfer the compressed or summarised data after it has been processed.  The downside of that however is that we can’t access the raw data to learn more about what is going on.  Sometimes IoT devices have a local data buffer that will retain data for a limited time before overwriting it, allowing for raw data to be recovered if it’s required in exceptional circumstances.

The Processing Challenge

AI can be computationally heavy and there may not be enough power to perform the necessary calculations on the target device.  Adding more processing power to an IoT device can lead to the unit cost of that device being too high. Not all AI is created equal in this regard.  Some algorithms require significant processing power to train but once trained the actual operation of the algorithm is computationally light.  Other approaches, which involve some kind of ‘learning’ during operation, can be much heavier, however.  

The Update Challenge

Another factor to consider in solving the ‘data problem’ is updates.  Sophisticated AI algorithms often need updating or fine-tuning.  IoT devices can be difficult to update.  If they are installed in a fleet of vehicles it’s not really practical to have to physically access each one, or each vehicle.  Instead, you would look to implement a mechanism for Over The Air (OTA) updates.  This is a non-trivial problem to solve.  For mobile devices, we have mechanisms through things like the App Store and Play Store, but for bespoke devices, you have to create this capability yourself.  Reliable OTA functionality is hard to achieve.  

The Latency Challenge

Connected to the Data Challenge is the Latency Challenge.  Let’s consider another example from our IoT-enabled truck fleet.  As I mentioned, the engine is fitted with a large range of sensors and this might well be used to make on-the-fly decisions about engine optimisation.  When the vehicle is cruising at a constant speed, the engine can be tuned for maximum efficiency.  When the vehicle is accelerating uphill, the engine can be tweaked to give a small performance boost.  These are decisions that need to be taken in the moment.  It would not be appropriate to send data to a server and have it make decisions about engine tuning.  The processing and the response need to be local. With satellite navigation, however, although you want a reasonably quick calculation of the optimum route, with live updates, a second or two is unlikely to make much of a difference.  Here the data coming from the vehicle is small and instantaneous response times are not required, so processing can take place on a server.  

Making Intelligent Things

We’ve considered the 5-layer model of IOT and the 4 key challenges that IoT architectures need to overcome.  We’ve seen how Artificial Intelligence contributes to the power of IoT to solve complex problems through a combination of data that is sensed and information that is known. Where are Intelligent Things heading?  One of the most talked about technology innovations of 2023 has been the arrival of ChatGPT.  This is an example of a generalised AI model, that is, a tool that can solve general problems, rather than specific ones.  Generalised AI models get somewhere close to natural language processing.  You can ask it a question in a natural way, as you would a person, and it will give you an answer.  How does this power add to the way IoT works? Let’s consider our example.  Some of the things we talked about can be impacted. For example, it would become possible to ask for directions rather than key in a postcode, but that has been available for some time and is not that useful for a logistics operation. Our vibration sensor example isn’t an example of a general model, it’s a very specific model that needs to be trained to the specific task assigned to it.  Generalised AI isn’t good at this kind of task. Where Generalised AI might become useful is at base, where decisions have to be taken about which truck to send where, and when to schedule maintenance.  It might be that the team responsible for managing the operation interacts with the software that supports them using more natural language and less keyboard/mouse.  

Autonomous Intelligent Things

Of course, the ultimate goal in our example is to develop a self-driving truck.  If there is no driver, the truck can be moving more of the time.  Communication between the truck and the logistics planning tools can be instantaneous.  Autonomous vehicles don’t need to be sent directions and they don’t take wrong turns.  Autonomous vehicles can communicate with each other to maximise use of road space, and they can communicate with traffic planning software to move through an area in the most efficient, least impactful way.  They can arrive at the lights just after they’ve changed to green. We are still some way from autonomous vehicles, but this is the direction of travel.  In all aspects of IoT, autonomous sensors will function increasingly without human intervention.  Efficient use of assets will be maximised and decisions taken seamlessly.  Humans will only be required to take the biggest and most important decisions.  We will dictate strategy, but the Things will be the ones who implement it. This has big implications for society,