Vision- and optics-based sensing in the Internet of Things era
Edo Noordermeer, Global Product Owner for Monitoring Innovations, Fugro
TotaLite vision-based bridge monitoring.
MONITORING is booming – the infrastructure sector in the Organisation for Economic Co-operation and Development (OECD) countries is experiencing a major shift in focus and budgets. After decades of focus on new construction projects, more and more effort is having to be allocated to operating and maintaining existing infrastructure, as well as renovation and replacement.
The underlying trend here is the combination of three factors; ageing infrastructure built in the aftermath of the Second World War between the 1960s and 1980s; a higher-than-anticipated utilisation rate; and years of deferred maintenance. In the USA alone, it is estimated that there are 46,000 deficient bridges being crossed by 172 million citizens every day1.
In this new environment, the major challenge for asset owners is to plan and prioritise their renovation works, to:
- Maintain the performance (uptime) of their assets
- Manage rising maintenance and renovation costs, to flatten the investment curve. In The Netherlands, for example, the annual cost for infrastructure refurbishment in 2040 will be 2.5 times higher than today’s budget2
- Make sure that ageing infrastructure remains safe for society to use, avoiding catastrophic events such as the collapse of Italy’s Morandi Bridge in 2018.
The trend described above represents an enormous opportunity for the global monitoring industry because prioritising the maintenance and renovation of infrastructure portfolios will require a better understanding of the condition of assets.
Fortunately, there is another global trend which helps to fulfil the needs of the monitoring industry, namely the development of the IoT.The traditional inspection-based approach to information-gathering will require an increasing number of experts to respond to this demand – yet they are already in short supply, particularly in countries with an ageing population.
Acutely aware of this growing challenge, many proactive asset owners have started transitioning to a data-driven approach to asset management. Demand for automated monitoring systems is rising fast – in the global structural health monitoring industry, it is projected to grow by around 15% a year3. This will bring new challenges.
The industry will need to respond to the growing demand for automated monitoring by developing sensing solutions that are more scaleable, easier to deploy and cheaper. It will need to develop new methods for processing and analysing ever-increasing volumes of data, so that the translation from raw data to actionable insights can be (partly) automated.
The IoT revolution
Fortunately, there is another global trend which helps to fulfil the needs of the monitoring industry, namely the development of the Internet of Things (IoT). The IoT describes the emergence of technology that allows physical objects (‘ things’ ), embedded with sensors and software, to connect to the internet. It enables seamless communication between these objects, anywhere on the planet, and central cloud-based databases and processes.
The development of the IoT was initially driven by other sectors, evidenced by smart household appliances and remote health monitoring applications. But it also offers great advantages for geotechnical and structural health monitoring.
IoT innovations have fuelled a new generation of small, low-cost sensors and wireless communication networks. Together, these simplify field installation; enable sensor systems to be configured and managed remotely; and help to fuse multiple data sources, giving users more holistic information about a project site or asset.
There is no doubt that these innovations will play a major role in facilitating the increased demand for real-time monitoring data. The ease of installation of the modern generation of sensors, coupled with the increased data volumes that can be processed in the cloud, enable more asset data to be collected with less effort.
Meanwhile, the price per sensor will inevitably fall, as will the cost per megabyte of data processed and stored, so the total expenditure on a given project for monitoring can still be reduced.
Deformation monitoring in the IoT era
Deformation is a key parameter in geotechnical and structural health monitoring. It is also a key indicator of (future) failure, with potentially disastrous consequences4. Deformation can be caused by natural influences, such as the natural settlement of underground layers or degradation of foundation piles; or by human activities, such as excessive load or nearby construction activities.
Many of the existing deformation monitoring technologies, like tilt sensors or strain gauges, can be adapted easily for the IoT by miniaturising sensors and implementing wireless data communication.Deformation monitoring has a long history and is done with a variety of sensor types. Many of the existing deformation monitoring technologies, like tilt sensors or strain gauges, can be adapted easily for the IoT by miniaturising sensors and implementing wireless data communication.
Most sensor manufacturers now offer wireless, low-power versions of these types of sensor, which send data automatically to a cloud-hosted data repository.
In sharp contrast, optical surveying equipment is not ideally suited to IoT. Devices like levelling instruments and total stations offer major advantages in terms of their contactless measurement principle, superior accuracy and versatility.
But with features like precision optics, rotating parts and self-leveling mountings, they are often large, heavy, power-hungry systems that must be installed by surveying experts. They are also expensive and prone to theft, so require extensive security measures.
SealSafe bar components.
Vision-based innovations for deformation monitoring
At Fugro, we believe that the future of the monitoring industry lies in scaleable and low-cost sensor systems. Only by improving the ease of installation and reducing the ‘cost per measurement’, will the monitoring industry be able to meet the growing demands of the construction and infrastructure markets.
Many of the traditional sensor types used in geotechnical and structural health monitoring are now available in an IoT format: small, power-efficient and wireless.
Meanwhile, increasingly powerful data management systems are being developed and more advanced data analysis and visualisation techniques are becoming available. These steps are helping us to deploy ever larger numbers of sensors and to manage the increasing data volumes that give our clients the insights they need.
Fugro is keen to achieve the same benefits for deformation monitoring, so it has started an in-house development programme of novel optics- and vision-based techniques for measuring displacements and deformations of assets and infrastructure.
The new systems combine the benefits of optics-based measurements (remote, contactless, longterm stability) with the benefits of the IoT. So, what do optics-based sensing innovations look like?
SealSafe: Scaleable monitoring of moving assets
SealSafe is a highly scaleable, compact and contactless measuring system that monitors changes in distance to a nearby object. It was recently deployed it in a tunnel in the Netherlands, where more than 1,000 sensors were installed to monitor the movements of rubber gaskets between the individual concrete elements.
Installing traditional displacement sensors in such large numbers would have been prohibitively expensive, but SealSafe is based on low-cost infrared and optical laser sensors that can be installed easily and efficiently in series of up to 100 sensors connected to a single data logger.
The acquired data is streamed via a 4G cellular connection to a cloud-based data engine, where it is processed in near real time and presented on an online dashboard, together with data from other sensor systems in the tunnel, such as strain, force and temperature.
The dashboard fuses the various data sources together to provide insight into the overall status of the tunnel. The combined data set provides information about the ‘normal’, natural behaviour of the tunnel.
Crucially, it also creates automated notifications if any anomalous conditions arise that may indicate increased risks of failure or damage.
The system’s applications are not restricted to tunnels – it can be used in any situation where two moving parts of an asset need to be monitored frequently and in a variety of places.
TotaLite: Vision-based displacement monitoring
TotaLite is Fugro’s novel IoT sensor technology for long-term remote optical monitoring of displacements and deformations. It shares some functionalities with robotic total stations, in the sense that it provides contactless measurements of displacements of targets (prisms) on an asset from a safe distance.
However, instead of the laser-based pointing technology used in total stations, TotaLite uses imaging technology. Its applications could help in risk-based asset management and construction monitoring of a variety of structures, including bridges, tunnels, quaysides and buildings.
Many of the traditional sensor types used in geotechnical and structural health monitoring are now available in an IoT format: small, power-efficient and wireless.Its optical assembly, combined with its image processing algorithms, delivers an accuracy of <1 mm at 50m distance and long-term stability that is comparable to a total station. Yet it is far less complex with no moving parts, making it easier to manufacture, calibrate and maintain.
TotaLite can also measure multiple targets simultaneously and at much higher frequency than a total station. It is compact and uses low-power optical sensors, which upload data wirelessly to cloud-based software systems for further post-processing with the displacement measurements displayed on an online dashboard. The sensors can be configured remotely through that same wireless link.
Fugro recently ran a pilot with an early version of TotaLite.
Deploying two of the sensors on either side of a railway bridge to monitor displacements of its abutments. It achieved around the same level of accuracy (<0.5mm) as the weekly manual measurements that the bridge owner had been relying on. It also took measurements more frequently, providing a deeper insight into the behaviour of the bridge under the influence of the weather, changing water levels or vibrations from passing trains.
TotaLite sensor monitoring railway bridge.
The future: Turning deformation monitoring data into actionable information
The IoT is transforming the geotechnical and structural health monitoring industry in a profound way. Scaleable and low-cost sensor systems are helping to meet the increased demands for automated acquisition of asset data.
But more data alone will not answer the questions that asset owners have about the condition of their assets. Data without proper interpretation will only lead to more questions.
The major challenge for the monitoring and wider asset management industries is to develop generic, scaleable data-analysis techniques that help asset owners to interpret the monitoring data and render it actionable. The bottom line is that asset owners need clear answers to fundamental questions about their assets in terms of safety, maintenance needs and remaining lifetime.
The engineering industry traditionally relies on custom-made, physical models of assets to interpret monitoring data. This approach applies especially to deformation monitoring, given that deformations can affect the structural integrity of an entire object. However, traditional methods require deep constructive engineering expertise, which is becoming increasingly scarce and expensive.
The IoT is transforming the geotechnical and structural health monitoring industry in a profound way. Scaleable and low-cost sensor systems are helping to meet the increased demands for automated acquisition of asset data.
There’s another, equally important issue.
Deformation monitoring experts require detailed information about the original construction design and how it was built, but this was often not recorded in the first place or archived inappropriately. Without this information, construction engineers are often left to guesswork about crucial details, which severely hinders a conclusive interpretation of measurement data.
The question of turning measurement data into actionable insights is not unique to our industry. The broader field of data analytics has seen tremendous advances in recent years, with new methods for machine learning and artificial intelligence.
At Fugro we believe the future lies more in these purely data-driven techniques, rather than in trying to optimise the physical modelling of assets. Modern, scaleable data acquisition systems allow larger volumes of data to be collected simultaneously: more parameters, at more locations and higher frequency, and from many assets within a complete infrastructure network.
With modern data analytics techniques, this wealth of data can be used on its own, without physical modelling, to help distinguish between normal behaviour and anomalous patterns. In effect, automated monitoring systems act as highly efficient early warning systems that trigger more detailed investigations, as and when needed.
Owners can focus their limited resources on assets that are in need of attention, safe in the knowledge that other assets do not currently need human intervention.
Edo Noordermeer, Global Product Owner for Monitoring Innovations, Fugro
SealSafe and TotaLite are registered trademarks of Fugro
---
1 Infrastructure Report Card; https://infrastructurereportcard.org/wp-content/uploads/2020/12/Bridges-2021.pdf
2 TNO 2021 R10440A, Instandhouding civiele infrastructuur. Proeve van landelijk prognoserapport vervanging en renovatie
3 Structural Health Monitoring Market by Component, End User, and Connectivity: Global Opportunity Analysis and Industry Forecast, 2020–2027; Allied Market Research; March 2021
4 Pre-Collapse Space Geodetic Observations of Critical Infrastructure: The Morandi Bridge, Genoa, Italy; P. Milillo et al., Remote Sens. 2019, 11, 1403; doi:10.3390/ rs11121403