NS3D set out to revolutionise monitoring of marine man-made structures (MMS), including offshore wind turbines and oil and gas platforms, by utilising routine ROV and diver footage to better understand how installation and decommissioning scenarios will affect wider ecosystem function.
Using cutting-edge 3D imaging, deep-learning AI, and statistical modelling, the project identified marine organisms and quantified their biomass. This revolutionary approach has the potential to transform years of existing video footage into valuable insights, revealing long-term ecosystem trends around offshore infrastructure without the need for manual review which requires great resource.
Using ROV and diver footage, NS3D programmed AI models to recognise different marine species automatically, piloting and testing their technology on local reefs. In order to quantify biomass these AI models were paired with 3D reconstruction techniques to generate lifelike digital models of MMS, enabling researchers to map where organisms live and calculate how much biomass they contribute.
Accurately quantifying the biomass growing on MMS is crucial because it provides evidence of the ecological value of MMS which could influence decisions about decommissioning options in the North Sea.
Video showing Automated species identification of UK temperate reef. Original footage (left) and species-annotated and reconstructed footage (right).
The project found that convolutional neural networks (a special type of computer programme that examines pictures and can recognise patterns), particularly SegFormer, achieved impressive ~92% accuracy in species identification (Marlow, et al, 2024).
However, while creating 3D models from underwater footage was technically feasible, the inconsistent quality of industry-sourced ROV videos made it difficult to accurately model or properly measure the volume of marine life growing on structures across the footage supplied.
Left: High-quality underwater image showing marine life on a man-made structure. Right: Example of typical ROV footage quality that researchers actually work with (image from John Halpin’s presentation at SIME 2024). Available here: https://www.youtube.com/watch?v=eZIAoTZ9Ylo&t=720s
In fact, only a small fraction of video footage was suitable for creating high-quality 3D models. Most videos lacked proper size references and did not clearly show where marine organisms ended and the metal structure began – both essential for accurately measuring the amount of marine life present.
Additionally, industry footage is typically collected to inspect the structures themselves rather than document marine life, meaning researchers cannot influence how the survey is conducted or direct the ROV’s path. This creates problems for 3D modelling, which requires images captured from many different angles and consistently following a robust method to work effectively.
Video: NS3D’s 3D modelling and automated species identification using industrial ROV survey footage of a North Sea Oil & Gas platform
To overcome these challenges and help make the AI and quantification of the ROV footage viable in the future, close collaboration with industry is crucial. The project has also highlighted the importance of early collaboration with modelling experts like SAMS to optimise both data capture and analysis from the outset. The NS3D team has been working with industry to help improve their footage quality, and methods for biomass data collection making asset inspections more effective for multiple data outputs. This has been done through developing a new methods and standard operating procedure for data collection that can be adopted by industry to ensure footage is suitable for both asset inspection and marine life analysis. Moving forward, widespread adoption of these methods and procedures could transform how the offshore industry monitors and manages marine life on structures by providing quality footage for the quantification of marine life biomass.
Imagine exploring the hidden ecosystems around man-made structures in the North Sea without ever getting wet. Through an innovative blend of art, science, and technology, NS3D and the University of Arts London are bringing these underwater worlds to the surface.
Rachel Pearl, a researcher exploring ecological systems through speculative design, created Project Archipelago to reimagine decommissioned North Sea oil platforms for boosting marine biodiversity. Developed in collaboration with NS3D, her installation features a 3D platform model with printed coral growth structures.
Image: Project Archipelago’s innovative 3D printed structures reimagine offshore platforms as biodiversity hubs
Building on these models, NS3D and the University of the Arts London to develop an immersive virtual reality experience that brings marine life on offshore structures to life for public audiences. This underwater adventure has been showcased at marine science festivals with thousands of visitors, providing an accessible way to explore these hidden ecosystems without getting wet. Such outreach is vital for raising public awareness about these hidden underwater ecosystems, highlighting the environmental value of offshore structures.
Left: Photo of an attendee at SIME Conference 2024 taking a virtual dive around a marine artificial structure; Right: picture of what someone sees when they take a virtual reality dive.
Looking ahead, NS3D is working much more closely with industry partners, joining survey cruises and using ROV downtime to conduct their own targeted research, resulting in better models and more accurate biomass estimates. With a current particular focus on Lophelia (cold-water coral) and access to extensive data archives, the NS3D project continues to expand, with exciting developments on the horizon.
Weighing up the pros and cons of different decommissioning options – studying how marine structures...
Scientific experts from around the world agree that complete removal of man-made structures (MMS) may...
We regularly publish updates on the projects that are directly funded by the Programme and other related research. If you would like to be kept up to date, sign up for our Connections newsletter.