AI can help run factories better: Thesis sharpens the 3D models

edited by Sadie Harley, reviewed by Andrew Zinin

This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:

Add to Preferred Sources Credit: Unsplash/CC0 Public Domain Digital twins are virtual 3D replicas of real spaces such as factories, warehouses and industrial plants. Companies use them to monitor equipment, plan maintenance and simulate changes without disrupting operations.

For a digital twin to be genuinely useful, the system has to understand what each object in the scene actually is: which part is a pipe, which is a container, which is a machine. Identifying and labeling those objects is known as the segmentation problem.

Ofia Rahman Alif's thesis examines this problem. The research was carried out in collaboration between the University of Helsinki and Twinverse, a Finnish company that builds digital twins of complex industrial facilities.

"This study was an opportunity we could not have otherwise taken. The results were insightful, and we remain open to continuing collaboration in the future. I encourage companies to be active and understand what an academic thesis is, how to guide and push thesis writers to do their best. There's little to no cost to the company other than the time investment, with genuinely useful results," says Arttu Julin, chief technology officer of Twinverse.

Alif's thesis provides a systematic analysis of where this approach succeeds and where it breaks down in real industrial settings. It examines how the quality of the 3D reconstruction affects segmentation accuracy, how the camera setup changes the results and how the consistency and accuracy of the object masks shape the final outcome.

"My analysis also looks at the conditions that make industrial environments difficult for AI: dark metallic surfaces, complex geometries, objects that are partly hidden from view and objects that look very similar to one another. Each of these affects different stages of the pipeline," Alif explains.

To make comparisons concrete, the methods for improving AI performance were tested both on real industrial datasets provided by Twinverse and on standard benchmark scenes commonly used with 3D Gaussian splatting (3DGS), a technique that reconstructs a real-world space as millions of tiny 3D shapes fitted to photographs taken from multiple angles. This allowed a direct comparison between idealized conditions and the messier reality of an operating industrial site.

The results map out which parts of the segmentation pipeline are ready for practical use and which still need work before object labeling in industrial digital twins can be reliably automated.

Alif's thesis builds on three existing methods. The first, 3DGS, produces photorealistic 3D reconstructions from regular camera images and their corresponding camera poses.

The second is FlashSplat, a research method that assigns each of these 3D shapes to a specific object using a voting mechanism across multiple camera views. The third tool is SAM2 (Meta's Segment Anything Model 2), an AI that can identify and outline objects in images.

"A central contribution of my thesis is a pipeline that connects these tools. SAM2 generates object outlines, or masks, from the images. SAM2's video tracking then keeps those masks consistent across different camera viewpoints, and the results feed into FlashSplat's 3D voting system."

The thesis also introduces an adaptive component that automatically selects the best SAM2 settings for each scene, based on characteristics such as lighting, how reflective the surfaces are and how complex the scene is.

"The most surprising result was that giving the AI more data actually made it perform worse, not better. A small, carefully checked set of inputs produced a far cleaner result than a much larger but noisier one. It turns out the system has no way to undo a bad piece of evidence once it's factored in, so a few mistakes can outweigh a lot of extra data. It was a good reminder that in AI, quality often matters more than quantity.

"Moving from controlled research datasets to a real industrial environment was a much bigger jump. The tools I started with worked better on clean, controlled scenes but broke in unexpected ways once I tested them in an actual industrial environment. The biggest lesson was that failure isn't a dead end if you can explain it, and understanding why something breaks is often as valuable as getting it to work," Alif sums up.

Ofia Rahman Alif, 3D Scene Segmentation for Industrial Digital Twin Applications. hdl.handle.net/10138/639508 Key concepts Computational 3D vision Machine learning methodologies Provided by University of Helsinki Who's behind this story? Sadie Harley BSc Life Sciences & Ecology. Microbiology lab background with pharmaceutical news experience in oil, gas, and renewable industries. Full profile →

Master's in physics with research experience. Long-time science news enthusiast. Plays key role in Science X's editorial success. Full profile →