Human in the loop 4.0: Human feedback guides AI approach that automates about 84% of decisions

by Anja Achenbach, Kühne Logistics University

edited by Lisa Lock, reviewed by Andrew Zinin

This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:

Add to Preferred Sources Credit: Unsplash/CC0 Public Domain How can we harness the potential of automation while retaining human oversight? And what does successful automation look like in practice?

In their paper "Targeted Automation and Sustaining Human-AI Learning," Kai Hoberg, professor of supply chain and operations strategy at Kühne Logistics University (KLU), Christina Imdahl, professor of machine learning in operations management at Eindhoven University of Technology, and William Schmidt, professor at Emory University, make the case for targeted automation.

Their approach is to automate decisions where human intervention adds little value, while keeping people involved where their input can lead to better outcomes.

The work is published in the journal Production and Operations Management.

"Our research shows that companies should take a selective approach to automation. Rather than automating all decisions, they should deliberately retain human involvement in those decisions where people can help improve the algorithm.

"At the same time, sufficient human decision-making must be maintained to ensure that the algorithm can adapt to changing conditions," says Hoberg, professor of supply chain and operations strategy at Kühne Logistics University (KLU).

This allows companies to strike a balance between short-term efficiency and continuous AI improvement, helping to ensure long-term success.

To put this into practice, the researchers developed a four-step approach to targeted decision automation.

1. Identify where human input adds value: First, they analyze when human corrections to algorithmic recommendations lead to better decisions.

2. Predict the value of human expertise: Based on these insights, the model developed by the researchers learns to predict when human expertise is likely to improve decision outcomes.

3. Automate decisions selectively: Decisions where human intervention is expected to offer little additional benefit are automated, while those where human expertise can add value remain in human hands.

4. Continuously improve the algorithms: Finally, human corrections are fed back into the system to refine and improve the algorithms over time.

"The crucial point is that it is not simply about leaving difficult decisions to humans, but about predicting which decisions actually allow humans to contribute to improving the system," emphasizes Imdahl, who earned her doctorate at Kühne Logistics University and is now a professor at Eindhoven University of Technology.

This ensures that valuable human expertise continues to feed into algorithm development while still allowing for a high level of automation. "In our empirical application, we were able to automate around 84% of decisions," she says.

Hoberg advises business leaders to take a gradual approach to automation, starting by establishing how they can measure the added value of human intervention.

"This provides a basis for making targeted decisions about which tasks to automate and where human expertise is still needed."

For their study, the researchers analyzed an extensive data set covering more than 475,000 order-related decisions processed over a four-year period at a major commercial vehicle manufacturer with annual revenues exceeding 4 billion euros. Their aim was to identify the circumstances in which humans outperformed the algorithm.

However, Hoberg, Imdahl and Schmidt were not primarily concerned with the traditional question of "Who performs better: humans or AI?" Instead, they sought to develop a model that views humans and AI as partners within a dynamic system. The goal is to relieve employees of repetitive tasks that add little value while ensuring that AI continues to evolve and improve through ongoing human feedback.

Christina Imdahl et al., Targeted Automation and Sustaining Human-AI Learning, Production and Operations Management (2025). DOI: 10.1177/10591478251381899 Key concepts Agentic consumer AI Artificial intelligence labor economics Provided by Kühne Logistics University Who's behind this story? Lisa Lock BA art history, MA material culture. Former museum editor, paramedic, and transplant coordinator. Editing for Science X since 2021. Full profile →

Master's in physics with research experience. Long-time science news enthusiast. Plays key role in Science X's editorial success. Full profile →