New Excavator Interface Lets Rookies Match Veteran Operators on Day One

A team led by Hermano Krebs, a principal research scientist in MIT's Department of Mechanical Engineering, has built a new excavator controller that compresses years of operator training into a single session. The findings appear in the Journal of Computing and Civil Engineering.

Standard joystick controls force an operator to translate every wrist movement into cabin rotation, arm extension or bucket movement — an unnatural mapping engineers call a mental model that typically takes years of practice to build. The team's device, called the World-Space Interface (WSI), is a miniature excavator arm and bucket that the operator physically grips and moves the way they would move their own arm while digging by hand, while a wraparound six-panel display projects a virtual excavator that mirrors every gesture instantly. "With this new interface, we can eliminate a lot of the mental maps that an operator would need to build in order to operate an excavator," Krebs said.

Krebs envisions the same controller eventually mounted directly in a real cabin, functioning like a prosthetic extension of the operator's arm, or even allowing an operator to sit off-site in a trailer and remotely tele-operate the excavator in a difficult or unsafe environment.

To test the concept, the researchers ran training experiments with expert and novice volunteers using both the WSI and a conventional joystick-based simulator across 15 virtual scenarios covering construction sites, highways, forest roads, riverbanks, mining areas, and urban and rural settings. Tasks included scooping and dumping sand or gravel, digging and grading trenches, clearing debris from roads, removing tree branches from water edges, and breaking up rocks. Volunteers trained one hour a day for seven days, mirroring a typical weeklong excavator driving course. With joysticks, novices performed consistently worse than experts, though they improved over the training period. With the World-Space Interface, that gap vanished entirely: novices matched expert performance from the very first session. "This is the first interface that does not require me to command the excavator with joysticks," said study co-author and MIT postdoc Joao Buzzatto.

The system currently gives the operator no physical feedback, so the team is now working to add haptics — force feedback that simulates the sense of touch — so an operator could feel the heaviness of rocks being picked up. Caterpillar, Hyundai and Komatsu are also developing their own simulators for training and eventual remote operation, but these remain largely based on traditional joystick controllers.