Keeping up with modern truck maintenance is a towering task. As trucks become more high-tech, it can be difficult for seasoned technicians to keep up, and even more difficult for novices to know where to start.

Augmented reality (AR) and virtual reality (VR) technologies are changing not only how technicians are trained, but also how they work. These mind-bending technologies are being deployed through various hardware and software solutions across multiple areas of the industry—from young students being introduced to what the commercial vehicle maintenance industry has to offer to real-time virtual collaboration during repair.

Understanding how AR and VR technology can be leveraged can help the industry successfully navigate advances in vehicle technology, recruit and onboard qualified technicians, and continue to move the maintenance bay into the 21st century.

Virtual immersion

AR and VR technologies offer multiple benefits to the maintenance industry, one being the ability to learn in a digitally enhanced environment that presents a level of engagement unattainable through other means.

The technology available to create these virtual environments has been developed beyond just costly headsets. There are now several entry-level smartphone and tablet applications available that interact with objects—from showing where all the sensors are hiding behind a bumper to overlaying step-by-step guides into a technician’s field of view. Using a smartphone app, a user can point a phone at equipment and see an augmented overlay of supplemental icons and other information on the phone screen that updates as the phone’s camera moves. And if that technician wants another set of eyes on the engine they are working on, an expert can log in remotely and virtually climb under the hood.

Photo courtesy of Wallace State Community College

Recruiting the next generation

Another benefit of AR and VR technologies is that they offer a different medium through which the maintenance industry can offer exposure to the next generation of potential technicians. During the American Trucking Associations’ (ATA) Technology & Maintenance Council (TMC) Spring Meeting, the organization announced the release of an AR game designed to attract, foster, and welcome a new generation of skilled technicians to the industry.

Based on TMC’s National Technician Skills Competition, the app—called TMCSuperTech: The Game—is geared toward younger students and is available as a free download for smart devices on Apple iOS and Android platforms.

“We thought that having a smart device game app would be a great way to reach a broad audience of middle and high school students to get them exposed to the idea that being a commercial vehicle technician is a valid career path,” said Robert Braswell, TMC’s executive director.

Design Interactive worked with TMC to develop and deploy the game. Matt Johnston, division head of commercial solutions at Design Interactive, explained that the game places the player in the role of a technician. Within the game, the player positions their phone screen toward a flat surface such as a floor or desk. Through the screen using AR, the player views a city landscape with trucks moving throughout. The player receives an alert when a truck is in need of repair, with the option to send the truck to the maintenance shop in the game.

Image courtesy of Design Interactive

“It has an animated sequence where the parts [are presented] and they have to make the right decision,” Johnston explained. “Then they get points for doing things right, and how quickly they make these repairs, and send that truck back to the road. “[The app] is intended to give them a little look at what running a maintenance shop might be like, what performing the repairs might be like, but leveraging this cool technology too makes it a little more fun—it’s a bit of a halo on the industry.”

Braswell explained that the game does more than just introduce the concepts and applications of vehicle repair management, but also presents resources to get students engaged in the industry.

“When you download the game, you can click and self-direct yourself [to resources providing information] about becoming a technician, find out about scholarships,” he said. “We are directing folks to the TechForce Foundation, which is our partner in this outreach effort, because they are very well suited toward social media that is attractive to students to get them interested in [the industry].”

Braswell added that information is available for students to learn how to talk to their parents, guidance counselors, and teachers about how this is a career they would like to pursue.

“The industry is exploring all kinds of fantastic technology from electrified vehicles to autonomous vehicles. This is just another bit of technology that can be used,” Johnston said. “I don’t know if those kids see this industry or the image of the industry as such that it reflects ‘high technology’ in comparison to the gaming industry or other places they might consider as they move forward. But it absolutely is just rife with next-generation technologies. They need to be aware of what the truth is—not just what the perceived or the assumed identity of the industry is.”

Engaged education

Alternate reality technologies can enhance the education process for those training to become diesel technicians. Educators are leveraging such technologies to further engage students with their respective curriculum. One such program is Diesel by Distance, based at Wallace State Community College in Alabama. The program integrated a VR service from TRANSFR in April.

“We are focused on our mission of helping to create that classroom to career pathway,” said Bharani Rajakumar, CEO and founder of TRANSFR.

The goal of the Diesel by Distance program has been to enable more engaging distance learning. Historically, the challenge with manufacturing and the skilled trades has been the need for hands-on training and distance learning has traditionally been thought of as online video, Rajakumar noted.

“We worked to understand what are all the things that someone would need to learn in order to become part of a diesel technology career pathway. We included things like replacing shock absorbers, putting together and taking apart engines, and things of that nature,” he added.

TRANSFR designed and created virtual training simulations within a virtual training facility, allowing users to don a headset no matter their location and learn the hands-on skills aligned with the program.

Rajakumar explained that the intent is to fuse education and industry learning into one experience. He said many technicians coming out of traditional degree and certification programs are showing up to the job site unable to do what the employer would like them to do on day-one. TRANSFR creates and designs virtual reality training simulations with input from employers, so when a student puts on the headset, they learn how to do the job before they get the job. That helps the employer make sure that someone is prepared for the job as soon as possible. It also helps the trainee get insight into what it is that they will actually be doing.

TRANSFR’s virtual reality experience also offers a digital coach within the program.

“It’s perfectly fine to make mistakes,” Rajakumar said. “We assume that everyone we are working with is a beginner, and in some way that is why they are using the service, so that they can acquire the skills to move up in their career path. The digital coach helps nudge you in the right direction when you’re making mistakes and not performing based on what an employer would expect.”

Having the ability to simulate that shop experience and bring it to the student can help expedite the onboarding process and provide first-hand real-world experience that caters to students who, generally speaking, are hands-on learners. Presenting this level of immersion can help drive home that learning experience and clarify expectations as to what the career will truly require.

“There are three different types of learners: visual, auditory, and kinesthetic,” said Beth Rutter, North American director of industry and customer engagement at Tradiebot Industries. “Guess what kind of learning is typically attracted to the repair industry?”

It is the kinesthetic—or tactile—learner, she said.

“All of a sudden, the demographic that’s attracted to collision repair and working on vehicles—whether they be automotive or heavy-duty diesel—get to learn in an environment that actually speaks to them,” Rutter said. “They get it way faster than the people who have to sit down, write stuff out, and review instruction.”

Enhanced on-site training

Implementing AR and VR resources beyond the classroom and into the service bay can make a major difference in the aptitude of technicians. Using such technologies for on-the-job training creates a continuous learning environment.

Design Interactive’s AUGMENTOR is a training platform that is enhanced through AR. An instructor-led feature, Classroom XR, allows a single instructor to livestream training from a first-person perspective populated with AR content to remote technicians who can view the training via a web portal.

Technicians can use AR to inspect equipment before ordering it.Photo courtesy of Design Interactive

“Our tagline is ‘replicate your experts,’” Johnston said. “You basically get the expert’s approach in that headset. You put it on the head of a brand new [technician], and you turn them into an expert immediately.”

A self-guided training aspect of AUGMENTOR allows technicians to download an AR-enhanced application that grants access to specific procedures. This can be used to brush up on certain repairs or can be applied in real-time as a step-by-step guide. With AR, content can be at the tech’s fingertips. AUGMENTOR also offers remote collaboration, an “over-the-shoulder” look from a trainer, allowing a remote expert to see what the technician is looking at. The trainer could test the technician’s skills, offer insight, or guide real-time troubleshooting.

Johnston posed an example of AR training and the development it can lead to for a technician.

“You’re repairing a brake and you need to make a decision whether during your wheel-on inspection to go to a wheel-off, or whether you need to repair or replace something,” he said. “Oftentimes, it’s corrosion, cracks, grooves that you see in things like the rotor or the brake pad itself. They have to make this decision, and if they make the right decision, then great. If they make the wrong decision, which could be replacing something that still had life, or not replacing something that they should have, that potentially causes downtime or costs into the system.”

To help mitigate these costly mistakes, fleets can present realistic 3D holograms instead of a line drawing in a PDF document, Johnston explained. This makes it easier to show technicians, “this is what you’re looking for. If you see something that looks like this, then you might want to replace it. If you see something that looks like this other one, it’s got some wear and tear on it, but that’s normal—keep that there.” Helping them make that decision and providing those examples, helps build that memory set, and ultimately technicians can make better decisions over time, he added.

Augmented and virtual reality can give students and veteran technicians the same training experience as working on an actual vehicle, according to Marlo Brooke, CEO and founder of Avatar Partners. She said her company discovered technicians have the highest memory retention through mixed reality. These days, this is a big asset as many skilled technicians are retiring, and the next generation has less experience.

“We’ve found the ability to literally take a Level 1 technician with zero to one month of experience on the job and enable that technician to train at the level of a senior technician,” Brooke explained.

Thanks to the reality-altering technology, the times to troubleshoot vehicle problems are reduced by up to 75%, she noted.

“Essentially, that’s because when the technician uses the augmented reality experience when performing the maintenance task, they essentially have X-ray vision into the vehicle or into the engine, for example,” Brooke said. “It really allows them to home in very, very quickly and find the problem.”

And while novice technicians certainly get a boost from AR, experienced technicians’ work is also sped up by 25%, she added. This gives fleets facing attrition the ability not only to train novices to become experts but also to increase their experts’ skills.

“The fleet industry needs these experts to actually become more skilled,” she said. “This tool works very effectively in that manner to improve both the younger generation as well as skilled technicians.”

Augmented collaboration

What seems to be one of the most promising aspects of AR technology is the idea of real-time remote collaboration—a technician in the bay working on a vehicle, wearing an AR headset, having a digital connection to a subject matter expert who can see what the technician is seeing, guide and troubleshoot repairs, and populate the technician’s field of vision with diagrams, charts, images, 3D models, and any information necessary to ensure an efficient and accurate repair. It may sound like a sci-fi dream, but today this is a reality already utilized by some within the commercial vehicle maintenance industry.

Photo courtesy of DTNAWith AR goggles, such as Microsoft’s mixed reality HoloLens, a skilled technician can see exactly what the technician wearing the goggles sees in real time.

“I can put the goggles on and call a more experienced technician who can see everything I see and then guide me through the repair,” said Alexandre Georgetti, director of manufacturing strategy for Eaton Vehicle Group. “You, as the more experienced one, can even draw circles and arrows on your screen, and everything you do I can see through my goggles.”

Daimler Trucks North America (DTNA) successfully piloted Microsoft’s HoloLens AR technology as a viable method of virtual collaboration with dealers and customers. DTNA is continuing to test the integration of AR into their operations, and throughout this testing, the company’s approach has been clear—creating value.

“We really obsess over how we are adding to the value proposition we have with our business partners, the dealers, and their technicians,” said Daoud Chaaya, director, field service, DTNA. “Technicians, being the hot commodity that they are, much like truck drivers these days, it’s hard to get one in the door, and then the training scale or getting them up to speed and making sure they’re hitting the ground running is also a concern we’re trying to assist with … We want to continue to be on the cutting edge of innovation. How do we add to that value proposition? We see where we’re at right now [with] HoloLens 2.0 and how much value we can add to our technicians to speed up repairs, increase uptime.”

DTNA has aligned the use of the HoloLens similarly to when a technician would contact the call center today. There is a certain “checklist” that is reviewed upfront when technicians contact the call center, with tickets filed. When a call center representative receives a call from a technician donning a HoloLens headset, the individual in the call center already has some knowledge of the technician’s situation.

“The focus is, ‘how do we minimize downtime and how do we maximize uptime?’” Chaaya said. “It’s not [achieved] through the technician teaching the call center representative what they’re seeing, so this [system of communication] allows the call center rep to come to the call prepared.”

A substantial benefit to the HoloLens system is the ability for the call center representative to bring in a subject matter expert when needed. The ability to have multiple people with “eyes” on the issue and particular expertise addressing the matter leads to clearer communication and expedited processes.

“From the moment the technician puts their headset on, at a minimum, they have a call center on the other line guiding the repair and leveraging augmented reality to be able to parlay into the discussion service bulletins, service manuals, anything necessary for that particular issue that they’re facing, and fault codes that they see that they want to show the technician documentation of on the fly,” Chaaya explained. “This also allows for subject matter experts to be a part of that call. If we’re talking about an HVAC issue and I want to rope in my HVAC engineer, they can also be virtually part of that call—this way, you get to bring in expertise, as needed, into a particular repair.”

Such a form of augmented collaboration offers opportunities in efficiency, accuracy, and success.

“I foresee a group of connected people helping each other out,” Chaaya said of the HoloLens AR technology. “Long gone are the days where I need you to send me an email that has a video … The video is clear, or the video is not clear, and we spend hours sending content back and forth, and people throwing their hands up in the air in frustration, not being able to address complex repairs … This is a special tool at the dealership that allows them to connect with DTNA in real-time and maximize uptime through quick goals … It can either end up being a cool tool that sits on somebody’s shelf, or, if leveraged correctly, really adds value, makes money, and leverages and pushes uptime.”

A new medium for a new normal

If COVID-19 has taught us anything it is how quickly industries need to be able to adapt to change. And through the pandemic, there have certainly been changes in the way fleets operate. As the end of the COVID crisis seems to be on the near horizon, industries must decide what best practices they implemented during this time they wish to continue, which they will leave behind, and which they would like to lean into further to create the best situations going forward. AR and VR technologies offer an avenue to pursue remote, digital engagement that many value as an established “new normal.”

People are excited to find alternative ways to learn, TRANSFR’s Rajakumar said. People naturally love to learn, but everyone understands the sentiment of “Zoom fatigue” and being tired of online meetings. That can be true for online learning as well, so having a new medium is an advantage.

“COVID has definitely been a catalyst because, as the saying goes, necessity is the mother of all innovation,” Rajakumar said. “Now that the vaccine rollout is underway and a lot of people are vaccinated, what we are hearing and what we are seeing is that folks are not interested in going back to the way we used to do things. The hands-on training simulations are just far more engaging. They do a better job explaining how things actually work, as opposed to just watching a bunch of videos or reading a bunch of textbooks. So, while COVID was a catalyst, I think that this medium can be here to stay, provided we continue to create exceptional experiences.”