Photo courtesy of istockphoto.com

How to Measure Fleet Effectiveness

In today’s data-driven world, it’s a given that fleets are awash in the kind of information that will allow them to measure their effectiveness with little difficulty.

The reality is that data is simply bits of information that have no meaning until they are collected, analyzed, and interpreted. While fleets can do much of this work themselves, there are a number of companies available to them that specialize in helping to interpret and measure the plethora of data.

Fleet Financials asked several of these services about the best ways to audit data to measure fleet effectiveness.

Don’t Call it an Audit

While the process of reviewing fleet data to discover the effectiveness of a fleet operation could rightly be called an audit, this label has some negative connotations that can impede the process.

“When we perform a comprehensive review of a fleet operation we typically refer to it as an ‘operational assessment of program effectiveness’,” explained Tony Yankovich, senior manager for Mercury Associates, Inc. “The term ‘audit’ can have a negative connotation to some people. An operational review is more of an assessment of current operations with the ultimate goal to identify opportunities for improvement. Benchmarking performance is just part of what we do in a typical operational assessment of a fleet operation.”

Sal Bibona, president of Chatham Consulting, uses the term “audit” to describe the process his company uses in evaluating a fleet operation.

“A management audit is a methodical examination of the agency’s efforts to plan, organize, coordinate, direct, and control the organization’s fleet activities,” he said.

In the fleet context, Bibona said that a management audit will have several goals and evaluate several fleet-specific areas, including but not limited to:

• Adequacy/Appropriateness of departmental goals and mission statement.
• Organizational structure and reporting relationships.
• Staffing levels.
• Management information systems.
• Budgeting processes.
• Communication systems.
• Vehicle assignment practices.
• Employee use of company vehicles.
• Vehicles meeting user requirements.
• Vehicle standardization.
• Vehicle age and condition.
• Preventive maintenance programs.
• Spare parts management practices.
• In-house versus outside contractor maintenance.
• Fuel management practices.
• Efficiency of unit rebuilding programs.
• Fleet maintenance costs.
• Average maintenance hours per vehicle class.
• Availability/Existence of management control mechanisms.
• Management summary and exception reporting.
• Written policy and procedures for fleet administration, operation, and maintenance.
• Procedures to monitor vehicle utilization and determine fleet size.
• Monitoring labor productivity and utilization.
• Economic criteria for repair vs. replace vs. rebuild decisions.
• Vehicle charge back systems.
• Motor pool operations.

No matter the terminology, a comprehensive overview of fleet effectiveness can be handled using a number of criteria to get to the same result, as Chris Shaffer, a partner with Utilimarc, noted.

“One of the things we do is an annual benchmark analysis and what that’s doing is capturing all of the financial data and all of the operational data associated with these fleets,” he said. “It involves looking at three buckets — the ownership bucket, the operating bucket, and what we call the support bucket. We look at it from a fleet perspective — fleet’s kind of simple in some ways, you have to buy a vehicle, you have to maintain it, you have to fuel it, and you have to have a support structure around it, and that’s what those buckets represent, so the ownership bucket is either depreciation or lease cost or licensing expense. The operating bucket is parts, labor, and fuel. The support bucket is kind of the supporting folks and the supporting costs around maintaining a fleet.”

The next step is to refine the items in these “buckets” to develop a consistent, logical methodology to measure the data.

“We take those line items in the various buckets and kind of build a methodology, a standardized methodology, that looks at those line items the same way and kind of makes sure that we’re including the same things in the line items and then we provide a trending analysis for our clients how are they performing for the past three years or 10 years and how do they compare to their peers and what’s changing and help discover things about what’s changing and why is it changing,” Shaffer said.

Consistency is the key element in establishing the methodology, according to Shaffer, in order to build a clear picture of the fleet’s operation.

Measuring at the Granular Level

Fleets are awash in data, but capturing all of it can involve going to a number of spigots. Many fleets have fleet information systems that include financial or other operational data, maintenance data, and telematics.

“With the use of fleet management information systems to capture, record, and report on fleet information (i.e., maintenance and repair costs, utilization, fuel, etc.) and other technologies (e.g., telematics) more and more information is available,” said Yankovich of Mercury Associates. “However, in this emerging era of Big Data, many organizations are simply collecting that data and do not necessarily have the ability (expertise or staff resources) to analyze the data to determine how it can help an organization become more efficient. However, telematics solutions and other on-board solutions can help by providing data and information often without human interaction therefore it can reduce the likelihood of errors such as data input errors when entering odometer readings, etc. Solutions can improve routing, maintenance diagnostics, utilization data capture, etc.”

Shaffer of Utilimarc said he collects data at a granular level. “We go right to the source and we pull it at a vehicle-specific level, so we’re pulling the data at the most granular level and then roll it up to a class level, and a fleet level, and that way we identify any anomalies, so if parts costs jump 37% or fuel spend goes down 42% in a year, because we’re capturing at that vehicle granular level, we can drill into the data and explain why,” he said. “One of the important things about an audit is when you identify something you have to be able to go to that next level and explain why.”

While consulting companies can do a deep dive on a fleet’s operation, it is a function that can be done by a fleet manager and his or her team as well.

“I absolutely think that a fleet manager can do that,” said Shaffer. “The only requirements are No. 1 the quality of the data they’re capturing, and No. 2 developing the line items they want to track. And just being consistent, having the same consistency year-in and year-out and you’re measuring those same critical line items with good quality data will give you a very reliable internal audit result.”

Benchmarking the Fleet

Benchmarking is an important part of the auditing process, but fleet managers need to take a number of elements into account when doing it. Crucial to the process — and really any fleet measurement process — is to make sure it isn’t an attempt to make it one-size-fits-all, but still remain in the context of fleet best practices.

“Every agency has its uniqueness and not every benchmark will be correct for that agency,” said Bibona of Chatham Consulting. “Initially the agency should benchmark areas with the highest costs and with the greatest shortfalls in performance that greatly impact customers.”

But there are appropriate starting points for the benchmarking process to put it into a larger fleet context.

“For benchmarking purposes, I like to measure performance in areas that will have the most impact for the fleet organization and the customers that they serve,” said Yankovich of Mercury Associates. “Typical measures/benchmarks include fleet availability — the industry target is 95% of fleet assets are available at any given time — preventive maintenance compliance rate, average repair turn-around time, total number of ‘comebacks,’ total number of direct hours technicians charge to work orders in a year, average age of the fleet by major vehicle/equipment classification, etc.”

Bibona noted that benchmarking has to include several factors in order to be successful.

“It is important to be reviewing more than one parameter when doing a benchmarking analysis,” he said. “For example, if an agency has a higher percentage of its staff in administration, this may be viewed negatively. However, if the value added by the administrative staff results in significantly lower costs, this is not a negative. When all or most of the indicators are in the wrong direction, then there may be a problem.”

There are a number of steps that fleets should also take in making sure benchmarking is effective and successful, according to Bibona, and include:

Involve all stakeholders and have executive support.

• Use a multidisciplinary project team.
• Have clear and focused objectives.
• Use a reasonable time frame.
• Commit to change and improvement.

Setting the Frequency

One question a fleet has to answer for itself is how often it conducts an audit or benchmarking study. As with the questions and priorities associated with the audit itself, there is no “right” answer to this.

“What we’re doing for most of our clients — we’ve turned on the data pipe, so most of these are coming to us on a regular basis,” said Shaffer of Utilimarc. “Generally we do one big audit annually, and then we create dashboards, KPIs, and metrics that are available monthly and that sort of thing. We still have clients for whom it’s a one-time deal. We go in and pull all year-end data and produce an annual data report online. Most of it is getting to be real time. We do lifecycle analysis and staffing analysis. That’s much different than an audit. So it really benefits the client if the data’s real time.”

Creating a Picture

Data points are simply data points. On their own, they may be interesting, but they won’t tell much about the organization that is being audited/benchmarked.

“Numbers do not give you the answers — they just point you towards them,” observed Bibona of Chatham Consulting.

One of the important functions of the audit or benchmarking analysis is that it creates a data-driven picture of the fleet and helps to bring context to where it stands among similar fleets.

 “I think one of the most important things that we do is bring context to a fleet’s data and its numbers,” said Shaffer of Utilimarc. “You can do an internal audit and have a bunch of numbers, and metrics, and costs and numbers, but if you don’t understand the context and how they relate to other people who are doing the same thing it gets really difficult. We bring an industry-accepted consistency of how we’re going to look at the data. We have an industry-approved methodology that says we’re going to do this apples-to-apples comparison. And the second thing — and this is critical, is bringing context to the numbers. We may say that your parts cost is 14% and it was 13% the year before that and 12% the year before that. Well, okay, so? We’d say that compared to other fleets doing the exact same thing it’s 27% and their costs have gone up from 27% to 42%. That gives the fleet we’re analyzing some context and shows its increase is much less than its peers, and that piece I think is so critical in getting that data to fleets.”

The context of looking at other fleets isn’t the only angle that the fleet should be evaluating. There’s also the context of the fleet itself within the organization it serves.

“Operational context is very important and should always be considered,” said Yankovich of Mercury Associates. “Some fleet assets are used in direct support of the mission of an organization (e.g., delivery trucks, refuse trucks, fire trucks, police cars, etc.) while others are used in supporting roles (e.g., transporting a manager to a meeting).”

While this contextual picture is important, ultimately the audit and benchmarking data needs to be used to quantify and answer two questions: How effective is the fleet and how efficient is it?

“Effectiveness relates to how well a set of results is achieved, while efficiency relates to how many resources were needed to achieve the results,” said Bibona of Chatham Consulting. “Thus, by measuring and comparing the ratio of output achieved (effectiveness) over input resources consumed (efficiency), within a specified period of time, we have a means to quantify performance. For most organizations, resources consist of capital, labor, material, and energy or some combination thereof.”

Bibona cautioned about how fleets should use these performance measurements.

“It should be noted that performance measurement should be used as a means to an end, improved performance, and not an end unto itself,” he said. “Moreover, best practice fleet operations recognize that performance improvement is a continuing process whereby management is involved in an ongoing effort to utilize resources more effectively to achieve quality results.”

Again, each fleet is unique in what needs to be benchmarked.

“While there are basic benchmarks for many areas of performance, each organization has unique challenges that must be considered and the benchmark for specific organizations may vary,” said Yankovich of Mercury Associates. “For example, an organization that must meet specific ‘roll-out’ requirements — meaning that they need a specific number of vehicles to meet mission critical operations each day — and may have a higher fleet availability target than a fleet of general purpose vehicles that are used primarily to perform ordinary activities. However, the number of spare vehicles that the organization owns can impact the availability requirement.”

The cause and effect relationships of decisions are another way benchmarking helps to deliver the context fleets need to improve their operations.

“There’s always a cause and effect, and that’s one of the things we try to provide — every decision has consequences — some are great and some are not so great. It’s really helping the fleet organizations to understand the consequences of their decisions,” said Shaffer.

Measuring the Advantages & Pitfalls

There are a number of advantages for fleets and some pitfalls to avoid when conducting an audit or benchmarking study.

“The biggest advantage for fleets in doing regular audits is creating that year-over-year trending that helps fleet managers argue their case to senior management,” said Shaffer of Utilimarc. “In the fleet space and the fleet world, doing an annual audit of your fleet and trending that year-in and year-out absolutely arms fleet managers with the ammunition to fight these battles. That’s the problem. Fleet unfortunately kind of gets smooshed in the middle, so you have the executive at the top saying ‘I don’t want to give you any money,’ and you have the user who is demanding ‘I need better service and better trucks. I want more, I want more, I want more.’ So fleet is smack dab in the middle of I don’t want to give you any and I want more. The only way to effectively prove that you’re doing a great job, and effectively argue your position to get what you need is with that financial audit information. If you don’t have that information, you don’t have a way to argue your point effectively. And that data is critical.”

Ironically, one of the biggest pitfalls fleets often encounter is not with the amount of data, but with not having the personnel to interpret it.

“What’s happened over the past 10 years is that the analytics support staff that fleet may have had is now gone. So there are no bodies left and no talent left to do that hard core analysis that’s needed to uncover the magic of that data and the potential of that data,” said Shaffer.

 

Source: Fleet Financials

Join to The Fleet Management Group

In this group, you can know and share knowledge, experiences and meet people interested in Fleet Management over the world. Each week we publish posts, conferences, news, scientific papers, and technology related to Fleet Management.

The Fleet Management group

I´m a Fleet Management expert, and the manager of Advanced Fleet Management Consulting, that provides Fleet Management Consultancy Services.