How Do We Measure a Lean Production System? – How Do We Know if Our System is Becoming Leaner?

How Do We Measure a Lean Production System? – How Do We Know if Our System is Becoming Leaner?

(Revised 7/15/18)

In my previous post, I discussed the importance of knowing “what we should do” versus only doing “what we can do”. But now I would like to delve into the next big question. How do we verify that we are doing the things we should do? How do we know if we are making actual progress in our Lean transformation efforts? How do we know if the things we are doing are actually turning into “Lean” improvements and are not just “busy work” generating cosmetic changes?

What can we measure that will tell us if we are doing the right things (versus just doing things right)?

The first criteria for this magic metric is that it must be applicable to the entire system that will be impacted by any changes we make. I can’t stress this enough. If we are implementing changes to a single or a few processes in our system, we must be able to measure the impact of these changes on the entire system. Only measuring the impact of changes to the subsystems we are trying to improve can and, in too many cases, will result in unexpected consequences somewhere else.

I learned this rule early in my career. I can still remember the many occasions where Senior Management would ask all of the departments under their control to come up with cost reduction targets for each department. Each department would then dutifully identify projects in their department that would result in significant cost savings. Everyone would get excited and work hard to implement these world saving efforts. But on too many occasions, when the end of the year financials were published, each department’s project savings were there for everyone to see – but the Plant-wide results were, all too often, less stellar than anticipated. Sometimes much less. It wasn’t until much later that I realized that what we had done, over and over, was just move costs around. Improvements in one area often caused deterioration in another area. The various departments were not independent of one another, but interdependent with each other.

So, what system wide metric is best for capturing our Lean progress? I’m sure those of you who have read other parts of my blog have already come to a conclusion as to my choice for this metric. Lead Time, right? After all, you can measure lead time for any process, but it’s also easy enough to measure lead time for the entire system. And it is the cornerstone of my guiding principle:

Thus the underlying principle behind TPS/Lean is the systemic creation of the shortest possible lead time for the continuous flow of materials and information in order to generate the highest quality and lowest cost.

Lead time?

Nope!

What?

While lead time should always be measured and is a cornerstone for identifying/quantifying improvements to the system, it has a couple of drawbacks to becoming our universal magic metric.

  1. Lead time is too relative.
  2. Lead time does not have a financial dimension.

Lead time is relative since there is no stated universal target for lead time. What lead time is too long? What lead time is too short? Can lead time be too short? The best we can say is that shorter is usually better than longer. When Boeing implemented a moving flow line around the turn of this century, lead times went from 22 days to 11 days. That’s great! But is an 11 day lead time good? I don’t know. Does the customer think it’s good? I don’t know. But I’m sure the customer would be happy if it was shorter. If I go to my bank and it takes 15 minutes to cash a check, do I think the lead time is good? No. What should the lead time be? I don’t know. But I want it to be shorter. That’s a relative judgement.

So, the best I can do with a lead time metric is to set a goal of a certain percentage reduction from the existing lead time. My goal is a relative measure. Can I calculate a financial impact for my reduced lead time? Not really. Not as a stand-alone metric. I know I should get a positive benefit with respect to cost, quality and delivery but, as a stand-alone metric, I can only guess at a financial quantification of that result. Do you think Upper Management or the accountants will pay any attention? Not likely.

So should I forget my deep understanding of the importance of lead time and move on to something else?

Hell no!!!

Stick with me. We just need to find a way to merge lead time into a broader system-wide metric that traditionalist thinkers can understand and accept. We have a paradigm we need to break. And that’s not easy to do.

One step at a time.

Value Stream Mapping is a tool we can use to better quantify lead time in absolute terms. If we map out all the process steps contained within a value stream system, and do it correctly, we will get a very good estimate of total lead time. But we also get something else. We can actually see the individual components that make up that total lead time. And we should also be able to measure or calculate the time consumed by each of these individual system components. The components are usually shown as individual process steps or groupings and any inventory waiting between steps, with shipment to the customer being the last step (actually receiving payment from the customer is the last step, but that never shows up on the maps). (But remember, inventory consumes time – always and forever more).

OK, now what?

Well there is a way to quantify these various time buckets in a manner that sheds a more revealing light on whether or not our total lead time is “good” or “bad”. This analysis technique is called Process Cycle Efficiency (PCE) by Value Stream mappers. Most of you are familiar with this technique even if you don’t recognize the name. It simply involves looking at each time bucket and quantifying how much of that time can be considered “value-added time” (VAT) with the rest being “non-value-added time” (NVAT). Add these various time buckets across the entire value stream and you will get a total lead time (TLT):

∑VAT + ∑NVAT = TLT

And the Process Cycle Efficiency (PCE) can be calculated as:

PCE = ∑VAT / TLT

(A video tutorial that demonstrates the entire calculation process can be seen at this Gemba Academy blogsite)

So now we have a ratio that quantifies the value-added time content of the total lead time of our Value Stream System. Now we are getting somewhere! We can quantitatively compare the “value-add efficiency” of our current value stream with other value streams. We can quantitatively compare our future state value stream with the current state. We can quantitatively compare Boeing’s lead time with my bank’s lead time. We have moved one step closer to an absolute measure of lead time from a purely relative one.

What is a good value for PCE? What is a bad value for PCE? It depends on the type of processes we are dealing with. Simple processes tend to have higher values and complex processes tend to have lower values. Manufacturing, for instance, will usually fall into the more complex category. A non-Lean manufacturing operation will usually have PCE values of 5% or less. A world class operation will approach PCE values of 25%. My bank, since cashing checks is a relatively simple system, should have comparable PCE values twice that for manufacturing.

OK, now we are making progress.

But…….

We still lack a financial dimension to our metric. Will Upper Management and the accountants pay attention? Maybe – but probably not for very long. They will be thinking: “Why do they keep bringing up this Time thing? They need to use calculators, not clocks. What will it mean to our bottom line? That’s what we need to know”.

But there is a way to turn Time into Money. Dr. John Little of MIT provided us with a tool we can use way back in 1961. A most basic form of Little’s Law can be expressed as:

Throughput (units/unit time) x Lead-Time = Work in Process (units)

All terms are averages. As I discussed in an earlier post, we can also express Little’s Law in terms of a financial entity:

Production Cost ($/unit time) x Total Production Lead Time = Cumulative Cash Invested ($)

Production cost can be obtained directly from the P&L. Just note the timeframe for the P&L statement (monthly, quarterly, etc.) to determine the cash input rate, then convert to a daily rate. Production Lead Time is the average total elapsed time from the start of production until the product is sold to a customer. Lead Time is usually in days. Just be sure the P&L and Lead Time have the same unit time dimension.

Cumulative Cash Invested is the sunk cost tied up in our production system. If we use my river system analogy, Production Cost is the river input. Lead Time is the flow time to traverse the river from beginning to end. Cumulative Cash Invested is the total river volume.

Let’s stop for a moment and reflect on what we have done here. We have linked up the P&L (Production Cost) with the Balance Sheet (Cash Invested). And we have done that by using “Time”. We have turned the clock into a calculator. And by doing so, we directed the focus from the P&L onto the much-neglected balance sheet. More on that in a minute.

There is one important caveat when using this cash flow version of Little’s Law. As written, all of the production cost is loaded at the beginning of the Lead Time bucket. All of that cash input flows through the entire length of the river. Most manufacturing operations are not constructed that way. There will usually be distinct cost inputs at different points in your manufacturing flow (departments, cost centers, etc.). A separate cash calculation can be done for each of these input points. The various cash calculations can then be added together to get total Cumulative Cash invested. Just make sure your Lead Time represents the total lead time from cash injection to the final system output (e.g., when shipped to the customer). If you are working off a Value Stream Map, this should be easy to do. And most P&L’s have production cost information for departments, cost centers, etc. But just remember, once your lead times become very short (the goal), the need to perform separate calculations becomes much less important. And if your cost inputs are somewhat frontloaded, just do the one calculation to get a very good estimate.

What now?

Well, you’ve probably figured that out already. Let’s go back to the “PCE equation” and convert those “times” into “cash”. But instead of Process Cycle Efficiency, a pretty boring term, let’s call it Cash Investment Efficiency (CIE). That’s still pretty boring, but probably not to the top guys and those accountants.

CIE = (∑VAT x Production Cost/unit time) / (TLT x Production Cost/unit time) = % of Cash Investment that Adds Value

Will Upper Management and the accountants pay attention to this? If they aren’t Lean, they probably won’t want to. But they really have to. When more than 95% of your cash investment doesn’t add any value to the customer – you must think about it. And that Balance Sheet may not seem so ignorable anymore. Will they know what to do about it? I don’t know….

But we know!

OK, we’ve got a metric that borders on the absolute – and we definitely have a financial dimension embedded into it – have we found our magic metric?

Not quite.

We still do not have a metric that encompasses the entire system. If we visualize Ohno’s River System analogy, we are still falling short. The P&L covers the river inputs and output, the Balance Sheet, Lead Time and CIE do a very good job describing the river body itself, but we don’t yet have a singular metric that covers the whole river system (although the CIE comes close).

Oh yes, we do!

ROI – Return on Investment.

Source

This is the DuPont Model of ROI. I covered the history and structure of this model in quite some detail in previous posts (here and here). I also contrasted the difference between the use of ROI versus the P&L in a Lean environment here. I urge everyone to read these posts if you have not already done so. The DuPont Model is not the ROI your accountant counterparts use to calculate whether or not to purchase a piece of equipment or a building expansion. This model was used to manage the entire business enterprise of DuPont and General Motors for almost a half century – along with most other corporations during that time period.

The best analysis of the use of ROI in a world class manufacturing environment that I have seen is in the 2013 book Demand Driven Performance by Debra and Chad Smith. To give you a flavor of why I think ROI is the magic metric for Lean Production, I will quote from an early part of the book:

“What happens when revenue is maximized and protected, inventory is minimized, and additional and/or unnecessary ancillary expenses are eliminated? Return on Investment (ROI) is high. …. The best, sustainable way to achieve that goal [high ROI] is to promote and protect flow. This is the very definition of an efficient manufacturing system. …. Once we realize the importance of flow, a few key principles emerge: Time is the ultimate constraint. Time is the most precious resource employed in the manufacturing process. …What we must always keep in mind is that the important time is the time it takes to move through the system.”

Sound familiar?

Here is a basic version of the above DuPont ROI model:

ROI = Earnings / Total Investment = (Sales – Cost of Sales) / (Working Capital + Permanent Investment)

Note that the numerator (Earnings = Sales – Cost of Sales) is obtained directly from the P&L. And this represents the real value added as defined by the customer. The various customers set the sales price by buying or not buying the offered product/service. It is up to the producer to maximize the value-added contribution by minimizing their cost. The denominator, obtained directly from the Balance Sheet, represents the amount of cash contribution the producer has invested at any given time to generate the value-added contribution represented by the Earnings.

ROI is a value-added ratio that encompasses the entire business enterprise. ROI is a total system metric. The river inputs and outputs, as well as the river body are all included in this one metric.

Is this metric a relative or absolute measurement? While all metrics are relative when you are measuring your progress (or lack thereof), ROI has been in use for almost a hundred years and has a long-documented history. You can find extensive detailed tabulations of ROI values online that list ROI by industry/business type and many other categories. And you can always lookup the ROI performance of your competitors. It’s usually included in their quarterly and/or annual reports. You can’t ask for more than that.

In order to better mesh with the other various metrics I have discussed in this post, I would like to make a few minor modifications to this equation. For our purposes “Cost of Sales” can be replaced by “Production Cost”. Since we are interested in the things we can control on a daily/monthly basis, I will remove the category “Permanent Investment” (buildings, equipment, etc.). “Working Capital” (inventory + accounts receivable + Cash), while a great term, is usually defined differently in today’s financial accounting world (versus management accounting – what we want), so I am going to rename this term as, no surprise, “cumulative cash invested”.

Here is the revised equation:

ROI = (Sales – Production Cost) / (Cumulative Cash Invested)

And if you think I’m going to forget the cash version of Little’s Law you would be mistaken. Here is the final version of my ROI equation:

ROI = (Sales – Production Cost) / (Production Cost x Total Lead time)

Voila!

[Note: Sales and Production Cost are a function of unit time. They are usually expressed per month, quarter or year on the P&L. But for this equation, the unit time measure must match the unit time for Lead Time. “Per day” usually works out well. But if you can get to a “per hour” metric – go for it! All costs are averages so this is a simple, unit time, conversion calculation.]

And when we say, “Production Cost”, we are talking about everything that is necessary to produce a product that, hopefully, the customer buys. Materials and labor, check. Overhead, sure. But what’s in overhead? Everything! Maintenance, Engineering, Purchasing, Production Planning, Quality Control, Accounting and Finance, Utilities, Rent, production Big Shot’s salaries, Janitorial Services, Insurance, Cafeteria, Uniforms and the all-important “Kitchen Sink”.

The only thing I would possibly leave out would be your standard S, G & A expenses (Selling, General and Admin). But if you want to throw those in also, that’s fine with me. It’s all part of “Cash Flow”. And it’s the “flow” of the river body that we are interested in.

And one final thing about “Total Lead Time”. It would probably be best if the end point is “getting paid” rather than “shipment to the customer”. “Accounts Receivable”, like inventory, is still “cash invested” until that check arrives in the mail. You want that conversion to “cash” to be as short as possible. Then you have something tangible to reinvest in “Production Cost” or pay off debt or buy some equipment or acquire a new business or etc., etc. Otherwise you just have another asset bucket that is just sitting there consuming time and money.

But back to our ROI equation.

If everything else stays the same, a reduction in lead time will inexorably give us an increased ROI. There is no way around that fact. I am a very firm believer in mathematics. A reduction in lead time gives us an immediate payback in our return on investment.

If your Lead Time is reduced, total cash invested (denominator) is reduced by an equivalent amount. You are paid back sooner. Again, you now have extra cash available to pay off loans, buy equipment, make acquisitions — whatever you want to use that free cash for.

Cut your lead time in half, a very achievable goal based on my experience, hold Sales and Production Costs constant, and your ROI will double!

Cut your lead time by 90%, a target I have achieved on many occasions, hold Sales and Production Costs constant, and your ROI will increase by a factor of 10!

But if you have read my last two posts (located here and here), you will also know that significant reductions in Lead Time almost always result in significant reductions in Production Cost (as well as significant improvements in quality and delivery). And ROI will show you this improvement in real time. You can then trace the numbers back through the ROI calculation sequence and see where those improvements are occurring.

As you reduce Lead Time, you are reducing time consuming waste in your processes (almost all waste consumes time). And that waste also consumes money in one form or another. Inventory is usually your biggest time related waste, and by reducing inventory you need to transport fewer items, inventory management is simplified, defects are discovered closer to their source, floor space is freed up for other uses, visibility is improved so labor can find what they need when they need it, etc., etc. Once “flow” begins to improve, there is less need for meetings to decide what to produce, problems are more visible and can be remedied on-the-spot, production planning is simplified – life becomes easier for everyone.

And when all these factors improve, Sales should also improve. And you will also see this improvement in real time directly from your ROI equation.

Thus, ROI is an ideal measure of your system’s overall financial health. Everything you need to know is right there in this one single metric!

But it all begins with Lead Time reduction.

I think Upper Management and those accountants will definitely notice this one.

Concentrate on Lead Time (i.e., flow) at the gemba where everyone can see it. And use CIE to wake people up. But measure ROI to confirm that your Lean Production System is in fact improving.

That’s a big WIN-WIN for everybody!!

One last thought. As you proceed along your Lean/TPS journey and you read all the myriad of blog posts, articles, books, videos, etc. based on today’s versions of Lean, just keep this quote from Mark Twain in the back of your mind:

“It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so.”


8 thoughts on “How Do We Measure a Lean Production System? – How Do We Know if Our System is Becoming Leaner?

  1. The MCT (manufacturing critical-path time a.k.a lead time) does have a specific goal… get it under the time in the P.O. Then you are only making stuff you have sold after and are not wasting resources on extra stuff you may not sell. Also you can use it as a competitive advantage against the competition and get 6-8 on overseas supplier competition.

    The MCT has no direct financial component but it impacts a lot of financial measures e.g. WIP, shrinkage, turn-over etc. and a lot of overhead numbers as well (forecasting, warehouse, firefighting). Lastly it is a measure of management headaches — cost/benefits ????

    1. Greg,

      You obviously have firsthand experience and knowledge of the advantages of short lead times. And, I agree, MCT is a more accurate definition of the concept. And if your customer is willing to agree to a P.O. time line, then a MCT below the P.O. time is ideal. But if your customer is used to “same day shipping” because you have it on the shelf, then the customer may not be happy.

      And it also depends on the process limitations you are stuck with. I had one product, a large bulky medical device, that was sterilized with ethylene oxide (ETO). The problem was that even though we had reduced the primary process lead time from weeks to hours, the product absorbed so much ETO that it had to be aerated for 5 days in a special ventilated room to get the residual ETO levels down to acceptable levels. The customers did not want to wait 5 days, so we had to keep some finished goods inventory on the shelf, but significantly less inventory than before the lead time improvements. Cash invested still came way down and ROI went way up.

      And I love your analogy of lead time being a measure of management headaches. As you point out, WIP, shrinkage, turn-over, forecasting, warehouse, firefighting, etc. does seem to facilitate the need for aspirin.

  2. Your note above is excellent. It should wake up accountants to the fact that Lead times do affect their numbers from 2 points: the length of time $ are invested and the total $ invested in the WIP inside the system. I am still of the opinion that looking at how customers respond to a shorter lead time, higher fill % etc. is a big but unknown factor that can pay off. The shorter lead times can totally change how the production strategy is done e.g. JIT and then even less $ is invested in production WIP.

    Here is a start on the path to shorter lead times which can then evolve into a better strategy once you see how customers respond.

    1. Greg,

      Thanks. I know you have a deep understanding of the importance of Lead Times from the work you have done. And your expert knowledge in Little’s Law and general queuing theory have paid dividends in your ability to demonstrate, through modeling, the impact of Lead times on overall system performance. Now if we can just pass that knowledge on to the finance guys and Senior Management in general, then today’s Lean efforts may develop a better success rate than they have shown recently. That’s what I have tried to do in my last three posts. Let’s keep our fingers crossed.

  3. Bill,
    I apologize in advance for the length of my response.

    I’d like to start out by presenting counter-arguments to the two short-falls you say lead-time has as a business management metric, as follows;

    First, it is not a negative attribute for a metric to be “relative” if the relativity is to the customer. In fact, (as an aside) in my opinion a short-fall of most corporate financial exhibits is that they have very little direct relationship to the specific industry — specifically customer demand characteristics — in which a company is operating. Anyway, I will assert that if having a two week lead-time gives you a competitive advantage in your marketplace, you are Lean. Yet if a competitor matches your two-week lead-time and you no longer have a competitive advantage, you have lost lean-ness. To use your example, if the lead-time for you to cash a check is (1) minute and that is measurably quicker than you can get from any other bank, your bank’s check cashing process Lean. My experience is that there must be a balance between “absolute” and “relative” metrics to maximize performance in specific markets.

    Second, it is not correct that lead-time cannot be directly related to financial metrics. The most obvious example of this is Inventory Turns (which is a metric on the radar screen of every OEM I’ve ever worked for or consulted with). Lead-time, of course, is the inverse of Inventory Turns and can easily be related to carrying charges (and other costs) associated with specific financial accounts. Going back to the IE case study you cite, reducing supplier lead-times had a dramatic impact on the required amount of Finished Goods inventory my employer needed to maintain Customer Fill Rates and it was easy to connect the “cause” and “effect” of the significant financial wind fall this delivered.

    Later in your article you make a point that I strongly agree with. Having isolated pockets of Lean practice does not make you a Lean company. In fact, companies can’t really be Lean. Only processes can be. The overall process of delivering product to a customer is comprised on multitudes of underlying processes, BUT (and this is an important BUT), if an underlying process is not on the critical path of the what is required to produce and deliver that product to the customer, it has no effect of the overall process Lean-ness. Hence, again, there is a discrepancy between current Lean practice — which measures Lean-ness at the micro level — and what is needed in business, i.e. to measure Lean-ness at the macro (Customer Fill Rate) level.

    Finally, even with my last statement above, critical-path lead-time is a metric that is additive, so it CAN be used to 1.) Quantify the Lean-ness of underlying processes; 2.) Quantify the Lean-ness associated with groups of underlying processes, or; 3.) Quantify the overall Lean-ness associated with producing and delivering a product to the customer. Because of this, it is relatable to financial exhibits of individual processing centers all the way up to higher level financial exhibits such as Inventory Turns. Processes not on the critical-path do not impact overall processing Lean-ness so again, this implies that there needs to be a “relative” aspect of a metric to quantify Lean-ness.

    I liked your article and agree with many points you make, but your arguments don’t effectively counter the experiences I’ve had relative to use of lead-time as a metric of Lean-ness. I suspect my commentary probably doesn’t move the dial much on your thinking either, but it’s fun to discuss, isn’t it?

    Paul

    1. I’ll start with both the beginning and the end of your comment. No need to apologize for your comment length, all are very valid discussion points. And, yes, it is fun to discuss Lead Time because it is not discussed enough in today’s Lean world.

      I think your example that having a two-week lead time, a competitive advantage over your competition, makes you Lean illustrates the problem of a “relative” metric. If your processes have the ultimate capability of a two-day lead time, then you are not Lean. But you are Lean-er than your competition – for now.

      As for the importance of having a financial aspect to my ultimate system metric, I am focused on the quality of the decisions made at the top of the hierarchy. Contrary to your experience, I have found few top-level managers who focus on inventory turns. Most of my management career was spent in businesses related to the medical device and pharmaceutical industries. Typical manufacturing margins were >70%. They never looked at the balance sheet. Inventory was just something they needed to insure sales dollars. They lived by the P&L. And that can lead to bad decisions in a Lean world.

      So, while Lead Time was always a major focus for me in the trenches where processes ruled, I needed a financial metric that would get the attention of the P&L guys but also foster an appreciation of the importance of flow. I needed the top guys to pay attention, not only to the inputs and outputs of the River System (P&L), but also to the actual River Body where flow takes place. That’s where ROI comes in. In a previous post, I covered several examples where the ROI metric leads to better Lean decisions than the P&L approach.

      So, in this post, I used Little’s Law to insert Lead Time into the ROI equation and get the best of both worlds. The P&L is there for the world to see and so is Lead Time. And as Lead Time is reduced, you can track the improvement in the P&L, all in the same metric. Mission accomplished!

  4. Well, I also understand your position but until there is a simple and easy way to answer the question(s);
    -How far along are we on our Lean journey?
    -When will we be Lean?
    In my opinion, anyway, Lean will never become a industry standard. I like what you’ve done with Little’s Law but it doesn’t answer those two questions.

    1. Paul, both of your questions have an implied assumption that our Lean journey has an end. It does not. Toyota has been at it for about 70 years – and they are still journeying. And the competition is still trying to catch up. The key question, and this is becoming more important as Lean continues to morph away from TPS, is “are we still on the journey?”. The P&L will not tell you. The number of Kaizen events will not tell you. The number of seminars everyone attends will not tell you. The number of teams you have formed will not tell you. But a close monitoring of Lead Time and/or ROI will tell you. But today, very few are actually doing that. (All the better for the consultants and academics to sell their wares).

Leave a Reply

Your email address will not be published. Required fields are marked *