“When You See Data, Doubt Them” – Dr. Kaoru Ishikawa

“To Measure = To Know” is one of the fundamental truths of quality management. That does not necessarily have to mean much: at some point in time people were fundamentally convinced that the earth was flat and/or that the sun turned around our planet. We now know that this is not really true or rather: really not true.

Putting an equality sign between to measure and to know is not correct either. Both terms are not equal: they relate to each other as a mean to a purpose. A measurement can help us to evaluate a situation more correctly but the measurement in itself does not do that. The obtained data are an input for people with knowledge, experience and imagination to work with. That process can lead to new and better knowledge, with the emphasis on can. Because there are conditions related to the measurement if it is to lead to knowledge.

First of all, the measurement has to be reliable and representative. What you measure must be a good representation of the situation that you want to investigate. How you will subsequently measure, is at least as important and very often treated with some neglect. Dr. Kaoru Ishikawa warned us for this a very long time ago, with his statement: “When you see data, doubt them”. The risk that you will enthusiastically draw the wrong conclusions out of wrong data is real. During the coaching of Six Sigma improvement teams I have very often experienced how unreliable existing data can be. And even more remarkable: how pretty much every team member also doubted the data and spontaneously proposed not to use them. One wonders why they have been collected for all those years. But before you, dear reader, doubt the quality of the companies that I work with: how confident are you that the data in your databases is truly reliable?

On the other hand: don’t keep on looking for the one measurement that is absolutely correct and totally reliable and captures the entire problem in full. Such a measurement simply does not exist. In other words: you can keep on measuring over and over again with ever different (and even more accurate) methodologies but without increasing your useful knowledge. It can then easily become an excuse: “we cannot do anything because we need to do some more measurements”. In Belgium – as in many other places in the world – we have a serious problem with traffic jams. Millions of euros have been spend on measuring traffic. The traffic authorities can now tell exactly where traffic jams occur and how long they are, in time, in distance and probably even in number of cars. The average commuter also knows this, undoubtedly less exact and probably only in time but a lot cheaper and coming to the same conclusions: traffic jams get longer every year and the critical locations are known and have been the same for years. So how about doing something about them?

What is true for technical measurements is equally true for management measurements like the well-known KPI’s or Key Performance (or Process) Indicators. The objective of KPI’s is to give an overview of the status of the company with a limited set of critical (key) numbers. Not everything is critical, so think very carefully before you add another indicator. Chances are the current ones are already overkill.

It is even more difficult to define a set of targets in such a way that achieving each one of them leads to a positive evolution for the overall system. To give just one example: the objective for purchasing to buy as much as possible as cheap possible is rarely a blessing for the rest of the organization. The lack of systems thinking quickly leads to sub optimization: while each part thinks it is improving, the whole is getting worse and worse.

But what worries me the most is that many measurements today are no longer aimed at improving the process or increasing knowledge. Under the motto: “we trust no one and certainly not our own staff”, there is an explosion of registrations purely directed at control and measurement of people. Actually, the current digitization makes this ever more easy. One would hope that these technical possibilities were used for better purposes. Especially in health care we should be very careful with this extreme efficiency and control thinking. Reducing a sick person to a product that is allowed a defined (and not be exceeded) processing time is not a sign of  progress or civilization. Rather something we ought to be ashamed about.