Gleaning wheat from chaff: What does this variation mean?

Question: Can one make too much of variation in a process? The answer: It depends.

Chicken Little, reacting to an acorn falling from a tree, spread the alarm: “The sky is falling!” This kind of over-reaction is highly recognizable. What about the panicked investor whose stocks go down by a percentage point one day, who wants to sell everything and get out of the market? Or the teacher whose classroom seems cold, turning the pre-set thermostat way up (and then later, when the room seems too warm, turning it way down)? Or a sales manager who calls the team together to bemoan “disastrous trends” after a week’s drop in sales revenues?

While these may all patently represent over-reaction, they are recognizable behaviors—perhaps even in our own responses to changing situations. Panic can easily set in without understanding the meaning of this kind of variation. Known as common cause variation, it represents natural movement in data points in any process. One needs to know whether the sky is really falling, or if seed cycles of oak trees are predictable and stable.

Aside from his inability to recognize an acorn, Chicken Little was a victim of what is known as “data point mentality.” A single incident made him leap to a giant conclusion in the same way that a slight drop in one’s checkbook balance might lead to filing for bankruptcy. To verify change in a process, enough data must be collected over time to see trends and establish limits. Many statisticians suggest 25 data points, but fewer are often acceptable.

Common-cause variation

Since control limits are created from the data itself, they provide clear guidelines for determining stability in a process. Notice in this chart that although the points may go up or down, they remain within the control limits. As Donald Wheeler, Ph.D., points out, “Every data set contains noise [or routine, normal variation]…and until you know how to separate the exceptional from the routine, you will be hopelessly confused in any attempt at analysis.”1

Once control limits have been calculated and reflected on a control chart, the easiest test for system stability is checking for points that fall outside these limits. Other indicators of instability include nonrandom patterns, including these:

  • A run of eight points above or below the center line
  • Six or more points in a row going in the same direction (up or down)
  • Two out of three points beyond the 2-sigma limits
  • Fifteen points within plus and minus one sigma (points too close to center line)

Patterns such as these suggest that attention must be given to the process to determine the source of the pattern that they suggest. Not every out-of-control situation is a cause for concern. If a system is rapidly improving, for example, one or more of these patterns may occur. The key lies in analysis of the causes, and eliminating special causes or unusual situations that produce unstable processes.

Routine variation exists in every system, and does not indicate instability. When a system is determined to be unstable, variation may represent an indicator of this instability. But that’s a different kind of variation, created by a specific circumstance that represents a special cause. Stay tuned for an examination of special cause variation, which provides a signal that must be analyzed further.

1Donald J. Wheeler, Twenty Things You Need to Know (Knoxville: SPC Press, 2009).