– Is the Measurement process good enough to guide process improvement efforts and to meet customer needs?

– Is there a formal process for measuring the variable?

– How have you determined you do not have sampling problems (when, how, sample stability, sampling the sample jar, etc)?

– What is the design of the MSA experiment?

– What measurement or sampling issues were resolved?

– Were all problems communicated to all appropriate people (local and globally)?

– Is there a control plan in place which includes ownership, calibration, procedures, troubleshooting guide, SPC (Statistical Process Control), etc…)?

Lean Six Sigma Measure Phase questions:

– How Would You Assess Your Measurement System Today?

– Talk to the individuals conducting the measurements?

– Have a few measurements taken and compare them?

– Have other individuals or “experts” verify our measurements?

– Hope your customers get the same measurements?

– Don’t assume computers are always right?

– Conduct a Gage R&R study?

Measurement Variation is broken down into two components: (The two R’s of Gage R&R)

Reproducibility (Operator Variability): Different individuals get different measurements for the same thing.

Repeatability (Equipment/Gage Variability): A given individual gets different measurements for the same thing when measured multiple times.

– The tool we use to determine the magnitude of these two sources of measurement system variation is called Gage R&R

– Reproducibility is the variation in the average of the measurements made by different operators using the same measuring instrument when measuring the identical characteristic on the same part.

– Repeatability is the variation between successive measurements of the same part, same characteristic, by the same person using the same equipment. Also known as test /re-test error, used as an estimate of short-term variation

Stability = If measurements do not change or drift over time, the instrument is considered to be stable.

Bias is the difference between the observed average value of measurements and the master value. The master value is determined by precise measurement typically by calibration tools linked to an accepted, traceable reference standard. Average of measurements are different by a fixed amount.

Bias effects include:

– Operator Bias – Different operators get detectable different averages for the same value,

– Instrument Bias – Different instruments get detectable different averages for the same measurement, and

– Other Bias – Day-to-day (environment), fixtures, customer and supplier (sites).

Discrimination is the capability of detecting small changes in the characteristic being measured.

– The instrument may not be appropriate to identify process variation or quantify individual part characteristic values if the discrimination is unacceptable.

– If an instrument does not allow differentiation between common variation in the process and special cause variation, it is unsatisfactory.

Acceptable Measurement Systems have properties that all acceptable measurement systems must have:

– The measurement system must be in control (only common cause variation; i.e., in statistical control).

– Variability of the measurement system must be small in relation to the process variation.

– Variability of the measurement system must be small compared with the specification limits.

– The increments of the measurement must be small relative to the smaller of:

a) the process variability or

b) the specification limits

Rule of thumb: increments are to be no greater than 1/10th of the smaller of:

a) process variability or

b) specification limits)

The Automotive Industry Action Group (AIAG) has two recognized standards for Gage R&R:

– Short Form – Five samples measured two times by two different individuals.

– Long Form – Ten samples measured three time each by three different individuals.

** Remember that the Measurement System is acceptable if the Gage R&R variability is small compared to the process variability or specification limits.

Preparation for a Measurement System Study:

– Plan the approach.

– Select number of appraisers, number of samples, and number of repeat measures.

– Use at least 2 appraisers and 5 samples, where each appraiser measures each sample at least twice (all using same device).

– Select appraisers who normally do the measurement.

– Select samples from the process that represent its entire operating range.

– Label each sample discretely so the label is not visible to the operator.

– Check that the instrument has a discrimination that is equal to or less than 1/10 of the expected process variability or specification limits.

Setting Up the Measurement Study:

– Assure that the gage/instrument has been maintained and calibrated to traceable standards.

– Parts are selected specifically to represent the full process variation

– Parts should come from both outside the specs (high side and low side) and from within the specification range

Running the Measurement Study:

– Each sample should be measured 2-3 times by each operator (2 times is the Short Test).

– Make sure the parts are marked for ease of data collection but remain “blind”(unidentifiable) to the operators.

– Be there for the study. Watch for unplanned influences.

– Randomize the parts continuously during the study to preclude operators influencing the test.

– The first time evaluating a given measurement process, let the process run as it would normally run.

Because in many cases we are unsure of how noise can affect our measurement system, we recommend the following procedure:

– Have the first operator measure all the samples once in random order.

– Have the second operator measure all the samples once in random order.

– Continue until all operators have measured the samples once (this is Trial 1).

– Repeat steps 2 – 4 for the required number of trials.

– Use a form to collect information.

– Analyze results.

– Determine follow-up action, if any.

If Process Tolerance and Historical Sigma values are not used in your statistcal software (i.e. Minitab), a critical assumption is then made that the sample parts chosen for the study, truthfully exhibit the true process variation. In this case, the acceptability of the measurement system is based upon comparison only to the part variation seen in the study. This can be a valid assumption if care is taken in selecting the study sample parts. AIAG states that “One element of criteria whether a measurement system is acceptable to analyze a process is the percentage of the part tolerance or the operational process variation that is consumed by measurement system variation”.

Remember that the guidelines are:

A. Under 10 % – Acceptable.

B. 10 to 30 % – Marginal. May be acceptable based upon the risk of the application, cost of measurement device, cost of repair, etc.

C. Over 30 % – Not Acceptable. Every effort should be made to improve the measurement system.

Repeatability is checked by using a special Range Chart where the differences in the measurements by each operator on each part is charted. If the difference between the largest value of a measured part and the smallest value of the same part does not exceed the Upper Control Limit (UCL), then that gage and operator are considered to be Repeatable

Reproducibility is best determined analytically using the tabulation analysis in the Minitab Session. Graphically it may be seen if there are significant differences in the operator patterns generated by each operator measuring the same samples.

This tabulation from Minitab builds the % of Study Variation that each source contributes to a calculated potential Total Variation seen in the study.

– The 6.0 * SD (Standard Deviation) is how statistically 99.73% of the Total Variation is calculated and this is assumed to equal 99.73% of the true process variation unless the Historical Sigma is input into Minitab.

– The %’s are used to grade the validity of the measurement system to perform measurement analysis using %’s already taught. If the process is performing well, the % Tolerance is then important.

– The sum of the %’s may add to more than 100% due to the math.

– The Number of Distinct Categories represents the number of non-overlapping measurement groups that this measurement system can reliably distinguish in the Study Variation. We would like that number to be 5 or higher. Four is marginal. Fewer than 4 implies that the measurement system can only work with attribute data

– Most physical measurement systems use measurement devices that provide continuous data.

For continuous data Measurement System Analysis we can use control charts or Gage R&R methods.

– Attribute/ordinal measurement systems utilize accept/reject criteria or ratings (such as 1 – 5) to determine if an acceptable level of quality has been attained. Kappa techniques can be used to evaluate these Attribute and Ordinal Measurement Systems.