Verification of form tolerances part I: Basic issues, flatness, and straightness

Kirsten Carr, Placid Ferreira

Research output: Contribution to journalArticlepeer-review


The ANSI Y14.5 National Standard on Dimensioning and Tolerancing definition for form tolerances requires the form error of a surface to be less than some set limit. However, most inspectors are interested in the minimum form error, known as the minimum zone solution. To compute the minimum zone flatness, an algorithm must determine the minimum distance between two parallel planes so that all datapoints are between the two planes. Therefore, the minimum zone solution minimizes the maximum error between the datapoints and a reference plane. Current coordinate measuring machine verification algorithms are based on the least-squares solution, which minimizes the sum of the squared errors, resulting in a possible overestimation of the form tolerance. Therefore, while coordinate measuring machine algorithms successfully reject bad parts, they may also reject some good parts. The verification algorithms developed in this set of papers compute the minimum zone solution of a set of datapoints sampled from a part. Computing the minimum zone solution is inherently a nonlinear optimization problem. The proposed algorithms solve a sequence of linear programs that converge to the solution of the nonlinear problem. The linear programs result from a novel combination of coordinate and scaling transformations and do not change the original optimization problem. Therefore, given adequate initial conditions, the sequence of linear programs will converge to the minimum zone solution. Implementation and test results demonstrate the correctness of these formulations. The implementation of these verification algorithms in a production environment can reduce the possibility of rejecting good parts, thereby reducing costs.

Original languageEnglish (US)
Pages (from-to)131-143
Number of pages13
JournalPrecision Engineering
Issue number2
StatePublished - Apr 1995


  • coordinate measuring machine
  • flatness
  • inspection
  • minimum zone method
  • straightness

ASJC Scopus subject areas

  • Engineering(all)


Dive into the research topics of 'Verification of form tolerances part I: Basic issues, flatness, and straightness'. Together they form a unique fingerprint.

Cite this