Precision CNC Machining: What "Tight Tolerance" Actually Means

CNC Machining · Tight Tolerance Precision CNC Machining: What "Tight Tolerance" Actually Means "Precision" is the most overused word in machining — and the least understood. This guide defines what precision machining actually is, how tight tolerances are achieved and verified, and how to specify only the precision your part genuinely needs so you stop paying for tolerance you cannot use. Request a quoteWhat precision means What precision machining actually means Precision machining is not a machine type; it is a capability level. It means holding dimensions, geometry and surface finish to tolerances tighter than general machining can reliably deliver — typically single-digit microns or better on the features that matter. The distinction is not the equipment alone but the whole system behind it: rigid machines, temperature control, correct tooling and fixturing, skilled setup, and — crucially — the metrology that proves the part is actually within tolerance. A shop that claims precision without a CMM and a first-article report is describing a hope, not a capability. The full machining picture. Precision is a chain, not a number A tight tolerance on a drawing is only as real as the weakest link in the chain that produces it. That chain runs: the machine's own geometric accuracy and thermal stability → the workholding that locates the part → the tooling and cutting strategy → the measurement that confirms the result. If any link fails — a warm spindle, a springy fixture, a dull tool, an uncalibrated gage — the tolerance on the print does not survive into the part. This is why two shops can quote the same ±0.005 mm and deliver different results: one controls the whole chain, the other only owns a precise machine. When you qualify a shop, ask how it holds each link, not just what machine it bought. How we prove the chain. How tight is tight? The tolerance ladder Tolerance bandWhat it suitsHow it is achieved ±0.05 mmGeneral features, clearancesStandard milling/turning ±0.02 mmFits, locating featuresCareful setup, stable process ±0.01 mmBearings, sealing facesControlled process, in-process checks ±0.005 mmPrecision bores, journalsFinish pass + CMM verification ±0.002 mm or tighterGround/EDM featuresGrinding or wire EDM Planning brackets, not guarantees. Each step down the ladder roughly doubles the process control — and the cost — required. Where precision actually lives in a part Most parts do not need tight tolerance everywhere; they need it on a handful of functional features. The precision usually concentrates in: bearing bores and journals that set concentricity and runout; sealing faces that must hold flatness and finish against a gasket or an O-ring; locating dowels and slots that position a mating part; and threads and pilot diameters that align an assembly. Everything else — the body, the clearance holes, the cosmetic surfaces — can run loose. The mark of a good drawing is that the tight tolerances are exactly where function needs them and nowhere else. Over-tolerancing is the single most common and most expensive error a buyer makes. GD&T, decoded. Typical parts and tolerances for precision work Precision machining serves a recognizable part profile: the features that locate, seal, rotate, or transmit force. The classic examples are bearing seats and journals, valve bodies and spools, pump and compressor housings, hydraulic manifolds, gear blanks, and the mating faces of instrumentation. What they share is a handful of critical features — a bore, a face, a shoulder — whose relationship, not just their size, carries the function. The tolerance you actually need follows from the feature, and the table below brackets what each process band delivers on a mature floor. Use it to sanity-check your drawing: if your callouts sit one band tighter than the feature needs, you are paying for tolerance you cannot use. FeatureTypical toleranceProcess Bearing bore / journal±0.005–0.01 mmFinish boring/turning + CMM Sealing face flatness0.005–0.01 mm over the faceFinish milling + CMM Locating dowel / slot position±0.01–0.02 mmMilling with controlled setup Ground feature (journal, seat)±0.002 mm and betterCylindrical grinding Sharp internal corner±0.005 mmWire EDM Thread (precision class)Class 2/3, gagedThread milling / turning Planning brackets, not guarantees — the achievable tolerance depends on material, geometry and feature count. Confirm on your drawing. How precision is actually achieved Tight tolerance is engineered, not wished. The levers are few and concrete. Machine rigidity and thermal stability: a machine that flexes under load, or that grows a few microns as it warms, cannot hold a micron tolerance. Workholding: the part must be located and clamped without distortion — a soft jaw, a vacuum plate or a dedicated fixture beats a generic vise for repeatability. Tooling strategy: a rough pass leaves stock, a finish pass removes a controlled, small amount, and the tool is sharp and correctly compensated. Measurement: the tolerance is meaningless without a CMM or a calibrated gage that can resolve it, and a first-article report that records it. None of this is exotic; it is discipline applied to a chain. The difference between a shop that holds ±0.005 mm once and one that holds it across a run is the difference between owning the equipment and running the discipline — which is why a capability claim should always come with a measured report, not a brochure. How Dongguan shops hold it. The cost of a decimal place Every step down the tolerance ladder costs more, and the cost is non-linear. Tight tolerance means slower feeds, more setups, more skilled attention, and — the biggest hidden cost — more inspection and a higher scrap allowance. A part held to ±0.005 mm on a bore may cost several times what the same part costs at ±0.05 mm, because the shop now has to control temperature, use a finish pass, and measure the bore on a CMM rather than trusting the machine. The buyer's lever is to specify tolerance only where function needs it: a clearance hole does not need a micron, and a bearing bore cannot do without it. The difference between a good drawing and a costly one is knowing which features are which. The rule of thumb: every band down the ladder can double the price of a feature, so a print with ten needlessly tight callouts is not a better part — it is a more expensive one. Where the cost goes. GD&T: the language that makes tolerance mean something Plus-minus dimensions tell you a size; GD&T tells you a relationship. Position locates a hole relative to datums; concentricity and runout control how a rotating feature relates to its axis; flatness and parallelism control surfaces that must mate or seal; profile controls complex curves. The value of GD&T is not precision for its own sake — it is that it expresses the function of the tolerance, so the shop knows which features actually matter and which can run loose. A drawing with proper datums and feature control frames lets a good shop quote tighter where it matters and cheaper where it does not, instead of guessing. It also gives the inspection report a defined meaning, so the number on the CMM report maps to the requirement on the print. The five symbol categories. Verifying precision: the metrology that backs the claim A tolerance is only as good as the instrument that verifies it. For single-digit-micron work, a coordinate-measuring machine is the standard — it measures position, form and profile in three dimensions against the CAD model, and it produces the first-article report that travels with the part. Supporting instruments fill the gaps: a profilometer for surface finish, a micro-Vickers tester for hardness, an optical comparator or laser microscope for small features, and X-ray fluorescence for material verification. Our floor runs two Hexagon GLOBAL S CMMs up to 900×1500×800 mm, a KEYENCE laser microscope, a SURFCOM roughness tester and handheld XRF, so the ±0.005 mm on the print is confirmed by a measured report, not a promise. The inspection floor. Process capability: precision that repeats, not a lucky part One good part proves nothing. Precision that matters is precision that repeats across the run, and the measure of that is process capability — Cpk. Cpk folds two things into one number: how wide the process spread is relative to the tolerance (Cp), and how centered it is (Ca). A process with Cpk 1.67 or higher is considered capable of holding the tolerance on a key characteristic, with margin for drift. Our floor targets Cpk ≥ 1.67 on key characteristics and ≥ 1.33 on general ones. Critical characteristics are held to Cpk ≥ 1.67 under SPC, general features to Cpk ≥ 1.33. That is the difference between a part that measures in tolerance once and a process that stays there. The capability baseline. Choosing a precision machining partner Ask for the metrology, not the machine list. A CMM with a calibrated record and a first-article report beats a brochure of machine names. Ask for Cpk, not just a pass mark. A real capability number on a real dimension proves the process, not the part. Check temperature control. Micron work needs a stable thermal environment — ask how the shop manages it. Send a hard test part first. A tight bore and a sealing face, before a container, tells you more than a month of email. Confirm the whole chain. Machine, workholding, tooling and measurement must all be under one quality system. The short version: precision is a system, and the shops that can prove it are the ones that can hold it. How to select a partner. Materials that precision-machines well Precision is easier in some metals than others, and material choice quietly sets the floor on what is achievable. Aluminum (6061, 7075) machines fast and holds tight tolerance well, with the caveat that it grows and shrinks with temperature. Stainless (303, 304, 316) is harder to cut but dimensionally stable and corrosion-resistant — the default for medical and instrument parts. Brass and copper cut beautifully and suit precision electrical and fluid parts. Titanium and nickel alloys hold tolerance well but cut slowly and heat the tool, so they cost more. Plastics such as PEEK and acetal can be machined precisely but are far more sensitive to temperature and clamping pressure than any metal. The rule: pick the material for the function first, then design the tolerance to what that material will reliably hold — a micron tolerance that is easy in brass may be a fight in a soft plastic. For the tightest work, dimensional stability matters as much as machinability — a material that moves with humidity or relieves internal stress after cutting will never hold a micron, no matter how good the machine. How to choose the material. Thermal effects: the invisible enemy of precision Every material expands when it warms, and at micron tolerances that expansion is not a rounding error — it is the whole game. A steel part 100 mm long grows roughly 1 micron for every 1°C of temperature change; aluminum grows about twice that. So a part machined warm, measured cold, or inspected on a different day, can drift in and out of tolerance for reasons that have nothing to do with the machine. A serious precision shop manages this three ways: it lets the machine and the part reach thermal equilibrium before finish cuts, it keeps the metrology area temperature-controlled, and it measures the part and the reference under the same conditions. This is also why a 20°C reference temperature is written into most precision drawings — the tolerance is only meaningful at the temperature it was specified for. Ask a shop how it handles thermal stability; the answer separates precision work from wishful machining. Precision processes compared: mill, turn, grind, EDM Different processes deliver precision differently, and knowing the hierarchy helps you specify the right one. Milling and turning hold the mid-to-tight range — roughly ±0.005 mm on a good floor — and cover the bulk of precision parts. Grinding takes the same feature to a finer tolerance and a finer surface finish — ±0.002 mm and better — because it removes material in tiny, controlled increments. Wire EDM cuts hardened steel and sharp internal corners that a mill cannot reach, to the same tight range, with no cutting force to deflect the part. The practical rule: machine to near-net, then grind or EDM only the features that need the last step. That keeps the fast, cheap process doing the bulk and reserves the expensive one for where it earns its cost. When grinding beats milling. · Wire EDM explained. A worked example: the precision bore A locating bore makes precision concrete. Suppose a housing needs a bore held to ±0.005 mm, with a sealing face flat to a few microns, and the bore must be concentric to the face. On the floor, the part is rough-milled to leave stock, then finish-bored with a sharp, correctly compensated tool; the sealing face is finished in the same setup so its relationship to the bore is set, not re-established. The bore and the face are then measured on the CMM — size, roundness, flatness and perpendicularity in one report — and the result is recorded against the drawing's GD&T callouts. If the process drifts, the SPC chart shows it before a bad part ships. That is what "we hold ±0.005 mm" actually looks like: a controlled sequence, not a machine setting, ending in a measured report. The measured envelope. Common precision failures and how a good shop avoids them Thermal drift. Parts measured at the wrong temperature. Avoided by thermal equilibrium and a temperature-controlled metrology area. Workholding distortion. Clamping that bends the part, so it measures good in the fixture and bad when released. Avoided by soft jaws, vacuum plates and dedicated fixtures. Tool wear mid-run. A finish tool that drifts as it wears. Avoided by tool compensation and in-process checks. Measuring with the wrong gage. A micrometer that cannot resolve the tolerance. Avoided by using a CMM for single-digit-micron features. Ignoring the tolerance chain. A part good alone that fails when mated. Avoided by GD&T that expresses relationships, not just sizes. Why precision machining in China makes economic sense Precision capability is expensive to build — it is the machines, the metrology and, above all, the skilled people who hold the chain together. China's precision shops built that capability at scale over two decades, and the cost structure that came with it means the same micron-level tolerance that is a premium in a high-cost country is routine in a mature Chinese machining belt. The saving is in labor and overhead, not material — metal costs about the same anywhere — which is why the gap widens as the part gets more complex and the tolerance gets tighter. The caveat is qualification: capability varies widely, so the tolerance on the print only matters if the shop's metrology and process control are real. That is why the questions above — CMM reports, Cpk data, thermal control — are the whole game, not the machine list. Sourcing precision in Dongguan. Frequently asked questions What tolerance counts as precision machining?Roughly ±0.01 mm and tighter, down to single-digit microns, on the features that matter. The tighter end needs finish passes and CMM verification, and grinding or wire EDM below about ±0.002 mm. What tolerance can you hold?±0.005 mm on CNC milling and turning, and ±0.002 mm on precision grinding and wire EDM, verified on Hexagon CMMs. Key characteristics are held to Cpk ≥ 1.67. Why does tighter tolerance cost so much more?It is non-linear: tighter tolerance means slower feeds, more setups, more inspection and higher scrap. Each decimal place roughly doubles the process control required. How do I stop over-paying for tolerance?Put tight tolerances only where function needs them — bearing bores, sealing faces, locating features — and let the rest run loose. A good DFM note will flag over-tolerancing before you commit. How do you prove a part is in tolerance?With a first-article CMM report tied to the drawing's GD&T callouts, plus SPC across the run. Critical characteristics are held to Cpk ≥ 1.67. Do you run prototypes before production?Yes — there is no minimum order, so a prototype runs first on the same machines and process, and the first-article data transfers to the production run. Why does China hold tight tolerance at lower cost?Because the capability was built at scale over decades, so labor and overhead are lower for the same micron-level work. Material costs about the same anywhere; the saving is in the process around it. What does a first-article inspection include?A CMM report against the drawing's GD&T callouts — size, position, form and profile — plus a surface-finish check, tied to the lot number. It is the document that proves the tolerance, and it travels with the first article of every run. Can you combine precision CNC with the electronics build?Yes — EMS and CNC sit under one roof, so a precision machined housing and the PCBA inside it come from one quality system and one tolerance chain. What is the difference between precision and general machining?It is a capability level, not a machine type. General machining holds roughly ±0.05 mm and up; precision machining holds single-digit microns on the features that matter, backed by temperature control, rigid workholding, and CMM verification rather than a machine setting. How do I specify precision without over-tolerancing?Put tight tolerances only on the features that carry function — bearing bores, sealing faces, locating features — and let the rest run loose. A good DFM note will flag over-tolerancing before you commit. Do you measure parts at 20°C?Yes — the metrology area is temperature-controlled and parts are measured at a stated reference so the number travels with the part. A micron tolerance is only meaningful at the temperature it was specified for. Sources & coverage note: Tolerance capability reflects our in-house CMM/SPC practice; machining-process references follow ASM International handbooks for achievable process capability, and GD&T interpretation follows ASME Y14.5. Cpk values are from our own measurement records, not a third-party audit. Holding a tight tolerance?Send the drawing to [email protected] — we will confirm what we can hold on the features that matter, and quote the tolerance your part actually needs.Request a quote Related articlesGD&T ExplainedCNC Grinding: When a Mill Is Not Precise EnoughCNC Swiss Machining: Precision Turning