Units and Measurements · Lesson 2 of 7

Precision and Significant Figures

“Read the precision carried by measured numbers and choose sensible measurement methods.”

Learning Objectives

• Distinguish precision, accuracy and resolution. • Count significant figures and interpret zeroes. • Use scientific notation to preserve precision. • Explain indirect and repeated measurement.

What the last digit means

Every physical measurement has limited precision. A reported value normally includes reliably known digits and a final uncertain digit. In a pendulum period reported as 1.62 s, the last digit communicates the finest reported detail. Writing 1.62000 s would claim additional information unless the instrument and method supported it. Significant figures describe how many meaningful digits are being reported; they do not by themselves prove that a measurement is close to the true value.

Resolution is the smallest change that an instrument can distinguish. Precision concerns the closeness of repeated measurements or the fineness with which a value is specified. Accuracy concerns closeness to the accepted or true value. A miscalibrated instrument may give very similar readings that are all displaced from the true value. Smaller scale divisions alone cannot remove calibration errors or poor technique.

Counting meaningful digits

Rule	Example	Significant figures
Nonzero digits count	2.64	3
Zeroes between nonzero digits count	6.032	4
Leading zeroes locate the decimal point	0.007	1
Trailing zeroes after a decimal can express precision	0.2370	4
Whole-number trailing zeroes may be ambiguous	4700	Use scientific notation to clarify

In 0.0006032, the initial zeroes only place the decimal point, but the zero between 6 and 3 matters. There are four significant figures. In 6.320 the final zero matters too. A change of units cannot change how well an object was measured; therefore the number of significant figures must be preserved even if ordinary decimal notation becomes ambiguous.

Preserving precision through a conversion

Problem
A length is reported as 4.700 m. Express it in millimetres without hiding its precision.

1.The original has four significant figures. The conversion is exact: $1 m = 1000 mm$ .
2.Write $4.700 \times 1 0^{3} mm$ . The coefficient explicitly retains all four digits.
3.Writing only 4700 mm can hide whether the final zeroes express measured precision or merely the size of the unit.

Scientific notation and scales

Scientific notation places one nonzero digit before the decimal point and expresses the scale separately as a power of ten. Count significant figures in the coefficient, not in the exponent. For a rough order-of-magnitude estimate, this chapter uses the convention of rounding a coefficient of 5 or less to 1, and a coefficient above 5 to 10. State the convention when an estimate lies near the boundary.

The Earth’s diameter is about 1.28 × 10⁷ m and a hydrogen atom’s diameter about 1.06 × 10⁻¹⁰ m. Their characteristic powers differ by 17. This comparison communicates a scale difference more clearly than writing long strings of zeroes. Exact counts, such as 20 oscillations, and exact unit definitions do not limit the significant figures of a calculation.

Measuring something too small for a ruler

To estimate thread diameter, wind many adjacent turns around a pencil without gaps or overlap. Measure their total width and divide by the number of turns. This enlarges the measurable length, although uneven thickness or stretching may still affect the result. Repeating measurements at several positions helps detect variation. More observations can reduce the influence of random scatter but cannot automatically remove a systematic offset.

An indirect measurement

Problem
A hair appears 3.5 mm wide under a microscope with linear magnification 100. Estimate its true thickness.

1.Linear magnification is image size divided by object size. Thus the thickness is $3.5/100 = 0.035 mm$ .
2.This is $3.5 \times 1 0^{- 5} m$ , or $35 μ m$ .
3.The estimate assumes the stated magnification is reliable. Repeated readings improve confidence in the observed image width.

For a screw gauge, least count is pitch divided by the number of circular-scale divisions. A pitch of 1 mm and 100 divisions gives 0.01 mm. Comparing a vernier requires knowing the main-scale spacing and the relationship between the two scales; the number of vernier divisions alone is insufficient. Real instrument accuracy also depends on calibration, contact pressure and alignment.

Writing scientific notation explicitly

Scientific notationLaTeX

x = a \times 1 0^{n}, 1 \leq ∣ a ∣ < 10

For a nonzero number, n is an integer. Count significant figures in the coefficient a.

Estimating an order of magnitude

Problem
Apply the chapter’s stated convention to 6.4 × 10⁻⁴ m.

1.The coefficient 6.4 is greater than 5, so round the coefficient to 10 for this rough estimate.
2.Then $10 \times 1 0^{- 4} = 1 0^{- 3}$ . The estimated scale is $1 0^{- 3} m$ and the order-of-magnitude exponent is −3.
3.For a coefficient at or below 5, this chapter retains the original exponent. This is a rough-scale convention, not the significant-figure rounding rule.

Comparing the resolving power of instruments

Least count relationshipsLaTeX

LC_{screw} = \frac{pitch}{circular divisions}, LC_{vernier} = ∣1 MSD - 1 VSD ∣

MSD and VSD mean one main-scale division and one vernier-scale division. Pitch is the axial advance in one full screw rotation.

Vernier, screw gauge and optical scale

Problem
Compare a conventional 20-division vernier, a 1 mm-pitch screw gauge with 100 divisions, and an instrument resolving a visible wavelength.

1.For the vernier, explicitly assume $1 MSD = 1 mm$ and $20 VSD = 19 MSD$ . Then $1 VSD = 0.95 mm$ , so its least count is $0.05 mm$ .
2.The screw gauge resolves $1/100 = 0.01 mm$ .
3.A representative visible wavelength is $500 nm = 0.0005 mm$ . An instrument resolving a length of that order resolves smaller differences than either mechanical instrument in this comparison.
4.The optical instrument is therefore the expected choice under these assumptions. Division count alone does not specify a vernier’s least count, and resolution alone does not establish accuracy.

Why more divisions do not guarantee accuracy

Problem
A screw gauge has pitch 1.0 mm and 200 circular divisions. Can arbitrary subdivision make it arbitrarily accurate?

1.Its nominal least count is $1.0/200 = 0.005 mm$ .
2.Finer engraved divisions do not remove backlash, imperfect screw threads, zero offset, object deformation or the limits of reading the scale.
3.Repeat observations and check zero error, alignment and contact pressure. Increasing nominal resolution is useful only if the rest of the measurement system can support it.

Mean of repeated readings and a thread estimateLaTeX

\overset{ˉ}{d} = \frac{d _{1} + d _{2} + \dots + d _{N}}{N}, d_{thread} \approx \frac{W}{N}

For the thread, W is the total width of N adjacent turns. N is an exact count.

For independent readings of a stable rod with random scatter, averaging 100 readings normally gives a more reliable estimate than averaging 5. Taking readings at several positions also reveals nonuniform thickness. An offset shared by every reading remains in the mean. Similarly, the oscillation period can be measured as total elapsed time divided by the exactly counted number of oscillations: τ = t/N. The measured time limits the precision, not the integer count.

Applying measurement to scale

Linear magnification from areas

Problem
A house image occupies 1.75 cm² on a slide and 1.55 m² on a screen. Find the linear magnification.

1.Convert the screen area to $1.55 \times 1 0^{4} c m^{2}$ .
2.For a uniformly enlarged image, area ratio is the square of the linear magnification: $M^{2} = A_{2} / A_{1}$ .
3.Thus $M = 15500/1.75 \approx 94.1$ . It is a dimensionless ratio, reported to three significant figures.

Apparent motion and distance

Looking sideways from a moving train, nearby objects change viewing direction rapidly while distant objects change direction slowly. For the same small sideways displacement, the angular change is roughly displacement divided by distance. This explains the apparent rapid motion of nearby trees and the near stationarity of distant hills. The comparison is about angular motion, not the distant objects actually travelling with the train.

Small angular changeLaTeX

Δ θ \approx \frac{Δ x}{D}

This estimate applies to a small transverse displacement Δx compared with distance D.

For the same transverse displacement, increasing distance tenfold reduces the angular change by a factor of ten. Nearby trees sweep across the view rapidly; a distant hill changes viewing direction very little. A sketch of two sightlines to a near and a far object makes the comparison clear.

Quick check

How many significant figures are in 0.0006032?

Key Takeaways

• Significant figures communicate reported precision. • Leading zeroes do not count, but internal zeroes do. • Scientific notation removes ambiguity in trailing zeroes. • Repeated measurements do not automatically eliminate systematic errors.

Previous · Lesson 1

Measurement and the SI System

Next · Lesson 3

Calculations with Measured Values