The CNT-90 timer/counter/analyzer employs linear regression techniques for improved measurement resolution. The inclination of the regression line corresponds to the estimated average period time of the input signal (T*), and is calculated as:

where:

xk = the number of cycles in sample k;

yk = the time-stamping value in sample k; and

n = the number of samples in the measurement

The uncertainty (variance) of the inclination T* is:

where:

s(y) = the normal root-mean-square (RMS) resolution, t_{RES}, for an individual time stamp and s(x) = the standard deviation of an approximate rectangular distribution,

(x_{k} = x_{0} + kN/n i.e.,

s(x) ≈ Σ = N/2(3)^{0.5}

for large values of n, which leads to the relative resolution in period or frequency:

s(T*)/T* = s(f*)/f* = 0.52t_{RES}>/0.5>