LensMTF

Now that these statistical results are available, Imatest will be doing more to promote the use of MTFAPN as a sharpness summary metric.

The plot below shows the edge and MTF from a raw image captured at ISO 12800. It is extremely noisy because of the very high ISO speed (dim illumination of the sensor, followed by very high gain) and because no noise reduction is applied to raw images.

Modulation transfer function

The tests were performed on the Panasonic Lumix LX100 camera: a high quality compact camera that has a relatively large sensor, excellent controls, and can produce both JPEG and raw files. We took 10 images of a large eSFR ISO test chart at each of three ISO speeds, saving them in both JPEG and raw formats, for a total of 60 images. (SFRplus could have been used with equally good results.) The lens was set to 50mm-equivalent focal length. Manual focus was used to eliminate variations due to autofocus.  Standard signal processing was selected. A 2-second time delay minimized camera shake.

This problem will not occur if MTF Area (a relatively unfamiliar summary metric, described below) is used instead of MTF50.

FOV and focal length

This image may look good on the small display of a camera phone, but it is far from optimum when exported and viewed on a large display or printed. When sharpened in this way, image quality won’t approach the potential of the lens and sensor. A better strategy would have been to sharpen with a radius between 1 and 1.5 (perhaps with a moderate oversharpening peak— see sharpening examples, below), then to add additional sharpening for images sent to the display.

There may be cases where moderate MTF overshoot (~25-50%) may be desirable. In such cases a modified version of MTF Area may be a better metric. For example (for sharpening between B. and C, above), an image with a spatial domain overshoot of 25% may appear sharper than it would without overshoot— and the halo won’t be very visible. This corresponds to a peak MTF of approximately 1.4.  For a desired peak MTF = MTFpk > 1, we define

The pattern of peaks and dips (near 0.42 and 0.84 C/P) suggests a sharpening radius of around 2.4, which would have its first peak around 0.24 C/P. The first system MTF peak is typically lower because of the MTF rolloff of the unsharpened image.

Running the image (using SFRplus with the original uncropped image; SFR with the crop) quickly showed the reason for the problem. A curious combination of excellent lens performance and large sharpening radius (appropriate for a mediocre lens) resulted in a long ramp in the MTF response around the 50% level. Slight changes in response could have major changes in MTF50.

When the system has strong response above the Nyquist frequency, low frequency artifacts such as Moiré fringing or stair-stepping may appear. Stair-stepping is clearly visible when the above image is magnified 5X, as shown on the right. These artifacts have an adverse effect on measurement consistency because an important step in the slanted-edge measurement is the estimate of the edge angle, which is quite consistent for smoothed (anti-aliased) edges, but which depends on the ROI position when there is extreme aliasing. A highly exaggerated example is shown on the far right, where the edge angle estimate derived from the two small red rectangles will be completely different.

We propose a new summary metric: the area under the MTF curve (up to the Nyquist frequency), normalized to the peak MTF value. This metric is relatively unfamiliar, but seems to be generally more stable than MTF50 and MTF50P.

As explained in the Sharpening page, the sharpening transfer function is periodic with a period of 1/Radius in Cycles/Pixel. The system MTF (shown on the right for the above image) is the the raw MTF response times the sharpening transfer function.

In the eSFR ISO Auto mode settings window, all Single-region plots and most Multi-ROI plots were unchecked. (When we needed to examine details we ran eSFR ISO Setup). Close figures after save and Allow CSV file output were checked.

A key observation (not obvious with normal images, which have much less noise) is that MTF50 is based on the first spatial frequency where MTF drops below 50% of the low frequency level— 997 LW/PH in this plot, where noise causes a sudden and misleading dip just under 1000 LW/PH in the (unsmoothed) MTF curve. Smoothing the MTF plot would improve MTF measurement consistency, but smoothing has never been addressed in the ISO 12233 standard. The optimum amount of smoothing, which depends on the width of the region (i.e., frequency increment or the number of points in the plot) would have to be determined.

The strong response above the Nyquist frequency (0.5 cycles/Pixel)  indicates that the lens is extremely sharp. Images from such lenses tends to look best with sharpening radii well below 2.4, which provides very little response boost in the important 0.35-0.5 C/P range, where the input MTF has sufficient energy to benefit from the boost.

MTFtesting Mastercard

In this page we analyze the consistency of slanted-edge MTF measurements, focusing on the effects of noise and region size on measurement consistency. We describe the test procedure in sufficient detail to enable Imatest users to perform similar studies for themselves.

Field of view

MTF

This number will be equal to MTF Area for measured MTF peak = 1 (which is the minimum possible value, since MTF is defined as 1 at the lowest spatial frequencies). It will continue to increase up to measured MTF peak = MTFpk, then it will flatten out. For example, for a desired peak MTF = MTFpk = 1.4 (corresponding to spatial overshoot ≅ 25%), MTF Area(1.4) = min(measured MTF peak, 1.4) * MTF Area will reach its maximum when measured MTF peak = 1.4.

In the eSFR ISO More settings window, MTF plot units were set to LW/PH, Speedup was checked, and MTF noise reduction (mod apod) was checked (except where indicated).

In running eSFR Auto, select batches of files to be analyzed together, i.e., don’t mix different ISO speeds or JPEG and raw, unless there is a specific need to do so (for example, for directly comparing different ISO speeds). No additional settings need to be entered, since they were saved during eSFR Setup (Rescharts).

A 25×40 pixel region size is sufficient for good results with low noise images. This may be the best that can be done with low resolution cameras, though larger is recommended if the geometry allows or if the image is very noisy.

ImageJMTF

A second observation is that MTF Area (short for MTF Area Peak-Normalized), which is equal to the integral of the MTF curve from 0 the Nyquist frequency  fNyq, would be closely approximate MTF50 derived from a properly-smoothed monotonically-decreasing MTF curve.

This is illustrated in the plot on the right, which shows MTF50, MTF50P and MTF Area for an edge (from one of the ISO 200 raw images in the above analysis). The edge was analyzed without sharpening, and with one and two applications of sharpening (Unsharp Mask in Picture Window Pro with Sharpening Radii = 1 and 2). MTF50 continues to increase as sharpening increases, even though the visible effect of sharpening— “halos” near edges— can degrade image appearance. MTF50P increases much more slowly: it’s a generally more reliable metric than MTF50. But it still depends on a single frequency. MTF area does not increase once a sharpening peak > 1 appears.

Another advantage of MTF Area is that it tracks MTF50, i.e., it increases as sharpening increases, up to the point where an overshoot appears in the sharpened MTF curve (peak MTF > 1), then it remains relatively constant. So it does not reward excessive sharpening with better numbers.

Related page: Correcting Misleading Image Quality Measurements: links to an Electronic Imaging paper that compares MTF summary metrics

Note: This page is not quite complete, but we felt that the results are important enough to be presented in its present (nearly complete) state.

In Batchview, enter the […]_sfrbatch.csv output files (from the eSFR ISO batch runs) into boxes A–D at the bottom. Up to four output files can be entered for comparison (though statistical results can only be viewed when a single batch is displayed: Display (lower-right) should be set to A, B, C, or D. The measurement (MTF50, MTF50P, etc.) should be selected from the dropdown menu at the top-left. For this particular analysis, Part way mean was selected in the Region dropdown menu, top-center. (Center mean will be selected in cases where the image is sharpest in the center.) Statistics (mean, sigma) should be selected in the dropdown menu on the top-right (only visible with a single batch is displayed).

Shipping Policy | Privacy Policy | Return Policy | Imatest Terms and Conditions

An Edge ID file with one ROI in each of the center, part-way, and corner zones of the image was used in eSFR ISO so that Batchview, which analyzes the mean of results for each zone, analyzes just one image per zone. (Otherwise you would get the mean of several measurements, which would have a lower variability). The edge ID shown on the right (format: x_y_LRTB) was chosen because the sharpest edge in the manually-focused image was the left edge of the slanted-square two squares to the right of the center. This square (region 3 in the figure below) was used for the analyses on this page.

A customer sent us an image whose MTF50 measurements varied unpredictably as ROI size changed. A crop is shown on the right. You can click on it to view a full-sized image that you can download and run.