Limits of Resolution 6 How Many Megapixels
Limits of Resolution 6 How Many Megapixels
Phil Service
Flagstaff, Arizona, USA
8 May 2015
Summary
Photosite spacing of 1.5 µm or less is common for smart phone cameras; and 1-inch
sensors in cameras such as the the Sony RX100 III have photosite spacing of 2.4 µm.
Diffraction-limited line-pair resolution is given for photosite spacing as little as 0.5 µm. Two
micron photosite spacing implies APS-C and “full-frame” (FF) sensors with 94 and 216 MP,
respectively. In order to approach the theoretical resolution limits of sensors with 2 – 3 µm
photosites, it will be necessary to have lenses that perform exceptionally well at apertures f/2.8 –
f/4. It is not clear if such lenses can be manufactured at reasonable cost for APS-C and FF
sensors. If it is, it may be necessary to sacrifice large maximum apertures, such as f/1.4, in order
to make “slower” but sharper lenses. With current technology, 2 – 3 µm photosites will entail a
trade-off between resolution and low noise, when compared to current FF and APS-C sensors.
For most image uses, 100 MP or greater resolution implies capture oversampling. That is,
images will be down-sampled for “final” use. It is suggested that such down-sampling may
produce a sharper and less noisy final image than could otherwise be obtained by capturing
images at lower initial resolution.
Key words: resolution limits, sampling frequency, diffraction, sensor pixel size, photosite size,
perfect lens, line-pairs, simulation, oversampling, sharpness, signal-to-noise ratio, down-
sampling, image noise, edge acutance
1. Introduction
All other things being equal, closer photosite spacing means higher image resolution (lp/
mm).1 On the other hand, diffraction degrades resolution. If we could pack photosites as closely
as we wanted, at what point would diffraction limit resolution? The answer to that question
arguably sets the useful upper limit to “megapixels” for a sensor of a given format.2
1 I will use the terms “photosite spacing” and “photosite size” interchangeably, with the understanding that
the former is more appropriate in the present context, which focuses on sampling frequency. Generally
photosite “size” is smaller than “spacing” or “pitch” because gaps must be left between adjacent
photosites.
2 At the risk of being eccentric, or worse, I use the term “photosite” when referring to an individual light
receptor on a camera sensor. A “pixel” is the smallest element of a digital image. However, given the
apparently universal practice of describing camera sensor resolution in terms of megapixels, it would
seem perverse to insist on using “megaphotosites” instead. Hence “megapixels” in the title of this paper.
Happlily, however, the usual abbreviation for megapixels, MP, also works for megaphotosites. So,
whenever you see the abbreviation MP, feel free to say megaphotosites in your head.
1
© 2015 Phil Service ([email protected]) Last revised: 6 July 2015
There is more to image sharpness than resolution, per se. Most images contain details
with a large variety of spatial frequencies. Images appear “sharp” not only when high-frequency
detail is visible, but also when lower frequency detail is rendered with crisp edges. Diffraction
unavoidably makes edges fuzzy, to a degree dependent on aperture, and sets a minimum width
for fuzzy edges. “Sampling” of edges by photosites cannot decrease the width of the fuzzy zone
below the minimum set by diffraction. However, low-frequency sampling — by large, widely-
spaced photosites — will increase the width of the zone. Thus, a second question: at what point
does packing photosites more closely together stop yielding useful gains in edge acutance?3
The sensor of the iPhone 6 camera has 1.5 µm photosite spacing. Some other smartphone
sensors have even smaller spacing, and sensors with photosites smaller than 1 µm are being
considered.4 A “full-frame”, 36 x 24 mm, sensor with 1.5 µm photosites would have 384 MP.
Whether we will ever see such sensors in consumer products, I cannot say. However, it is
probably safe to say that the 1 – 1.5 µm photosite will eventually find its way to sensors larger
than those used in current smartphone cameras.
1.1. Terminology
As in the previous papers, it will be useful to define a few terms at the outset. The lens
image is the image that is formed by the lens. The lens image, or image field, is an effectively
continuous, analog representation of the external world — the object field — in front of the lens.
The sensor image is the digitized image recorded by the sensor. It is absolutely crucial to
understand that this is a sample of the image field. Blur affects the lens image, not the sampling
process. Therefore, there is not a simple relationship between diffraction blur and photosite pitch
— a fact that seems generally to be misunderstood. Resolution in the present context means line-
pairs per millimeter (lp/mm). In general, a contrast ratio will be associated with a resolution
measure. The contrast ratio is the maximum difference in lightness values of alternating light
and dark lines, divided by their sum. A perfect lens is a lens with no optical aberrations. In the
absence of diffraction, it would produce an image with no blur. A perfect lens is assumed in
everything that follows, and diffraction is the only source of lens image blur. Nyquist rate
resolution is the maximum resolution (lp/mm) achievable with a given sensor. A minimum of
two rows or columns of photosites is required to record a line-pair. Thus, if photosite spacing is
4 µm, then 8 µm (= 0.008 mm) are required to sample one line-pair. The Nyquist rate resolution
is then 1/0.008 = 125 lp/mm.
3 Acutance is defined here as the rate of change of brightness with distance. If a transition zone (edge)
between white and black areas is wide, edge acutance will be relatively lower than if the transition zone is
narrower. Resolution and acutance both contribute to the perceived sharpness of an image.
4 Agranov, G., et al. 2011. Pixel continues to shrink....Small Pixels for Novel CMOS Image Sensors.
2011 International Image Sensor Workshop (IISW), Hokkaido, Japan. http://www.imagesensors.org/Past
%20Workshops/2011%20Workshop/2011%20Papers/R01_Agranov_SmallPixel.pdf
Tian, H., et al. 2013. Architecture and Development of Next Generation Small BSI Pixels. 2013
International Image Sensor Workshop (IISW), Snowbird, Utah, USA. http://www.imagesensors.org/Past
%20Workshops/2013%20Workshop/2013%20Papers/01-4_080-Tian-paper.pdf
2
© 2015 Phil Service ([email protected]) Last revised: 6 July 2015
2. Methods
The methods are the same as used in a previous paper, and need not be repeated in detail
here.5 As in my previous papers, a “perfect” lens is assumed. Diffraction blur is taken to be 70%
of the diameter of the Airy disk for wavelength 550 nm.6 Complications arising from a color
filter array are ignored. Post-capture sharpening, which will increase micro-contrast, is not
considered. Finally, I assume that there are no limits to fabricating sensors with ever smaller
photosites, and no bandwidth limits with respect to processing sensor data.
3. Results
1024
f/1.4
f/2
f/2.8
512
f/4
Resolution, Log (lp/mm)
f/5.6
f/8
256 f/11
2
f/16
Nyquist
128
64
32
0 1 2 3 4 5 6 7 8
Photosite Pitch, µm
Fig. 1. Line-pair resolution at 50% contrast ratio. “Nyquist” is theoretical maximum resolution
achievable with a given photosite pitch. Note the logarithmic scale for resolution.
3
© 2015 Phil Service ([email protected]) Last revised: 6 July 2015
Fig. 2. Simulation and graphical representation of a blurred border between black and white areas.
The un-blurred border would be at position 0 µm. The width of the blurred zone is approximately 3.76
µm, which corresponds to the diffraction blur circle diameter for f/4.
4
© 2015 Phil Service ([email protected]) Last revised: 6 July 2015
resolution or edge sharpness by using apertures larger than f/5.6 on 5 µm photosites,as will be
discussed below.) (3) The converse of (2) is that small apertures seriously degrade resolution
when photosites are small. Consider, for example, the effect of using f/11 with 4 µm photosites
— relative to f/5.6 or f/4.7
7 Lest anyone think this example extreme, it is worth noting that the photosite pitch of the 16 MP m4/3
sensor in the Olympus OM-D E-M1 is approximately 3.7 µm. With a reasonably good lens and suitably
detailed subject matter, the degradation in image sharpness for f/11 compared to f/5.6 would be obvious.
8 Figs. 3 – 5 are contained in separate documents that can be accessed by clicking on the embedded
links.
9 Whether one views this as a silhouette of a dark edge against a light background, as I tend to, or vice
versa, is immaterial.
5
© 2015 Phil Service ([email protected]) Last revised: 6 July 2015
demonstrate would not be visible in images viewed at 100% magnification — only that they
would be much less obvious.
3.2.2. Diffraction. Simulations of the effect diffraction blur on edge acutance are shown for
f/2.8 and f/8 (Figs.3b and 3c). The diameter of the diffraction blur circle is about 2.6 µm for f/
2.8 and about 7.5 µm for f/8. None of the major “features” of the edge are completely obscured
by diffraction at f/8, although the edge is very “soft” compared to the f/2.8 example. Diffraction
blur at f/16 (and possibly f/11) would most probably leave the narrow black lobe in the bottom
third of the image unresolved.
3.2.3. Photosite Size (Sampling Rate). Four- and two-micron photosites are superimposed
on the image field for the f/2.8 diffraction blur case (Figs. 4b and 4c). A given photosite can
encode only one color, or shade of gray — obtained by averaging within the borders of each
simulated photosite shown in Figs. 4b and 4c. The resulting simulated sensor images for the 4
µm and 2 µm photosites are shown in Fig. 5. It is clear that, for this particular example at least,
2 µm photosites do a much better job of reproducing the edge. It’s not that the 4 µm photosites
do not capture the major “features” of the edge, it’s just that they do so very crudely. Note
particularly that in the 4 µm case, contrast is reduced between the interdigitating black and white
lobes.
116 MP 29 MP 13 MP 7 MP
One-inch (13.2 x 8.8
(13,200 x 8,800) (6,600 x 4,400) (4,400 x 2,933) (3,300 x 2,200)
mm)
4. Discussion
The illustrations of the effect of diffraction blur (Figs. 3b and 3c) show that diffraction
sets an upper limit to realized edge acutance. Edge width (in the absence of sharpening) cannot
be less than set by diffraction. Large photosites may increase the width of an edge, and therefore
decrease acutance. On the other hand, very small photosites may sample an edge with relatively
high fidelity. But if the edge is wide — perhaps because a small aperture has introduced
substantial diffraction blur — small photosites will do nothing to enhance edge acutance — a
detailed sample of a blur is still a blur. The unsurprising conclusion is that for maximum
resolution and acutance in the plane of focus, use the largest apertures possible and sensors with
6
© 2015 Phil Service ([email protected]) Last revised: 6 July 2015
the smallest photosites. Remember that we are assuming perfect lenses, and that depth of field is
not a consideration. As already mentioned, some smart phone cameras have sensors with 1.5 µm
or smaller photosites. The 20 MP 1” sensor used in the Sony RX100 III, for example, has 2.4
µm photosite spacing. A 24 MP APS-C sensor has 4 µm photosites. Table 1 shows the
resolutions of larger format sensors for various photosites pitches. I have no idea if we will ever
see 864 or 216 MP full-frame camera sensors. However, 96 MP does not seem out of reach in
the reasonably near future.
4.2. Noise
The most commonly voiced objection to small photosites is that, all else being equal, they
are noisier than larger photosites. However, the story of imaging sensors over the last 15 years,
or so, is that “all else” is seldom equal. In particular, while there has been a progression to
higher megapixel counts, and therefore smaller photosites, sensors have simultaneously become
less noisy. There is some suggestion in recent data, however, that a plateau may have been
reached.
Table 2 shows specifications for a number of camera sensors together with data on signal-
to-noise ratios (SNR), and low-light ISO (SNR and ISO data published by DxO Mark). SNR
results are for illumination equivalent to 18% and 1% gray-scale, with the camera set at ISO 200
(manufacturers’ setting). 1% gray-scale illumination is approximately 6 2/3 EV below 100%
gray, which, I believe, corresponds to sensor saturation. Thus, the 1% gray SNR (Table 2,
10Recall that the actual, object-field size of the x µm feature in the image field depends upon image
magnification (i.e., lens focal length and object distance).
7
© 2015 Phil Service ([email protected]) Last revised: 6 July 2015
A B C D E F G H I J
Full-frame (FX)
D3 2007-08 12.20 8.40 70.48 118.85 20.42 0.29 2,290 32.49
D3x 2008-12 24.59 5.90 34.86 109.65 14.29 0.41 1,992 57.14
D3s 2009-10 12.20 8.40 70.48 134.90 21.38 0.30 3,253 46.15
D4 2012-01 16.43 7.21 52.01 138.04 22.39 0.43 2,965 57.01
D810 2014-06 36.37 4.86 23.66 130.32 14.96 0.63 2,853 120.57
A7S 2014-04 12.21 8.30 68.93 134.90 26.92 0.39 3,702 53.71
APS-C (DX)
D70 2004-01 6.12 7.89 62.33 49.55 10.12 0.16 529 8.49
D50 2005-04 6.12 7.80 60.78 55.59 12.45 0.20 560 9.21
D300 2007-08 12.48 5.42 29.41 69.18 12.30 0.42 679 23.09
D90 2008-08 12.36 5.48 29.98 83.18 14.79 0.49 977 32.59
D7000 2010-09 16.37 4.73 22.36 81.28 13.34 0.60 1,167 52.19
D5200 2012-11 24.26 3.91 15.29 94.41 13.18 0.86 1,284 83.99
D7200 2015-03 24.16 3.91 15.26 95.50 13.34 0.87 1,333 87.36
One-inch
RX100 III 2014-05 20.18 2.40 5.77 57.54 8.13 1.41 495 85.81
* SNR and Low-light ISO data from DxOMark. Original SNR data was in dB. I have transformed it to linear scale.
Low-light ISO (column I) is defined as “the highest ISO setting for a camera that allows it to achieve an SNR of
30dB while keeping a good dynamic range of 9 EVs and a color depth of 18bits.” [ 30 dB = 31.6 SNR (linear)]. Note
that dynamic range “corresponds to the ratio between the highest brightness a camera can capture (saturation) and
the lowest brightness it can capture (typically when noise becomes more important than the signal, i.e., a signal-to-
noise ratio below 0 dB).” It should be pointed out that this is a very generous definition of dynamic range — 0 dB,
or a linear SNR of 1, means that noise is equal to signal. ISO 200 refers to the manufacturers’ on-camera ISO
setting.
column G) gives us information about shadow noise. Low-light ISO (Table 2, column I; also
referred to as the “Sports” score) is a measure of the usefulness of the camera in poorly lit
situations. The 1% gray SNR and low-light ISO are given in absolute terms (columns G and I);
and also “standardized” for difference in nominal photosite area (columns H and J).
With respect to APS-C sensors, manufacturers appear to have been unwilling to sacrifice
shadow SNR (at low ISO) for increased resolution. Thus, the 1% gray SNR (column G) has
remained quite steady over time in absolute terms (actually increasing slightly), despite the fact
that pixel area has decreased by about 75% (compare the D5200 with the D70). This reflects real
8
© 2015 Phil Service ([email protected]) Last revised: 6 July 2015
9
© 2015 Phil Service ([email protected]) Last revised: 6 July 2015
minimize total blur from uncorrected lens aberrations and diffraction. My speculation is that
maximizing lens speed conflicts with maximizing image sharpness at smaller apertures, say f/2.5
– f/3.5. If that is true, then it should be possible to design a lens with a maximum aperture of,
say, f/2.8 that would be sharper than a much faster lens, when both are used at f/2.8. Our
hypothetical, highly-corrected f/2.8 lens would most probably be lighter and more compact than
its f/1.4 competition, and possibly less expensive, although not cheap. A perfect f/2.8 lens could
make good use of photosites as small as 1 µm (Fig. 1); whereas a faster lens stopped down to f/4
(or f/5.6) — and performing perfectly — would be close to its 50% contrast resolution limit with
2 (or 3) µm photosites (Fig. 1). An obvious objection to this lens strategy, assuming it is
technically feasible, is that the market for such lenses would be too small. That is, most
photographers would not be willing to invest in relatively expensive, relatively “slow” lenses.
Additionally, if small photosites mean low maximum ISO, as would appear to be the case, slow
lenses would be adding insult to injury. Nevertheless, to take maximum advantage of very high
resolution sensors I suggest that is will be necessary to re-think lens design, or at least to re-think
the trade-off between lens speed and resolution.
4.3.2. Matching Lens Speed, Sensor Size, and Photosite Size. My impression is that it
is more difficult to design fast lenses for larger sensors than for smaller ones. If that is true, then
it is another reason to abandon the attachment to f/1.4 lenses for full-frame sensors. Leave the f/
1.4 lenses for smaller sensors, and scale maximum aperture accordingly. For example, a perfect
f/1.4 lens in front of a 1” sensor with 1 µm photosites would have a 50% contrast resolution of
about 4,000 line-pairs per picture height (LP/PH). A perfect f/4 lens in front of a full-frame
sensor sensor with 3 µm photosites could capture about 3,700 LP/PH — or about the same total
resolution.11
The “cameras” described in the preceding paragraph are approximately equivalent: the
“crop factor” for a 1” sensor is 2.73, or about 3. The photosite sizes also differed by a factor of
3, and the numerical f-values — 4 vs 1.4 — differed by a factor of 2.9. The example is meant to
illustrate how aperture, sensor size, and photosite size can be scaled to yield images of similar
resolution. However, even if we accept that f/4 is a desirable maximum aperture for lenses
designed for full-frame sensors, there is no necessary reason to limit photosite size to 3 µm. For
a perfect f/4 lens, 50% contrast resolution increases from 154 to 184 lp/mm when photosite size
is reduced from 3 µm to 2 µm (Fig. 1). A further reduction in photosite size to 1 µm would yield
a more modest additional improvement in resolution, to 203 lp/mm, indicating that increases in
resolution due to higher sampling rate are being opposed by f/4 diffraction blur. On the other
hand, if we could produce a perfect f/2.8 lens, 1 µm photosites would yield 281 lp/mm, and 2 µm
photosites would give 227 lp/mm with 50% contrast. Both are substantial improvements over
the f/4 case, although for a full frame sensor, 2 µm photosites would mean 216 MP (Table 1). If
we decide that the smallest practicable photosites for a full-frame sensor are 3 µm, then there is
negligible resolution benefit to increasing aperture to f/2.8 from f/4 — 161 vs 154 lp/mm with
50% contrast.
11 These calculations are based on resolutions (lp/mm) presented in Table 2 of Limits of Resolution. 3.
Diffraction and Photosite Size.
10
© 2015 Phil Service ([email protected]) Last revised: 6 July 2015
Whether or not a full-frame sensor, for example, can make maximal use of 3 or 2 µm
photosites will depend on reasonably-priced lenses that are up to the task. As I write this (July,
2015) the highest resolution full-frame sensor currently available is the 50 MP sensor in the
Canon 5DS and 5DS R, with 4.1 µm photosite spacing. Presumably, some careful testing will
tell us if any current lenses can extract the full potential of that sensor. The Nyquist limit
resolution of the Canon sensor is about 120 lp/mm or 2,880 LP/PH — theoretically achievable
with 50% contrast by a perfect lens at f/4.
11
© 2015 Phil Service ([email protected]) Last revised: 6 July 2015
4.5. Conclusion
Given that 2.4 µm photosite spacing is already being used with 1” sensors, it seems
reasonable to assume that 2 - 3 µm photosites will eventually be used for APS-C and “full-
12This is a direct consequence of the fact that the number of photons arriving at each photosite is
Poisson distributed. For more details see this page at ClarkVision.
13 Service, Phil. 2015. Limits of Resolution. 4. Image Capture for Maximum Detail Printing.
14Assuming, of course, that image blur is controlled well-enough to take advantage of the “oversampling”
sensor.
12
© 2015 Phil Service ([email protected]) Last revised: 6 July 2015
frame” sensors, implying total resolutions >100 MP. In order for such sensors to perform close
to their theoretical resolution limits, it may be necessary to sacrifice lens “speed” in order to
optimize sharpness at f/2.8 - f/4. Even if such sensors do not achieve their theoretical resolution
limits — perhaps because the necessary lenses are not available — they may still provide
advantages over current 24 – 50 MP sensors. In particular, most >100 MP images will be down-
sampled for “end use”. I suggest that such down-sampling may result in a sharper and less
“noisy” final image than would otherwise be obtained with lower resolution sensors.
13