Rethinking the Process of Vision

This explanation of light interaction with the retina of the eye makes it absolutely (actually geometrically!) clear that this surface evolved to detect, and does so detect, the 2-D Fourier transform of the perceived visual image. It does not directly detect the “camera photograph” that has been assumed for so long in the history of vision science. The retina is not an array of “red, green, and blue dots” as characteristic of color photographic film or the silicon imaging chips of contemporary digital cameras. The historically measured plan (Osterberg and following) of cone and rod receptors on the retinal surface doesn’t even look like this but the field persists in ignoring these data!

I have tried to explain in this work the nature of a 2-D Fourier transform in non-mathematical terms but have invariably found that the mere mention of the term “Fourier” causes the listener to nod off! I believe that this situation represents a fundamental problem for vision science. Matters “Fourier” are generally the province of the disciplines of electrical engineering and/or image processing who articulate the subject in mathematical terms not generally accessible to biologists, geneticists, etc. It seems that this is a “cross-disciplinary problem and that these EE groups must increase their involvement in the vision field.

The “Fourier equation” that describes the imaging process defines that light detection at the “Fourier plane” involves two terms that correspond to light intensity and phase detected at each point (or “pixel”) of the visual image. This plane corresponds to the focal plane of an imaging system and not the intensity-only sensitive image plane used in camera etc. Even our most advanced imaging technologies – both film and digital imaging – are capable of detecting and processing only the intensity of light falling on the imaging surface. We do not possess imaging technology that can simultaneously detect both light intensity and phase.

But the eye does!

I have attempted to describe in non-mathematical terms that detection of light phase at the retina corresponds to the direction of the impinging light ray with this factor, in addition to detection of light intensity at that point hoping that this would lead one’s mind to how the eye reconstructs the visual image. But, the concept of 2-D Fourier imaging remains difficult. A quote from Brian Hagan’s brilliant paper (“The Mathematical Transformation of Growth and Form”, Medical Hypotheses, 6, 559-609, 1980) under the heading Dissimilarity Between the Object and It’s Transform, “Under Fourier transformation, the big becomes small and vice versa…..”.

I suppose that the oft referenced attempt to portray the Fourier transformation of 2-D images is Kevin Cowtan’s Book of Fourier. These are beautiful illustrations of transformations that use color to encode light phase. They give some general idea of how Fourier transformations appear and should be studied.

A perhaps even better idea might be gained by viewing the images of a site authored by John Brayer of the University of New Mexico http://www.cs.unm.edu/~brayer/vision/fourier.html. If you wish, disregard the mathematical formulas at the beginning and scroll down to the images and their transforms. A quote from the site: “Now, with the above introduction, the best way to become familiar with Fourier Transforms is to see lots of images and lots of their FTs”. I would suggest reading this work. The analogy with a “radio station” is particularly good. It should become clear in viewing the images and their transforms how the transforms appear to relate to the retinal response posited in this work. The intensity of the central “spot” corresponds to the response of the fovea (solely at long wavelengths) and “sets” the overall brightness of the image for the visual system.

To conclude with two more thoughts: 1.) it seems clear that the retina processes color (in the manner described by Land) deriving this information from an “overlay” on the retinal surface of the spatial frequency 2-D Fourier transform. This is an exciting thought as it may portend entirely new technological imaging modalities that for the first time duplicate the process of vision, and 2.) I have described elsewhere in this work how the inter-receptor light detection devices of the retina, that are the basis of this explanation, simultaneously possess the ability to detect both the intensity and phase of incident light.

GCH,

Tucson, AZ

11/13/07

The surface of the retina of the eye extending from the fovea to the peripheral retina detects (or is “dimensionally tuned to”) only three wavelengths that correspond to what Young over a hundred years ago discerned to be the “primary” colors determining  that vision was trichromatic. In physics terms the light detection sites on the retinal surface that accomplish this are the center-to-center spaces between adjacent receptors where light is absorbed as the wave of classical physics. The lateral dimension of cone and rod receptors then has a spatial, dimension-controlling function. In physics terms these sites act as “quantum-confined electron” spaces. Light interaction with the retina can be viewed either as a classical or quantum event but it is clear that the eye fundamentally evolved to detect light as a wave. The function of the various rhodopsin complexes within receptors is to act as dimensional spacings in the process of conducting wave absorbed light energy to the signal-producing retinal molecule within the body of receptors.

It is then seen that the retina should be viewed abstractly as an array of quantum confined electron sites whose spacing has been determined by an elemental rule of geometry.

The three absorbed wavelengths to which the retina is sensitive are seen to have been refracted by the lens of the eye in an effect that has historically been termed an optical “aberration” – precisely “longitudinal chromatic aberration”. This is not arm waving!  The geometrically defined light detection pattern on the retina defined by this work  finally explains  Osterberg’s historic 1935 measurements of the distribution of cone and rod receptors on the retinal surface. The long held view that it is different pigments within receptors that determines wavelength discrimination is shown not to be valid. No “laboratory spectroscopy” concept need be conjured up – nature employs the simplest principle of geometry to code for the refraction of light through a condensing lens. One can almost imagine at some primordial time the evolution of the eye as the result of solar radiation transmitted through a drop of water in contact with self-assembling polar long carbon chain molecules – and, over eons of time, this fundamental retinal structure evolves!

 The eye geometrically encodes the physical laws of the refraction of light - nothing more!

(The referenced geometric principle can be stated as simply as “an admixture of circles of two diameters results in three center-to-center distances”. It is these three distances that correspond to the primary colors. On the retina these two circles correspond to the diameter of  the cones and rods, and, moreover, the ratio of their diameters of 1.8:1 corresponds to the bandwidth of visual response).

This explanation also asserts that the precise long (R) and short (B) wavelength limits of visual response are determined by the lateral cone-to-cone and rod-to-rod dimensions. But more importantly, the concept provides a geometrically determined center (G) wavelength determined by the cone-to-rod dimension. This is the fixed reference point that Edwin Land brilliantly deduced from external color vision measurement must be present. The many hues of color that we see will be found to result from the eye comparing intensities (Land termed them “lightnesses”) on either side of this fixed reference. It can now be seen at last that Land’s color vision theory is valid!

Additionally it is shown that these light responsive retinal outer segments detect both the traditionally assumed intensity of these three predetermined wavelengths but also, and crucially, light phase information. What the three geometrically determined retinal receptor regions actually encode are intensity and phase information of these three predetermined wavelengths or primary colors.

The ability of each receptor apposition light detection site on the retina to detect both light intensity and phase proves that the Fourier equation is operative, a finding that has fundamental meaning to understanding of vision. The diffractometric (and not “photographic film-like”) retina is then actually the focal or Fourier and not the intensity sensitive “image” plane of the optics of the eye. Processing of the visual image therefore involves solution of a 2-D Fourier transformation by the underlying retina/brain interface.

I believe that the rod receptors in the peripheral retina (i.e., beyond approximately twenty degrees of retinal angle), in addition to providing high spatial frequency information for the Fourier transform of the visual image, act in parallel as the “wide angle light meter” of the eye. This integrated signal will be found to control mechanical constriction of the pupil that is the fundamental light-limiting element of the eye. Again, the rod-rod apposition distance in the peripheral retina represents the absolute short wavelength limit of visual response. This wavelength at 400 nanometers is very close to the biologically-damaging UV region. An individual having smaller diameter rods (genetically determined) would potentially allow these shorter wavelengths to enter the eye – that might lead to macular degeneration?

There are many aspects of the vision process that have never been adequately explained beyond the glaring misinterpretation noted above of the asymmetric distribution of receptors on the retina. Perhaps the most important is providing an understanding as to how vision is capable of detecting single light quanta (photons). This represents an extraordinary capability that is not currently possible in the field of photonics even using the most advanced methods that require either cryogenic cooling of the detector or use of very high electronic values of electronic amplification to detect single quanta. Neither of these is apparent in the biological eye that goes these methods one better in detecting single quanta at elevated body temperature! I have proposed that, following the realization that the eye is composed of approximately one hundred million individual receptors per square centimeter, an assumption that each retinal light detection “device” (or “receptor apposition”) acts independently leads to an understanding. These nanometer dimensioned structures operating in the very fast (femtosecond or 10^-15 second)) time domain that seems never before to have been considered is the factor that reduces signal-obscuring electronic noise. Taken together, this posits that the process of vision really lies in the spatio-temporal domain of quantum physics and any further insights into the process are going to require application of this disciple of physics.

I would refer the reader to a comment that I have previously posted On the Quantum Limit Sensitivity of the Eye that discusses in more detail the above ideas.

GCH

11/12/07

I had been aware of the low density of cone receptors extending to the peripheral retina but did not give them much thought other than thinking that their function might be to slightly alter (due to their low density) the blue response that I had seen for this region. In retrospect, however, I see that this cannot be their function and that they are probably involved in detecting the high spatial frequency part of the 2-D Fourier transform that the retina encodes. As we develop the details of how the eye performs this transform the role of these receptors will be clarified.

Details about these peripheral cones abstracted from a web site:

“Although peripheral cone density falls as low as 4000/sq mm, versus
200 K/sq mm for the maximum rod density, it is important to realize
that the cone diameter at these eccentricities is as large as 5 - 9 um.
The cone inner segments therefore occupy as much as 1/3 of the area
of peripheral retina, and catch as much as 1/3 of the light captured by
the two types of peripheral photoreceptors.” (underline is mine)

It seems obvious that these cones probably play a more significant role than I had thought by virtue of their capturing a great deal of light in the peripheral retina. Note that the large 5-9 micron dimension refers to the inner segments and not the light-detecting outer segments of these cones. I do have (but will find) the size of the important outer segment then being able to assign a wavelength response to the centers. This response, however, must lie at short wavelengths as these are the only wavelengths that are refracted to these high retinal angles.

Recall that I had proposed that the peripheral retina acts as a “wide angle light meter” controlling the (mechanical) constriction of the pupil and thus light entrance into the eye. There seems to experimental evidence supporting this proposal noting that receptors in the peripheral retina are “ganged together” in parallel. And too, that I propose that it is the physical diameter of the rods, and thus their characteristic rod-to-rod appositional distance, that determines the absolute short wavelength of visual response. (If, for example, one has genetically-determined , slightly smaller rod receptors one ventures into the biologically-damaging ultraviolet region – not good!

Knowledge of sub-retinal “circuitry” of the peripheral retina (that undoubtedly exists) will probably shed more light on this subject.

GCH

11/07/07

Tucson, AZ

Vision science must stop thinking of the retina of the eye as the image plane of a camera or as the analogue of photographic film or the array structures used by contemporary silicon (CCD or CMOS) imaging chips!

A condensing lens (as  in the eye )  has two  image planes.  The nearest is the focal or Fourier plane where image information is encoded in two terms as light intensity and phase, and, the “image” plane that is sensitive only to light intensity. The latter is the point where film is placed in a camera and forms the basis for most of contemporary imaging technology. The eye, however, evolved to image in the Fourier domain that never seems to have been recognized in a hundred years of vision science

See the retina as the traditional, intensity-only sensitive, image plane  is  fundamentally at variance with the historical measurements  of retinal morphology made by Osterberg in  1935  (recognized and quoted ad infinitum  in the literature of vision) of the spatial arrangement of the rod and cone receptors of the retina. It is just  illogical to persist in this thinking.

It is seen from this work that  from a simple geometrical premise the retina is a diffractive array  of receptors that is demonstrably the focal or Fourier plane of the optics of the eye. Further, I have proposed retinal devices that are capable of encoding  both light intensity and phase as required by the Fourier equation.

The  diffractive pattern of light detection on the retina is  not the uniform array of “red/green/blue detection centers” that would be required by film  Moreover, it is  clear that this pattern  derives (or evolved) directly from the chromatic aberration of the lens of the eye where different wavelengths of light are refracted through different paths within the lens to different positions on the focal or Fourier plane behind the lens. THIS WORK DEFINES WITH GEOMETRICAL PRECISION WHERE THE WAVELENGTHS OF LIGHT WITHIN THE VISUAL BAND FALL ON THE RETINAL SURFACE FROM THE FOVEA TO THE PERIPHERAL RETINA.

The vision process is in the “Fourier” domain and not the “intensity only” domain of the film camera. This means that the individual light detection centers of the retina possess the capability for encoding both light intensity and phase as required by the Fourier equation. Phase encoding implies that the direction of the impinging light ray is retained and can be used eventually to process the image. In a sense the Fourier plane is the most efficient location to encode the image – a property that nature would certainly make use of.

Moreover, the plan of light interaction on the retina undoubtedly directly reflects the, heretofore termed chromatic “aberration”, of the solar spectrum transmitted through a condensing lens. It becomes clear (at least to me) that the evolution of the eye derives beginning somewhere way back there when a combination of sunlight incident on that first condensing lens (a drop of water?) and the molecular self-organizing character of some polar long chain hydrocarbon molecules that happened to be in the vicinity. All that was needed beyond that were the physical laws of light refraction…and…in due course of a long long time…….the eye and vision!

This explanation opens a great many new vistas in vision science. Perhaps a critical insight is that an exact mid-point of the visual band is defined on the retinal surface is geometrically-defined! This would correspond to 550 nm if the band is assumed to extend from 400 to 700 nm. No “laboratory spectrometer” is needed. This wavelength – defined as the “green” primary color – is determined by the geometric ratio of the size of the cones and rods. It provides the genetically determined (through the sizes of cones and rods) “universal reference standard” where all of us “see the same green”. This fixed reference is also used in retinal computation to determine the many hues of color that we see. The existence of such a fixed reference was presciently and brilliantly seen by Edwin Land from measurements made external to the eye. He termed it the “fulcrum” of retinal response.

Beyond this, the work provides an explanation for the remarkable (and never explained) ability of the eye to sense single quanta (or photons). I have written about this – see previous Comments. In summary, the vision process, at the point of light interaction with the retina, will be seen to lie in the spatial and temporal domains of quantum physics. Details cannot be presently seen but the beginnings of an understanding may lie in the recent Comment proposing the formation of standing waves within retinal receptors (or, alternatively, within the inter receptor devices that I propose) involving the frequency of light. I would note, again as I have written, that any fast, quantum processes on the retina are probably slowed to “human nerve scale” by the lengthy transmission of image information through the lengthy optic nerve.

I have been reading “The Silicon Eye” by George Gilder (Norton, 2005) recounting developmental work on silicon imaging chips that are based to some extent at least on replicating the light response of the human retina. I will have more to say about this shortly. It seems, however, that the fundamental basis of the effort is the assumption, again, that the retina is the analogue of photographic film, i.e., intensity-only detection! Beyond this, the work aims at besting contemporary silicon imaging chips by designing each single light receptive pixel space to process the three primary RGB colors instead of the separate R, G, and B, spaces employed by contemporary imaging (CCD or CMOS) chips (“that waste two thirds of the incident light”). The impression is given that these new “single RGB sensitive pixel” imaging methods approach the behavior of the human eye. I would propose that this is not nearly the case.

The “Fourier retina” of this work is divided into precisely three RGB wavelength-tuned regions each of which is composed of an array of single devices that are capable of detecting, and processing a usable electronic signal from, single quanta interactions. I have proposed the structure of, and the physics underlying, these devices. This, in turn, implies that the quantum efficiency within each region is very high – near unity. Moreover, the density of individual receptors in these arrays approximates one hundred million per square centimeter, i.e., one hundred “megapixels”. I think that we have much further to go before we replicate the biological retina on silicon – but I know the first steps to take!

GCH

Tucson, AZ

11/4/07