Jul 03 2011

Computational Photography

Lightfielddemo

Lytro, Inc, a technology spin-off company founded by Ren Ng, have been in the news recently with their announcement of a re-focusable camera: take one “image”, and change where the focal plane lies after the fact. This is illustrated in the images above, generated from a single shot from the prototype camera. As you move from left to right across this sequence you can see the focus shifting from the front left of image to back right.I saw this work a few years ago at the mighty SIGGRAPH conference, it comes out of a relatively new field of “computational photography”.

All photography is computational to a degree. In the past the computation was done using lenses and chemicals, different chemical mixes and processing times led to different colour effects in the final image. Nowadays we can do things digitally, or in new combinations of physical and digital.

These days your digital camera will already be doing significant computation on any image. The CCD sensor in a camera is fundamentally a photon-counting device – it doesn’t know anything about colour. Colour is obtained by putting a Bayer mask over the sensor, a cunning array of red, green and blue filters. It requires computation to unravel the effect of this filter array to make a colour image. Your camera will also make a white balance correction to take account of lighting colour. Finally, the manufacturer may apply image sharpening and colour enhancement, since colour is a remarkably complex thing there are a range of choices about how to present measured colours. These days compact cameras often come with face recognition, a further level of computation.

The Lytro system works by placing a microlens array in the optical train, the prototype device (described here) used a 296×296 array of lenses focusing onto a 16 million pixel medium format CCD chip, just short 40mmx40mm in size. The array of microlenses means means that for each pixel on the sensor you can work out the direction in which it was travelling, rather than just where it landed. For this reason this type of photography is sometimes called 4D or light-field photography. The 4 dimensions are the 2 dimensions locating where on the sensor the photon lands, and the direction in which it travels, described by another two dimensions. Once you have this truckload of data you can start doing neat tricks, such as changing the aperture and focal position of the displayed image, you can even shift the image viewpoint.

As well as refocusing there are also potentially benefits in being able to take images before accurate autofocus is achieved and then using computation to recover a focused image.

The work leading to Lytro was done by Ren Ng in Marc Levoy’s group at Stanford, home of the Stanford Multi-Camera Array: dispense with all that fiddly microlens stuff: just strap together 100 separate digital video cameras! This area can also result in terrible things being done to innocent cameras, for example in this work on deblurring images by fluttering the shutter, half a camera has been hacked off! Those involved have recognized this propensity and created the FrankenCamera.

Another example of computational photography is in high dynamic range imaging, normal digital images are acquired in a limited dynamic range: the ratio of the brightest thing they can show to the darkest thing they can show in a single image. The way around this is to take multiple images with different exposures and then combine together. This seems to lead, rather often, to some rather “over cooked” shots. However, this is a function of taste, fundamentally there is nothing wrong with this technique. The reason that such processing occurs is that although we can capture very high dynamic range images, displaying them is tricky so we have to look for techniques to squish the range down for viewing. There’s more on high dynamic range imaging here on the Cambridge in Colour website, which I recommend for good descriptions of all manner of things relating to photography.

I’m not sure whether the Lytro camera will be a commercial success. Users of mass market cameras are not typically using the type of depth-of-field effect shown at the top of the post (and repeated ad nauseum on the Lytro website). However, the system does offer other benefits, and it may be that ultimately it ends up in cameras without us really being aware of it. It’s possible Lytro will never make a camera, but instead license the technology to the big players like Canon, Panasonic or Nikon. As it stands we are part way through the journey from research demo to product.

Jun 28 2011

Book review: Map of a Nation by Rachel Hewitt

ordnance“Map of a Nation” by Rachel Hewitt is the story of the Ordnance Survey from its conception following the Jacobite Uprising in Scotland in 1745 to the completion of the First Series maps in 1870. As such it interlinks heavily with previous posts I have made concerning the French meridian survey, Maskelyne’s measurements of the weight of the earth at Schiehallion, Joseph Banks at the Royal Society, William Smith’s geological map of Britain and Gerard Mercator.

The core of the Ordnance Survey’s work was the Triangulation Survey, the construction of a set of triangles across the landscape made by observing the angles between landmarks (or triangulation points) ultimately converted to distances. This process had been invented in the 16th century, however it had been slow to catch on since it was slow and required specialist equipment and knowledge. Chromatic abberration in telescopes was also a factor – if your target is surrounded with multi-colour shadows – which one do you pick to measure? The triangles are large, up to tens of miles along a side, so within these triangles the Interior Survey was made which details the actual features on the ground – tied down by the overarching Triangulation Survey.

A second component of this survey is the baseline measurement – a precise measurement of the length of one side of one triangle made, to put it crudely, by placing rulers end to end over a straight between the terminal triangulation points.

The Triangulation Survey is in contrast to “route” or “transverse” surveys which measure distances along roads by means of a surveyor’s wheel, note significant points along the roadside. There is scope for errors in location to propagate. Some idea of the problem can be gained from this 1734 map showing an overlay of six “pre-triangulation” maps of Scotland, the coastline is all over the place – with discrepancies of 20 miles or so in places.

The motivation for the Ordnance Survey mapping is complex. Its origins were with David Watson in the poorly mapped Scotland of the early part of the 18th century, and the Board of Ordnance – a branch of the military concerned with logistics. There was also a degree of competition with the French, who had completed their triangulation survey for the Carte de Cassini and were in the process of conducting the meridian survey to define the metre. The survey of England and Wales was completed after the Irish Triangulation and after the Great Trigonometric Survey of India – both the result of more pressing military and administrative needs. As the survey developed in England more and more uses were found for it. Indeed late in the process the Poor Law Commission were demanding maps of even higher resolution than those the Ordnance Survey initially proved, in order to provide better sanitation in cities.

The Survey captured popular imagination, the measurements of the baseline at Hounslow Heath were a popular attraction. This quantitative surveying was also in the spirit of the Enlightenment. There was significant involvement of the Royal Society via its president, Joseph Banks, and reports on progress were regularly published through the Society. Over the years after the foundation of the Ordnance Survey in 1791 accurate surveying for canals and railways was to become very important. In the period before the founding of the Ordnance Survey surveying was a skill, related to mathematics, which a gentleman was supposed to possess and perhaps apply to establishing the contents of his estate.

Borda’s repeating circle, used in the French meridian survey to measure angles, found its counterpart in Jesse Ramsden’s “Great Theodolite“, a delicate instrument 3 feet across and weighing 200lbs. The interaction with the French through the surveying of Britain is intriguing. Prior to the French Revolution a joint triangulation survey had been conducted to establish exactly the distance between the Paris and Greenwich meridians, with the two instruments pitted against each other. There was only a 7 foot discrepancy in the 26 miles the two teams measured by triangulation between Dover and Calais. In 1817, less than two years after the Battle of Waterloo a Frenchman, Jean-Baptiste Biot, was in the Shetlands with an English survey team extending the meridian measurements in the United Kingdom.

The accuracy achieved in the survey was impressive, only one baseline measurement is absolutely required to convert the angular distances in the triangulation survey into distances but typically other baselines are measured as a check. The primary baseline for the Triangulation Survey was measured at Hounslow Heath, a second baseline measured at Romney Marsh showed a discrepancy of only 4.5 inches in 28532.92 feet, a further baseline measured at Lough Foyle, in Northern Ireland found a discrepancy of less than 5 inches in 41,640.8873 feet.

The leaders of the Ordnance Survey were somewhat prone to distraction by the terrain they surveyed across, William Roy, for example, wrote on the Roman antiquities of Scotland. Whilst Thomas Colby started on a rather large survey of the life and history of Ireland. Alongside these real distractions were the more practical problems of the naming of places: toponymy, particularly difficult in Wales and Ireland where the surveyors did not share the language of the natives.

Overall a fine book containing a blend of the characters involved in the process, the context of the time, the technical details and an obvious passion for maps.

Footnotes

In writing this blog post I came across some interesting resources:

Jun 28 2011

Universities and knowledge

The Higher Education White Paper is published today, in common with all other commentators in this area I have not read it either. One thing which seems to have attracted comment is the idea that there should be a market in higher education. The academics don’t seem to approve.

But knowledge doesn’t belong to universities. Universities provide qualifications, accreditation, and they provide personalised teaching.

For many students, such as myself 20 years ago, a university education was a given: it was the middle class way of easing myself out of the parental home and the gateway to the career I have now – first as an academic and now as an industrial research scientist. It was available to a relatively small fraction of the population. Things have changed now, increasingly university is seen as the gateway to most careers. Students do not go to university for the love of knowledge, they go because they must to get the careers they want. Pragmatically many careers do not require three years of post-18 education but we are manoeuvring ourselves in to a position where we say they must.

Students will no doubt see themselves in a market – even before this white paper they were being asked to commit significant future income in paying for three years of education, they are foregoing three years of paying work for the promise of a better future. If I were a student I’d be a bit peeved that the university sector were not at least showing willing in making that burden lighter.

Universities don’t give us knowledge – that’s down to us as individuals to hunt out, universities give us the tools to do that and the bit of paper that says we can do that.

Jun 26 2011

The Weekly Rage

Every week I listen to the Sunday programme on Radio 4, largely through inertia. Most weeks it manages to wind me up. I was a bit worried that I may be repeating myself here, so regular is the rage that I thought I must have written about it before. It turns out I have, but on a different topic.

The specific cause of my ire this week is the Church of England, the Equalities act and the inadmissibility of gay bishops. Forced by the Equality Act 2010 the Church has sought legal advice on how it should treat its gay clergy, it turns out they think that they may be obliged to accept gay bishops but that they can demand that they are celibate. You can read the BBC report here.

Why should this concern me, as a British atheist? Several reasons:

  • the Church of England is an established church, it takes (unelected) part in our legislation through the Lords Spiritual, it has a special position in teaching our children;
  • the Church of England claims moral authority, it specifically claims that it’s views on morality are superior to mine because they are faith-based. See the Bishop of Oxford’s comments this week on the Today programme;
  • I am ethnically Christian and English, so their position reflects badly on me;
  • the church’s position puts us all on shaky ground when we argue against inequality in other communities.

The Church could take a principled position that any group should be able to follow it’s faith: that the BNP should be allowed to exclude non-Caucasians from their number, for example. It could take the principled position that it should be subject to the same laws as the rest of us, without exemptions. It choses to do neither of these things, it choses instead to lobby for exemptions from the law and work out the minimum they can get away with in complying with that watered-down law.

What is the Church trying to tell us through this position? That the gays are OK, but not for them and not for positions of power?

Can you imagine a company, such as the one that I work for, demanding of it’s employee’s that they not only reveal their sexual orientation but also their sexual activity and if they confessed to the wrong sort of sexual activity they should be denied promotion?

Jun 25 2011

“Ridiculously long vacations”?

Lord Adonis, former education minister, is reported here as saying universities should:

…just abandoning these ridiculously long vacations … That only really makes sense as far as I can see if you want to travel the world or you need to get a job…

This is to misunderstand what happens during the long university vacation – the teaching staff, who are also research staff are getting on with doing research or, more painfully, trying to get funding for research. His point is not entirely without merit: universities have a distinctly schizophrenic attitude to teaching. If, as I have, you have applied for a number of lectureship positions you will learn that the time in interview dedicated to discussing your teaching experience, aspirations and ideas is approaching zero. Status in a university department depends largely on your research achievements, not your teaching achievements. This means there is scope in the market for universities that make teaching their priority, rather than research.