May 13 2018

Book review: Other minds by Peter Godfrey-Smith

other_mindsOther Minds by Peter Godfrey-Smith is about consciousness through the lens of cephalopods, a group comprising octopuses, cuttlefish, squid and the nautiluses.

Godfrey-Smith is a philosopher, rather than a scientist. This reflects the theme of the book, Godfrey-Smith’s idea is to understand our consciousness by looking at a creature with as radically different a consciousness as he can find. So his book is more a philosophical rather than a scientific view on consciousness which to my mind is no bad thing.

I was particularly impressed by Charles the octopus, one of three which experimenters used to try to measure octopus intelligence. The other two octopuses in their study put in some effort to carry out the tests presented whilst Charles insisted on squirting water at the experimenters and being otherwise uncooperative. It does make you wonder whether measures of animal intelligence are more a combination of willingness and intelligence. Perhaps the smarter animals just can’t be doing with intelligence tests. Squirting water at lights in aquariums to put them out seems, apocryphally, to be quite common behaviour amongst octopuses.

Mrs H is undertaking a doctorate in education and it strikes me that her mode of doing research, qualitative with relatively few subjects has more in common with this type of study than the experimenters care to admit. In this type of experiment the narrative rather than the result may be more important. So measuring the intelligence of the octopuses, or the time it took them to complete a learning task, is less important than the narrative of how they performed in the experiment.

The book weaves together cephalopod biology with Godfrey-Smiths own observations of cephalopods in the wild. I was intrigued to learn that the octopus brain is wrapped around the digestive tract which has been observed to lead to problems when attempting to eat particularly spikey foods. More generally, the neural material of an octopus is not all to be found in the “brain” it is distributed around the body. Octopus legs appear to have a degree of autonomy in at least deciding how to achieve a goal, even if the “brain” decides what the end goal might be.

Cuttlefish and octopuses have amazing abilities to change their appearance both in terms of colour, but also physical texture – they can make themselves bumpy. Interestingly, as far as we can tell they are unable to see the colours they produce – the photoreceptors in their eyes are of a single type. However, there is clearly something else going on – photoreceptors are found throughout the body of the cephalopod, and they are able to match background colours exquisitely. Furthermore, with the chromatophore and other colour producing structures also present a different mechanism for producing colour sensitivity is quite feasible – for example using those chromatophores as a filter “wheel” that sits in front of the light sensitive cells.

Godrey-Smith highlights that, although the cephalopods have a huge ability to make signals, because they are not a social species their ability to use those signals is limited. This is contrast to species such as baboons who have a more limited ability to generate vocal signals but, as a social species, have a much greater ability to interpret those signals in terms of establishing their position in a hierarchy and understanding how the hierarchy has changed.

Close to the end of the book, there is a terrible denouement: the typical cephalopod lifespan is only a couple of years. These creatures, so full of potential, are but brief inhabitants of the earth. Godfrey-Smith draws parallels with the Replicants in Bladerunner here. Their brief lives are understood in terms of their dangerous environment which has led to an evolutionary strategy of large broods of young, easily lost.

The book finishes with a discussion of “Octopolis” a location in Australia where octopuses, unusually, congregate and where Godfrey-Smith did a large part of his observations discussed throughout the book. Also we find here that he is involved in scientific publication.

Overall, one is left with the feeling of cephalopods having been an opportunity missed in the consciousness stakes. They have all the mental machinery but their truncated lives and limited social behaviour means that in all likelihood the opportunity is unfulfilled. This is a consciousness-centric human view, no species is striving for consciousness or intelligence they are doing what is needed for there species to continue in the niches they find themselves.

Apr 26 2018

Book review: Hands-on machine learning with scikit-learn & tensorflow by Aurélien Géron

machine-learningI’ve recently started playing around with recurrent neural networks and tensorflow which brought me to Hands-on machine learning with scikit-learn & tensorflow by Aurélien Géron, as a bonus it also includes material on scikit-learn which I’ve been using for a while.

The book is divided into two parts, the first, “Fundamentals of Machine Learning” focuses on the functionality which is found in the scikit-learn library. It starts with a big picture, running through the types of machine learning which exist (supervised / unsupervised, batched / online and instance / model) and then some of the pitfalls and problems with machine learning before a section on testing and validation. The next part is a medium sized example of machine learning in action which demonstrates how the functionality of scikit-learn can be quickly used to develop predictions of house prices in California based on census data. This is a subject after my own heart, I’ve been working property data in the UK for the past couple of years.

This example serves two purposes, firstly it demonstrates the practical steps you need to take when undertaking a machine learning exercise and secondly it highlights just how concisely much of it can be executed in scikit-learn. The following chapters then go into more depth first about how models are trained and scored and then going into the details of different algorithms such as Support Vector Machines and Decision Trees. This part finishes with a chapter on ensemble methods.

Although the chapters contain some maths, their strength is in the clear explanations of the methods described. I particularly liked the chapter on ensemble methods. They also demonstrate how consistent the scikit-learn library is in its interfaces. I knew that I could switch algorithms very easily with scikit-learn but I hadn’t fully appreciated how the libraries generally handled regression and multi-class classification so seamlessly.

I wonder whether outside data science it is perceived that data scientists write their own algorithms from scratch. In practical terms it is not the case, and hasn’t been the case since at least the early nineties when I started data analysis which looks very similar to the machine learning based analysis I do today. In those days we used the NAG numerical library, Numerical Recipes in FORTRAN and libraries developed by a very limited number of colleagues in the wider academic community (probably shared by email attachment).

The second part of the book, “Neural networks and Deep Learning”, looks at the tensorflow library. Tensorflow has general applications for processing multi-dimensional arrays but it has been designed with deep learning and neural networks in mind. This means there are a whole bunch of functions to generate and train neural networks of different types and different architectures.

The section starts with an overview of tensorflow with some references to other deep learning libraries, before providing an introduction to neural networks in general, which have been around quite a while now. Following this there is a section on training deep learning networks, and the importance of the form of activation functions.

Tensorflow will run across multiple processors, GPUs and/or servers although configuring this looks a bit complicated. Typically a neural network layer can’t be distributed over multiple processing units.

There then follow chapters on convolution neural networks (good for image applications), recurrent neural networks (good for sequence data), autoencoders (finding compact representations) and finally reinforcement learning (good for playing pac-man). My current interest is in recurrent neural networks, it was nice to see a brief description of all of the potential input/output scenarios for recurrent neural networks and how to build them.

I spent a few years doing conventional image analysis, convolution neural networks feel quite similar to the convolution filters I used then although they stack more layers (or filters) than are normally used in conventional image analysis. Furthermore, in conventional image analysis the kernels are typically handcrafted to perform certain tasks (such as detect horizontal or vertical edges), whilst neural networks learn their kernels in training. In conventional image analysis convolution is often done in Fourier space since it is more efficient and I see there experiments along these lines with convolution neural networks.

Developing and training neural networks has the air of an experimental science rather than a theoretical science. That’s to say that rather than thinking hard and coming up with an effective neural network and training scheme one needs to tinker with different designs and training methods and see how well they work. It has more the air of training an animal the programming a computer. There are a number of standard training / test sets of images and successful models trained against these by other people can be downloaded. Such models can be used as-is but alternatively just parts can be used.

This section has many references to the original literature for the development of deep learning, highlighting how recent this new iteration of neural networks is.

Overall an excellent book, scikit-learn and tensorflow are the go-to libraries for Python programmers wanting to do machine learning and deep learning respectively. This book describes their use eloquently, with references to original literature where appropriate whilst providing a good overview of both topics. The code used in the book can be found on github, as a set of Jupyter Notebooks.

Mar 18 2018

Book review: The Philosophical Breakfast Club by Laura J. Snyder

breakfastThe Philosophical Breakfast Club by Laura J. Snyder is an ensemble biography of William Whewell (pronounced: who-ell), Charles Babbage, Richard Jones and John Herschel who were all born towards the end of the 18th century and died in the later half of the 19th century. Their joint project was the professionalisation of science.

The pattern for their reform was Francis Bacon’s New Atlantis published in 1626 which fictionalised a government-funded science institution whose work was for the public good, and whose philosophical basis was the systematic collection of facts, including experimentation, from which scientific theories would be gleaned by induction. As such they follow in the footsteps of the founding fathers of the Royal Society who also took Bacon as their guiding light.

By the earl years of the 19th century the Royal Society had drifted in its purpose since its founding, it was more a gentlemen’s dining club than a scientific society with your position in society a more important factor than your scientific achievements in gaining entry.

Prior to reading this book I recognised the names of Whewell, famous for coining the term “scientist”, Herschel and Babbage – the former the son of William Herschel, the astronomer, the latter the inventor of the Difference Engine. I also knew that Herschel and Babbage had been involved in attempts to reform the Royal Society.

Richard Jones was unknown to me. His contributions were in the foundations, or at least building, of the field of economics. In particular he proposed an economics based on induction, that is to say one should go out and collect facts about the economy and from that point infer rules about the operation of economies from the data. The alternative is to hypothesis some simple rules, and elaborate the consequences of those rules – this is known as deduction. In economics Ricardo and Malthus had been early proponents of this deductive method. Jones went on to become one of the commissioners under the Tithe Commutation Act 1836 which converted the payments in kind of the old tithe system into what was effectively a local tax.

Babbage, Jones and Herschel all came from moderately wealthy backgrounds for whom the path to Cambridge University was relatively smooth. Whewell, on the other hand, was the son of a carpenter which although a respectable trade would not fund attendance at the university. Whewell was educated at a grammar school in Lancaster, his home town, as a result of being spotted by the local gentry who also smoothed his path into Cambridge. This appears to be the route by which the lower middle class entered university – chance encounters.

The four men met at Cambridge University where they formed the Philosophical Breakfast Club. It was at a time when gathering together and discussing politics was seen as borderline seditious. It was not long after the French revolution and the Great Reform Act was yet to come. They corresponded throughout the rest of their lives but there is no feeling from the book that their collaboration to change the face of science was at all formal (or even subject to an overall plan).

At Cambridge Babbage and Whewell were responsible for driving the use of Leibniz’s notation for calculus, in place of Newton’s notation to which the university had adhered for some time. Leibniz’s notation is the one in use today, generally it is seen as clearer than the Newtonian version and more amenable to extension.

Babbage post-Cambridge started work on mechanical computing, managing to extract large quantities of money from the government for this work, exceptional at the time, although he did not deliver a working device. The first Difference Engine was designed to calculate mathematical tables. The later Analytical Engine was very much like modern computers in its architecture. Neither of these devices were ever fully constructed. Babbage could best be described as a mathematician which put him into some conflict with others in the Breakfast Club since mathematics is rather more deductive than inductive in its basis. Later in his life he seems to have become involved in codebreaking, quite possibly for the government, although the evidence for this is circumstantial.

Babbage also led a ferocious attack on the Royal Society in his book Reflections on the decline of science in England. The British Association for the Advancement of Science (BAAS) followed on from this although Babbage, Herschel and Whewell did not attend its first meeting. The BAAS annual meetings became rather large, and there was muttering at the time about the attendees penchant for fine dining. Unlike the Royal Society, it was open to all, even women! I was interested to read about the foundation of my own professional society, the Royal Statistical Society. It started as a section of the British Association for the Advancement of Science where it proved contentious because it was concerned in the collection and analysis of social data which surely leads to politics. Babbage and Jones set up the London Statistical Society which was to become the Royal Statistical Society.

After Cambridge Herschel spent some time in South Africa measuring the location of stars in the southern skies, following on the family business. He became president of the Royal Astronomical Society and published several books on astronomy as well as star catalogues. As well as this he was involved in the development of photography, he was an enthusiastic chemical experimenter and appears to have guided Henry Fox Talbot in fixing his early photographic images.

Whewell remained at Cambridge University for the rest of his life, where he later became the Master of Trinity College. As well as his efforts in changing the teaching of calculus he introduced the Natural Sciences Tripos (parts of which I have taught). His publications were mainly in the history and philosophy of science. He was involved in some scientific endeavours – the measurement and analysis of the tides, for example. Although he coined the term “scientist” in 1833 it wasn’t to gain much currency until much later in the century.

Snyder identifies the period 1820-70 as one where there was a great transition in science from being a gentleman’s hobby to a (sort of) mass participation activity with at least some regard for practical application, a defined career path at least for a few and some more regular government funding.

I found The Philosophical Breakfast Club very readable. It covers a period of great transition in science in the UK, and makes a nice companion to Henrietta Heald’s biography of William Armstrong.

Feb 10 2018

Book review: William Armstrong–Magician of the North by Henrietta Heald

A return to industrial history with armstrongWilliam Armstrong: Magician of the North by Henrietta Armstrong. Armstrong was a 19th century industrialist who spent his life in the north-east of England around Newcastle. His great industrial innovation was the introduction of hydraulic power to cranes and the like. His great wealth, and honours (a knighthood and then a baronetcy) derived from his work in the invention and sale of armaments principally artillery and ships. His home, Cragside near Rothbury some 30 miles north of Newcastle upon Tyne, was the first to feature electric lighting amongst many other technical innovations.

Armstrong was a contemporary of Robert Stephenson, Isambard Kingdom Brunel and Joseph Whitworth – they were all born near the beginning of the 19th century, Armstrong dying in 1900 outlasted them all with Brunel and Robert Stephenson dying in 1859.

Armstrong was born in 1910 his parents started him on a career in the law. However, he had always been fascinated by water. This led to his realisation that the power that could be extracted from a head of water in a sealed system. A water wheel extracts energy from water falling the height of the wheel, a matter of a few metres. A sealed iron pipe, such as could now be manufactured allowed you to capture the energy from a fall of tens of metres or more. In Newcastle upon Tyne the local landscape could provide this head of pressure but with a little ingenuity the head of pressure could be created with a steam engine or other mechanical means. This energy could be used to drive all manner of machinery, Armstrong initially used it to power cranes, and lock gates, to be used in docks and the many factories springing up around the country. Ultimately his hydraulic mechanisms drove London’s Tower Bridge.

In the aftermath of the Crimean War, Armstrong switched his attention to building artillery. During the Crimean War the British artillery was found wanting in terms of accuracy, destructive power and firing rate. His innovations were to move from cannonballs to shells (shaped like bullets), and from muzzle loading to breech loading. He gave up the patents for his artillery pieces to the government but made a fair business on them. His activities with ordnance led to his knighthood and baronetcy although ultimately he withdraw from the close relationship with the British government in armaments as a result of political manoeuvrings by competitors.

The manufacture of artillery led to the manufacture of warships, which incidentally also carried the artillery. The Japanese Navy were particularly important.

He was a leading light of the Literary and Philosophical Society of Newcastle upon Tyne (Lit & Phil), and contributed to founding what is now Newcastle University. Late in his life, in 1897, he published Electric movement in air and water based on his experiments and featuring cutting-edge photographs of the phenomena he described. From a scientific point of view, Armstrong is not a name you will hear in physics classrooms (at any level) today – I don’t know if the same holds for his engineering innovations. Also late in his life he bought Bamburgh Castle, and spent a fair amount of money refurbishing it.

Magician of the North is a somewhat sympathetic view of Armstrong, along the lines of Man of Iron by Julian Glover about Thomas Telford. This contrasts with Samuel Smiles biography of George Stephenson and Rolt’s of Brunel which are much more effusive about their subjects. The Armstrong’s arms trading is discussed in some detail, it seems the company sailed somewhat close to the wind legally in supplying both sides in the American Civil War. A second blemish on Armstrong’s reputation came from industrial disputes with his, and other workers on the Tyne, asking for shorting working hours. That said, he was clearly a pillar of the Newcastle and north eastern community and highly regarded by most of the people most of the time. Many buildings in Newcastle bore his name as a result of his donations both whilst he was a live and after he died.

As usual the author of this biography bemoans the limited attention their subject has received. In the case of Armstrong they put this down to his extensive involvement in the arms trade which, never the most popular, was to fall further out of favour following the Great War. I’ve never seen a quantitative analysis of what makes the right amount of attention for figures in the history of science and technology.

William Armstrong died in 1900, after his death his company went into a slow decline. The Great War led to a distaste for the arms trade, and then came the Great Depression. With Armstrong gone there was no strong, capable leader for the company. The Armstrong name lived on in various spin off companies such as Armstrong Siddeley and various amalgamations with Whitworths and Vickers.

Jan 08 2018

Book reviews: Christmas Extravaganza!

I’ve given up writing my full length posts for my Christmas book haul which, I show below, was rather fine.This is in part a result of the type of books one gets for Christmas, and in part the conditions under which they are read – in a Christmas cake induced haze, for my part.

walking_thumb1A Philosopy of Walking by Frédéric Gros

A Philosopy of Walking by Frédéric Gros was, like the best presents, something I wouldn’t have got for myself but nevertheless enjoyed.

The book interleaves chapters on various walking related thoughts with some walking oriented biographical content. This covers Nietzsche, Rimbaud, Rousseau, Nerval, Kant and Ghandi. The predominant feeling from these biographies is a bit grim, several of the protagonists died young or after prolonged illness – Nerval committed suicide. Their walking in feels compulsive. Ghandi lived to a ripe old age but was ultimately assassinated. Kant took the same walk every day, he lived to 80 but it sounds pretty dull!

The chapter on pilgrimage struck a cord with me, I’ve been meditating for a while which often involves focusing on a mantra or physical manifestation, like breathing. Some pilgrims take a similar approach combining walking with a prayer-like mantra.

Somehow the author has missed our family favourite walking habit – humming the Imperial March from Star Wars as a rhythm to walk to along broad well-made paths in the Lake District.

The book is translated from French, I learned this on reading in a footnote that the French word témoin which I knew meant “witness” also refers to the baton in a relay race.

New views by Alastair Bonnett

new_viewsNew Views by Alastair Bonnett is a different manner of Christmas book, a coffee table book – as are the rest of the books in this post.

New Views  is a collection of world maps illustrating different data in three broad areas which could be described as physical, human and animal, and trade. The pattern is the same in  each case – a double page contains a map with key, and on the following double page is some text describing the context of the map and another, different graphic. The maps are very much on the global scale, cities may be mentioned here and there but the overwhelming impression is of the world as a whole, not individual countries.

I liked the map of lightning strikes which highlights odd areas, particularly in the east of the Democratic Republic of Congo which has the highest rate of lightning strikes in the world. The maps of amphibian and bird diversity are fun too – they map out features of the underlying geography like rivers and mountains but in different ways.

I was surprised to learn just how big an exporter of nuts the US was, I should have known this since I constant in my last job was the monthly scrape of the reports of the almond board of California for a customer. Also I learned that Brazil exports no Brazil nuts because they don’t grow there!

Sometimes the colour keys are a bit cryptic, that’s to say I couldn’t distinguish between two categories on the scale. On another map countries where there is no data are omitted completely which makes the map difficult to parse unless you have a photographic recall of the shapes of the countries of the world. I was puzzle to learn that the viper was the only poisonous snake in the United Kingdom – I always called them “adders”.

This is a creditable work of this genre.

Bird by Andrew Zuckerman

birdBird by Andrew Zuckerman is an immense book, comprised entirely of photographs of birds shown against a pure white background. There are a few words, and a pictorial index which names the birds, at the back but the main body of the text is completely wordless.

The pictures are gorgeous but I found myself wanting more having flicked through to the end of the book. The style is intentional and is contrasted with that of Audubon who included much more context in his famous paintings of birds. Technically the photographs are very good to excellent.

Zuckerman has produced a number of books in this style, I’m most interested to see his works on flowers and creatures.

Vermeer: The Complete Works by Karl Schütz

vermeerVermeer: The Complete Works by Karl Schütz. I was surprised to read that there are only 35 works attributed to Vermeer. This may be because he fell out of popularity after his death in the 17th century and interest was not revived until the 19th century.

Canaletto & the art of Venice by Rosie Razzall and Lucy Whitaker

canalettoCanaletto & the art of Venice by Rosie Razzall and Lucy Whitaker. The authors names are very discreetly displayed on this volume. I’m a fan of Canaletto – I love the almost CAD-like precision of his architectural paintings.