Dr Administrator
Author's posts
Oct 23 2013
A place in the country
This post was first published at ScraperWiki.
Recently Shelter came to us asking for data on house prices across the UK to help them with some research in support of campaign on housing affordability.
This is a challenge we’re well suited to address, in fact a large fraction of the ScraperWiki team have scraped property price data for our own purposes. Usually though we just scrape a local area, using the Zoopla API, but Shelter wanted the whole country. It would be possible to do the whole country by this route but rate-limiting would mean it took a few days. So we spoke nicely to Zoopla who generously lifted the rate-limiting for us, they were also very helpfully in responding to our questions about their API.
The raw data for this job amounted to 2 gigabytes, 34 pieces of information for each of 500,000 properties for sale in the UK in August 2013. The data tell us about the location, the sale price, the property details, the estate agent details and the price history of each property.
As usual in these situations we fired up Tableau to get a look at the data, Tableau is well-suited to this type of database-table shaped data and is responsive for this number of lines of data.
What sort of properties are we looking at here?
We can find out this information from the “property type” field, shown in the chart below which counts the number of properties for sale in each property type category. The most common category is “Detached”, followed by “Flat”.
We can also look at the number of bedrooms. Unsurprisingly the number of bedrooms peaks at about 3 but with significant numbers of properties with 4, 5 and 6 bedrooms. Beyond that there are various guest houses, investment properties, parcels of land for sale with nominal numbers of bedrooms culminating in a 150 bedroom “property” which actually sounds like a village.
What about prices?
This is where things get really interesting. Below is a chart of the number of properties for sale in each price £25k price “bin”, for example the bin marked 475k contains all of the houses priced between £475k and £499,950 – the next bin being labelled 500k containing houses priced from £500k to £525k. We can see that the plot here is jagged, the numbers of properties for sale in each bin does not vary smoothly as the price increases, it jumps up and down. In fact this effect is quite regular, for houses priced over £500k there are fewest for sale at the round numbers £500k, £600k etc most for sale at £575k, £675k and so forth.
But this doesn’t just effect the super-wealthy – if we zoom into the lower priced region, making our price bins only £1k there is a similar effect with prices ending 4,5 and 9,0 more frequent than those ending 1, 2, 3 or 6, 7, 8. This is all the psychology of pricing.
Distribution of prices around the country?
We can get a biased view of the population distribution simply by plotting all the property for sale locations. ‘Biased’ because, at the very least, varying economic conditions around the country will bias the number of properties for sale.
This looks about right, there are voids in the areas of the country which are sparsely populated such as Scotland, Wales, the Peak District and the Lake District.
Finally, we can look at how prices vary around the country – the map below shows the average house price in a region defined by the “outcode” – the first group of letters in a UK postcode. The colour of the points indicates the average price – darkest blue for the lowest average price (£40k) and darkest red for the highest average price (£500k). The size of the dots shows how many properties are for sale in that area.
I’m grateful to be living in the relatively inexpensive North West of England!
There’s plenty more things to look at in this data, for example – the frequency of street names around the UK and the words used by estate agents to describe properties but that is for another day.
That’s what we found – what would you do?
Oct 06 2013
Trento
Normally travel for work is a less than enjoyable experience but this week I’ve been to Trento*, and it was very pleasant. Sadly I only had my mobile phone to take photos.
Trento is in the north of Italy, the bit that is positively Germanic. The second language appears to be German and the cuisine is more alpine than pizza. It’s a small town with a substantial university. It sits in a broad, bottomed steep-sided valley an hour on the train from Verona on the line that heads up to Bolzano, the Brenner Pass and Austria. I’d not heard of Trento before, I’d heard of the Council of Trent (which refers to the Ecumenical Council of the Roman Catholic Church which was held in Trent between 1545 and 1563).
A short walk from the railway station and you are in the heart of the old city, narrow streets with marble pavements faced with buildings which in large part seem to date from the 16th century. The majority of the shops, restaurants and bars embedded in the lower floors are rather swish and classy looking.
The heart of the old town is the Piazza del Duomo, featuring the city’s cathedral, the fountain of Neptune and other fine buildings.
The cathedral seems to date to some time around the start of the 13th century. It’s been well conserved and the square itself is largely in character. The most similar British cities I know in terms of old architecture are probably Wells and Canterbury most other British cities either never had substantial buildings of such age, or they were replaced at some point since.
The fontana del nettune is quite blingy:
Next door to my hotel, Chiesa di Santa Maria Maggiore:
Alongside the ecclesiastical buildings are some fine townhouses. Romeo and Juliet was set in Verona, an hour down the railway line – I wonder if this is how the balcony Juliet stood in looked:
This is the Palazzo Quetta Alberti-Colico. There’s also the rather nice Palazzo Geremia (pdf)
On the edge of town, close to the bridge over the river, the Torre Vanga, erected originally in 1210, is palimpsest of masonry and brick.
And as well as that there are some impressive entrances:
Definitely worth a day trip if you are in the area, and great if you have business as the university!
*Unsurprisingly the Italian wikipedia entry is much more extensive.
Oct 05 2013
Book Review: Backroom Boys by Francis Spufford
Electronic books bring many advantages but for a lengthy journey to Trento a paper book seemed more convenient. So I returned to my shelves to pick up Backroom Boys: The Secret Return of the British Boffin by Francis Spufford.
I first read this book quite some time ago, it tells six short stories of British technical innovation. It is in the character of Empire of the Clouds and A computer called LEO. Perhaps a little nationalistic and regretful of opportunities lost.
The first of the stories is of the British space programme after the war, it starts with the disturbing picture of members of the British Interplanetary Society celebrating the fall of a V2 rocket in London. This leads on to a brief discussion of Blue Streak – Britain’s ICBM, scrapped in favour of the American Polaris missile system. As part of the Blue Streak programme a rocket named Black Knight was developed to test re-entry technology from this grew the Black Arrow – a rocket to put satellites into space.
In some ways Black Arrow was a small, white elephant from the start. The US had offered the British free satellite launches. Black Arrow was run on a shoestring budget, kept strictly as an extension of the Black Knight rocket and hence rather small. The motivation for this was nominally that it could be used to gain experience for the UK satellite industry and provide an independent launch system for the UK government, perhaps for things they wished to keep quiet. Ultimately it launched a single test satellite into space, still orbiting the earth now. However, it was too small to launch the useful satellites of the day and growing it would require complete redevelopment. The programme was cancelled in 1971.
Next up is Concorde, which could probably be better described as a large, white elephant. Developed in a joint Anglo-French programme into which the participants were mutually locked it burned money for nearly two decades before the British part was taken on by British Airways who used it to enhance the prestige of their brand. As a workhorse, commercial jet, it was poor choice: too small, too thirsty, and too loud.
But now for something more successful! Long ago there existed a home computer market in the UK, populated by many and various computers. First amongst these early machines was the BBC Micro. For which the first blockbuster game, Elite, was written by two Cambridge undergraduates (David Braben and Ian Bell). I played Elite in one of its later incarnations – on an Amstrad CPC464. Elite was a space trading and fighting game with revolutionary 3D wireframe graphics and complex gameplay. And it all fitted into 22kb – the absolute maximum memory available on the BBC Micro. The cunning required to build multiple universes in such a small space, and the battles to gain a byte here and a byte there to add another feature are alien to the modern programmers eyes. At the time Acornsoft were publishing quite a few games but Elite was something different: they’d paid for the development which took an unimaginable 18 months or so and when it was released there was a launch event at Alton Towers and the game came out in a large box stuffed with supporting material. All of this was a substantial break with the past. Ultimately the number of copies of Elite sold for the BBC Micro approximately matched the number of BBC Micros sold – an apparent market saturation.
Success continues with the story of Vodaphone – one of the first two players in the UK mobile phone market. The science here is in radio planning – choosing where to place your masts for optimal coverage, Vodaphone bought handsets from Panasonic and base stations from Ericsson. Interestingly Europe and the UK had a lead over the US in digital mobile networks – they agreed the GSM standard which gave instant access to a huge market. Whilst in the US 722 franchises were awarded with no common digital standard.
Moving out of the backroom a little is the story of the Human Genome Project, principally the period after Craig Venter announced he was going to sequence the human genome faster than the public effort then sell it! This effort was stymied by the Wellcome Trust who put a great deal further money into the public effort. Genetic research has a long history in the UK but the story here is one of industrial scale sequencing, quite different from conventional lab research and the power of the world’s second largest private research funder (the largest is currently the Bill & Melinda Gates Foundation).
The final chapter of the book is on the Beagle 2 Mars lander, built quickly, cheaply and with the huge enthusiasm and (unlikely) fund raising abilities of Colin Pillinger. Sadly, as the Epilogue records the lander became a high velocity impactor – nothing was heard from it after it left the Mars orbiter which had brought it from the Earth.
The theme for the book is the innate cunning of the British, but if there’s a lesson to be learnt it seems to be that thinking big is a benefit. Elite, the mobile phone network, the Human Genome Project were the successes from this book. Concorde was a technical wonder but an economic disaster. Black Arrow and Beagle 2 suffered from being done on a shoestring budget.
Overall I enjoyed the Backroom Boys, it reminded me of my childhood with Elite and the coming of the mobile phones. It’s more a celebration than a dispassionate view but there’s no harm in that.
Sep 27 2013
Book Review: Georgian London–Into the Streets by Lucy Inglis
I saw the gestation of Georgian London: Into the streets by Lucy Inglis, so now it is born – I had to buy it!
Lucy Inglis has been blogging about Georgian London for much of the last four years, and I have been reading since then. Her focus is the stories of everyday folk, little snippets from contemporary records surrounded by her extensive knowledge of the period.
The book starts with some scene settings, in particular the end of the Restoration (1660), the Plague (1665), the Great Fire of London (1666) and the Glorious Revolution (1688). These events shape the stage for the Georgian period which covers the years 1714 to 1837, named for the succession of King George’s who reigned through the period marked by the death of William IV (don’t ask me).
London is then covered geographically, using John Rocque’s rather fabulous 1746 map as ornamentation. What is obvious to even those such as myself who are broadly ignorant of the geography of London is how much smaller London was then. Areas such as Islington, which I consider to be in the heart of London were on the edge of the city at the time, rural locations with farming and so forth. The period saw a huge expansion in the city from a population of 500,000 at the beginning of the period to 1.5 million by 1831 which much of the growth occurring in the second half of the 18th century.
Georgian London is somewhat resistant to my usual style of “review” which involves combining the usual elements of review and a degree of summary to remind me of what I read. Essentially there is just too much going on for summarising to work! So I will try some sort of vague impressionistic views:
It struck me how the nature of poverty changed with urbanisation; prior to a move to the city the poor could rely to some extent on the support of their parish, moving to London broke these ties and, particularly for women supporting children, this led to destitution. Men could easily travel to find work, either back home or elsewhere – a women with a child couldn’t do this.
The role of the state was rather smaller than it is now, when the time came to build Westminster Bridge, there was no government funding but rather a series of lotteries. The prize for one of these was the Jernegan cistern, a wine container made from quarter of a ton of silver with a capacity of 60 gallons! Another indicator of the smaller size of the state was that in 1730 a quarter of state income was from tax on alcohol, much of it on gin. Currently alcohol duties account for about £10billion per year which is about 1.5% of the total government spending.
Businesswomen make regular appearances through the book, for example such as Elinor James who was the widow of a printer, Thomas James but published under her own name. She was both a speaker and a pamphleteer, working at the beginning of the 18th century. At the end of the century, the younger Eleanor Coade, was running a thriving business making artificial stone (Coade stone). She’d first come to London in 1769, with her mother, also Eleanor following the death of her father.
At the same time that a quarter of all government revenue came from alcohol duties, a quarter of all gin distillers were women. Alcohol caused many social problems, particularly in the second quarter of the 18th century, as recorded by Hogarth’s “Gin Line”. The vice of the upper classes in the second half of the 18th century was gambling.
The Tower of London housed exotic animals for many years, providing a money-raising visitor attraction through the Georgian period, only losing it status in 1835 on the creation of London Zoo in Regent’s Park. A few years earlier, in 1832, the Tower of London hosted 280 beasts of varying types but it was becoming clear it was an unsuitable location to keep animals. The British were also becoming more aware of animal cruelty, with animal baiting becoming less popular through the Georgian period – culminating with the Act to Prevent the Cruel and Improper Treatment in 1822, and the formation of the RSPCA a couple years later.
It seems useful to know that London’s first street numbers where introduced in 1708.
The voice of the book is spot-on, conversational but authoritative, providing colour without clumsiness. There are no footnotes but there are extensive notes at the end of the book, along with a bibliography. For someone trying to write a blog post like this, the index could do with extension!
It’s difficult to write a review of a book by someone you know, all I can say is that if I didn’t like it I would have not written this. Don’t just take it from me – see what the Sunday Sport thought!
Aug 31 2013
Pevsner Architectural Guide – Liverpool by Joseph Sharples
I reach Pevsner’s Architectural Guide – Liverpool (additional material Joseph Sharples) by a somewhat winding route: I take the Merseyrail train to Liverpool; my normal route is changed and I must walk across the city; it turns out the buildings are spectacular; I take photos and then I want to know what I have photoed. This is where the Pevsner Guide enters the picture.
Sir Nickolaus Pevsner was a German born art historian who moved to the UK in 1933, he felt that the academic study of architecture in Britain was lacking, and furthermore there were was no convenient source of information on the many and wondrous buildings of the country. In 1945 he proposed a series of books: Buildings of England to address this lack. The series ultimately ran to 46 volumes, 32 written by Pevsner, a further 10 which he co-authored and 4 written by others.
This Guide – Liverpool is a city specific reworking of the original guides. The book is large-pocket sized, well produced with a fair number of images. It starts with an overview of the history of Liverpool. I have to admit, shame-faced, that I was woefully ignorant of the city I now work in. For nearly 10 years I have lived just down the line in Chester, and yet I had visited Liverpool a handful of times, in the evening for works dinners. My perceptions of Liverpool are coloured by the time I grew up, in the 1980s, when Liverpool was host to riots in the Toxteth area of the city, mass unemployment and far-left politics. Walking around now what I see is completely at odds with my perceptions, you can see in my earlier post. To add some decoration to this post, below is the Royal Liver Building, one of the Three Graces, built on the waterfront at the beginning of the 20th century.
Royal Liver Building, wide angle view with perspective “correction” applied
And St George’s Hall:
St George’s Hall
Liverpool has long been Britain’s second port and probably has a strong claim for second city status (both following London). Initially it grew through exporting Cheshire salt, than as part of the triangle route carrying slaves, then as a point of exit for Britain’s manufactured goods and finally as a passenger terminus. Liverpool is not blessed with the best of conditions for shipping, this meant it was an early pioneer of gated docks. This was significant engineering work, only possible through the collective action of the city Corporation. It’s worth noting that one of the first railways in Britain was between Liverpool and Manchester, providing a link between the manufacturing centre and the port. Liverpool remained preeminent until the sixties when British manufacturing declined, and shipping became containerized, much reducing the labour required. It had no “second fiddle” so with the loss of shipping it went into rapid decline.
Nowadays Liverpool is making a resurgence, the fine buildings from its early high water mark are joined by some excellent new ones.
After a historical overview the Guide covers six major buildings / areas of the city: the Town Hall (dating to mid 18th century and the oldest major building in the city), St George’s Hall and the Plateau (up by Liverpool Lime Street Station); the William Brown Street Group and St. John’s Gardens; Pier Head where the outstanding Three Graces are to be found and finally the two cathedrals (Anglican and Catholic) both built in the 20th century. The majority of the buildings date from the latter half of the 19th century and early 20th century, the burgeoning wealth of the city having little time for preserving the relatively meagre past. The city suffered significant bombing during the Second World War, as a result of its importance as a port.
After the major buildings the remainder of the the Guide is broken down into a set of 10 walks around the central area of the city, spanning a few miles with an interlude covering the city centre. I’d spotted the grand building of the Marks & Spencer store in the centre of town on my previous perambulations, it is in fact Compton House (see below) built as one of the earliest department stores in 1867.
Compton House, Church Street
As a bonus the book finishes with three short pieces from areas outside the city: Speke Hall, Port Sunlight and Hamilton Square.
The Pevsner Guides are not really designed to be read sitting on the train, as I did, they are to be held as you walk around with a map. Despite the relatively large number of photos it doesn’t feel like enough, I was frustrated by reading the words but not being able to see the buildings. I think a few more walks with the camera are in order.
The Guide is a staccato recounting of what you can see, listing locations, architectural features, architects and the occasion blunt opinion, this is his comment on the re-development of the Prince’s Dock:
“The results so far, though, are inadequate. The architecture is both bland and overly fussy”
It feels like an excellent opportunity for a smartphone app. The current publishers seem a bit bewildered by this newfangled app world and have produced a digital companion in the form of a glossary of architectural terms. Elsewhere someone is selling a database of all of the Pevsner entries, the Guide is a database rendered in prose form.
It seems the components are there for a Pevsner App, who is with me in making it happen?
Footnote
Here‘s a Google Map of the Major Buildings from the Guide.
About
I've worked as a scientist for the last 30 years, at various universities, a large home and personal care company, a startup in Liverpool called The Sensible Code Company (formerly ScraperWiki Ltd), GBG and now as a consultant in data science.
I write about:
* the books I have read, typically science and history (or both), partly as a reminder to myself and partly as a review;
* science, things I have done or things I find interesting;
* technology, programming and gadgets;
politics, and current affairs;
* ...and other stuff as it takes my fancy - holidays, photographs and things I want to remember.