Open Knowledge Conference

ScraperWiki: We eat Data

And run open data websites for breakfast! From making The Public Whip to joining TheyWorkForYou, Julian Todd has been at the forefront of big data scraping projects. He’ll tell you where the difficulties lie in retrieving, formatting and maintaining scrapers; and how his creation, ScraperWiki, provides the solution.  Find out why Open Australia are running their planning alerts off ScraperWiki, the UK government used ScraperWiki in their AlphaGov project and how OpenCorporates scraped 10% of the world’s companies in two weeks using ScraperWiki.

by Julian Todd on July 1st at 10:30, Track II

Since he gained his degree in mathematics, Julian has earned his living writing software for machine tools. But all this changed in 2003 when he teamed up with Francis Irving to write PublicWhip to screen-scrape the votes of the UK Parliament in order to find out what his representative was doing in his name. Following his involvement in many other screen scraping and public data projects over the years, he eventually saw what was common between them and invented ScraperWiki in 2009. He is an active caver and kayak diver and continues to work on his cave mapping software TunnelX