Sarah Booker


Programming for the public (@frabcus) #hhldn

Francis talking about two different stories on the internet.

It used to be the case you had to check the division list to find out how MPs voted.

Created a web scraper pulling out the information and created The Public Whip, showing how MPs voted.

Have to be a parliament nerd to understand, even when it’s broken down.

They Work for You simplifies the information even more, it tells you something about your MP.

Bring the division information together. Take a list from public whip and create a summary of how they voted.

Checking how one MP voted on the Iraq War. Voted with the majority in favour of the war on three votes and abstained from the first and then the final three. It’s almost a deal with electorate.

MP asked to have “voted moderately” removed because found it misleading. A number of MPs have complained, but checked the votes.


Richard Pope founder of Scraperwiki made a website after the demolition of his local pub (a fine-looking establishment called The Queen) and created Planning website.

It helps people access information from outside the immediate catchment area. He wrote lots of web scrapers. Example of different councils’ planning application systems.

Scraperwiki is like Wikipedia but for data. It’s a technical product for use when you’re not technical. Can look at different data scrapers and copy what others are doing without learning Pearl or Python.

Planning Alerts is being moved over to Scraperwiki. Can tag it on Scraperwiki and find information. Can find stories and in-depth information.

Can request a dataset and have something built for you.

Francis was asked,  is it legal? In the UK if it’s public data, not for sale, you can reuse it. Would take things down if asked, but it’s open stuff.

Could it be stopped? Would be ill-advised to stop people, and journalists, reading public information.

Public whip and They work for you, look at numerous votes.

Looking at ways to fund it such as private scrapers, or scrapers in a cocoon. Looking at white label for intranet use. There’s a market for data and developers who want to give data. Want to match developers with data. Currently funded by Channel4. Want to remain free for the public.

Does it make people lazy? No, it’s already published but it makes it easier. Movement of people trying to get publishers of data to change. Always a need to pull out in a variety of formats.

Running Hacks and Hackers days working together finding stories and hunting around.

Have had data scraped from What do they know site.





Heather Brooke at The web data revolution #iweu live blog

Filed under: journalism — Sarah Booker Lewis @ 4:47 pm
Tags: , , , , , ,

As questions became more complex, the only way questions could be answered was through data.

First issue was with police not showing up when called 999. Wondered how often they didn’t turn up

Didn’t get an answer. Formed basis for Your Right to Know,

Want people to stop being deferential, but get the data before making a decision.

Made 52 FOI requests into response times to 999 calls.

Lots of data coming back as spread sheet, complex with 999 calls divided into priorities. Sometimes say how many incidents there were and how responded at the time.

Found inaccurate even if had the facts at all.

Deferential conclusion, if meant to be responded to in 10 minutes, wasn’t kept track. Assumed it had been successful.

Kept coming across attitude public can’t be trusted with the raw data.

Thought Parliament ought to uphold laws. Found stonewalling of expenses requests frustrated.

If MPs won’t disclose, why should councils or hospitals. Taking hold as a symbol.

AK – How MPs get around?

HB – In Silent State, data collected in the name of the public collected but public can’t use it.

Who working for? The public. Seeing a change in attitude. They work for us and have to give out this data.