Read on Twitter
So this morning, Google launched Community Mobility Profiles for 131 countries https://www.google.com/covid19/mobility/. Big thanks to Google and engineers for producing really useful data to support Covid-19 response. Here's my home town #Brighton UK
The data is non-sensitive, and uses some good data-privacy methods for safety. The reports are based on data from users who have opted-in to location history, and aggregated by google up to local areas and countries. More info on the Google blog https://www.blog.google/technology/health/covid-19-community-mobility-reports
But here's a thing. Google released the data as PDF. Here's the report for the UK - 78 pages, 150 areas, coming up for 1,000 trend charts. And there are similar PDF reports for all 131 countries https://www.gstatic.com/covid19/mobility/2020-03-29_GB_Mobility_Report_en.pdf
We would love to use this data to improve our understanding of how social distancing is changing behaviour. The government's response to Covid-19 is crucially dependent on good information. Here's @ONS daily round-up of Covid outputs we're publishing https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/articles/coronaviruscovid19roundup/2020-03-26
But we *really* need the raw data to build into models, analysis, dashboards etc. Copy pasting chunks of PDFs doesn't work for us for reuse. As @timberners_lee said "Raw Data Now" https://www.ted.com/talks/tim_berners_lee_the_next_web/transcript?language=en
So some of the @DataSciCampus team got up early, revised their image classification, PDF scraping skills, and got to work. There was quite a tussle over the best approach - turning trend charts into raw data is not totally straightforward, especially with nearly 1,000 of them
But the team made good progress, and we shared our data and analysis with partners across government earlier today. We've got some more to finish-up tomorrow, so some of the team are back in over the weekend to tidy-up. We'll also make our code available for others to do the same
As I know many people interested in this, we've made our working set of data available on GitHub. We'll update this again tomorrow. Data provided on as-is basis, based on the Google reports https://raw.githubusercontent.com/datasciencecampus/google-mobility-reports-data/master/google-mobility-profiles ons-data-science-campus-extraction 2020-04-03 6pm.xlsx
Finally, I'd again like to commend Google and teams for putting this data out there to support Covid-19 responses across the world. It's a powerful addition to the 'data for public good' toolkit. Look forward to analysing the data properly, and seeing how trends change over time
So tweet #fail for me. URL with spaces in it fails. Twitter #hivemind how do I fix that - can't see anything obvious from searching
Here's the corrected URL: https://raw.githubusercontent.com/datasciencecampus/google-mobility-reports-data/master/google-mobility-profiles%20ons-data-science-campus-extraction%202020-04-03%206pm.xlsx h/t @puzzlesthewill @GeogDave
And here's the Google data presented in this evening's press conference https://twitter.com/peterkwells/status/1246110343673393154?s=20 h/t @peterkwells @thomasforth
