Both of the experimental tools I built are publicly available – and are on github, here and here – but do be aware that they are experiments. They are not guaranteed to be either reliably useful or legal. I’ve not looked at the code and have particular concerns over how the UK version handles copyright.
With those reservations in mind, here’s some thoughts I had after using Replit to build those two tools:
the experience of building the tools was pretty easy, and at times astonishing,
but Replit didn’t give me confidence that the tools would be reliable and
it neither checked, or encouraged me to check, whether what I was doing was legal.
It was a heck of a lot easier to build an address validator for France than the UK.
The experience of building the two apps was pretty easy, and at times astonishing
I am *not* a software engineer but I can do some coding, understand software engineering practices, and have some experience working with public sector data like addresses. Within that context the tool development experience was pretty straightforward and at times astonishing to my tiny mind.
I could both use the chatbot-like interface to tweak small things – like content on the front-end – and large things – like working out how to download and make use of specific address data sources that it was pointed at. That latter bit felt astonishing.
Replit taking on the task of integrating data from Datadaptive
At one point I needed to go and create an API key, but didn’t feel the need to go and look under the hood at the code or database.
But Replit didn’t give me confidence that the tools would be reliable
At the start of the build process for the UK address validator Replit said it had built something that worked, but it obviously did not. It took three cycles of me saying things like “that doesn’t work, look harder” before the tool started working and even then it had not done what it had been asked to do.
Replit had been told to build an address validator, but instead it built a postcode validator
After that initial confusion Replit reported that it was running more tests, but it never showed the results. I had to specifically ask it to show test results before it reported that it was loading a testing skill and running some tests.
Despite this Replit had been happy to publish the app with no indication either to myself or to potential end users of the app that it might not be reliable.
Replit might give the feeling, but not the reality, of reliability. They should try to fix that.
And Replit neither checked, or encouraged me to check, whether what I was doing was legal
Replit struggled to understand or communicate data protection risks, which are important in France, or copyright risks, which are important in the UK.
It happily built functionality to collect and republish the addresses that people validated using the tool without telling me this should be made clearly visible to users. Eek. Don’t test it with your home address!
When I suggested Replit should use some public sector data containing addresses that was released under the UK’s OGL (Open Government Licence) it told me that using the data was fully permitted. This misses that the UK OGL contains exemptions both for personal data and for third party rights.
This is an incomplete summary of the legal position. The UK OGL has a set of exemptions that are important to understand.
There have been multiple cases in the UK where people have been threatened with legal action over infringements of copyright when using address data. Replit had even suggested I use a service – https://getaddress.io – that recently closed because it lost a court case over third party rights in address data. Silly Replit.
To look deeper into copyright complications Replit was told to look at the UK Land Registry’s Price Paid data. This data is published under the UK OGL and has an explicit warning that the Royal Mail and Ordnance Survey reserve some rights.
This time Replit communicated the restrictions but suggested they could be worked around by showing the residential property price when validating the address. I don’t think the courts would agree with this interpretation.
The purpose of the tool was to validate addresses, not to provide residential property price information.
After a bit of prompting I got Replit to start communicating the various data protection and copyright risks to potential users of my experiment, but it did leave me wondering.
How many other Replit users are happily producing apps that unhappily break the law with risks to themselves and other people?
Whether as well as the law potentially needing to become more machine-readable that these new coding tools need to get better at communicating legal requirements and risks to the people that use them?
And should governments play a role in making that happen?
After all, the increased ease of using this wave of coding tools seems likely to increase the number of people who produce software, whether it be in tools like Replit or in real-time when using an AI agent. I suspect it will be increasingly important that AI-generated software and the humans that are responsible for it follow the law.
It was a lot easier to build a French address validator than a UK one
Finally, there was just one more thought. One that is likely to be obvious for anyone who works with UK geospatial data.
It took me several hours of to and fro to produce a useful looking UK address validator that did not completely rely on expensive licences and that could communicate to users the legal requirements that came with reusing the data. And I already knew quite a bit about how to do that.
It took me just 10 minutes to do the same for France, and that came with considerably less risk.
This is partly because the French government has already put in the operational and technical work to build an open address database and provide an API that tools like Replit can use. But it is also because the French government has put in the legal and financial work to ensure that they could provide this data for free and under an open licence which is more permissive than the UK’s OGL. The French government – and others – have done this for many other public sector datasets too.
If we are moving to a world where AI-enabled coding tools, like Replit, are more widely used then the work that countries like France have done could prove invaluable in helping many more people produce software tools that work, are reliable and are legally safe to use. The UK has some catching up to do.
The survey is not particularly easy to respond to, for example it asks people to provide information about all of the public sector data they use. That would be a lot of information for many organisations and people!
There is a page listing all of questions and a separate page with an online survey to answer the questions. It does not say that you can email answers to the team so I would recommend filling in the online survey.
If you focus on the key questions 12-14 it might only take about 5-10 minutes. You can save progress as you go, but you can’t change your answers once you have submitted the questions
I recommend that you respond politely and honestly throughout. Do use your own words and thoughts. In case it’s useful to help your thinking then here are some tips on what to expect and how I responded.
Questions 1-7 are about your organisation and job.
I both work with multiple organisations / sectors and use data as a citizen so found this section confusing as it assumes people only use data for a single organisation. Others may find it simpler.
Questions 8-11 are about public sector data you use in your work.
Again, it wasn’t obvious how to answer these. I use, experiment with, imagine, and campaign to improve lots of data. But this wasn’t why I was responding to the survey and it wasn’t clear what this information would be used for.
So I kept it simple and skipped questions 8 and 9, they’re optional questions, and answered don’t know/other to questions 10 and 11 as they are both mandatory questions.
In response to question 11 I gave a couple of suggestions of other issues that affect both businesses and individual citizens, specifically “inability to contact data holder” and “inability to request data corrections”.
Questions 12-14 are about existing public sector data sets that you or your organisation would like to access that you currently do not. This is where I chose to respond about address data
For Question 12 I responded “Administrative address and mapping data created and maintained by public sector organisations as part of their public task“.
Question 13 is about the access issues. I responded with “Cost”, “Data Quality” and “Legal Frameworks”. In the “Other” section I gave a short description of the legal risks and current business models, saying:
“1. The intellectual property status of many public datasets that contain this data is unclear. This is because of unclear licensing and derived database rights. Public sector organisations can interpret this situation differently, for example Ordnance Survey’s recent contact with 34 local authorities https://www.owenboswarva.com/blog/post-addr44.htm. This risks a chilling effect where people are unwilling to invest to innovate.
2. Some of the public sector organisations that create and maintain this data, for example the Ordnance Survey, are run as government companies. This creates an incentive to protect their rights and monetise the data, rather than to maximise the public good. They should instead be funded to publish this data for free.“
Question 14 is about the benefits. I responded:
“Address data, and geospatial data more broadly, are foundational datasets that are used extensively across the public and private sectors.
There are many current use cases that would be improved by making the current datasets freely available under an open licence; use cases that will be improved because effort and money that is currently spent buying datasets and understanding licences will be spent on other more useful activities, and new use cases that will be created by innovators – in the public, private and third sectors – who currently feel hindered by the cost and/or legal uncertainty.
The EU estimates it will receive up to €2 billion per year in economic growth from making this data available for free under an open licence. “
Questions 15-16 are about new datasets that the government could collect.
I chose not to respond on these topics in this survey.
Questions 17 – 19 is about public sector data that would help people develop or use AI.
My response to question 17 pointed out the existing set of complex copyright questions about data and AI and how open data would help:
“Publishing address data, and geospatial data more broadly, as open data for free would help me develop or use AI as it will reduce one element of the current complex set of copyright issues that surround AI products.“
Questions 20-21 are broad free-text questions
Questions 22-25 are about staying in touch for more research.
In its 2024 manifesto the UK Labour Party promised to build a National Data Library that would:
“bring together existing research programmes and help deliver data-driven public services, whilst maintaining strong safeguards and ensuring all of the public benefit.”
Great, there’s a lot to do on data in the UK. But, unfortunately the manifesto commitment is a pretty broad scope and the term ‘library’ can be confusing. It is not surprising that people are finding it hard to agree what to do next.
Here’s a suggestion for what the “National Data Library” should be and how it could get started.
The National Data Library should help public sector teams deliver trustworthy data services by providing guidance, reusable platforms / technical components, and growing communities of practice.
It should start with practical work to build and improve the government’s existing data services. This will help it learn what is needed to deliver trustworthy data services across the public sector.
The visions and priorities for the library are broad, that is making it hard to define
At a political level the description in the manifesto might seem to make sense, but when you poke at it the sentence starts to fall apart.
The concepts of bringing together “existing research programmes” and helping “deliver data-driven public services” are different.
Public services tend to need access to reference data, information about rules, and to collect and process data from and about individuals and businesses that use the service. They are shaped by democratic debate and a range of legislation including data protection and administrative law. There are many existing public service-focussed data initiatives inside the government.
Research programmes tend to need access to large datasets from one or more organisations. They have oversight, such as research ethics committees, and researchers are likely to be accredited. A research programme is a lot more likely to seek informed consent, for both participation in the research and data use, than a public service. There are many existing research programmes running in universities, philanthropic and public sector organisations.
The original proposal for a UK National Data Library, from centre-right think tank Onward, said that it should be “a centralised, secure platform to collate high-quality data for scientists and [AI] start-ups”. While data for scientists might mean the same thing as data for research programmes, data for AI start-ups certainly does not. So, that is another set of needs to understand, prioritise and design for.
Finally, the Labour government places an emphasis on mission-led government with a set of initiatives that cut across government departments, other parts of the public sector and wider society.
These missions will rely on data to understand problems, make change, and report progress to the public (perhaps as official statistics) and senior politicians. To deliver on their missions teams are also likely to need to use data in ways that create impact that differ from people’s classic conception of a public service. So that is another set of things to think about.
And the library will prioritise between all of these different areas and their possible use cases while maintaining strong safeguards and ensuring all of the public benefit?
And the idea of a data library has been leading people in confusing directions
Meanwhile there’s another problem. The term ‘library’ seems to be confusing things.
Like most people who does policy – and other things – I can take a strange joy from exploring definitions and meanings, but the term ‘library’ seems to be proving unhelpful. People seem to be thinking of it as a single object or thing that contains all the data.
Box-and-wire diagrams are in fashion this year, aren’t they?
A central data portal with a catalogue for all of the data seems to be a popular idea.
Yet a lesson we have learnt is that a single, big portal will not meet people’s varying needs when publishing data, searching for it, or making use of it. A broad scope like the one set out in the Labour manifesto needs lots of catalogues, portals and other things to ensure that data gets to people who need it and are allowed to use it.
A less popular idea, but an idea that is still visible in policy circles, is a single technology platform – such as the one described by Onward. This would be a platform where all of the data is accessible with common governance, technology and standards.
Unfortunately a single platform for data would be a great target for hackers, stifle innovation, and change democratic accountability in ways that are hard to predict.
Government is not one organisation. It is thousands of organisations with varying goals, that fulfill their democratic lines of accountability in different ways. This means that different governance, technology and standards will often be appropriate. The world of using health data for medical research is pretty different to the world of using data to improve local authority services for planning applications.
And, just like a central portal, a central platform would not be able to meet the wide range of needs of data users.
Given the focus on the word ‘library‘ I’m even expecting someone to be daft enough to suggest library cards, perhaps with fines for people who don’t ‘return’ a particular piece of data on schedule…
The UK needs an approach that works for many different contexts and that builds on the work that is already being done by teams across the country.
There is not much discussion of the need to improve the government’s existing data services
Lots of data services already exist across the UK that might fall into the National Data Library’s broad scope.
Each of these services has stakeholders with different behaviours, needs, motivations and skills. They also have delivery teams with different capabilities, strengths, and weaknesses. In the different services data might use different standards, because they meet different needs. There are multiple legal and governance frameworks. Data protection is not the only law at play here.
Some of these existing services are great, some are heading in the right direction, some don’t seem to understand their stakeholders, and some simply don’t exist even when they should.
Rather than building a single thing, the National Data Library should make it easier for teams across the public sector to deliver data services like these.
The National Data Library needs to help people deliver trustworthy data services
Despite the variations between these services there will be some shared problems and lessons that have been learnt and shared for how they can be tackled. This is an area where the National Data Library could usefully focus.
Here are some ideas for the support capabilities that could be provided:
guidebooks and manuals for how to design trustworthy data services and their associated governance and oversight mechanisms. Organisations can then use these core practices or adapt and build on them in their own context.
a shared library of user research to make it easier for data service teams to understand the range of people impacted by their work such as researchers, data analysts, policymakers, and members of the public.
a design system and patterns, the data design patterns that currently exist tend to have been developed outsidethe public sector.
reusable, tested and well-maintained components,for example platforms – or simpler technical libraries – for publishing data or information about research projects, tools that make it easy to create data portals, verify data against standards, or understand data bias and its potential discriminatory effects.
a data linking service, guidance, public engagement, transparency and approval mechanisms for linking data across government and/or non-government research infrastructures. Perhaps London shows a way?
data transparency and control services, that empower individuals, communities and regulators to understand and control how data is used at times and places that are relevant to them.
a National Data Academy that provides training courses and coaching in data skills
funding, to support experiments, pilots and discoveries in under-resourced public sector organisations.
communities of practice to maintain and improve all of the above. Some of these communities already exist, both formally and informally, but many need more support. Communities of practice could usefully exist in teams building data services and in the groups of people – like researchers and start-ups – that use them.
Design patterns are a way that digital team teams communicate repeatable solutions to common problems. These images are from IF’s design pattern catalogue.
To stress. These are just ideas.
Other people might have better ideas. A team that helped people deliver trustworthy data services would have to learn what was needed by:
doing the practical work to build/improve some data services themselves,
working with existing teams to understand their challenges,
and listening to various viewpoints from outside the public sector.
This was roughly how GDS got going in improving digital services.
A data library team would also need to be careful of the overlap with existing things that help with the goal of delivering trustworthy data services. For example the ICO’s regulatory guidance on data protection, HRA’s advice on health research, or even the OSR regulatory guidance and ONS guidebook’s on statistics.
But these are not insurmountable challenges and, from my own work across government, I’m confident that there are many needs for support that are not being met and many opportunities to tackle the problems together.
The existing data service teams across government can then use these new support capabilities to help them build trustworthy data services better, cheaper and faster.
Getting things started
It is important that one of things a national data library team that helps people deliver trustworthy services starts with is some practical work to help public sector organisations improve or build some data services. This would deliver some early impact, create momentum, and generate learnings and capabilities in that core team.
In selecting these data services it will need to look for some variety. This will help illuminate the problem / opportunity space.
The final decisions for where to start should be based on government priorities but – from my own knowledge about common problems / opportunities – perhaps it could include:
a data service that openly provides access to authoritative, non-personal data held by the public sector that could be widely used by public services and startups, for example address data
a data service that provides authorised organisations and researchers with secure access to attributes about individuals, for example their age or eligibility for benefits
a platform that helps research programmes publish accessible information about research projects throughout their lifecycle so that individuals and communities can understand how they are impacted by research activities
a platform that helps research programmes publish accessible information about the data they hold and types of research they support so that researchers can find what they need more easily
a service that makes it easier for multiple local authorities to publish data locally and then aggregates it for use nationally, for example information about elections or places where clean energy infrastructure could be built
By doing this work the national data library team can start to develop the much-needed guides, components and communities of practice that can deliver more trustworthy data services across the public sector.
The National Data Library should help people deliver trustworthy data services
The commitment behind the National Data Library provides an opportunity to improve how the UK public sector, researchers and startups use public sector data, but this opportunity will not be realised if the UK ends up in endless abstract policy debates, workshops and roundtables or – even worse – building big new central portals and technical platforms that everyone is told to use.
Building capabilities that support existing teams to deliver more trustworthy data services across the public sector will be far more impactful and start to deliver the change that is needed.
Want to read more stuff?
If you’re interested in the National Data Library then here are some links I found useful when forming my views:
The MHCLG Planning Data platform manual’s approach to data quality, which brings the government framework into the planning data context
And thanks to Ellie, Steve, Andy and others who I bounced around these ideas with as I was writing them up. All mistakes and idiocies are always my own.
Yesterday I was skimming the UK Department for Work and Pensions (DWP) latest whitepaper, called “Get Britain Working”, to see how they planned to use technology to help them deliver the policies and public services in it.
In doing so I was struck, not for the first time, by the wide number of ways in which the words ‘digital’ and ‘data’ are used across the public sector. As an experiment I thought I’d try to categorise them. This post is about ‘data‘, I might publish a similar post about ‘digital‘.
The categories in this whitepaper will be far, far, away from a complete list and all categorisations are loaded in some way, but some light grouping came up with eight categories that were interesting to me:
data as an input to and output of scientific research
data as an input to and output of a tool or public service
data produced as an official statistic
data produced for policy development
data produced for service planning
data to describe a type of computer system
data as something that needs governance and could be used for many purposes
It’s interesting to consider which of these categories of data uses the UK has the capability to do repeatedly to a decent standard, and which are the ones where more work might be required.
Data as an input to and output of scientific research
There was one case of data being an output from a scientific research study. Data will be an input to this study too. There are a range of legal, ethical and professional frameworks guiding scientific research.
Research is one of the purposes that the National Data Library is meant to support .The current text of the Data Use and Access bill includes a change to the legal definition of research. The change broadens the definition of research beyond scientific research in the public interest.
“a place-based real-world evidence study … aims to evaluate the effectiveness of tirzepatide on obesity and its impact on obesity-related conditions in a real-world setting… As well as data on patient outcomes, such as a reduction in rates or even reversal of conditions such as diabetes, CVD and poor mental health, the study will also …”
Data as an input to and output of a tool or public service
There was one case of a new tool that uses data to produce data.
In this case the tool would provide a prediction that an individual is at risk of not being in education, employment or training. This might also be referred to as a public service that includes automated decision making (ADM).
This kind of tool comes with multiple risks, such as: unfairness and discrimination against some communities, or that the prediction is treated with too much certainty and used inappropriately by other tools, people or organisations when it might turn out to be flat out wrong.
Data-driven public services is one of the purposes that the National Data Library is meant to support and that the DSIT digital centre works on with departmental teams like DWP Digital.
“We will publish new guidance on using a Risk of NEET Indicator (RONI) approach and provide a new data tool so that local authorities can better identify those at risk of becoming disengaged and put preventative measures in place”.
“Under the accountability and data sharing frameworks of a future Youth Guarantee, the college informs the Mayoral Combined Authority that Luca is at risk of not being in education, employment or training.
A local, youth-focused community organisation commissioned by the Mayoral Combined Authority reaches out to Luca to offer support and encouragement to re-engage and explore his employment or further education options”
Data produced as an official statistic
There were several references to data that, when you follow the footnotes, has been produced as an official statistic.
Official statistics are independently regulated by the Office for Statistics Regulation and there is a community of practitioners in and around the public sector.
“Data shows that only around 31% of prison leavers are in employment 6 months after release and 46% are employed in the 6 months following completion of a community sentence.”
“While many mothers want to care for their children full time, survey data indicates around half of non-working mothers would prefer to work”
“The latest available data shows that the relative poverty rate (after housing costs) of children in households where all adults work was 14%, compared to 75% for children living in households where no adults work”
Data produced for policy development
There were multiple references to data being used to help develop policies. It is unclear whether this data would be produced to similar methods and standards as official statistics or whether some other approach would be used.
While official statistics are openly published, data for policy development might be kept within the public sector and not published transparently. Often data for policy development comes from multiple sources and is linked together and analysed to find insights.
“The government wants local areas to have improved data to understand local population needs and to help design future programmes. We also need better data to track outcomes and develop the evidence base”
“We will continue to engage the expert Labour Market Advisory Board announced by the Secretary of State for Work and Pensions to provide the government with insight, ideas, and challenge. The immediate priorities of the Board…include job quality and progression, opportunity and equalities, health and inactivity, regional inequalities and data”
“By linking migration data with skills and employment policy, we will ensure that training in England is aligned to labour market needs”
“It will draw on local and regional vacancy data and Local Skills Improvement Plans (LSIPs) to inform its skills needs assessments”
Data produced for service planning
There were a couple of places where data appears to be being used to help plan and carry out operational services.
This might be produced using similar methods as data for policy development, but it typically has a different audience with different capabilities and needs.
“This will work with Integrated Care Board leaders to further reduce waiting times and improve data and metrics and referral pathways to wider support services.”
“In August, the Department for Education introduced new statutory guidance for schools and local authorities on improving attendance, supported by comprehensive near real-time data in England”
Data to describe a type of computer system
There was one reference to a type of computer system for data.
A team developing and maintaining a system like this might follow guidance from the DSIT digital centre or, as this particular system is in the health sector, guidance produced by the Department for Health and Social Care or NHS England.
“as well as exploring opportunities to utilise the data platform created by Our Future Health in partnership with the NHS”
Data as something that needs governance and could be used for many purposes
There were a couple of broad references to data being information that could be used for many things, but that needed appropriate governance.
Local Get Britain Working Plans trailblazers that require “governance and management – including accountabilities and responsibilities across partners, and arrangements for data sharing”
Youth guarantee trailblazers that require “Governance and management – including accountabilities and responsibilities across partners, arrangements for making the best use of data, and management structures.“
Data as an enabler for other unspecified things
And a couple of references to data being something that could enable other, loosely specified, things. These examples seemed to differ from the above as they did not explicitly mention aspects of governance.
“Developing further tools will also provide the foundational data to enable further opportunities to transform the services”
“To enable [more enhanced collaboration between Jobcentre Plus and the National Careers Service], a new England-wide data sharing agreement between the Department for Education and DWP will be put in place from winter 2024”
The UK has an official list of building addresses and their locations – ‘address data’. This data is a vital resource for building public and private services that rely on locations, and is part of our national data infrastructure. At the moment, the UK’s address data is expensive, hard to access, not always accurate, and hard to correct. This causes problems for businesses and other organisations that rely on address data – and ultimately it affects us all.
The debate had contributions from Labour, Liberal Democrat, Green and Conservative backbenchers. The Minister for the Conservative government then rejected the amendment.
Reading and watching back the debate made think about three things:
The government agreed to share deeper analysis, which is good news
But it misunderstands why previous attempts to recreate UK address file failed, that is bad news – and not just for addresses
The risks of openly publishing address data, or of not publishing it, are misunderstood
The government agreed to share deeper analysis
The Minister said that they were “very happy to share deeper analysis” of address data. This is good news, both because better evidence can create a better debate but also as it indicates that the government actually has some analysis.
The Geospatial Commission said they had no analysis
In 2022 the Geospatial Commission responded to a Freedom of Information (FOI) request by saying that it did not assess address data when preparing its strategy. Similarly in 2023, when the Geospatial Commission was agreeing a £31m contract with the Royal Mail, they said that they did not perform any analysis of the costs, benefits or alternative options.
There were some previous projects that did do deeper work. For example, in 2017 the government spent £500k, out of a potential budget of £5m, investigating how to create an open address file.
A map of open address data around the world from OpenAddresses.io
A misunderstanding of why previous attempts to recreate UK address data failed
The Minister referred to previous attempts to recreate the UK’s address data, saying
“the resulting dataset had, I am afraid, critical quality issues”.
Viscount camrose
As someone who spent part of 2014/15 working on a project to recreate the UK’s address data that was not why our project was stopped. The Minister might want to ask officials for more details as we learned some interesting lessons that the government needs to learn too.
The kind of innovation that government policy wanted to support
Our approach to recreating the UK’s address data was to start with data that the UK government already publishes. In line with the government’s “open by default” data policy, organisations like the Land Registry, Companies House, and the Valuation Office Agency spend money to make the data they hold available for other people to use. Some of this data contains address information.
We intended to make the bulk data available for free, and then generate just enough revenue for sustainability – perhaps from high volume users of the API. We set ourselves up as a not-for-profit company.
It was the kind of innovation that the government’s open by default policy is intended to support.
Much of the government’s open data was not ‘open’, this creates legal risks
Unfortunately we found that much of the government’s open data was not actually ‘open’.
The government’s copyright licence (the Open Government Licence, or OGL) excludes third party intellectual property rights. The third parties who hold IP rights in address data, Royal Mail and the Ordnance Survey, are litigious and many of the government organisations that published the data were unable to be clear on whether or not there was Royal Mail or Ordnance Survey rights in the data they published. We only used datasets where the publishing organisation told us it was ‘safe’.
But even though it was government organisations publishing the data they would not be liable if there was a legal issue. We would be. So we needed insurance cover.
But given the risks only one insurance company was willing to offer cover and that was on unrealistic terms. So, we stopped the project.
To put it another way, an innovative, not-for-profit business could not use the data that multiple government organisations published to support innovation, because another government organisation might take legal action.
There are new plans to publish more government data, they risk the same problems
Zooming forward in time from the ancient history of 2014/15 and back to the present day various UK government departments are currently making new plans to publish more government data.
This is because of initiatives like the Vallance report on pro-innovation regulation of technologies and a desire to support the UK’s AI industry. High-quality, authoritative government reference data is one way of reducing the hallucinations that the current generation of AI models suffer from. Sounds sensible, right?
But publishing widely used address data is a lot simpler and safer than much of the planned work, yet the government failed to do so in a way that allowed organisations to clearly understand what they legally could, or couldn’t, do with it. Will this new wave of government data come with instructions telling AI models and engineers not to do anything with addresses? And what other third party rights might be lurking in there? Or will government just make AI’s copyright issues even more complicated.
If the government does not understand why its previous attempts to publish data did not yield the desired benefits then I fear a lot more wasted money in the future.
The risks of openly publishing address data are misunderstood
In the debate Lord Bassam said
“there is a balance to be struck between privacy issues and the need to ensure that service delivery and commercial activity operate on a level playing field”
LORD BASSAM
It is good that politicians consider privacy issues, but this misunderstands the risks.
Address data does not create new privacy risks
The list of addresses does not tell us where specific individuals live, the only personal data involved is likely to be those of people who name their business address after themselves. Instead address data tells us where people might live, work and play but not who is living, working or playing there.
(As an aside: I don’t want to imply that there are no risks of privacy, or other human rights, breaches with non-personal geographic data. For example in a separatist war in Sudan in 2011 atrocities were carried out because satellite data showed where particular groups of people were. But, hopefully, the UK is a long way from a separatist war and, let’s be honest, truly harmful actors will either simply buy the address data or use an illegal copy.)
The harms created by the lack of access to address data are more pressing
By contrast Lord Clement-Jones pointed out that
“The harms created by the lack of access to address data are more pressing”
LORD CLEMENT-JONEs
While Baroness Harding pointed at the issues with the current data quality saying:
“the quality of the data is not good enough….Anybody who has tried to build a service that delivers things to human beings in the physical world knows that errors in the database can cause huge problems. It might not feel like a huge problem if it concerns your latest Amazon delivery but, if it concerns the urgent dispatch of an ambulance, it is life and death.“
BARONNESS HARDING
Elsewhere the National Audit Office has pointed to the challenges of creating and using the shielding list of people with extreme clinical vulnerabilities during the pandemic. One of the challenges was inconsistent address data in different formats in different IT systems and organisations. This is one of the many challenges that opening up the official list of address data will help with, because over time more organisations will refer to and use the same reference data.
If the funding model changes then will quality drop?
There is one risk that was not discussed in the debate though.
If the maintenance and publication of address data is not funded from licence fees collected by Royal Mail and Ordnance Survey then will the quality drop?
This is where there is an important balance to be struck as people and organisations need the correct incentives to publish useful data.
Bluntly, this is the risk I worry about the most. Money is only one type of incentive but it is an important one in this context and it is one of the reasons why I’m so keen to see some deeper analysis of the current costs.
Experience tells me that the current costs are significantly overstated – particularly the Royal Mail who claim costs of ~£25m/year for ~300,000 changes/year. But however much the costs can be reduced it will still cost money to publish quality address data.
Making the publication of the data a statutory duty, as this amendment would have done, is one way to help tackle this risk. It requires the government to fund and do the work.
Perhaps the money might come from general taxation, and the increase in economic activity that will come from publishing the data? Or perhaps from a small increase in registration fees collected by local authorities who do most of the work to create addresses? Or a small increase in the Land Registry transaction fees, after all they handle nearly 50 million transactions per year?
Other countries have changed legacy business models, the UK should too
Whatever the final decision it will need some coordination and activity from a few public bodies willing and able to work together to publish address data as a public service.
And that’s where I hope the government is really focussing its analysis. Not on whether to publish address data for free, but on how to do it.
Because in the 21st century it is pretty sensible for high-income countries to make reference data, like addresses, as widely available as possible. That is why peers from so many different parties supported this amendment, and why so many other countries are doing the work.
The hard part of the work is changing the legacy business models and incentives of government organisations so that they make it happen. Other countries have done that, and it’s long past time for the UK to do the same.
The UK’s Data Protection and Digital Information Bill continues to work its way through Parliament. The UK government hopes to get it completed in the first half of 2024.
The bill is complex with lots of different parts. When the UK government first started promoting the bill they said that one of the ways it would help the public was by reducing cookie pop-ups, reducing the chance of people being pestered by seemingly unnecessary alerts.
Unfortunately, the bill will do little to cookies – that’s a problem that industry is trying to ‘solve’ – but it looks like it could significantly increase the amount of unwanted spam and letters that people receive. From some figures it looks like there could be a 25% increase. Uh oh.
Image by DALL-E and me.
The bill makes it easier for more organisations to send unwanted mail
Current UK legislation and guidance effectively says that unless organisations have consent then they need to carry out a number of tests to decide whether they have a ‘legitimate interest’ in sending direct marketing to people.
The data protection regulator say that those tests mean organisations need to consider things like the nuisance factor of unwanted adverts and the effect they might have on people in vulnerable situations. Sounds sensible.
If the regulator’s guidance is not followed then organisations can be fined. That also sounds sensible.
But the new bill explicitly says that direct marketing – a category that includes things like posted or emailed adverts – is an example of a legitimate interest.
Industry thinks this will mean that a lot more money is spent on adverts
At a conference last year the CEO Direct Marketing Association said that this change is an important clarification and that they expect it to mean that an extra £250m will be spent on printing and posting adverts through people’s letterboxes.
The Advertising Association said that £1.1bn was spent on direct mail in 2021 so an extra £250m means about a 25% increase in the amount of printed adverts that we’ll all get.
If those estimates are correct then it seems reasonable to think that there’ll be a similar 25% increase in the number of emailed adverts.
Obviously some adverts are useful, but people hate spam
Last year I worked on a project with the team at IF that researched how people felt about advertising.
It was pretty clear that most people like some advertising, I mean who wouldn’t want a discount for their favourite food in the middle of a cost of living crisis.
But it was also very clear that people hated unwanted advertising, particularly when it came through their letterboxes and into their emails, and that there was too much of it already.
Another image by DALL-E and me. I have a lot less hair than this.
Unwanted adverts makes life harder for everyone because we need to wade through them to find meaningful things, like the increasing number of notifications that public services send us about our taxes, health, or benefits.
It makes it particularly hard for people in vulnerable situations. Some people find it harder to sift through the volume of letters and emails to find the important things, while more people with specific vulnerabilities might be targeted by bad organisations.
That is why the ICO recommends those tests under the current legislation. It helps reduce the proportion of unwanted, or actively harmful, adverts that people receive.
In IF’s research we also found that people wanted other ways to reduce unwanted adverts, for example by using their legal right to object. Unfortunately that legal right is not being respected.
Instead of fixing these things – and giving people more useful and controllable advertising – the government seems to be changing the legislation so that more adverts can get sent.
Do the industry’s figures on increased spend on advertising feel ‘right’?
It is genuinely hard to tell if the industry’s figures are accurate.
Government has published an impact assessment for the bill. It says that this change will save organisations about £4.5m per year and notes the potential risks to people in vulnerable situations.
The impact assessment does not attempt to quantify those risks whether in monetary terms, in terms of the number of people affected, or the number of extra unwanted adverts that people will receive. It does not bring to life how an increase in marketing will affect people.
But will advertisers really spend even more money on advertising? Or just shift it between different types of advertising like direct mail, email and online adverts? Perhaps they will spend more money but it will simply get swallowed up within the opaque online advertising industry?
Who knows.
But given that the expected benefits are a tiny £4.5m a year in reduced costs, perhaps more people should be asking how much extra spam people will get in return?
This post is based on desk research, conversations with various people in national and local organisations, and a talk I gave at an OpenDataSavesLives meeting. For more Coronavirus stuff that I’ve worked on see the Ada Lovelace Institute’s “Exit Through The App Store“.
Coronavirus is a pandemic. For a couple of centuries we have known that data is one of the most powerful tools in a pandemic. The UK prides itself on being a world leading nation in the use of digital, technology and data. Yet in England, the largest of the UK’s four nations, we are struggling to get data to local places so that they can use it to help save lives.
The role of local places in a pandemic
In England local authorities are responsible for public health in their area. They also play a vital role across many services including housing, business support, health and social care. They work with a range of partners to do this. Hospitals, doctors, care providers, police forces, charities, businesses and citizens (through both existing and new structures).
At the moment England can see the end of the first wave of the pandemic and is starting to relax lockdown measures. The focus has shifted to what is called test, trace, and isolate. Widespread testing to understand where the disease is, contact tracing to track down who else might have it, and isolation to contain new outbreaks of the disease.
These are tasks where national decisions and health research play a role, but a similarly important role is played by local places.
Having good data about the spread of the virus in local places might help a community group to tailor hygiene advice to meet language needs, a business organisation to distribute hand sanitisers to shops, care homes to take extra precautions, public health officials and statisticians to produce local predictive models, or a local authority to manage a local lockdown.
Local organisations are often the most appropriate organisations to do this because their staff know their places and the people who live there. They are trusted, or not, in different ways than the central government.
Data and information about the pandemic
But to take these decisions they need information.
Some of this information will come from these organisation’s connections with their places – a community organiser might hear of an outbreak because a friend is affected by it, or people might see complaints about shop hygiene on a local social media page.
Other bits of information need to come from data, for example the number of people tested in an area and how many were positive, or the number of contacts traced and whether there is a difference between demographic groups.
Local places are struggling to get access to this data, but it does exist.
The national government has set up national programmes like the Covid-19 data store, NHS Test + Trace, the NHS symptom tracking service and Project OASIS – which brings together data from various symptom tracking apps. As an aside this seems to be an exceptionally English approach, most other nations of a similar size seem to have built on existing regional and local structures.
All of these national programmes use data, for example to improve operational performance, to inform national decision makers, to support medical researchers, and to inform national media debate.
But the data they collect and steward is not getting to local places and those local places need it too.
Charities collecting and publishing data about social care because of government failure. Local academics being told that their research needs to conform with national health needs. Regions exploring whether to launch their own symptom tracking services. Businesses offering data services that may be of lower quality than that which the national government already holds. Local officials and community groups struggling to find out who to speak with to even start a conversation about data access.
In May there were reports that an interim operational review by a cross-government team highlighted the problem of data access. Tom Riodan, the CEO of Leeds Council, was given a role in the national Test + Trace programme after that review. His role is not only about data access but, as a result, some progress seems to be happening.
Despite this the national programmes still lack urgency and there are now concerns that the government will supply local places with dashboards that it and its national partners design, rather than giving local places access to data so that they can use it to design and operate whatever decision making tools they need.
Meanwhile the public complaints will continue and the opportunity to make decisions that could save lives will be lost.
Accessing and using data in trustworthy ways
When data access is provided then it will need to be used in trustworthy ways.
Local public sector organisations have had the legal power to use personal health data since COPI (Control of Patient Information) notices were issued back on 1 April 2020. The notices were passed to support this kind of use.
Other organisations, such as charities or businesses, can use open data which is aggregated to a safe level.
For these organisations then daily publication of symptom, testing and contact tracing data at the level of LSOAs (Lower layer Super Output Areas) is likely to have the right balance between data protection and usefulness for public health. It is hard to be certain without access to the data.
If the national programmes do not have the expertise to navigate these issues then they could get help from the Office of National Statistics who can both work through how to publish the data and help to communicate how this data for local operational decision making has different characteristics to statistical data.
The power of networks
When the data is available then it can start to rapidly be put to use.
Some local authorities are already working with their communities to prototype what they can do when, or if, the data arrives.
Networks like these can help get the data used in building tools for local places, evaluate the outcomes to discover what works and what does not, and share their learnings across the nation.
But they need the data
There are lessons to be learnt here, and not just about public health programmes in a pandemic.
If the UK wants to level up across the country it will need to do a lot more work on devolving data governance and learning how to get both local places and citizens represented in decision making about data. Perhaps the plan for the UK’s recovery after the pandemic or the national data strategy will tackle that particular challenge.
But there are also immediate steps that need to be taken.
We urgently need to get data out of these national programmes and to local places. It will help save lives.
I used to lead the Open Data Institute’s work on data institutions. The team both piloted data trusts and explained that a range of approaches existed – including things like data representatives and data cooperatives – that can change how decisions are made about data. Hopefully to make those decisions more trustworthy. There are many other people working on data institutions in the UK, in Europe and around the world. I’m often surprised by how many.
Over the last couple of weeks I have been talking with people about data institutions. Many of the conversations surface similar implicit assumptions.
There can be only one
In many of the conversations people assumed that there could be only one data institution within a particular context. They had not thought about whether and when there might be multiple.
Some data institutions will exist to steward data for which you might want there to only be a single source of truth[mfn]I know. I do love a bit of epistemology and discussions about the nature of ‘truth’ but that would be an unnecessary diversion in this blogpost[/mfn] – for example the list of Prime Ministers of a country, the list of websites that exist, or who you are married to.
Many others will steward data or have a purpose where there might be multiple things doing roughly similar jobs but, perhaps, with different methodologies or priorities. Maybe one has a purpose of “for the benefit of the people of Newcastle”, another has “for the economic benefit of the people of Newcastle” and a third has “for the benefit of the businesses of Newcastle”. A single word can make a big difference.
Sometimes there should be only one data institution but multiple will exist. That’s life. We live in a wonderfully imperfect world.
Being open to the need to work with other people and other institutions is a better starting assumption than there being only one. Institutions might compete with each other, cooperate with each other, or both, but do expect it to happen.
Rip it up and start again
Another assumption was about the need for something new.
The way we steward data at the moment is not working, therefore we must need a new institution to fix the problem, right? Maybe…
Sometimes we need to fix things that are not working, or at least try to make them better. An existing institution might provide vital services, it might contain valuable knowledge, or it might do things that – shock! horror! – are only loosely related to data. Creating a new institution might break existing and important things.
I do not know of a good methodology to help people decide when to try a revolution and when to try evolution, but do make sure that it is a conscious decision
You forgot government
Many people thought that they needed a new type of data institution – like a data trust or data cooperative – when actually they might just need to improve a simple, old-school democratic institution like a bit of government.
I am very conscious that I live in the UK, a high-income country with an old and (relatively…) stable democracy. Not everyone does. I’ve worked a lot internationally, but mostly in similar countries. In these countries we have many institutions that are already legally responsible and democratically accountable for stewarding data for a particular purpose.
There will be institutions responsible for land registries, local places, criminal justice systems, welfare payments and – in a country with a national health system like the UK – health and social care. Perhaps, rather than working around those government institutions you need to use democratic processes to change their behaviour to make them more useful and trustworthy.
Some people seemed to forget the government and implicitly assumed that they needed to take responsibility into a new institution that they would build and run.
Sometimes we do need to take responsibility away from the government, but at other times we need to add new responsibilities to government or just make existing bits of government work a bit better.
Again, make it a conscious decision.
Building institutions takes time
Building institutions takes time. Not just your time, but other people’s too. It will take even longer if you do not think about why you are doing it and do not surface and challenge assumptions about what any new institutional arrangements should look like.
Assumptions like whether there will be multiple institutions, whether there should be something new, whether the institution should be part of the government, what approach you need, or even whether that approach is suitable for your particular context.
Making those assumptions explicit and challenging them is likely to help you move a bit faster and be a bit more effective at actually making people’s lives a bit better.
Hi, I’m Peter. I currently work at the ODI (Open Data Institute) where I am Director of Public Policy. I will start with my usual warning, particularly for an audience where English is not the first language. Sometimes I speak too quietly and too fast and I often make bad jokes and obscure references. I’m bad like that. This is my last public talk for the ODI so I am even more likely to do that than normal. Please tell me off if you cannot follow what I am saying. I will stop and get better.
About the ODI and about me
The ODI is a not-for-profit that works with businesses and governments to help build an open and trustworthy ecosystem. The ODI believes in a world where data works for everyone. As simple to describe, and as hard to achieve, as that.
In that world data improves the lives of every person, not necessarily every business or every government. Some businesses and governments are deliberately building new monopolies or causing harm to people. Sometimes it is not possible to fix that behaviour by working with organisations, instead it needs other ways to change behaviour. I will talk about those later.
At the ODI I have been heading up the public policy function — I’ve been responsible for the ODI’s views on the role of data in our societies.
I am a technologist by background and I somehow stumbled into the world of public policy a few years ago. One of the things I have been focussed on in that time is making sure that public policy is informed by and tested in practical research and delivery (and vice versa, that delivery work aligns with policy thinking). Data, technology and people are always changing. A strong link between practice and policy helps make stuff useful.
I am here to talk about practical data ethics. I would like to start by talking about how we create value from data; why we need to change the behaviour of people and organisations that collect, share and use data; and finally to talk about some possible interventions to change behaviour — including practical data ethics.
Creating value from data
Value is created from data when people make decisions.
To maximise the decisions that can be made we need to create tools that meet the needs of different decision makers — for example a mapping app to help me find the building that we are in today, a bit of sales and customer analysis to help a business decide whether to invest in a new product, or a research project to help a government decide whether and where to build a new road.
To create this range of tools we need to make data as open as possible.
This needs stewards — the people who decide who can get access to data — to make it accessible in ways that creators can use. There are a number of reasons why they might do this but it is (hopefully!) always driven by the need to use the data to tackle a problem by making a decision.
The problems with data
Unfortunately there has been a rush to collect data, open up data, share data, or make more decisions using data without thinking about whether or not we should.
This is an ethics event so I am going to start by talking about harms. Rather than organisations making data work for people, they make it work against them.
Harm to groups of people is not always caused by personal data. The excellent book Group Privacy contains many examples. One that sticks in my head is from the South Sudanese Civil War. The Harvard Humanitarian Initiative published analysis created from satellite imagery to help people find and get aid to refugees. Unfortunately terrible human beings used the same analysis to find and attack those same refugees. The tools that the team had available had helped them think about mitigating the risk to individuals from the release of personal data, but not the threats to groups of people created by non-personal data.
And as a final example there has been damage to our democracies. The use of data in political advertising, to spread misinformation, or most famously in the Facebook/Cambridge Analytica debacle. Personally I do not think that the data collected by Cambridge Analytica had much effect, I reckon they sold snake oil, but the fear of it having had an effect is damage in and of itself.
Left unchecked these harms will lead us to a data wasteland where organisations do not collect or use data, people withdraw consent and give misleading data, and as a result we will get poor conclusions when we try to make decisions based on data. It reduces the social and economic value that data could create.
But there is another type of harm. Where people and organisations collect data but use it only for their own purposes. They don’t make data work for everyone. They just make it work for themselves.
This is data hoarding. It is the attitude that “data is oil and I must control it”. Data is collected and used within a single organisation for too narrow a purpose.
A simple example comes from Google. In recent years Google have encouraged people to crowdsource data about wheelchair accessibility in cities so that it is easier for people in wheelchairs to move around. But the data is only available in Google Maps. The people who contributed the data would surely have wanted it made more widely available so that people in wheelchairs who used Apple Maps could find their way around, or that the data was made available to civil society and city authorities who might have been able to use it to improve wheelchair accessibility in cities. Instead the data is hoarded by Google to create a competitive advantage and bring in more customers
There are vast amounts of data locked up in data monopolies like Google, Facebook, Apple, and legacy organisations like big multinational corporates or national mapping agencies.
This leads to lost opportunities for innovation. Innovation that might have created better outcomes for people. As a result lotsofpeople are looking at data as a competition issue at the moment.
It also leads to lost opportunities for understanding and tackling major societal challenges like understanding the impact of the internet and web on our democracies, how to cope with aging populations or increasing urbanisation, or how to prevent or reduce the impact of climate change. We need to be careful of vital data infrastructure becoming over-reliant on the private sector firm, and the excessive data collection caused by some business models, but just imagine the data held by governments and businesses that could be made safely available to help with these problems.
The challenge is finding a path between the data wasteland and data hoarding. If we make data too open and available then it causes harm, if we do not make it open enough then we lose benefits and concentrate power in monopolies.
We need to move from a world where people are rushing to collect, share and use data to one where societies have more strategic decision making about data. Where data is as well maintained and useful as other forms of infrastructure like road, rail and energy. Where there is better legislation, rules, guidelines, and professionalism.
In doing that we need to recognise that different societies will make different decisions about data. Just like they make different decisions about other forms of infrastructure. People’s needs and social norms vary.
As long as we stay within democratic norms and respect fundamental human rights then we should accept those differences. Many of my examples today are from high-income countries but personally I am excited to see what new futures emerge from the rest of the world. That would be a different talk though.
Anyway, moving to a better data future will require constant monitoring and intervening by a range of people and organisations. The ODI is one of the organisations doing that monitoring and intervening. The strategy for how and when we do it is on the website.
Possible interventions
It is essential to think about the ecosystem around data and to think about multiple points of intervention. To create a world where data works for everyone many forms of intervention are needed. I am going to touch on some before getting to practical data ethics.
Many people start by thinking that better choices by citizens and consumers can change the world. Consumer power is the answer. Consumers will pick services from organisations that cause less harm and create more benefits.
Many people say that consumers are happy with the current situation — why else would they be using these organisations and services? Unfortunately work in the US by the academics Nora A Draper and Joseph Turow on digital resignation and the trade-off fallacy, and our own recent piece of work on how people in the UK feel about data about us, shows that most people do care and want a different future but that they feel unable to get there.
One of the things that is lacking is choice for consumers. The previously mentioned work on digital competition, and things like interoperability and data portability, will help but it will take time. It is not going to reduce some of the harms we can all see right now.
Regulators can intervene. In the UK the Open Banking movement designed a framework which was adopted by the UK’s banking regulator. It tackled competition issues, by giving bank customers more control over data about them, and had measures to protect against harms. Rather than open banking being solely down to consumer choice a regulator approves who bank customers can share data with. I helped a bit both with the framework and the persuasion to get it adopted. The process has taken at least four years and is just starting to see changes that benefit people.
Another necessary point of intervention is legislation. This is essential and can radically change the behaviour of businesses and governments. But again legislation takes time. That is a feature, not a bug, of democracy. Democracy comes with debate and compromise. GDPR took six years from the first legislative proposal until it came into force.
For more immediate change there is existing legislation that could be used — for example anti-discrimination legislation and worker’s rights — but that legislation is likely to need updating as, like any legislation, we will learn that there are gaps and changes to be made.
But these new institutions are in a research and development stage. We have to be realistic that it will take more time to determine if they are useful, where they are useful, and how to build and regulate them.
Practical data ethics
There are many other possible points of intervention but one important and often overlooked one is the people within the organisations that collect, share and use data. Which brings me (finally!) to practical data ethics.
In the USA there have been growing protests by tech workers against the decisions made by their employers, in the UK research by DotEveryone found that “significant numbers of highly skilled people are voting with their feet and leaving jobs they feel could have negative consequences for people and society.” Meanwhile consumers and citizens are saying that they do care and do want more ethical technology, organisations respond to that. The need to retain both workers and customers creates a need to change.
We should never forget that, as my friend Ellen Broad put it in her book, decision are made by humans. Humans decide to fund or stop projects, to buy technology, they make design and development decisions, and they decide whether and how to evaluate its outcomes.
These decisions are influenced by consumers, governments and regulators but they are also influenced by other things such as professional codes, training courses and organisational methodologies.
This does not mean that principles are useless, within an organisation they can demonstrate values and help create space for challenge, but we need to look at other techniques to make them more useful at the systemic level where the ODI is looking to intervene.
When Ellen Broad and Amanda Smith looked at this for the ODI a few years ago. They came to the conclusion that the most useful thing for the ODI for to do was something a bit more practical and a bit more like the tools that people already use.
In the two years since then various other people — like Fiona, Anna and Caley — have worked with me to iterate it and helped turn it into to what you can see today. Not all of those people work for the ODI. We have been iterating it based on feedback from our own users and audience too.
The canvas does not give easy answers it ask questions. It encourages people to take responsibility for coming up with their own answers in their own contexts. The questions are inspired by the problems we and other see.
It prompts people to think about their existing ethical and legislative context — perhaps they are already covered by health ethics or anti-discrimination legislation, or one of the many sets of AI and data ethics principles— and the limitations of data.
The canvas prompts people to think of both possible positive and negative effects, but it encourages them to think more deeply about which groups of people win and lose.
The canvas is designed to be used by multi-disciplinary teams of people, not just individuals. We have seen it used by groups including lawyers, developers, programme managers, user researchers, policy analysts, designers and product managers. It encourages people in organisations to create space and time for debate, and then to make and act on decisions.
The canvas also encourages transparency and openness. That way people outside an organisation can see how it plans to use data, what benefits and risks are expected, and what mitigation plans are in place. It encourages people in organisations to listen to people who they might affect.
But is it having any effect?
I have used it in public training, private workshops and conversations with a range of organisations. I have seen it broaden people’s minds about the range of ethical issues that they should consider before making a decision. I have seen senior people in organisations try it in a few projects then go on to implement it in their standard project governance.
I have also seen individuals sneak it into a few projects within a large organisation with the goal of proving its value before talking more with their bosses. You normally don’t need permission to try a new methodology. Give it a go in your own organisations.
It is hard to track usage of something that is openly published on the web but I know from our own research and surveys that hundreds of people in public, private and third sector organisations at local, national, and global levels are using it because of that decision to make it openly available.
Those people work in multiple sectors like academia, civil society, public service, health, finance, engineering. Some are in large corporates, some in small startups. People tell me that some organisations have stopped projects because of questions raised by the canvas. Others say that they have redesigned products and projects. Brilliant. It is causing some decisions to be made.
I can only share those stories vaguely, because I respect the confidence and privacy of those people.
One organisation, the UK Cooperative Group, have talked most about their use of the canvas. It forms part of their standard product development model. Because the canvas has an open licence they could adapt it to suit their own needs. Perfect. I hope some of the many, many others will share their stories too. I think it will be less scary than they might think.
I am always wary of over-confidence. At a place like the ODI we get listened to and the canvas could actually be making things worse. Is the effect overall positive and how big is it? Only time and more detailed evaluation will tell. But from my own checks I am reasonably confident that it is helping.
Obviously this approach to practical data ethics is only one type of intervention. Accountability — through organisational processes, professional codes, regulation and legislation is still very much needed. But practical data ethics can create some practical change now. If we can get people to be more open with their tales it should also inform policymakers on where the biggest problems are and what regulation and legislation is needed.
Building a better future for people with data will take quite a while. There are some obvious problems, some of which have obvious answers, but there also less obvious problems and no easy answers for all of the problems. We all have to keep monitoring and intervening at multiple points in the system.
We need to stay optimistic and believe that it is possible. I believe being optimistic is a political act that makes it more possible that we will build a world where data works for everyone.
Anyway, I have rambled on too long. It is time for less talking from stage and more talking with each other. Grab me if you want to chat or email me on peterkwells@gmail.com if you do not get a chance.
At the Open Data Institute we use a theory of change. It is one of the tools that we use internally to help us make decisions and externally to explain to people what we do and how we do it.
Our theory of change describes the farmland, oilfield and wasteland futures and helps us try to steer between the extremes of the oilfield and wasteland futures to get to the farmland.
The wasteland future emerges when there are unaddressed fears arising from legitimate concerns — such as who has access to data and how it might be used.
We frequently talk through the theory of change to explain what we do and how we do it. We try to provide pauses in the conversation to get other people to give their opinions. It helps people to think and learn for themselves. It helps us learn too. We hear what other people think happens in the wasteland future. How they think people and organisations will react to their fears being unaddressed.
Most of us the people we talk with think that the wasteland future has a lack of data. They realise that with a lack of trust then many people and organisations will reduce how much data they share. They imagine people refusing to use services because they don’t trust them, and that organisations similarly refuse to share data because they fear being punished. They think the data stops flowing.
These tips—based on insurers using social media data to set premiums—are stunningly dystopian. pic.twitter.com/uceFFIThyO
A smaller group of people realise the wasteland is more complex and weird. People’s behaviour will change in many different ways. Humans are fun like that.
Some people might post inaccurate data. Perhaps you will post fake claims of jogging exploits to social media if it is the only way to get a fair life insurance deal. Other people will hide in the data. Maybe we will give our children common names so they are hard to identify or so they appear to be from an ethnic group that is not discriminated against.
I’m sure that even if you hadn’t thought of them at first you can now think of many more things that happen in the wasteland future.
You can see some of this future now. There are already people and organiastion hiding in the flows of data. Some of those people need and deserve help to hide because they have a genuine fear of harm, perhaps due to their political beliefs, ethnicity or sexuality. Equally there are others who are trying to evade fair scrutiny, for example tax dodgers and other criminals, and organisations providing services to help them do so. But if we increasingly fear harm then more people will want and need these services and, inevitably, they will become ever cheaper and used by more of us.
As this behaviour becomes widespread we will see data that is massively biased and misleading. People and organisations that use data-enabled services to tackle global challenges such as global warming, to price a life insurance premium in a way that doesn’t unfairly discriminate, or to decide whether or not to take a job will struggle. That would not be good for any of us.
Navigating the a route between the wasteland future and a different future where we get more economic and social value from data will not be easy. There will always be some people who need to pollute and hide in data to protect themselves from harm, we need to allow that to happen. Understanding and addressing people’s fears is not only a technical challenge, it is also a social and political one. To retain trust we need businesses and governments to adapt to people’s ever-changing expectations in a range of cultural contexts.
An increasing fear of how data is used will not simply stop people using services or sharing data, it will change peoples behaviour in a range of ways. If that happens we can expect data to be increasingly poor quality, biased and misleading. And that pollution will make data less useful to help people, communities and organisations make decisions that hold the potential to improve all of our lives. Some of that potential is false — the use of data required is too scary and people do not want or need it — but that is why it is important to understand and address the concerns we can if societies are to navigate towards the farmland.
Hello. This is the personal website of Peter K Wells. I do politics, policy and delivery to try to make data and technology benefit everyone. I also do bad jokes and music references.
This website stores cookies on your computer. These cookies are used to provide a more personalized experience and to track your whereabouts around our website in compliance with the European General Data Protection Regulation. If you decide to to opt-out of any future tracking, a cookie will be setup in your browser to remember this choice for one year.
