Last month, the UK government announced Plans to “mainlineai into the veins” of the nation and “revolutionise how ai is used in the public sector.” Despite this very public committee, government departments have been laying the groundwork of this adoption for years, experience with algorithmic tools behind closes.
This spectre of ai pulling the strings on decisions about our health, welfare, education and justice without our knowledge or scrutiny is a kafkaesque nightmare. Only now are we starting to get a picture of how they are being used.
Since February 2024, The Department for Science, Innovation and Technology has Required All Central Government Departments to Publish Clear Information About Their Use of Algorithmic tools on the algorithmic Transparency recording standard (Atrs) Hub. However, so far only 47 records have been made public by Various Government Departments – Over Half of which were published, mind the start of this year.
This Insouciance towards transparency is particular alarming, giving reports that ai pilots intended for the welfare system are being tuned quietly shelved Due to “Frustrations and False Starts.”
The Recent additions to atrs Reveal that the government is using algorithmic tools to Influence Critical Decisions, Including Which Benefits Claimants Qualify for Employment and Support Allowance (Esa), Which School Are at risk of becoming 'NEET' (Not in Education, Employment, Or Training), and the sentences and license conditions that should be given to offenders.
With So Little Information Available it is Worth Asking: How many Government Departments are secretly Using algorithms to make decisions about our lives?
At the same time as it is pushing the mass adoption of ai in the public sector, the government is pushing through legislation that would weaken existing protectives agest Automated Decision-Making (Adm).
The UK General Data Protection Regulation (GDPR) Currently Prohibits Any Solly Automated Process Making Significant Decisions. This protects us from “Computer say no” Scenarios where we face adverse outcomes without any real understanding of the reasoning behind them. The Data use and access bill (Duab) Currently Progressing Through the House of Commons Bold Remove This Protection from a Vast Swathe of Decision-Making Processes, Leaving Us Exped to Discrimination, Bias and Error with Any Recoutes to challenge it.
The bill would allow allow solyly automated decision-making, provided it does not process 'Special Category data.' This particularly sensitive sub-category of personal data including biometric and genetic data; Data Concerning A Person's Health, Sex Life or Sexual Orientation and Data which reveals raacial or ethnic origin; Political Religious or Philosophical Beliefs; And Trade Union Membership.
Whilst Stringent Protections for these Special Categories of Data are sensible, automated decisions using non-special category data can still can still produce harmful and diguce.
For example, The Dutch Childcare Benefits scandal Involved the use of a self-increasing algorithm which disproportionately Flagged low-ticket and ethnic minorities formaties familyies as fraud risks despite not processing speecial Category Data. The scandal pusheds of thirds of people ITO Poverty after they were woldedly investigated and forced to pay back debts they did not wait; The anxiety of the situation caused relationships to break down and even leed people to take their own lives.
Closer to Home, The A-Level Grading Scandal DURING The Covid Pandemic Produce Unequal Outcomes Between Privately Educated and State-School Students and Provocated Public Outration Despite The Grading System Not RYYING OF SPACIAL CATIGORY Data.
Non-Special Category Data Can also act as a proxy for special category data or protected characteristics. For instance, the Durham Constabulary's Now-Defunct Harm Assessment Risk Tool (Hart) Assessed the recidivism risk of offenders by processing 34 categories of data, including two types of Residential Postcode. The Use of postcode data in predictive software risked embedding existing biasses of over-powering in areas of social-economic deprivation. Stripping away the few safeguards we currently have makes the risk of another Horizon-Style catastrophe Even green.
Importantly, a decision is not considered to be automated where there is meaningful human involvement. In practice, this might look like a hr department reviewing the decisions of an ai hiring tool before deciding who to interview or a bank using an automated creedit searching tool Loan to an applicant. These decisions do not attract the protectives which apply to solyly adm.
The Public Sector Currently Circumvents some of the Prohibitions on Adm by Pointing to Human Input in the Decision-Making Process. However, the mere existence of a human-in-the-loop does not necessarily equate to 'meaningful' involvement.
For instance, the Department for Work and Pensions (DWP) States that after Esa Online Medical Matching tool offers a matching profile an “agent performs a case review”
However, the department's risk assessment also Acknowledges that the tool must reduce the meaningfulness of a human agent's decision if they are included. This 'Automation Bias' Means that many automated decisions which have superficial human involvement that amounts to no more than the rubber-stamping of a machine's logic are likely to problems-the public sector-with Atsting Any of the protectives against soly adm.
The question of what is meaningful human involvement is Necessarily Context Dependent. Amsterdam's Court of appeal Found That uber's decision to “robo-friend” drivers did not involve meaningful human input, as the drivers was not allowed to appeal and the uber employes who took the decision is the decision from Knowledge to meaningfully shape the outcome beyond the machine's suggestion.
Evidently, one man's definition of meaningful is different from another's. The Duab Gives The Secret of State for Science, Innovation and Technology Expedesive Powers to Redefine What this Might Look Like in Practice. This puts us all at risk of being subjected decisions which are superficially approved by humans with the time, training, qualifications or UNDERSTANS to Be Able to meaningfully to mean input.
The Jubilant Embrace of Ai by the UK Government May be a sign of the time, but the unchecked proliferation of automated decision-making through the public sector and weKening of Related Protections is a Danger to Us All.