What: policy-orienting document

Impactscore: 4

For who: Companies, sector organizations, academics and other persons involved in the auditing of algorithms



Note: The DRCF is a cooperation between several British regulators including the Competition and Markets Authority (CMA), the Information Commissioner’s Office (ICO), the Office of Communications (Ofcom) and the Financial Conduct Authority (FCA).

In this discussion paper, the DRCF identifies current issues relating to the auditing of algorithms ( encountered by regulators but also by organisations themselves, civil society, academia, etc.). Algorithmic auditing is useful for many different parties, such as the involved organisations (to improve their systems), the affected persons, regulators, etc. On the basis of the identified issues, the DRCF explores the future ecosystem of algorithmic auditing and the role of regulators herein. The DRCF also describes the different types and outcomes of audits.

The DRCF identified the following current issues:

  • A lack of rules and standards (for auditing) in the algorithmic auditing landscape (with the exception of some highly regulated sectors such as aviation and health);
  • Auditors entering the market without quality assurance and without commonly agreed upon auditing standards;
  • Inconsistency in what the audits focus on;
  • Limited access to systems by auditors (especially academics and non-profits);
  • Incomplete cooperation and inconsistent documentation from organisations;
  • Legal risks for auditors scaping information from systems;
  • Insufficient means of redress for affected persons even after the audit.

The DRCF recognises the important role of regulators in addressing these issues and considers several plausible solutions. For example providing guidance on the timing of audits, establishing best practices (who needs to disclose information to whom?) and assisting with the development of standards (for processing, auditing and performance). Regulators should dulyconsider when they should act themselves and when they should act as facilitator. In that regard, they may accredit third-party auditors to certify systems in compliance with with regulatory standards.

Regulators can also help to ensure sufficient access to systems for auditors and to ensure appropriate (regulatory) actions after an audit. They may make it easier to report harms and provide incentives to come forward with relevant information. The DRCF also considers the need to balance the benefits of these new auditing requirements with the regulatory costs for organisations.

In that regard the DRCF welcomes cooperation and collaborative action with industry to address these issues and launches a call for input to inform their future work.