A new lab measures how algorithms impact different populations

New York-based independent research institution Data & Society is opening a new lab to study how automated decision-making systems are impacting the world, the organization exclusively told Semafor Wednesday.

The Algorithmic Impact Methods Lab will develop new tools that can measure how algorithms are affecting different populations.

It will likely focus on systems being used for high-stakes purposes, like making hiring decisions, allocating public benefits, or determining criminal justice outcomes, said Jacob Metcalf, who leads Data & Society’s existing AI on the Ground Initiative.

Metcalf compared the new lab’s work to conducting environmental impact assessments, which real estate developers are often required to complete by law before starting new projects.

Several bills introduced at both the state and federal level would similarly require tech companies to conduct algorithmic impact assessments before deploying new technological systems. But the problem, Metcalf said, is that no one knows how to do them. Data & Society is hoping its new lab will help to change that.

“We’re really worried that if these bills pass and there aren’t established, tested methods that really take into consideration the needs and interests of impacted populations, then the tech companies will get to grade their own homework,” said Metcalf.

• Data & Society has previously published research about how to design algorithmic impact assessments and concluded that a one-size-fits-all approach would not be effective.
• U.S. Senate Democrats introduced a bill last year that would require tech companies to “conduct impact assessments for bias, effectiveness and other factors, when using automated decision systems to make critical decisions.”