Could low-cost, multipronged testing of innovative ideas—a strategy that Amazon CEO Jeff Bezos has used with stellar success—fix our justice system? NYU Prof. Angela Hawken has pioneered a public policy app that, she tells the ‘New Thinking” podcast, will prove it can.
Addressing the problems of our criminal justice system, and particularly analyzing whether or not justice policies actually work, can be time-consuming—and costly. But what if we rethought the way such evaluations are done?
Matt Watkins’s podcast, New Thinking, for the Center for Court Innovation, focuses on just that. His latest podcast episode features a professor who has applied a strategy used successfully by Amazon CEO Jeff Bezos to test multiple ideas with minimum cost.
Angela Hawken, a professor of public policy at New York University, is the founder of BetaGov, a program that offers free and fast evaluations of public policy programs.
Hawken says that too many research projects use the “Cadillac” model, employing expensive, time-consuming tests that make it difficult for researchers to halt the study if their results aren’t productive.
In developing her model, Hawken looked at research models of successful companies like Amazon. She cites Jeff Bezos, the CEO of Amazon, as saying that he attributes the success of Amazon to its ability to run thousands of low-costs tests of ideas or apps at one time to test whether they attract an audience, instead of a few dozen or so.
Prof. Hawken set out to make what she calls an alternative, “exploratory” track of research that’s “inexpensive to do and can be shut down very nimbly if the outcomes aren’t moving in the intended direction.”
BetaGov uses mostly randomized controlled trials, or RCT’s. Hawken says RCT’s are the “gold standard” of research. Because they allow for causal claims, which determine the effect of the intervention being measured between two similar populations.
RCT’s are especially valuable in criminal justice research, Hawken said, because it often suffers from selection bias, when two things are compared that are different, which skews the results.
“So you’ll see many evaluations in criminal justice that compare, for example, treatment completers to people who did not go to treatment,” said Hawken. “The sorts of people who complete anything are different from the sorts of people who either don’t start or drop out along the way.”
Another major innovation of BetaGov utilizes practitioners on the front lines of the public policy and justice system across the country, instead of using a large research team. Hawken refers to them as “pracademics”—people who work in government organizations and want to test ideas that might be of value.
Hawken describes them as people “in the public sector who have looked around their offices and their buildings and the clients that they were serving and asked themselves the what-if question. What if we did it differently?”
If a practitioner runs a test and finds promising outcomes, it’s quickly replicated in another location to see if it holds up.
A byproduct of this democratizing approach is that it allows for more diverse input on what gets researched. “We wanted to shatter the monopoly over who gets to decide what is being tested,” said Hawken.
“Very few people typically get to weigh in on what will become an evidence-based program or practice.”
This approach extends further to what Hawken calls “collaborative design,” which is listening to ideas from people who are closest to the problems facing the criminal justice system.
Hawken spoke about her time in Washington state meeting inmates in solitary confinement. Their ideas were so useful that Hawken says they turned into policy reforms in several states.
Hawken said other inspiration for her approach with BetaGov came from a paper written by a roboticist at MIT in the 1980s. The paper argued that instead of sending massive probes or satellites into space, we should send thousands of little “bugs,” that would signal when they detected something valuable.
“And I thought, wow, what if we did that in the government sector?” she said. “We blasted out thousands of innovations and then let the promising ones cluster…We tend to send out the big mission first, then find it’s dark, and then get mad.
“We need to change how we do this. That doesn’t mean there (isn’t) room for the traditional academic model, of course there is. But it’s a parallel track.”
Accepting that most attempts will fail is central to the approach behind BetaGov as well. Its low-cost structure helps to engender this line of thinking.
“When no money’s at stake, people are also more willing to fail,“ Hawken said. That’s so important, because there’s failure all around us.”
Dane Stallone is a TCR news intern. Readers’ comments are welcome.