The ‘Amazon Approach’ to Criminal Justice Reform

Could low-cost, multipronged testing of innovative ideas—a strategy that Amazon CEO Jeff Bezos has used with stellar success—fix our justice system?  NYU Prof. Angela Hawken has pioneered a public policy app that, she tells the ‘New Thinking” podcast, will prove it can.

Addressing the problems of our criminal justice system, and particularly analyzing whether or not justice policies actually work, can be time-consuming—and costly. But what if we rethought the way such evaluations are done?

Matt Watkins’s podcast, New Thinking, for the Center for Court Innovation, focuses on just that.  His latest podcast episode features a professor who has applied a strategy used successfully by Amazon CEO Jeff Bezos to test multiple ideas with minimum cost.

Angela Hawken, a professor of public policy at New York University, is the founder of BetaGov, a program that offers free and fast evaluations of public policy programs.

Hawken says that too many research projects use the “Cadillac” model, employing expensive, time-consuming tests that make it difficult for researchers to halt the study if their results aren’t productive.

In developing her model, Hawken looked at research models of successful companies like Amazon. She cites Jeff Bezos, the CEO of Amazon, as saying that he attributes the success of Amazon to its ability to run thousands of low-costs tests of ideas or apps at one time to test whether they attract an audience, instead of a few dozen or so.

Angela Hawken

Angela Hawken

Prof. Hawken set out to make what she calls an alternative, “exploratory” track of research that’s “inexpensive to do and can be shut down very nimbly if the outcomes aren’t moving in the intended direction.”

BetaGov uses mostly randomized controlled trials, or RCT’s. Hawken says RCT’s are the “gold standard” of research. Because they allow for causal claims, which determine the effect of the intervention being measured between two similar populations.

RCT’s are especially valuable in criminal justice research, Hawken said, because it often suffers from selection bias, when two things are compared that are different, which skews the results.

“So you’ll see many evaluations in criminal justice that compare, for example, treatment completers to people who did not go to treatment,” said Hawken. “The sorts of people who complete anything are different from the sorts of people who either don’t start or drop out along the way.”

Another major innovation of BetaGov utilizes practitioners on the front lines of the public policy and justice system across the country, instead of using a large research team. Hawken refers to them as “pracademics”—people who work in government organizations and want to test ideas that might be of value.

Hawken describes them as people “in the public sector who have looked around their offices and their buildings and the clients that they were serving and asked themselves the what-if question. What if we did it differently?”

If a practitioner runs a test and finds promising outcomes, it’s quickly replicated in another location to see if it holds up.

A byproduct of this democratizing approach is that it allows for more diverse input on what gets researched. “We wanted to shatter the monopoly over who gets to decide what is being tested,” said Hawken.

“Very few people typically get to weigh in on what will become an evidence-based program or practice.”

This approach extends further to what Hawken calls “collaborative design,” which is listening to ideas from people who are closest to the problems facing the criminal justice system.

Hawken spoke about her time in Washington state meeting inmates in solitary confinement.  Their ideas were so useful that Hawken says they turned into policy reforms in several states.

Hawken said other inspiration for her approach with BetaGov came from a paper written by a roboticist at MIT in the 1980s. The paper argued that instead of sending massive probes or satellites into space, we should send thousands of little “bugs,” that would signal when they detected something valuable.

“And I thought, wow, what if we did that in the government sector?” she said. “We blasted out thousands of innovations and then let the promising ones cluster…We tend to send out the big mission first, then find it’s dark, and then get mad.

“We need to change how we do this. That doesn’t mean there (isn’t) room for the traditional academic model, of course there is. But it’s a parallel track.”

Accepting that most attempts will fail is central to the approach behind BetaGov as well. Its low-cost structure helps to engender this line of thinking.

“When no money’s at stake, people are also more willing to fail,“ Hawken said. That’s so important, because there’s failure all around us.”

Dane Stallone is a TCR news intern.  Readers’ comments are welcome.


Navajo Justice Officials: Funding Shortages Worsen Public Safety Crisis

Covering an area of about 27,000 square miles, the Navajo Nation is the largest Native American territory in the US, yet its legal infrastructure is severely understaffed and underfunded, according to senior tribal law enforcement officials, speaking on the “Native America Calling” podcast.

Covering an area of about 27,000 square miles, the Navajo Nation is the largest Native American territory in the US, spanning Arizona, Colorado and Utah. Yet its legal infrastructure is severely understaffed and underfunded.

In an exploration of tribal justice issues, leading Navajo justice officials said on the podcast Native America Calling that the lack of sufficient resources for law enforcement infrastructure was intensifying the “public safety crisis” facing Navajo people.

With less than 200 police officers to cover a territory bigger than many states, Navajo Police Chief Phillip Francisco said that despite some increases in funding, more than twice that number was needed to provide a suitable amount of protection.

“We’re trying to rebuild and hire more officers,” said Francisco. “We’ve established our own police academy, (and) we’re the only Indian law enforcement agency that has that.”

But he said the sheer distance between police districts, combined with staff shortages, meant that it could take up to an hour and a half to respond to some calls.

“When we get there a lot of the evidence is gone, or the suspects are gone, so we have to do a lot of follow ups.”

Data consistently show that tribal nations suffer among the nation’s highest crime rates. According to FBI statistics, the Navajo Nation has a violent crime rate higher than most major US cities.

Francisco said the tribal police communication infrastructure is a major issue as well.

“We don’t get 911 data, so we can’t locate where people are calling from,” he said “Sometimes people call but they can’t get through our dispatch center, so crime is probably underreported because people can’t get through to the police department.”

Because the Navajo Nation has so few officers, sometimes there are only two officers on shift at a time.

“If two officers are on a domestic violence call, you don’t have anyone else, so we have to wait until we have someone else to respond (to other calls coming in),” said Francisco.

Ethel Branch, Attorney General of the Navajo Nation, said 40 percent of Navajo territory is in a “cell dead zone.”

“That makes it difficult for people to call for help when they need it,” he said. “60 percent is without two-way radio coverage, which is how officers communicate and respond.”

Describing the direness of the situation, Branch said, “Our homicide rates, which I find very alarming, are consistently meeting or exceeding high national homicide rates.”

Branch thinks that Congress needs to step up and ensure that Indian country citizens are getting the resources they deserve.

“Native Americans serve at the highest per capita rates in the military,” he said. “We ensure American families are kept safe, and we expect our families to be kept safe, even if we choose to live within our Indian nations.”

Branch went on to say that she thinks the Navajo Nation needs direct funding from the federal government, instead of funneling dollars through the states.

Gertrude Lee, chief prosecutor of the Navajo Nation, agreed that the Navajo Nation needs more resources, but says things have gotten better during her time there.

“We have enormous caseloads, but I’ve seen much improvement from what I walked into,” she said.”We’re doing the best to work with the resources we have available.”

Lee said that when she started, five of the nine district offices didn’t have prosecutors in them. She had seven prosecutors on her staff, but now there are 14, and a prosecutor in every office. Lee said that in 2017, her staff went to a combined 3,100 more hearings than the previous year.

Lee also noted that the office of the prosecutor received 3,000 more reports from police districts in 2017 than 2016.

“It’s testament to the Navajo police and the amazing work they have done since Francisco came on,” she said. “Having key leadership positions filled and directing resources to the type of crimes that need to be addressed has a huge impact.”

The police leaders said that despite signs improvement, the Navajo Nation is still needs more resources allocated from the Federal government.

“Federal funds haven’t been revised in 15 to 20 years, so we’re really trying to plead our case to Congress and Washington,” said Francisco. “We are critically understaffed for the area and demographics we need to police.”

See also: Justice Returns to the Navajo Nation

Dane Stallone is a TCR news intern. He welcomes readers’ comments.


One Man’s 22-Year Search for Justice

Calvin Buari, convicted of a double murder he didn’t commit, was a casualty of over-zealous prosecutors in New York’s tough-on-crime era of the 1990s. In a conversation with The Crime Report about his new podcast, “Empire on Blood,” investigative journalist Steve Fishman tells the story of the battle to clear his name.

Calvin Buari spent 22 years in prison for a double murder he didn’t commit. He was a notorious crack dealer in The Bronx, N.Y., when he was arrested, which made the struggle to prove his innocence─and find allies who could help get his case heard─that much harder.

But when he persuaded veteran journalist Steve Fishman to investigate, his fortunes changed.

Buari was exonerated in May 2017, seven months after his conviction was vacated and he had left prison. The story of Buari’s search for justice is told in a recently released podcast, “Empire on Blood.” The seven-part podcast takes listeners back to the New York of the 1990s, when the city was reeling from a crack epidemic and 2,000 murders a year, and newly elected Mayor Rudy Giuliani came to power promising “tough-on-crime” strategies. But it also offers troubling lessons for today about the collateral damage of those strategies.

In a conversation with TCR Deputy Editor Victoria Mckenzie, Fishman reflects on his seven-year-long investigation, on how he battled his own doubts about the case at the beginning, and on what his podcast reveals about the workings of the criminal justice system.

The Crime Report: Why were you obsessed with this case?

steve fishman

Steve Fishman in the studio (courtesy Panoply Media)

Steve Fishman: I resisted the obsession for a long time, but I guess the origin story, because I’ve thought about this a bunch, is the first call with Cal. He had been put in touch with me by a guy I knew, Emel McDowell, who in fact had won his innocence after being convicted of a murder at the age of 17. Emel says to me ‘I think Cal is innocent,’ so Cal picks up the phone and cold-calls me. And there he is at the other end of the line. Having had some experience with prisons, I can picture Cal on the payphone, standing with these inmates behind him waiting on line for their 15 minutes. And he’s kind of racing through the details of his case and all of this evidence, and it’s a blur—even if I really wanted to I couldn’t have deciphered it.

Maybe it was one of those moments where I was particularly open or vulnerable. I mean, his voice was filled with despair despite the fact that Cal, as I later came to know him, really learned patience and emotional control in prison. I’m imagining this guy with the whole system stacked against him running this campaign for freedom from this prison payphone. It was one of those moments where I was able to imagine, to whatever extent I could, that it must be excruciating to face those kinds of odds.

[I said] “OK Cal, send me the transcripts,” and then these 1,100 pages arrived. It turns out that not only is this one prosecution witness, Dwight, Cal’s “great friend,” really organizing the prosecution, but the prosecution is handing out deals. All of the witnesses who are brought in by Dwight have received an order of protection, which means that Cal and his defense attorney cannot know the names of witnesses who are testifying against him until they walk to the stand. So I’m reading this and I’m thinking— this is crazy! This is insane, this is unfair.

And then I read that in chambers, the judge is saying the same thing. It’s the first day of trial, and the prosecutor in chambers says he just got a phone call from a new witness, and it’s like raining witnesses… and the judge says, for all I know there’s a call out to the entire northeast, any drug dealer wanting to testify will have the charges dismissed against them!

And it just became a moment for me where you start to feel like—the system’s not fair.

It became a road that I was willing to walk. And I have to tell you, it wasn’t easy. I mean, Cal’s no angel. I liked him, I think he’s changed— but he was a guy who was boasting that he helped bring crack to The Bronx. He was a very successful drug dealer and he was ruining neighborhoods, and he needed to be off the streets.

TCR: It seems challenging to get the general public to care about the wrongful conviction of an admitted crack dealer

Fishman: It was a real problem for the audience, and it was a problem for editors— it was a problem for the lawyers that Cal reached out to! He got on the phone and told them the evidence, and when they heard he was a drug dealer, they said: “OK, best of luck.” But I came to the belief that the criminal justice world and maybe the world in general is a complicated place, and the podcast really takes that head on, and I’m proud of that.

oscar michelen

Courtesy Panoply Media

Calvin Buari and his attorney, Oscar Michelen (courtesy Panoply Media)

I think we say, yeah he’s no angel, but what does that mean? What right does that give you as the criminal justice system [to] say, “Well, he didn’t do this crime but he did another— so let the wardens sort them out?” I think that’s a pretty live question for the criminal justice system. People are flawed. Does the past always determine the future? If you believe in rehabilitation and psychotherapy, then no.

TCR: The story also epitomizes a key moment in New York City history, the Giuliani era. Where was your crime reporting at the time? How was this different from the other wrongful convictions?

Fishman: I had reported the wrongful conviction of a guy named David Wong, who was actually in prison for a robbery that he did commit, and then was accused and convicted of a prison yard murder he did not commit. Again, it’s a guy who’s not exactly the high school honor student that you want to root for. The difference for me is that I came in after he had been exonerated and I was telling that story retrospectively. So I was involved in it as a piece of drama that I had all the pieces for.

Cal’s case was very different, because in many ways I was really a catalyst. As a journalist you’re kind of supposed to be a step back, and you’re supposed to be objective. And I certainly was a journalist, as I had to remind Cal at several points. But I was also involved in the story, and I’m going to contend that’s where some of the richness of the narrative comes from.

I started a long time ago in an eastern Connecticut newspaper called the Norwich Bulletin, and the first big story that I ever did was about a rapist-murderer. Part of my portfolio has always been writing about crime, and that’s the reason I would get these envelopes from time to time.

TCR: Who was responsible for Cal finally getting the charges dismissed?

Fishman: Maybe the podcast put a little weight on that side of the scale, because the DA knew it was coming out in a few days and maybe it was a good idea to get ahead of the story. But it was really that Cal was incredibly, incredibly persistent and disciplined. Cal’s a smart and really gifted entrepreneur. One of the things he did on the street that he could not help but be proud of— even when he’s saying “oh, I didn’t realize all the bad I did” — when you get him talking about the crack trade, he says “I was always ahead of the game, I had these marketing gimmicks, I would do sales, two-for-ones,” and there’s a certain kind of pride in his voice. But it is true that he became this quite intuitively gifted businessman.

calvin buari

Calvin Buari (courtesy Panoply Media)

He goes into prison at the age of 24, and spends the next 22 years there. So he really spends his adulthood in prison. And he still has this entrepreneurial gift. He’s writing business plans. He starts a fashion business in prison. And at the same time, he’s maturing. He was all about self-gratification when he was on the streets, and he learns delayed gratification, and patience, and he learns discipline— to the point where he stops even leaving his cell. He just takes his meals in his cell, and reads his books and works on his case, with a level of single-minded devotion that is way beyond my capabilities.

And then, by the way, when he comes out— he’s been out seven months or so, because his case was vacated before the charges were dismissed—he started a business.

TCR: What was your role in the reinvestigation?

Fishman: I will admit that there were a lot of moments when I wavered because Cal’s story was not something that I always believed, not in all of its details. He was a drug dealer in an extremely violent time and he operated on a corner that was dubbed the Corner on Blood [Although] Cal said he was never violent, I always found that to be a little unsettling.

The private eye on the case was a guy who actually had been the private eye on the David Wong case. You meet him in episode 5. He’s a really weird, interesting guy, I mean very colorful. He was on this case and he knew that I was interested in Cal’s case, and he said, “Do you want to take a train ride? There were these new witnesses, and he didn’t know what they were going to say. We’re at a kind of restaurant at a Holiday Inn [in North Carolina], where we meet two sisters. One of the sisters had flashbacks of the scene right in front of me, because she had been 25 feet away [from the murders]. It solidified for me Cal’s innocence. And it certainly helped Cal’s case.

myron beldock

Myron Beldock (courtesy Panoply Media)

So Cal gets the affidavit from her and he manages to get hooked up with Myron Beldock, the legendary attorney who got Hurricane Carter out of prison. Beldock, then 85 years old, comes off his deathbed to basically take up Cal as his last case—and then he dies. And that’s another moment that really pushes me ahead on this path. Cal is a guy who can do 100 pushups without a stop—he’s a tough dude—and he hears that Beldock has died, and he’s in tears. It was just a very emotional moment. It was emotional for him, it was emotional for me. It was one of those moments where I could actually imagine what it might be like to suddenly have this… not just a setback, it’s like your life feels it’s on a course toward freedom, and then the tunnel you’re going through collapses on you.

Cal needed an attorney afterwards. He can’t make phone calls from prison except to people whose numbers are approved. so he would call me and I would conference in people, and I ended up conferencing in Emel McDowell, the guy who initially introduced me to Cal.

Emel had written the appeal that had gotten him out of prison, and then went to work for a law firm. [Emel offered his attorney] and he finally wins Cal’s case. I was a little bit of a provocateur, a little bit of a catalyst, and a little bit of a witness.

TCR: What was your role with the new witnesses?

Fishman: It’s a good question because I was there as a journalist, and I was introduced as a journalist. Journalists don’t usually get to ride along with investigators interviewing witnesses for the first time. I was also there as somebody who had been involved in the case for a long time. I would say that I was not there as an advocate, but I was certainly there as a participant. I did not see my role as urging them to come forward, but I grilled her on her story. It was a journalistic interview, but maybe there’s not too much difference between that and another kind of interview. I wanted to know how close she was, and I wanted to know what she saw, and who she saw, and how could she be sure?

Her story was very, very credible, and frankly I was kind of pleased by that, just from the point of view of having some assuredness about the story that I’d been on by that time for three or four years. it reassured me as a journalist, and because I always occupied a kind of strange space; it buoyed me as somebody who had been involved with Cal for a long time, because I knew, I knew as she said that. I knew that this really could change Cal’s life. It took another two years to get there.

TCR: One of the most striking moments for me is learning what happens to the witness who comes forward after 20 years to testify, how this innocent bystander gets punished by a pretty blunt-edged justice system.   

Fishman: This goes I think to a larger question about how justice works. As you’ve pointed out it was the Giuliani era, the 1990s, there were 2000 murders a year. These guys— the prosecutor, the detective— felt they were the good guys. And they were the good guys. Somebody had to clean up the streets. And it wasn’t always pleasant work to do.

I don’t think that these guys are corrupt. I think that in the context of the times, they felt they had a mandate to act, in a sense, by any means necessary. And if he didn’t do this crime, well he did another crime. and you know what? With Cal, it was true. He didn’t do the murder, he did the drug [crime]. They were going to do whatever it took to make sure that someone like Calvin Buari was not on the streets. But the system can’t work that way.

TCR: Earlier this year, in a conversation about conviction reviews, Brooklyn DA Eric Gonzalez, commenting on the flood of murder cases of the 1980s and 1990s, said many of them were not very well investigated. The attitude was “We’ll just let the jury decide.”

Fishman: That could well be true. He probably mentioned, or maybe he didn’t, if you went before a jury in The Bronx in the 1990s, you didn’t have much of a chance. The DA had a huge advantage, and that advantage was that people were on the side of the DA. It was not a Black Lives Matter moment; it was a moment where Giuliani rolled tanks into the streets. People forget, the city was under siege. I don’t think [prosecutors] felt they had as high a burden of proof as they must feel now.

TCR: There were times during the podcast that you said either prosecutors or detectives intentionally neglected to investigate.

Fishman: I do think that there were times that… I want to choose my words very carefully here. I guess I can’t really use the word conspiracy, but listen— the main witness against Cal absolutely and definitively lied on the stand. And he lied on the stand in two regards: one is that he denied being involved in an attempt to murder Cal, and two, he lied in saying that Cal did it.

Dwight, the main witness, admitted to me that he lied about both of those things. And I really grilled him on the first one. Dwight had actually tried to murder Cal. And the fact that the chief witness against Cal had previously tried to murder him— that really seems like it would be an important fact for the jury to consider. Dwight was asked on the stand, did you have anything to do with the attempted murder of Cal, and he says no. But subsequently Dwight narrated for me how he tried to kill Cal– and so did the chief investigator for The Bronx DA! I asked Dwight, if he was surprised they let him get away with it, and he said no— that’s how the game is played. If you play by the rules, you’re always going to lose.

For me that was actually the most chilling moment in the whole walk through the criminal justice system. Because here you had the insider, the guy who really organized the prosecution, who was the chief witness. And basically he was not only saying that he lied, but he was basically saying that the system worked by allowing witnesses to lie. And I think one of the things the podcast does is really let you know how the system works.

Victoria Mckenzie is Deputy Editor (Content) for The Crime Report. She welcomes comments from readers.