In this New Thinking podcast, Harvard Law School Professor James Greiner talks with Aubrey Fox about why he
launched the Access to Justice Lab, which has researched topics as varied as how to provide self-help materials to
defendants involved in civil debt collection cases to the effectiveness of commonly used pretrial assessment interview
tools in criminal court. Greiner also discusses what he sees as a strategy for improving the legal profession’s
openness to evidence-based thinking.
AUBREY FOX: So this is Aubrey Fox. I’m today’s
guest host for The New Thinking Podcast, which is the Center for Innovation’s podcast where we interview legal
experts and researchers. And I’m very fortunate to be joined today by Professor James Greiner, who is the William
Henry Bloomberg Professor of Law at Harvard University. Does this mean that you’re connected to our former Mayor
Bloomberg?
PROFESSOR JAMES GREINER: Actually, I think that, if I have it right, the Chair was
named after his father and he established it on his father’s behalf. It’s a rotating chair, so I have it
for another few months before it rotates to someone else. But that is the connection.
FOX: So
you’re not dining with Mayor Bloomberg on a regular basis?
GREINER: I haven’t had the
pleasure yet. I’m sure it’ll be terrific. I’m guessing he probably sets a good table, but I haven’t
had the pleasure yet.
FOX: Among many things, Jim, is the faculty director of something called
the Access to Justice Lab, which is a really exciting new initiative that he started where he’s trying to bring
evidence-based research to the study of criminal justice and civil justice. And so we want to talk to Jim today about
his work with the Access to Justice Lab and also some of his thinking about why it’s important to introduce
this evidence-based approach, and want to talk to him a little bit about some of the particular issues he’s
investigated.
Jim, maybe where we’ll start is just to let you talk about how you got interested
in this area of criminal justice and civil justice, and tell us a little bit about your background.
GREINER:
Sure. So when I graduated from law school, I hadn’t taken any statistics courses or any sort of quantitative
courses in college or anything like that. I just wanted to be a regular practicing lawyer. So I did that for 3 years
for the Justice Department and then 3 years for a private law firm, a law firm called Jenner & Block in DC
and, at the time, I was litigating cases that involved some numbers. So, employment discrimination class actions,
litigation about the 2000 decennial census, whether you could use statistical techniques to try to correct for over-counts
and under-counts according to racial groups. I’d always been interested in social justice issues and criminal
law issues.
When I moved over to the private law firm, I did a couple of pro-bono cases that we
picked up. And basically, through the practice, redistricting was in the area I did, got interested in numbers and
what numbers and evidence-based thinking could bring to the table in terms of social justice issues. And so after
6 years as a practicing attorney, I left the practice of law for 5 years to get a PhD in statistics.
And
then when I got my PhD in Statistics I started researching in a bunch of areas, but the one I was having the most
fun with, really enjoying and the one that I thought I could potentially do some good with was access to justice
issues, more on the civil side than on the criminal side, but some on the criminal side also. So issues like, how
do you make law work and be accessible for folks who can’t afford to hire lawyers to interface with the legal
system. And on the criminal side, how can we make existing institutions perform what we want them to perform better.
How can we release more people who are not dangerous, how can we reduce the criminal footprint on communities and
still maintain a system of law and order, etc.
So those are the big-picture issues that I found
most interesting.
FOX: So, having a statistician as a professor
of law. Is that an unusual development, or is that something that’s been going on for decades and decades?
GREINER: No, I think there are only 2 of us still in the legal academy that have stats degrees– stats PhDs.
I know the other person, unsurprisingly. But having someone as a law professor who knows, as we say, “how to
count”, that’s not all that unusual. It’s just folks typically have economics degrees or political
science degrees or something like that. But the hardcore statistics is what I really wanted to get invested in. And
the reason for that was I pretty much knew when I was going into the statistics program that I was a litigator through
and through. I was always going to be a litigator. I was going to care about courts and administrative agencies and
the practice of law at a person by person level.
And so I didn’t feel like I wanted to learn
a lot about the big-picture political science theory and big-picture economic theory that those folks do. And so
I just said let’s go straight into the heavy numbers.
FOX: And so tell us a little bit about
the Access to Justice Lab. What does it do, when did you get it started?
GREINER: We’re talking
in December of 2016. It’s only been in existence since July of 2016, and it is funded by a generous grant from
the Laura and John Arnold Foundation. The foundation itself cares about evidence-based thinking across a lot of different
spheres, and it funded the lab. And basically the lab has 2 overall missions. One is to produce useful, rigorous
evidence that would assist policy makers and judges and lawyers in what they do. Especially with respect, again,
to access to justice related to folks who can’t afford to hire lawyers to interface with the legal system for
them.
And then the second overall purpose is to try to tear down the resistance to rigorous empirical
evidence, especially via the randomized control trials, or randomized field experiment, that resistance that exists
within the legal profession and within the judiciary.
I think that in law, especially with the
practice of law, we are in roughly where the medical profession was in about 1938 or 1940. Basically, we are beginning
to engage in the debate about whether our profession should be evidence-based in the way that medicine was engaging
in a debate about whether drugs and medical devices should have evidence behind them before they are allowed to be
sold to the public.
And there are many folks in law, on both the bench and the bar, who think
that randomization in law is unethical or is unnecessary. Unethical because randomization takes away the professional
judgment about who should get what, and unnecessary because we know that, as lawyers and judges, all of our professional
judgements are perfect, or very, very good and, therefore, we don’t really need to investigate them all that
much. And so the Access to Justice Lab’s second purpose is to try to suggest to folks that scientific-based
thinking and evidence really can bring a lot to the table and it may end up overturning accepted truths within the
bench and the bar.
FOX: I’m glad, in your comparison to the medical world, you didn’t
bring up leaches. So at least we’re… We’re 100 years ahead of where we could be.
GREINER:
Right.
FOX: And, just to follow up for a moment, I kind of want to get your sense of the state
of play on this because I understand that there may be resistance to RCTs, which are the most rigorous form of evidence-based
analysis and require dividing a control group from an experimental group and giving some people something that you
don’t give the control group. But there is a history of doing research into criminological issues. So it’s
not like you’re starting from square one. So I guess in this mix of some history and some resistance, where
would you say we are at the moment?
GREINER: I’d say more resistance than history with respect
to courts and judges and lawyers. So you’re absolutely right that there is a reasonably well developed literature
in the criminology field. And a good portion of it, not as much as many criminologists would like, but still a good
portion of it backed up by randomized studies to try to figure out whether, say, afterschool programs prevent people
from getting arrested, or whether certain types of treatment as a condition for probation are effective in preventing
recidivism. But if you notice in the 2 examples that I gave you, one of them is sort of prior to the involvement,
the intervention is prior to the involvement of the criminal justice system, and one of them is post sentencing,
sort of after the involvement of what we call the lawyer-based or court-based criminal justice system.
And so there has been substantially less in that portion of the criminal justice system where the lawyers
and the judges get involved to the point where, when a co-author and I try to catalog all of the randomized studies
that have been done in United States law, criminal or civil, that involved randomization of a decision that would
otherwise have been made by a judge or a lawyer, so it’s how we defined randomized studies in law. We’re
taking a decision that otherwise would’ve been made by a judge and a lawyer and randomizing it to find out whether
the decision is worthwhile or would be an effective style of treatment or something like that. We could only find
about 50 in the history of the United States, based upon all areas of law, all sorts of settings.
So
if you think about medicine starting from 1938 or 1940, the number would probably be uncountable, be tens of thousands
or hundreds of thousands of randomized studies that have involved replacing a judgment that would’ve otherwise
been made by a medical professional with randomization in order to figure out whether the treatment or the intervention
works. Whereas we could find fewer than 1 per year in law. So again, I absolutely agree with you. Criminology has
much more of a history. It’s a much more developed field. But they’re basically working in their rigorous
evidence prior to the formal legal system or after it in the lifetime of a particular defendant or person or crime.
And where we think the evidence really needs to be focused is, at least in terms of a new effort, we need to focus
on where there isn’t as much evidence right now, which is inside courts and law offices.
FOX:
One of the reasons that I wanted to speak to you, Jim, was the Center for Court Innovation, while we’re known
more for our work on criminal court programs, we also have been developing a civil justice portfolio. And so I’m
curious to hear you talk about some of the work you’ve done on issues, and you’re continuing to do, like
debt collection and the assignment of council in housing court cases. And I guess the framework I have for my question
is, making the case for this kind of research may rest on finding kind of practical implications that flow from the
research. So I’m wondering if you can give us some examples of research you’re doing and what you see as
the things that a practitioner would immediately identify as, well that’s really interesting and useful.
GREINER: Absolutely. So I guess, two examples. And I don’t want to go on for too long, so stop me if
this turns into a monologue, but two examples with respect to financial distress work which you mentioned, the debt
collection work that you mentioned, one of the premises of that work is that there is just never going to be enough
money, there’s just never going to be enough of a social commitment to provide a lawyer at state expense for
every individual who can’t afford to hire a lawyer who has a legal problem. Even if we were to restrict that
set to people who are facing court adjudications that affect basic human needs by some definition, which is the ABA’s
proposed definition. It’s called the Civil Gideon Project, this idea that we can provide lawyers at state expense.
I just don’t think the resources are ever going to be there. Some smart economists have tried to calculate that
and it’s in billions upon billions of dollars would be needed.
With that in mind, it’s
probably going to take a battery of lots of different things to address the access to justice problems that we’re
having in United States society, and one of those things, one of those many things that we need, is going to be self-help.
How’re you going to make complex subjects accessible, complex procedures and complex principles accessible to
folks who can’t afford to hire lawyers for them so they can try to address legal problems they have on their
own?
So the financial distress work that I’m doing is an attempt to try to say, well, if
what we need are self-help materials, and we need basically to educate people about how to solve their own problems,
what sorts of principles can we draw from non-law fields that would help us make self-help materials more effective,
or would we hypothesize that would make self-help materials more effective?
For instance, if we’re
trying to get people to a particular location at a particular time of day and on a particular date to do something
they would rather not do because they decide it’s good for them, how do we persuade people to do that?
Well, if you pose the question at that level of generality, you recognize that… Voting. Political scientists
study how you get people to vote on a particular day at a particular time. Public health officials study how to get
people out to take flu shots, or to get colonoscopies on a particular day. These are all studies that have been run.
But in law, our self-help materials don’t actually use any of the lessons from any of those other fields
about how to get people to a court date. We think that’s bad. We think that if good lessons are learned from
those fields, we should try them in law and see if they work.
Similarly, there’s an entire
field called adult education, or just education period, about how you communicate complex concepts to people who
need to be able to use them. But in law, our self-help materials don’t currently use the lessons from adult
education. And you can see that because if you read the legal self-help materials, they’re word-heavy. There’s
just lots and lots of text. Whereas adult education folks, or education folks generally are saying, no, no, no. You’ve
got to use images. So, in our case, we’re using cartoons.
And so, the idea there is to say,
let’s try to create the best possible self-help materials that, under the state of the art, we currently know
how to create using the lessons from all these different fields. But then, let’s don’t declare victory
there. Because maybe there’s something about law, or maybe there’s something about the way that we did
the self-help materials that would make them not effective, or maybe there’s something they’re not understanding.
Or maybe the best possible self-help materials are just not enough in a system that is so complex that self-help
is just not an option.
So we go test them. We go put them into the field and we do a randomized
experiment. In one randomized experiment, we might randomize people to either get self-help materials or an offer
of representation from an attorney. Or in another experiment, we randomize whether people get self-help materials
versus no further assistance. Our self-help materials versus existing self help materials which, in many cases, just
don’t exist.
So that’s one example. I’ve got another one but, again, I don’t
want to go on and on.
FOX: I’ll just cut in for a moment to say the research you’re
doing on self-help or pro se council is relevant to us because we run a program called Legal Hands, which essentially
employs volunteers who are not lawyers. So it’s a bit of a blended model. This is people who are physically
talking to a community member, so it’s not something that you go online to get self-help materials. But they’re
not lawyers. And it speaks to your general point which is, it may not be possible to have a lawyer for every person
on every case of relevance to them.
GREINER: Absolutely. I think what your program is doing is
mixing a partial service-based solution with a partial self-help so it might be a sort of guided self-help. Absolutely.
Again, I think part of the problem, one of the reasons why I find this field so much fun to operate in but also so
frustrating is that we just don’t have any evidence to suggest, to guide us as to when we need a partial service-based
solution, partial self-help, and what settings a total self-help solution is sufficient. Or maybe it’s not the
legal setting, maybe it’s the type of person. Which people can take care of it on their own if guided appropriately
either by self-help materials, or which ones are going to need some service along with it. So absolutely.
FOX: Let’s hear your other example.
GREINER: Another example that comes to mind
is the problem of triage which is basically, whenever there is more of a particular type of case, or another way
to put it, more clients than you have the capacity to provide the highest level of assistance to. So in our particular
study that we’ve been trying to get underway and still in the planning stages for, this setting is victims of
domestic violence who come to a legal services provider and seeking assistance with obtaining civil protection orders
from courts, and the legal services provider has the capacity to assist in terms of providing full representation
to. Around 1/3 of the folks who come to it who are eligible for the help, income and asset eligible, and who are
actively seeking civil protection orders.
Now that by the way, that number, 1/3, would go down
substantially if the legal services provider did more outreach in the community. So we’ve looked in the court
systems where this LSP is operating and in fact a substantial number of people are already in the court system seeking
civil protection orders without council. And so you only see the tip of the iceberg, and you can’t even handle
the tip of the iceberg, in terms of providing full representation, the traditional attorney/client relationship to
these folks, much less the whole iceberg. So what do you do?
And it turns out that in medicine,
again, to use that analogy, there’s been lots of study about how to make these triage decisions. It’s both
an ethical question and an operational question. Ethically, what do you want to do when you’re in that sort
of scare situation, and that comes up both in medicine in terms of something like when you don’t have enough
organs for transplant, it comes up when you have Hurricane Katrina knocking out power in a hospital and you don’t
have enough evacuation capacity to get all of your patients out of your hospital. That was the subject of a recent
Radio Lab episode. So there’s an ethical dimension to that question.
There’s also an
operational dimension. Suppose what you wanted to do was to say, we should use these resources to save the most lives.
Or, in a civil protection order context, we should use these resources to try to get as many people as possible civil
protection orders because that’s what they’re for and that’s best for the community. How would you
do that? How would you go about doing that? And one way you would go about thinking about it, the way that the military
thinks about it when we’re talking about battlefield casualties is, you say, well try to distinguish people
into 3 groups. One is a group of people who are going to be able to succeed on their own. So in battlefield casualties,
these are going to be people who are going to survive the battlefield even if not assisted immediately. You don’t
provide immediate assistance to them. In the legal services provider, people who are going to be able to get civil
protection orders on their own, you don’t offer full representation to them.
For people who
are never going to be able to survive on the battlefield, or never going be able to get civil protection orders on
their own, even if given the full representation, you don’t give them full representation either, because it’s
not affecting the outcome.
The people you do give battlefield assistance to, or full representation
to, are the folks that you can change their outcome. If you give assistance, they’ll survive or they’ll
get the civil protection order. If you don’t give assistance, they won’t survive or they won’t get
the civil protection order.
So that’s one way to think about it. But if you want to do that,
you face the problem of, how do you go about distinguishing those people. Because a lot of study and a lot of different
settings, again, emergency room setting is one example, turns out that’s a very hard thing to do. To figure
out who you can actually change the outcomes for. So we’re pursuing a randomized experiment that’s designed
to try to add to our knowledge of how do you distinguish those folks. And most importantly, whether lawyers, given,
exercising professional judgment, can in fact predict which people really would benefit from the help.
FOX: And I guess what you’re talking about raises a big issue for me, which is, let’s give a concrete
example, in New York city they’re talking about investing potentially $200 million more a year in hiring lawyers
to represent people facing eviction in housing court. So there is actually more money now in areas of the law there
hasn’t been a lot of money in the past.
However, you’re no doubt correct that there
isn’t enough money to cover every case at every moment. And so I guess having an RCT or a research study is
great progress towards this habit of thinking in a kind of, I would use the term problem solving way. But how do
we get to more fundamental change? What’s the way to inculcate this way of thinking in a legal services organization
in a more fundamental fashion?
GREINER: It’s a really hard question and one that we’ve
been scratching our heads about in the Access to Justice Lab about. Because that’s our overall mission. One
way of thinking about this is, if the Access to Justice Lab is going to be one of the few institutions around the
country that’s going to be doing randomized studies in the law and we’re going to depend on those few institutions
to produce evidence, we should stop. We’re never going to be able to produce enough evidence. Even as a lab
and even as an institution, 1 or 2 institutions are never going to be able to do it. Just the same way one center,
Johns Hopkins can never produce enough randomized experiments in medicine to make progress in medicine possible.
So we have to start a movement. We have to get people to think this way and to demand this evidence. In
terms of the strategies we’re pursuing in the Access to Justice Lab, we’re, first of all, going to try
to produce studies and try to educate folks about, hey, this evidence from this study can be useful in your practice,
it can get you to think about ways other than the way you were thinking before, or ask new questions about what you
should be doing to be more effective.
We’re in the process of producing a short course so
that folks who, either want to do these studies as researchers or to form partnerships with researchers to get the
study done, so these field operators will know more about what’s required and how to do them. We will be doing
persuasion in terms of just appearing at conferences and giving speeches and talking to anyone who will listen to
us about why this is so critical and, quite frankly, we think that there’s an ethical duty to do these studies.
Because if you think about it in the civil protection order context, our view is that if we triaged more
effectively, more people would have civil protection orders than currently do with no additional injection of resources.
When you think about it that way, there may be an ethical duty to do these studies, as opposed to, oh, well they’re
just curiosities. We think ultimately that we’re going to need to persuade funders to demand evidence. Maybe
not demand the evidence, show it to me right now or I’ll cut your funding, rather say, if you don’t have
the evidence, I want you to, this is a funder speaking to a legal services provider, I want you to engage in an evidence
gathering study. And sure, I’ll fund your efforts, your services as long as you are gathering evidence about
whether those actually work.
FOX: Well, this is fascinating work. And thank you so much for agreeing
to speak with us. Again, I’m Aubrey Fox. I’m the guest host of the New Thinking Podcast for the Center
for Court Innovation and joined today by Professor James Greiner who’s a Professor of Law at Harvard University
and the Faculty Director of the Access to Justice Lab. They have a great website that you can go online and take
a look at some of the studies that they’re participating in. Thank you so much, Jim.
GREINER:
My pleasure, thanks for the opportunity.
Podcast: Play in new window | Download