When I asked Aaron to talk about the upcoming NFC Championship game between the Seattle Seahawks and Carolina Panthers, I was hoping that he would give me a few minutes of matchup information. What actually occurred was the definition of a pleasant surprise – he went into Deep Analysis Mode in the following five-part, forty-five minute audio interview. The audio links have been posted for your enjoyment, with a full transcript to follow presently:
.NET: This is Doug Farrar with Seahawks.NET, and we are very pleased to welcome Aaron Schatz, the Editor-in-Chief of Football Outsiders and the lead writer and statistician for the Football Prospectus Annual volume. Aaron is a Massachusetts resident, and he would like us to know that he is the second-most eastern-most writer in the country. We’re here to talk about the NFC Championship Game and some general statistical concepts that our listeners may not be familiar with.
Aaron Schatz: Thank you, thank you…yes, the joke is that I have the most East Coast media bias because as far as I know, the only national football writer who lives to the east of me is Michael Smith from ESPN who lives about three exits east of here on I-90. Which Seattle residents will know well, but we’re on the other end of it.
.NET: We’ll be sure to torture you about that later. Before we get into the matchups between Seattle and Carolina, can you take a minute and give us a thumbnail sketch of Football Outsiders for those who haven’t seen the site or the Football Prospectus book and tell us what is different about what you do from an analysis standpoint.
Aaron Schatz: Football Outsiders is an attempt to do for the NFL what people like Bill James and Rob Neyer have done for baseball, which is also happening in other sports. Seattle people will recognize the name of Dean Oliver who works as a consultant for the SuperSonics and does this kind of work for basketball. We try to take statistics and use them to break through conventional wisdom rather than trusting sort of the old hack things that have always been said about football like, “you need to establish the run”. We want to take the numbers and show whether these things are really true. Sometimes they are. Sometimes they’re not. We try to break through contexts of numbers. You know, 100 yards isn’t always a great thing if it takes you 33 carries to get there, that sort of deal.
We also do tape analysis and general writing. It’s not all numerical and certainly this kind of analysis is in the early days compared to where baseball is so we don’t have all the answers, but we do have some interesting ones. They do give us some good insights on a lot of teams and how teams win Super Bowls in the NFL.
.NET: The whole concept started, if I remember correctly, with something about New England’s running game.
Aaron Schatz: Yeah, it started in 2002. I’m a New England Patriots fan and that was the year they didn’t make the playoffs. Ron Borges is a local writer here and he said the reason they didn’t make the playoffs was that they could not establish the run. At the same time, the team that he said would probably win the Super Bowl that year was Oakland. Oakland ran less than any other team in the NFL.
So I said, “Well you can’t have it both ways. You can’t both have it that the Patriots need to run more and that Oakland is the best team.” So I went to investigate whether teams really do need to establish the run, and I went through box scores and play-by-play counting the number of rushing attempts in the first quarter and whether that correlated to winning and the answer is that it didn’t.
The way that you usually build up rushing numbers is by being ahead in the third and fourth quarter and then you run out the clock to solidify your win. At that point I realized I had created this database of plays and I had sort of copied all of the plays over, not just the running plays, so I had this database with everything for the 2002 season and I could sort of play with it and six months later Football Outsiders was born and here we are three years into my research and now lots of people are reading it. It’s a lot of fun and I never would have expected that this is what I would be doing for a living, but here I am.
.NET: It’s not just Football Outsiders and Football Prospectus, you also are with Foxsports.com now. You’ve also written for the New York Times, and you provide analysis for the New York Sun.
Aaron Schatz: The New York Sun is interesting. It’s a conservative newspaper that started a couple of years ago as sort of a conservative alternative to the New York Times that wasn’t “tabloid-y” like the New York Post, but that left them with the question of what to do with their sports sections, so they decided that they were going to have an all intelligent analysis sports section.
Baseball Prospectus writes for them, John Hollinger, who does basketball analysis for ESPN, writes for them now, Michael David Smith, and our college writer Russell Levine, and a number of us are writing for FoxSports.com now. What started out as just me and a couple of my fraternity brothers writing a fantasy column has turned into a site with a dozen different writers writing for it and the yearly book, the Football Prospectus, and our writing appearing in a lot of places. It’s really been remarkable.
.NET: For those who are familiar with your work, specifically the sometimes-controversial DVOA Rankings, can you confirm once and for all that these rankings are based on your complex stats and game charts and not some “Omerta” from the East Coastsports writing mafia. How in the heck can you have playoff teams who lose, rated ahead of playoff teams that win? I think that is the main confusion people have here.
Aaron Schatz: That’s the confusion now - ten weeks ago, the confusion was something else. I do make fun of Seattle fans and their obsession with the East Coast bias. It’s a statistical system based on the fact that we want the ratings to correlate the best with winning for every team over a long period of multiple years. So we try not to tweak the ratings just because we think one team looks weird. We know that a couple of teams are going to look weird, but we want it to be as accurate as possible for as many teams as possible and not just be close to winning that year.
We want it to carry over from year to year to sort of try to filter out elements of luck and randomness and really try to get at the real quality of teams. Because as much as NFL teams vary from year to year, I think they vary less than it looks. A lot of the variance is randomness. The thing is that what happens when we consider every play in our rating, every single play of every single game, and then plus we have a weighted rating that drops the strength of earlier games. But now that we’re getting into the playoffs and doing ratings in the playoffs for the first time, we’re still considering 12 or so weeks of games even in the weighted rating.
So what happened this week was that the Colts are still at number one, because if you consider 12 weeks worth of games, they still come out as the number one team. The way that I put it was if the Colts played Pittsburgh a hundred times, they would probably win the majority of them, but you don’t get to play a hundred times in the NFL. You only get to play once, and you’d better game-plan correctly that once and you’d better give it your all that once or if it’s the playoffs it’s over and that’s what happened to Indy. They may be better inherently than Pittsburgh, but that day they got out-coached, out-thought and out-played and they’re out.
.NET: Now, how many of those 99 games would they win if Peter Morelli was not officiating?
Aaron Schatz: (laughs) Well that’s the other thing about the ratings this week is my rating actually came out with Indy as being higher for that game than Pittsburgh, which sounds ridiculous until you realize Indy actually had more yards, more yards per play and won the turnover battle thanks in part to the overturned Troy Polamalu interception.
If you have that interception be an actual interception and you take away the rest of that offensive drive for the Colts the Steelers end up with the higher rating for the game, which is I think what we all saw with our eyes.
.NET: So you don’t go back into your rankings and say, “The NFL has admitted this was an interception, so at that point it becomes one.” You say, “This is what happened in the game, this is the play-by-play and this is what we’re calling it.”
Aaron Schatz: Right. That actually had an effect on the Seahawks when you’re talking about the game with the Giants, where you had those very questionable touchdowns. My ratings for those is as touchdowns because that’s what they are in the official stats.
We have a charting project where we’re trying to chart new stats in games that are not in the regular play-by-play and if we can eventually turn that around quickly, we could do things like mark up plays where we think the officials got it wrong, but for now we can't. So we just go with what’s in the play-by-play, and according to the play-by-play, Troy Polamalu did not have an interception - even though we all know that’s bogus.
.NET: And the other way the Giants – Seahawks game was affected was that the Giants had better stats than Seattle in a lot of different ways and they were, I believe, ranked ahead of Seattle the week after they lost to Seattle. Is that correct?
Aaron Schatz: Yeah, they had a better rating for that game because of the fact that the Seahawks can’t be credited for (Jay Feely’s) missed field goals. The Giants can be penalized for them, but the Seahawks can’t be credited and the Giants did play better than the Seahawks in that game.
That was around the time that the Seahawks had just beaten San Francisco very narrowly (27-25 on November 20, a week before the Giants game). That was sort of the dip in Seattle’s season in the middle where they dropped in our rankings a bit, but they sort of settled after that. The San Francisco close win was a fluke really, and after a couple more weeks of having big wins, that game became a lot less important.
I do think that the Giants were the biggest threat to Seattle in the NFC until all their linebackers got injured. I think if their linebackers had been healthy, they would have made it to the championship game against the Seahawks. They would have been Seattle's biggest threat (on the way) to the Super Bowl.
.NET: So you think that healthy linebackers would have mitigated Eli Manning’s faulty GPS?
Aaron Schatz: That was the amazing part of (the Giants-Panthers playoff game) - how badly the New York offense played. However, the healthy linebackers would have stopped Carolina’s running game, giving the Giants more opportunities to move the ball and stopping Carolina from scoring and turning it into a real defensive battle rather than a rout.
Editor's Note - Many thanks to Associate Editor Scott Eklund for his help in transcribing this interview.
Doug Farrar is the Editor-in-Chief of Seahawks.NET. Feel free to e-mail him at firstname.lastname@example.org.
Aaron Schatz is the Editor-in-Chief of FootballOutsiders.com, Lead Writer and Statistician for the “Football Prospectus” annual volume, and an NFL analyst for FoxSports.com. He has also written for the New York Times, the New York Sun, the Boston Globe, The New Republic Online and Slate, and has done custom research for NFL.com and a number of NFL teams. Before creating Football Outsiders, he spent three years tracking search trends online for the internet column, “The Lycos 50”. He has a B.A. in economics from Brown University and lives in Framingham, Massachusetts, with his wife, Kathryn, and daughter, Mirinae.