By The Numbers

There's that famous quote from The Bard, "The fault, dear Brutus, lies not in our stars, but in ourselves if we are underlings." As my fork in the Yellow Brick Road tracks more towards (what's now called) Data Science, various notions bubble to the surface. One lies in an age old (within my age, anyway) dispute between traditional (often called frequentist) math stats and those who follow the Bayesian path. From my point of view, not necessarily agreed to exist by those on the other side, Bayesian methods are merely a way to inject bias into the results. Bayesians refer to this "data" as prior knowledge, but, of course, the arithmetic can't distinguish between objective prior knowledge and fudging the numbers.

So, I set out this morning, being Columbus Day (a day honoring Discovery for some, invasion for others), to see whether there're any papers floating about the intertubes discussing the proposition that our Wall Street Quants (those who fudged the numbers) bent Bayesian methods in their work. As I began my spelunking, I had no prior knowledge about the degree to which Bayesian had taken over the quants, or not. Quants could still be frequentists. On the other hand, it is quite clear that Bayesian is far more mainstream than when I was in grad school. Could Bayes have taken significant mindshare? Could the quants (and their overseer suits) abused the Bayesian method to, at least, exacerbated, at most, driven The Great Recession. It seemed to me likely, any crook uses any available tool, but I had no proof.

Right off the bat, search gave me this paper which references one (at a pay site) from the Sloan Management Review. The paper puts the blame on risk management that wasn't Bayesian. You should read this; while the post does discuss the SMR paper on its merits (which I couldn't read, of course), it also discusses the flaw in Bayes (bias by the name of judgment) as it applies to risk management.

Continuing. While I was a grad student, the field of academic economics was in the throes of change. The verbal/evidence/ideas approach to scholarship was being replaced by a math-y sort of study. I say math-y because many of the young Ph.D.s were those who flunked out of doctoral programs in math-y subjects. Forward thinking departments recruited them to take Samuelson many steps further. These guys (almost all, then) knew little if anything about economic principles, but department heads didn't care. These guys could sling derivatives (initially the math kind, but eventually the Wall Street kind) on the whiteboard like Einstein. I noted the problem then, the 1970's. This paper touches on this issue (linked from here). "These lapsed physicists and mathematical virtuosos were the ones who both invented these oblique securities and created software models that supposedly measured the risk a firm would incur by holding them in its portfolio." Nice to know it only took 40 years for the mainstream pundits to catch up.

And, while not specifically about Bayesian culpability, this paper makes my thesis, which I realized about 2003 and have written about earlier: "Among the most damning examples of the blind spot this created, Winter says, was the failure by many economists and business people to acknowledge the common-sense fact that home prices could not continue rising faster than household incomes." One of those, D'oh! moments. McElhone, the Texas math stat, introduced me to the term 'blit', which is 5 pounds of shit in a 4 pound sack. By 2003, and certainly following, the US housing market had become rather blit-y. The article is well worth the reading. There are links to many other papers, and it does raise the question of the models used by the rating agencies. Were these models Bayesian? Were the rating agencies injecting optimism?

Which leads to this paper, which I'll end with, as it holds (so far as I am concerned) the smoking gun (which I found to be blindingly obvious back in 2003): "Even in the existing data fields that the agency has used since 2002 as 'primary' inputs into their models they do not include important loan information such as a borrower's debt-to-income (DTI)..."

This few minutes trek through the intertubes hasn't found a direct link between Bayes and the Great Recession. I know it's out there. I need only posit such as initial condition to my MCMC (look it up).