You Like Me! You Really Like Me!! - part the second

During my earlier task of letting my fingers do the walking through the Yellow Googles on the topic of DSGE, the Yellow Googles failed me in a spectacular way. For many years, I have viewed Joe Stiglitz as the Godzilla of macro analysis, yet they didn't show me this paper. The Failing Yellow Googles.

In the discussion below, I shall illustrate the inadequacies of the DSGE framework by focusing on the 2008 crisis. Some advocates of DSGE models say these models were not meant to address "once in a hundred year floods." There are several responses to this defense. The first is by way of analogy: what would one think of a medical doctor who, when a patient comes with a serious disease, responded by saying, "I am sorry, but I only deal with colds."

You really should read that paper, if you care about macro quant. I'll offer a few more salient quotes.

Those readers with good memories will remember the Viagra graph:

[...] when house prices didn't increase as expected (and it should have been clear that they couldn't increase forever at those rates) and homeowners faced constraints in refinancing, the bubble broke, and the crisis ensued.

In a later footnote, he makes the same argument made here:

The Congressional inquiry into the 2008 crisis called itself the Financial Crisis Inquiry Commission and focused on aspects of the financial sector like credit rating agencies and the role of CDS's, derivatives, and other complex financial instruments. The standard DSGE models have nothing to say about either of these: these are failings related to its inadequate treatment of the financial sector.
[my emphasis]

Once again, with vigor, I give you Blythe Masters

One of the key reasons that representative agent models fail in enhancing understanding of macro-fluctuations is the pervasiveness of macroeconomic externalities -- the actions of each agent (in the aggregate) has macroeconomic consequences which they do not take into account.

Or give a shit about! Externalities, ignoring them, is where the profit is.

And, in a footnote to this part of the text, he Drops The Big One:

Complexity in financial structures may make it even impossible to ascertain whether a system is systemically stable.

Ooops.

The bete noir of my professional existence is the notion that macro-analytic structure is just the sum of some set of micro-structures.

The micro-economics of the basic competitive model—as formulated in Arrow and Debreu-- has been shown to be flawed by forty years of economic research. Why should we expect a macroeconomic model based on such micro-foundations to work?

Well, we don't.

And, finally

Defenders of DSGE models counter that other models did little better than the DSGE models. That is not correct. There were several economists (such as Rob Shiller) who, using less fully articulated models, could see that there was clear evidence of a high probability of a housing bubble. There were models of financial contagion (described earlier in this paper, developed further since the crisis), which predicted that the collapse of the housing bubble would likely have systemic effects.

What's that about history? Does it repeat itself, or just rhyme?

Write Once, Go Home And Sleep [update]

May be it's just me, but the news that Intel has released XPoint as DRAM replacement seems to mean we can go Dr. Codd one better. May be it's just me, but Optane DC Persistent Memory offers the choice to write transactions just once, beyond the cpu caches. Can't we dispense with all those extra tiers? What do we gain with sync-ing all those tiers? What do we gain by eliminating them? Let's go look.

This could also be the nail in the coffin of MVCC, which was devised as a way around locking transaction semantics. The cost, in resources, of MVCC is not trivial. For many years, it's been widely understood that Oracle (and PG, of course) wants the whole machine! And it needs it. With single level storage, the engine has so much less to keep track of. As does the OS.

Yowza! A bit of sending my fingers through the Yellow Googles reveals that others have the same idea.

With Intel Optane DC Persistent Memory, you can put an entire Redis database in-memory and have sub-millisecond latency without the vast expense of doing so with DRAM. This has led Redis CTO and co-founder, Yiftach Shoolman, to say that 'we believe the next-generation server architecture will be all persistent memory. This is going to change the entire database market.'

Sounds like the call for single level storage to me.

Another report confirms that OS twerking will be needed:

Speaking of which, the researchers also benchmarked how the product performed as persistent storage. In this scenario, the Optane DC modules are treated as a storage block device located in memory, while the DRAM fulfills its conventional role as volatile memory. One of the challenges here is that file systems typically do not support memory-based I/O, so options are somewhat limited.
[my emphasis]

Ah, but there is some good news (and expected by Your Humble Servant)

The NOVA-Relaxed variant, (which allows in-place file page writes in order to improve write performance for applications that do not require data consistency for every write) delivered the best performance of the various file system setups.

Ah, but yet again yet more good news

However, the best results were obtained by rewriting three of the applications (RocksDB, Redis, and MongoDB) to use memory mapped Optane DC instead of going through the file system loads and stores. This requires a greater level of effort, since the cache consistency normally provided by the file system has to be re-implemented in the customized software. But as the researchers pointed out, the potential for performance gains are larger.

IOW, to some small delta, single level storage blows everything else away. As the report says, both the database engine and OS have to be twerked to run with a single level storage. But getting there is most of the fun.

[update]
Well, that didn't take long. Yet more fingers wandering through the Yellow Googles finds this paper. You should read it. Among other things, it makes the case that single level storage means RDBMS logging can get out of the way.

The write-ahead logging (WAL) protocol supports efficient transaction processing when memory is volatile and durable storage cannot support fast random writes [58, 39, 33]. But this assumption causes unnecessary performance degradations in a DBMS with NVM storage [14]. Consider a transaction that inserts a tuple into a table. A DBMS first records the tuple's contents in the log, and it later propagates the change to the database. With NVM, a DBMS can employ a logging protocol that avoids this unnecessary data duplication

You Like Me! You Reallly Like Me!!

Well, I know I'm not Sally Field. I've not been given the top award in my chosen field of endeavor. But, in the last week or so, a few of the main themes of this endeavor have been supported by those who have.

I do not think that the currently popular DSGE models pass the smell test. ... The advocates no doubt believe what they say, but they seem to have stopped sniffing or to have lost their sense of smell altogether.
-- Robert Solow/2010

So, it all started with a question: is there a specific version of econometrics tailored to macro? When I was in school loe those many decades ago, there was just econometrics, exemplified by Theil's Godzilla book (you know, in the sense that it conquered all). I used Kmenta. Macro folks drifted to time series analysis, again with a seminal text, this time, by Box and Jenkins.

So, I let my fingers do the walking through the Yellow Googles. Turns out there is at least one text, "Structural Macroeconometrics". I got a copy and have been browsing. I hesitated in ordering, since the ToC had that hated word, Bayesian, sprinkled throughout. But, girding my loins as Taras Bulba commanded Andrei to learn the ways of the Poles, I went ahead with it. The hubris of the text is astonishing! Now, the copyright date is 2011, more than enough time for these authors to know that The Great Recession had occurred. And to know that the method they propound, DSGE (dynamic stochastic general equilibrium), was just as much a bust in predicting as all the other quant models.

The simple fact is The Great Recession was driven by The Rich seeking low-risk, high-return instruments rather than physical investment. They, lemmings all, settled on residential (and, to a lesser degree, commercial) mortgages. That's the matter. The anti-matter was the CDS, invented at a specific time and place by one person. A woman, of course. Pandora for our times. The CDS turned the metaphorical Wall Street Casino into a real one: anyone could bet on anyone else's assets. More bets than holdings. Oops.

Here's one view of DSGE and The Great Recession.

The failure of economists to predict the Great Recession of 2008 - 09 has rightly come under attack. The areas receiving most criticism have been economic forecasting and macroeconomic modelling. Distinguished economists - among them Nobel Prize winner Paul Krugman - have blamed developments in macroeconomic modelling over the last 30 years and particularly the use of dynamic stochastic general equilibrium (DSGE) models for this failure.

Which is not to say that merely reading the New York Times and Wall Street Journal, looking for, to use econ jargon, 'technology shocks' in the form of dangerous changes to the rules of engagement is sufficient. But the quant approach of predicting the future based on the past, even a recent one, won't spot such problems. A good job for a grad student RA.

So, we get this reporting. The like me quote:

The problem the major economies in Japan, Europe, and the United States have today, [Richard C. Koo, chief economist at Nomura Securities] said, is that despite low interest rates, investment opportunities in domestic markets don't offer sufficient returns to lure borrowers to go into debt, using the vast pools of available savings.

IOW, nothing has changed since the lemmings' surge into residential mortgages. The CxO class is still too stupid at finding physical investment. You like me. You really like me.

Another, truly rare, observation:

[federal budget deficits aren't] necessarily a forerunner of inflation. ... [other causes are] a shortage of labor, raw materials, and factories.

IOW, cost push. Greedy union workers aren't the only cause of inflation. Again, you like me. You really like me.

Most of the front page of the Business page is taken up with Neil Irwin's reporting on Australia's 'miracle'. You should read it.

Divide and Conquer - part the second

It's approaching the 50th anniversary of Dr. Codd's initial paper on the RM, internally at IBM. A year later it went public. At the time, there were three protocols for I/O: flat files, e.g. the COBOL COPYBOOK; IDMS, the network database system; and IMS, the hierarchical database system. That latter was made by IBM, apparently to avoid paying royalties and such to use IDMS. Yes, the more primitive (single parent) XML-like database came after the more structured (multi-parent) approach. Both hard-wired said structure, needing a byzantine syntax to maintain.

Ever since, coders have resisted the RM/SQL approach. Some, esp. since the advent of XML, still prefer dumb files. Even when there's a RDBMS on the server, they prefer to "do transactions in the client". With the innterTubes bandwidth now available, it's a truly lame excuse. The main basis for client-centric coding is the notion that all that compute power "out there" is good reason to move transaction control "out there". It's really, really stupid, but coders don't think beyond the ends of their noses.

A recent piece from AnandTech rhymes.

About That Statue

"Keep, ancient lands, your storied pomp!" cries she
With silent lips. "Give me your tired, your poor,
Your huddled masses yearning to breathe free,
The wretched refuse of your teeming shore.
Send these, the homeless, tempest-tost to me,
I lift my lamp beside the golden door!"
-- Emma Lazarus/1883

Written as fund raiser to support, literally, The Statue of Liberty. Added to the monument in 1903. Widely believed to express the liberalism of the USofA. The fact is, the USofA has been hostile to the non-WASP populations of the world from inception. Today's neo-Nazis are of a piece with common 19th century practice.

So, this recent reporting should be something of an eye-opener:

According to a story published in the Washington Post, on March 14, 1891, a crowd of 8,000 assembled on New Orleans' Canal Street, "almost filling up the large space from curb to curb on each side of the boulevard." The crowd, possessed by an "ungovernable" fury, had guns and arrived at the parish prison at 10:30 that night. Prison guards let the mob into the prison, where they eventually found the Italian prisoners. "The shotguns belched forth and the slayers of Hennessy fell dead in their tracks," the story says.

Good, God fearing, Christian, Real, Americans.

How Dumb Can You Be - part the second

Dumb enough to start a NFL 'development' league without negotiating access to 'developing' NFL players. Can you believe this? And capitalists are the self-appointed Smartest Guys in the Room!!

One players' union official said it would violate the terms of the NFL's collective bargaining agreement if active players are lent to the AAF.

[update 3 April]
Well, that didn't take long.

Unfortunately, after careful consideration, the board has decided to suspend operations of the Alliance of American Football, effective immediately.
-- Aditi Kinkhabwala

War is Hell

A frequent theme over the years of these missives is the answer to the question, "why was growth and equality (modulo race) so much greater in the 50s and 60s than since?". The answer was generally in response to some nonsensical blathering from the Ayn Rand crowd. Well, Tex, this time it's different. This time a mainstream pundit has finally struck the Mother Lode. Not too surprisingly, it's David Leonhardt:

Failing New York Times
[When I got back from my greasy spoon reading of the Times, note in hand as usual, I went to the site expecting to extract a sentence or two. But, NO. They have instituted an even more foolish and Draconian access. So, starting tomorrow, I'll be typing in those few sentences. I could do with the typing practice, in any case.]

So, if you've seen his piece, he says (in his words) what has been said here. The reason for the great growth in the 50s and 60s was due to the fact that the men (and they were 99.44%) who had the hands on the levers of power in both the private and public sectors were the product of both the Great Depression and WWII, thus a pervasive "we're all in this together" attitude. Taxes on the rich were high, unions were strong, and the middle class got most of the moolah. And, thus, aggregate grew nicely. Laffer be damned, real corporations only increase output when there is clearly unmet demand. All the money printing, so called, by the Fed and ECB has gone to the 1%, thus no demand pull inflation. Commodities are in restraint, so no cost push. And, in case you haven't noticed, median income isn't going anywhere North. So, no inflation.

Those who ignore history are doomed to repeat it, and all that.