Conflict of Interest

A theme of these endeavors, which emerged from them rather than motivated their creation, is that when policy conflicts with data, policy wins and data loses. Now, this observation is, generally, not true when data of the physical world is the subject. Brownian motion and microarrays spring to mind. In general, then, one can observe that data about physical processes is mostly unambiguous, in the sense that the data reflect rules which are external to the process of interest. You can't fool Mother Nature (or God, if you're so inclined).

Data about human processes is a whole other story. As The Great Recession demonstrated, both the data and the rules of engagement of the activities behind the data are subject to change by the humans exercising the engagement. Both data and rules are fungible. In such a circumstance, is data analysis even worth the trouble? To earn a paypacket, I suppose so. To understand, better, the real world, perhaps not so much.

I've been making a desultory trip through Kuhn and Johnson and Hastie, et al over the last while. In a nutshell, the former is more narrative while the latter is more algebra.

What's germane here are a couple of snippets from K&J Introduction (some, courtesy of my ten fingers). First, they quote from Rodriguez:

Predictive modeling, the process by which a model is created or chosen to try to best predict the probability of an outcome, has lost credibility as a forecasting tool.

Rodriguez continues his paragraph thus:

Overly simplistic models have failed to account for the sheer complexity of human interaction and the degree to which most people behave irrationally. Most predictive economic models presume that people behave rationally most of the time, a premise which is terribly flawed but which serves as the intellectual foundation of many current economic models (See the Wall Street Journal article on this issue, here).

But they then end the section thus:

While the primary interest of predictive modeling is to generate accurate predictions, a secondary interest may to interpret the model and understand why it works. The unfortunate reality is that as we push towards higher accuracy, models become more complex and their interpretability becomes more difficult. This is almost always the trade off we make when predictive accuracy is the primary goal.

IOW, we know this is crap, but we're going to do it anyway. I'll grant that the authors work in biotech, not finance, so they're (likely) coming at the issue from a justifiable Mother Nature perspective. Still.

This imposed bifurcation didn't exist when I sat my stat and econometrics classes. What was clearly understood then was that human processes aren't Brownian motion, and that prediction was restricted to the range of the independent variable(s). Over the years, quants have been willing to ignore that last restriction and predict way beyond the range of data. It turned out to be a lucrative practice. Until it blew up the world. One of the reasons they gave us The Great Recession. Today mostly looks like yesterday, and so on.

As more failed out physical scientists moved into the social sciences (the maths are easier over here, and many social scientists don't bother to look behind the curtain), they brought with them the paradigm of inviolate rules of engagement. Thus, house prices can continue to rise beyond reason because they have been and there must be some mechanism, which we don't understand and don't need to understand, which makes it all work out just fine. After all, The Invisible Hand is just like Mother Nature, yes? Until it doesn't.

Without understanding, we get leeching to cure disease and virgin sacrifice to palliate The Gods. Which brings us Krugman today.

I've been thinking a lot lately about the power of doctrines -- how support for a false dogma can become politically mandatory, and how overwhelming contrary evidence only makes such dogmas stronger and more extreme.

He's ranting about Republicans and climate change (mostly), but the point is more general. I've long forgotten which class/teacher/professor said it but, "Data doesn't displace a theory, only another theory does that". Or words to that effect. Contradictory data may bring a theory into question, but doesn't replace that theory. The problem with policy and data is that policy doesn't rise to the importance of theory. Policy is merely imposition of will. Sometimes that will is of the majority. Sometimes not.

Do It Yourself

Today's news includes the disturbance that ExOne, a 3D printing company, got hosed. Bad quarterly. 3D Systems and Stratasys have been trading at or near 52 week lows of late. Why is anyone surprised? Some are, of course. Once the hype started, the lemmings nodded sagely and ran off the cliff.

Here's the point: if 3D printing actually made sense for production manufacturing, then all computing devices would be built from FPGAs. The semantics is the same: small volume and high cost, both relative to full-on manufacturing. Not gonna happen.

Which led me to another leg of the stool. While most java-ites know the acronym jit, it came about in manufacturing. Patient zero might be Ford or Taylor, depends on how one measures. Most attribute the current meme to Toyota post WWII. I'd add Deming as a major influence. The current outsourcing to Asia by American manufacturers could be viewed as a repudiation of jit, from the point of view of Americans, since the usual method is to make up a batch of 10,000,000 or so widgets, stuff into containers, put 'em on a boat, and wait three weeks. I wonder if one could make those 10,000,000 widgets at a split level in Ventura with some 3D printers in the course of three weeks?

Tiny Tim

Was out getting the meager needs have we for suppa, when Geithner pops up on "All Things Considered", examined by Robert Siegel. Three doses of self-absorbed tool of the Banksters in just two days!!?? He's flogging his self-defense tome, "Stress Test", the point being his stress, of course.

Sunday had him in the Times Magazine, written by Sorkin. Then, again in the today's Times, a review of sorts (right under the section heading which is "theArts"; a bit Freudian, what?), of the book itself. Until the radio, I had decided to let it all pass. Kind of like a quant/psycho/macro kidney stone.

But then, Timmy had to go and spoil it all. Siegel finally gets around to the point, which no one in the media has in my experience. Timmy had been yapping about how there was a confligration, and he was valiantly trying to put it out, not that he was in any personal danger, unlike real firefighters, of course. Siegel says (paraphrasing, I wasn't carrying my NSA issue recording device), "but the home buyers didn't create these exotic instruments, the banks did for their own benefit". And Timmy just lies! It was all an accumulation of belief that housing was safe. No mention of Viagra at the Home. No mention of gaming Timmy's System. The Banksters did nothing wrong. If I'd had anything left in my stomach, it would have ended up on the inside of the windshield. That's one evil motherfucker.

I have a funny, funny feeling that this attempt to set the record straight isn't going to work out well for Timmy.

An Attic Full of Boxes

So, what's going on with the SSD invasion? Is it the "Dave Clark Five" and not "The Beatles" as we all expected? Shouldn't the vendors be raking in the moolah as everyone segues from HDD to SSD? Doesn't look that way.

From a Wall Street point of view, it's all tanking. Who knew? Fusion-io, Violin Memory, and Nimble Storage all trade near lows. Oops?

Nimble is the most interesting case. It sells a Seagate-like hybrid drive, but it's market performance is shaky. Fusion-io parts are also server-centric. Which brings up the question: what of memory channel storage? Diablo Tech has been offering flash in DRAM slots for a bit. Again, a tad odd, and server-centric.

None of this would matter much, in that all of these vendors are after the same goal; speeding up data movement within a server. So, why did EMC scarf up DSSD? Read up this write-up, which is largely sound-bytes with EMC's Jeremy Burton. Sounds very much like Linus' observation is beginning to come to fruition. May be.

The DSSD machine is attached through the PCIe connector on a server. However, unlike PCIe devices from Fusion-io, the DSSD machine is connected remotely, rather than sitting inside each server. And so it acts as a pool of storage that fulfills some of the function of a traditional array, but without the latency introduced by traditional connections through fibre channel, ethernet, and interfaces such as "iSCSCI," says Burton.

What neither Burton nor the author address is the elephant in the room: Big Data and Big Analytics remain in the RBAR world, and for that purpose, 15K discs spitting out flatfile record after flatfile record is the bee's knees. Not to mention that analytics software (I'm talkin' to you, R) tends to do its iteration in its high-level code. A few milliseconds of latency won't matter. Refactoring those flatfile images to something more efficient, and with an engine which is inherently parallel (not, alas, Postgres), gains one a lot of speed. Except for corner cases such as median, the only reason quant software has been stuck with RBAR paradigms is that most practitioners still have Σ tattooed in their brains.

Interestingly, Violin has some quotes on its site's pages. I like this one.

If we had invented flash 50 years ago would we now package it up to look like disk drives? The short answer is "NO".
~Robin Harris, StorageMojo

While Harris doesn't, that I can find, display Linus' assertion, reading some of his posts might lead one to conclude that he agrees.

Organic Normal Form™ databases should be next on the agenda. Get all that silly, flabby, flatulent, flatfile Big Data under control.

Joe Sixpack

The key to those wonderful six-pack abs is a solid core. And light, right eating. How's Apple's core doing? For some time, there's been a continuing argument that either a) Steve died, so did the company or b) Steve took the company out of the world of computing into one of gadgets with the iPod, and this decision is now biting Tim in the butt.

For rather a long time, in these endeavors and elsewhere, I stood alone (so far as my extensive reading shows me) in diagnosing the issue as one of Apple being a toy company, e.g. Mattel rather than some kind of computer company. Alone. So alone. Until Today!!

Well, it was "announced" Thursday that Apple would slurp up Beats for $3.2 billion. And the die was cast. But the slew of journalism got into full song today.

This piece shows up, among a host of others, as one might expect. You can find, if not already, your favorite.

Having broken the ice, Apple now must embrace its newly discovered and more realistic identity as a higher end consumer electronics company and continue to move aggressively to extend complementary offerings.

AKA, it's now Mattel time. The Piketty income concentration evidence is also material to Apple's move, while I really, really doubt that Tim would ever admit it through word or deed. As wealth and income concentration continues apace, the TAM (top 20%, by USofA dollars, of global earners) doesn't grow much, and will eventually shrink. Again, as Dr. McElhone observed, "the world is not linear". Income concentration has a black hole aspect: once it gets its act together, the event horizon expands and accumulation accelerates. The Apples of the world have to move ever more product into, at best, static TAM. Burberry (remember her?) does it by the meme of a coat for every occasion. But only a Burberry is good enough for those with the moolah. Apple doesn't have such a product. No one really needs multiple phones or pads or pods. So, just as Mattel has to keep creating new games to create sales, Apple has to offer yet more devices to soak up the 20% disposable income. The problem for Apple is that Beats phones sound like drek, and on purpose. They're ghetto blasters for your head. If Apple goes through with it, it will end badly.

Mine Is Bigger Than Yours. Don't Look!!

One submeme (if that's a word, which isn't a cylindrical meat sandwich) of these endeavors is that today's "innovative" economy is evermore advert pushers exploding from the body a la Alien (yuck!). Or, in line with Krugman (2005):

In other words, a fuller answer to my former neighbor would be that these days, Americans make a living selling each other houses, paid for with money borrowed from the Chinese. Somehow, that doesn't seem like a sustainable lifestyle.

Then it was swapping houses. Now, it's an endless loop of advert makers selling adverts for their advert businesses. A horde of whirling dervishes on nicotine, crack, and LSD.

Today's NYT business section has a very long piece on innterTubes adverts. A j'accuse moment, if it were not so terrifying. Innovation boils down to Mad Men? And, I only know the show from hearsay; never had any interest in watching a fictionalized version of real stupidity. The piece is way too long to bother with making a precis' and claiming authorship.

I'll settle for suggesting you owe it to yourself to go read it, and offer up the best bon mots. The ones which echo my previous lamentations.

Turns out that one is, largely, buying a pig in a poke:

But disappointment turned to rage when she read the list of domain names where the ads were running; it included pornographic websites. The team opened one site with an especially lewd name and gaped in horror. "Oh my God," some shouted. Others cursed. Ms. VanHeirseele picked up her phone to call the media buyer in a fury.

I had, before wading through the piece, a vague notion that the advert business is much like Apple: an advert is largely composed of bought in parts, despite the name of seller of the advert pushing service. There is a long list of the kind of companies twixt the advert buyer (i.e., the company whose product is being flogged) and the advert viewer. Even with sharp language in the contracts, your Bible flogging business could end up on a porn site. Maher would be tickled. (No, that's not explicitly in the piece; just my musing. But it tickles me!)

When ads are sold, even the media buyer that was initially given the contract to place the ads may not know where they are running.

Kind of like buying a Gucci purse from a table in the Bowery?

What's the problem? Much like the subprime ARM mortgages buying McMansions, most just want to keep dancing as long as the music keeps playing. They all hope to either get one of the diminishing number of chairs, or squirreled away enough moolah to have gone home already, when the music stops.

"Except for the advertisers, no one has a vested interest in spending less money," said Jeff Semones, president of M80, a direct-marketing company, who was at the conference. "Whether it's the publishers, the ad platforms, the agencies that manage these activities. Right now, it behooves almost no one to clean up this mess."

I guess someone rewound the ARM movie and it's playing to a new crowd. When there's rules to be gamed, they will be. Data? Don't look behind that curtain!!

For nearly two decades, the meme from the In People on the innterTubes was that the innterTubes was a better way to push adverts. Spend only to get the eyeballs of the folks you want. Newspapers? Magazines? Network TV? Pointless legacy. A sniper rifle rather than a shotgun. Turns out, not too surprisingly, not so much. I suspect the NSA knows how well the gag is working, but they're not telling.

The chasm between the value of such ads to brands (negligible) and their value to publishers and ad networks (considerable) is the reason that many say this medium is at an inflection point.

Once again we find that "honest business" is the second most important oxymoron, right behind "happily married".

Things Go Better With Koch

Wow. Lots to babble about today. Let's get to it.

Since Nate left the NYT, the Times has created a group that they call "The Upshot", which has the remit to talk about data, graphics, and policy. Today was the release of April's employment numbers, and the mainstream pundits are aghast. While the job increase number was higher than the pundits predicted, at 288K, the plunge in the participation rate has them spooked. Just as corporations, the ones left in the USofA at least, make mo money by firing employees; so too does the macro-economy. Lower the denominator, and the percentage (employed, the inverse) goes up. Imagine that. The referenced link, among many others, trots out the "retiring baby boomers" meme as explanation. Baloney.

It's going up for geezers. Economists generally only care about the cohort up to 55, and that's the one that's been dropping. So, kind of right, but for the wrong reason. Boomers can't afford to retire early. Been there done that. Not that I have any interest in retiring. Deming worked, if running seminars in one's chosen passion can be called work, well into his 80's. I attended a couple.

So, it was with some amusement that I read my dead trees NYT early, before I saw the BLS reporting. "The Upshot" folks do a decent job of reviewing, as these endeavors have advised more than once, the details. All of this data, and nearly all of the data produced by OECD, and any other first world source one can name, come from samples. You should find a copy of Snedecor's Little Yellow Book on the desks of BLS mavens. About the only population data is the weekly unemployment number. That's a census, and not always squeaky clean.

A couple of other pieces in the Times which bear on the two major themes of these endeavors, data and policy.

First, Floyd Norris takes on interest. In particular, the demand for mo money for lending money. He begins:

When buyers demand something, the supply will be created.

Especially on Wall Street.

He could have mentioned WS's willingness to create toxic mortgage bundles, at that point in the piece. Whatever.

"It is a very favorable time for bond issuers," said Martin Fridson, the chief investment officer of Lehmann, Livian, Fridson Advisors and a longtime analyst of the high-yield market. "There is just a lot of money sloshing around out there. There are simply not a lot of alternatives, and money managers are under pressure to put that money to work."

The problem remains: there are only two sources of cash to pay the vig on a loan; deferred consumption and increases in productivity. The former is a less than zero sum game, and should generally be avoided whenever possible. The latter is true, organic growth. Back in the 19th century, we had a lot to learn in science and engineering, so as we learned we devised ever more productive assets. Not so much today. We've found all of Earth that there is. We know all the elements, and the ways said elements can become molecules (there's a reason organic chemistry, based on carbon, encompasses far more of the field than that of the rest of the periodic table). The reason interest rates are in the toilet has nothing to do with the Fed. If corporations' CEOs actually had the skill for which they are titularly paid, superior asset allocation, then the Fed's assault on short-term rates would have worked. It didn't.

Krugman closes his piece today with:

Meanwhile, powerful political factions find that bad economic analysis serves their objectives. Most obviously, people whose real goal is dismantling the social safety net have found promoting deficit panic an effective way to push their agenda. And such people have been aided and abetted by what I've come to think of as the trahison des nerds -- the willingness of some economists to come up with analyses that tell powerful people what they want to hear, whether it's that slashing government spending is actually expansionary, because of confidence, or that government debt somehow has dire effects on economic growth even if interest rates stay low.

Whatever the reasons basic economics got tossed aside, the result has been tragic. Most of the waste and suffering that have afflicted Western economies these past five years was unnecessary. We have, all along, had the knowledge and the tools to restore full employment. But policy makers just keep finding reasons not to do the right thing.

Back to Norris. In discussing some new, innovative(?), techniques:

The funds also have the freedom to move in and out of different kinds of bonds, which is supposed to produce superior performance -- and will do so if the managers do turn out to be able to call market turns in advance.

Good luck.

Norris is generally pretty laid back. But then adds this zinger, Vulcan mind meld with Your Humble Servant

All this easy credit may not do as much as it could for the economy. Loans that finance corporate acquisitions, or dividends to private equity funds, won't do anything for employment, or at least a lot less than loans that pay for the purchase of new plants and equipment.

Finally, he closes with (making for a Three Musketeers moment)

At least some of the criticism might be better directed at the Congress. Had it been willing to provide more fiscal stimulus, the Fed might not have been forced to follow the course it has.

Once again, whenever data contradicts policy, and policy is controlled by those with the Big Bucks, policy wins.