The Winning Incentive

Today's NYT has a bumper crop of new data/facts surrounding the overarching themes of these endeavours. So, with minimal gloat, here they are.

First, there continues to be the Rightwing cabal trying to pin The Great Recession on the Dems. For the record, once again, Bush was President, Rehnquist/Roberts ran the Supremes, the House for the whole time, and the Senate save for 2 years. So, the mess is squarely Rightwing. I didn't keep a cite, but one knuckledragger asserted that The Great Recession was caused by QE. That's one deeply stupid pencil neck, or just vigorous liar.

One of the enduring myths from the Right was that the housing mess was caused by the Dems, citing the CRA quite often. What they don't admit is that the mortgage companies, CountryWide spectacularly, weren't banks and thus had no recourse to the CRA. Another myth was that the GSEs led the pack. Again, it was the privates, MGIC and the like, which led and the GSEs, seeing market share eroding, took up the sword.

What is new today, is documentation of how deeply corrupt the investment banks were in creating these loans. As asserted here, a lot, whenever the data don't make sense, follow the incentives. In the case of The Great Recession, it was the need to sop up all that idle money when American CxOs couldn't, or wouldn't, put fiduciary capital to work as physical capital.

Now, though, a trove of emails and confidential documents, filed in court, reveal the extent to which one of Wall Street's leading banks, Morgan Stanley, actively influenced New Century's push into riskier and more onerous mortgages, and brushed aside questions about the ability of homeowners to make the payments.

Who were these loans defined by and created to benefit? The answer to both isn't folks of marginal wealth and income. It's the CxOs.

But the bank remained an important backer of New Century to the bitter end. In March 2007, after other banks had withdrawn their credit lines from New Century, Morgan Stanley gave it $265 million in financing. Soon after that, Morgan Stanley withdrew the money. New Century filed for bankruptcy a few weeks later.

Should we blame all of the investment banks, commercial banks, mortgage companies, and private mortgage insurers? Getting close to, "Hell, yes".

Second, the USofA isn't the only country with a housing/investment problem. China's had one for a long time, for somewhat different reasons. It's long been the case, and known but not often mentioned in the mainstream press, that Chinese have fewer ways to get vig. Housing has been one, and by all accounts with the encouragement of Beijing. Generally, the only one. "60 Minutes" has video of ghost towns. Now, Joe Plunger (I don't know the Mandarin equivalent) can bet on stocks more easily. The inevitable result, of course, is more like a chain letter.

Although the Chinese leadership has long hoped to see a rebound in the country's stock markets, the current frenzy carries risks that could stick investors with heavy losses. Much of the trading is also being done on margin, or by using borrowed funds to buy shares. So the boom could unwind even faster if sentiment sours.

Not surprisingly, since China's capital market is largely closed, housing prices have moved in the opposite direction. It's largely the same Giant Pool of Money (again, no Mandarin), so the seesaw swings.

Third, is domestic, for now, Chinese smartphone market. Thanks to Foxconn, and the like, China is a major (if not majority) manufacturer of smartphone parts. Apple makes none of the parts in an iPhone, just so ya know. With all that capital sunk into part making, the time would come when marginal and variable cost pressures would win. Xiaomi is the result.

Xiaomi, founded in 2010, has overtaken both Samsung and Apple in China by offering inexpensive, high-quality phones through clever online marketing campaigns that appeal to China's growing ranks of young and affluent consumers. Around 500 million smartphones are expected to be sold in China in 2015, more than three times as many as will be sold in the United States, according to the research firm IDC.

Apple has always, at least since Jobs II, ignored all but the upper X% of demand. But you can't run an economy or even a sector that way. Especially in a time of out of control automation of production. Fixed costs eventually overwhelm materials and labour, so the need to shift widgets at any price rears that ugly head.

But the start-up smartphone maker's fast growth, competitive pricing and innovative marketing have struck a chord with Chinese consumers. The company hopes this approach will translate into success in overseas markets, too. Mr. Lei and his co-founders, who include the former Google executive Lin Bin, Xiaomi's president, are considering expansion into large developing markets like India and Brazil.

Correlation isn't causation, necessarily. Sometimes, but not always. For the quants, micros, and macros far less often than any of them would prefer. After all, a magic algorithm would be as valuable as Rumpelstiltskin. Data helps in predicting gross money flows, but even then, the meandering of that flow is caused by changes, generally implemented by those who stand to benefit first and most, in the rules/incentives. The only way to know you will spin straw into gold is if you know about the new incentives before anyone else. Just like the mortgage companies invented Liar Loans, and took their winnings up front, leaving the occupants and insurers up a creek without a paddle. All of those who chased the new rules were just lemmings headed off the cliff, although lemmings aren't actually that dumb. Disney made it up.

Marky Mark

No, not Wahlberg, although you should see "The Departed" if you've neglected it. No, this musing is all about real books, and how to really highlight them.

Years ago, I grew tired of the huge Marks-A-Lot type with the felt tip which broke down soon enough. I used the Pentel Data Checker for a long time, but they're harder to find in the flesh these days. I've got a supply of the yellows, if anyone's interested. I used them "upside down", using the back of the chisel to draw through the text.

I discovered that Noodlers make highlight inks (the page is for firefly yellow, which isn't called out in the swatches) for fountain pens. But two issues: only firefly is a bit fluorescent, and it fades rather quickly. The others are "just" semi-transparent inks, so far as I can see. The other thing is that the others are dark enough that the drawn line stands out enough from the page as to be distracting. I haven't found a pen/nib that I'm willing to pay for that's as wide as a traditional highlighter, so one highlights through the text, not over it. Firefly does blend well enough with white book paper that the meandering line still highlights but doesn't distract.

As to how to apply? Pelikan offers up broad nibs, but the pens are obscenely expensive, and may not ship with the broadest nib; purchase separately(!). Well, the M205 is a bit cheaper, if double broad is OK. Both way more than I wanted to spend, so I've ended up with Lamy AL-Stars with broad nib. I don't remember where I got them, but Goulet is an alternative to Amazon at about the same price.

I tired of the Noodlers firefly fading, and stumbled on Pelikan yellow. Now, that's more like it. Very bright, hasn't had a fading issue, so far.

Go read a real book. You'll feel better in the morning.

Baby You Drive My Car

Once in a while, a random comment on a random post on a random thread on a random site strikes me as worthy of a post here in these endeavors. What follows is such.

IBM's (and American corps generally) problem is that they've lost their marbles. Corps managements (the CxO class) are paid those big bucks because they are supposed to be smarter allocators of fiduciary capital into physical capital, and thus earn a real return. They've all, by and large, stopped doing that. There's a reason that up to $7 trillion is sitting in corporate coffers. Buy backs and dividends only move money from one subset of the 1% to another subset of the 1%. There's no growth, which all corps need, in the general economy. It's just circling the wagons against the 99%. A Robocop world, in fact. No idea how to generate real growth yields to accounting manipulation, aka financial engineering.

The latest scheme, not one IBM is party to that I've read, is 100+% car title loans to poor people. Such a great way to build growth into an economy.

Have a nice day.

Call Me Some Time

Thanks to r-bloggers, we get to read Dirk's rant on a Julia paean. Comments can't be made there, so here's mine.

While the post referenced by Dirk is bogus for the reasons Dirk decries, he's (purposely?) missing the point, one I've made once or twice along the way. To wit (and to woo): the R zealots continue to push R as both (not just one!!!) a Functional Language and an Object Oriented Language. Chambers demurs a bit in his book, but he's the exception. The OO bits are silly on the face, for anyone who's spent more than a semester's worth of time with an industrial strength OO language.

The Functional mantle is a bit murkier, and more deceitful, to my mind. Yes R has functions. So did/does Fortran, from which it sprang. So does every language on the planet, even 4GLs, although they tend to be called Procedures and the like. The point of the Julia post referenced is that R, native, bites the big weeny when asked to be recursive. The thing is, Functional Languages are all about recursion; they live by it. Be shitty at recursion, and you're shitty at Functional. It's that simple. There's a reason Rcpp (Dirk's baby, for those not up to speed) exists. But, just as Guido insists (last time I looked, anyway) that Python will never implement tail call optimization, so too, at the moment for R.

A Functional Language without explicit support for tail call will be fragile. Some structure will blow the stack or heap just when you least expect it.

So, yes, the post Dirk rants about is bogus, but that post's point is true: R isn't really a Functional Language if only because it misbehaves under recursion.

Kicked to the Curb

In some businesses, it makes sense to kick to geezers to the curb when downsizing. Newspapers isn't one of them since it takes a bit of time listening to the liars to figure out what a lie sounds like, but they do that nevertheless. This is Floyd Norris' last piece for the NYT. You should read it.

He opens with:

What happens when you turn over regulatory responsibilities to people who think there is really no need for regulation?

Of course, the Right Wingnuts will likely dismiss, and not bother to read, what follows. They're always right, of course.

To a significant extent, derivatives enabled risk to be shifted from those who understood it to those who did not. Securities deemed risk-free by the rating agencies turned out to be worthless. Much of the financial innovation that so impressed Mr. Greenspan had been designed to let banks find ways to reduce their capital levels without the regulators noticing.

And for the quants:

Bank capital rules came to allow the banks to use their own -- presumably sophisticated -- models to calculate how much capital was needed for any asset they owned. Countries like Ireland and Iceland developed large banking systems and were hailed for finding high-paying, nonpolluting jobs.

Naturally, they ended up polluting the global economic structure. And the cleanup Superfund comes from, in the case of Ireland, almost wholly on the backs of small taxpaying citizens. Moral hazard? Of course not, from the banks' point of view.

Peeing Into the Wind

Thanks to r-bloggers, thence this Revolution post, a skip to this post, finally to the posting on the "evils" of p-value. Yet again. Comments are closed, but between the piece and the pissing contest amongst the commenters it's a worthy source of the giggles.

One deborah mayo (no link) had this to say:

p-values are NOT likelihoods, however, they permit computations that Bayesian likelihoods alone cannot. They allow evaluating the probability that the testing procedure would have resulted in a less impressive departure (from the null) under the assumption the null is true, and also under the assumption of varying discrepancies from the null. It's a small part of the panoply of methods that use error probabilities. Guess what? Bayesians are the ones who only use likelihoods conditional on the observed value! So no error probabilistic assessments are possible. Oh, but there's a prior you say? No error control there either--just what someone believes, and very few scientists want to mix their prior beliefs into the study. The point of the research is to test claims--not beg the question by imputing prior beliefs!

So, yes, once again those villagers out to slay the p-value Monster begat by Dr. Frankenstein turn to the turgid ravings of an ancient Presbyterian minister. Oy vey!!

I Loves Olive Oyl

The problem with being dependent on data to make decisions: what do you do when there is no data? Well, punt. Let's turn our attention to the oil patch. Crude is quoted at $61/barrel (and change) today, and the blue-eyed Arabs in the Red states are whining poor mouth. Already. Gad.

Here's the dirty secret about oil, or any extractive resource for that matter: marginal cost pricing doesn't work in the short to medium term. Depending on the resource, even long term. The reason is that marginal cost <> variable cost. The latter is what determines whether to power the pumps on existing wells, or not. It is just the cost to run the motors, keep them running, and so forth. Marginal cost is how much to bring *a new* well into production. It's clear that the second number is much higher than the other. And, it explains why the Saudis are willing to keep the pumps running; an existing well is profitable at a very low barrel price. Yes, the petro companies would like to fully amortize all the sunk costs on a well, but they have no control over sunk costs. They're sunk, after all. You're not getting the money back. You kissed it goodbye long ago. So long as you get more for the crude than it cost to get it to the surface, you're ahead. It's just second-grade arithmetic. You could shut the well, hoping that the price will rise soon enough, but so do all the other owners. Who'll blink first? Shut the well, and you lose the moolah, but the other owners don't.

But being this is a .44 Magnum, the most powerful handgun in the world and would blow you head clean off, you've gotta ask yourself one question: "Do I feel lucky?" Well, do ya, punk?

How low Mr. Natural? Hard to say, since nobody seems willing to divulge the number. Easy oil comes from a fresh pipe with sufficient gas pressure to lift the crude to the surface from the deposit. All those gushers from old movies. Variable cost of a barrel from such a well: $0. Nada. Zilch. And so forth.

The Wiki explains the succession of involvement to get crude to the surface. One of the reasons Peak Oil came to be a meme was that USofA production was well (pun intended) into tertiary recovery. That costs rather more to lift. In any case, once the infrastructure to aid lift is in place, the variable cost is the electricity to run the pumps (into and out of the well) and the supply of material to inject. That's mostly water, which is ironic in the case of middle east oil, since there's so little of it nearby.

In sum then, there's money to be made from existing wells even at $50 or $60 a barrel. That's my guess, of course. And, it seems, for the Saudi's too.

Putin on the Schitz [update 2]

Incentive.
Incentive.
Incentive.

As always, when there's a disruption in the process which created the time series the quant relies on to forecast future values, ignore the time series and look to the changed incentive proposal.

By now all but the most self-absorbed Fox News watcher is aware that something is going on in resource extraction economies, both foreign and domestic. Yes?

The ruble seems to have steadied from its leap off the cliff, but the flight of moolah from Russia appears unabated, and so from Brazil as well.

Why might all this be? Of course, there's the simple finances of it: lower $$$/barrel of petro means lower $$$ for Vlad. Thus, the ruble drops relative to the Buck, and the Money Men decide that Vlad isn't the savior they told him he was. Not that Vlad actually put all that petro and gas in the ground with his own two hands, of course. Lower $$$ for Vlad means he has to call on his police state to keep the lid on. "How much gasoline for that bag of carrots?"
[update]
Here's a snippet from a CBS News report today

With the ruble hitting record lows, many Russians rushed to unload their shrinking bank accounts on high ticket items like refrigerators and dishwashers.

(Remember: trade is always barter, just that "modern" economies use currency as a kind of lube job.)
[end update]
You read similar here not too long ago. "His" interest rate gag won't work: domestic moolah is leaving by the boatload along with foreign. With the ruble now basically a worthless domestic currency, and Russia not nearly a self-sufficient domestic economy, Vlad could well go to war someplace. Stoke, once again, the vision of Greater Russia (USSR) for Real Russians. Stay tuned.

The fundamental problem for all extraction based economies is that, by the nature of the beast, they have to be fascist. And the reason for that is simple: the value lies in the ground, so control of the ground determines control of the resource, which means control of the moolah. That fascism may be direct, as in Russia where Vlad and his buddies "own" the petro, or it may be indirect, as in the USofA where pliant government moats "private" ownership. The USofA, you say? Beacon of democracy? Not so much in the resource states. The oil, coal, and farm states have been very Red since the Founding. The few that stole the land, not always knowing what resources lie within (beyond soil and timber and rocks), from the Natives wanted to keep the value unto themselves. Pliant local and state (and, on occasion, federal) governments saw to it.

While it does cost more to get the stuff out of the ground once the easy X% has been taken, the value of the stuff is determined by the use of the stuff in production. There is no value-add to extraction; I don't care what Vern Smith bloviates. In the case of petro, cracking towers turn raw petro into various different compounds, with attendant different uses. Value-add exists for that, certainly. If folks can't afford to use, or they need bags of carrots to eat, petro price drops.

So, what does this all mean Mr. Natural? It means that interest rates here in the USofA are about to tumble from their already painful (if you're a coupon clipper) lows. How can that be? All that Russian and Brazilian and such moolah is looking for a safe haven. That'd be us. All that USofA moolah that might have gone to resource extraction also needs to find another home. Treasuries are lookin' mighty good. Expect the next auction to dip even further (the 10 Year Note is 13 January; will be interesting here's a concise report). Supply and demand, Econ 101: mo money chasing diminishing number of chairs as the music plays on.
[update 2]

In a stunning analysis this week, Goldman Sachs found almost $1 trillion in investments in future oil projects at risk. They looked at 400 of the world's largest new oil and gas fields -- excluding U.S. shale -- and found projects representing $930 billion of future investment that are no longer profitable with Brent crude at $70. In the U.S., the shale-oil party isn't over yet, but zombies are beginning to crash it.

And, I'll bet most readers laughed out loud when I said that the next 10-year Note auction would be instructive??

Ides of December Nudge

Another in the desultory reminders that you need to read this guy.

Crazy Schizo Qaunts

OK, so back in 2012 the Right Wingnuts were blaming Obambi for hiking gasoline prices (and, by necessity, petro price). The lobbyer doesn't have a link to the advert, anymore, so here's the quote of note:

Since Obama became president, gas prices have nearly doubled. Tell Obama we can't afford his failing energy policies.

Today, Mr. Market, led by the same Right Wingnuts is going out of its mind over falling gasoline and petro prices: here (and, of course, nearly every news source).

U.S. benchmark crude scythed below $58 a barrel for the first time since May 2009 for a weekly loss of nearly 12 percent in the midst of a tumble whose severity and magnitude matched what happened during the 2008 financial crisis.

Makes one wonder who the USofA economy is run for? All of us?? Or hedge funds?

Those Crazy Quants

Regular reader, by now, is aware that I take the view that quantitative analysis is for both fun and profit, but that it should be attempted somewhat gingerly when the venue is human behavior. We get to make the rules up as we go along, unlike the other animals, minerals, and vegetables which live by God's (or Nature's, depending on your -ism) rules. The rules don't change. We ain't Brownian motion, fur shur.

Well, I just found this post on r-bloggers.

The eBook reprints several unpublished articles and reports from the Econophysics Group at ETH Zurich.

"Econophysics"???!!! And all this time, I'd viewed the Europeans as, generally, more astute. Guess not.

On The Fast Track

Recent musing said that BDUF was the way I understood how real engineers built real projects. Agile type projects didn't exist in the real world. Wrong. Sort of. Real engineers don't always BDUF, although the alternative isn't exactly Agile, either.

Of course, memory failed. Today came a story about Seattle's tunnel troubles, which jogged my memory. Nothing in the story references fast-track construction design and build, but that is the method which emerged from some space deep in the lower brain stem. I don't recall when I first heard of it, but fast-track construction projects have been around since the 1960s, apparently.

"A whole block just went down an inch," said Todd Trepanier, the administrator of the project for the State Department of Transportation, at Monday's City Council hearing. "We don't like an inch."

An inch? Well, yeah.

Anyway, here is piece on fast-track.

Most design professionals believe that only nonspecialized building types with repetitive spaces and standard construction make suitable candidates for fast-track construction. "In cases where you are doing a one-off, customized building, the more you speed up the construction time, the harder it is to get things coordinated, the less iteration in design, the more mistakes, and the longer punch list you will end up with," explains James Timberlake, FAIA, of Kieran Timberlake in Philadelphia...

In other words, since software projects are, kind of by definition, one-off, may be we shouldn't do them 'fast-track'.

Now, wouldn't you love to have this for application development?

Sarah Slaughter, CEO and president of Models of Construction Activities (MOCA) Systems, Inc. (http://www.mocasystems.com), has developed a microsimulation environment that models the entire construction process. It provides comparative information on design and construction alternatives at a high level of detail, usually experienced only during actual construction.

Finally, this paper includes a classic observation:

At one of my presentations, an audience member shared the quip that "If building engineers built buildings with the same care as software engineers build systems, the first woodpecker to come along would be the end of civilization as we know it."

Widget Wonderland

Three semi-related posts went past my eyeballs today. The thread which hangs them together is the value of value. How do we figure it?

First, Blankenhorn takes on Amazon, among my favorite fubar companies. As is often the case, the comment stream is as significant as the post. The poster child for the "we lose money on each sale, but make up for it in volume" approach. Either Amazon attains monopoly, i.e. pricing, power over retailing or it goes belly up.

Second, one Peter Greulich (not familiar) takes on IBM. Here the issue is both the danger of goodwill and the structure of companies. How to value IP, services from IP, and goodwill (e.g. the sale of WhatsApp)? IBM has been shedding real capital and real production for many years. It wants to see the ROI of software driven services. The WhatsApp home run as paradigm for Big Companies, if you will. So do many corporations. Good luck.

Third, Rob Hyndman takes on Data Science. What is it? Who does it? Is it just stats with a new name? Is it just OR, with a new name? Is it more, in some sense, than just stats (Janert says so)? Likely.

Taken together, some musings on the value of value. Micros and quants invariably practice reductio ad absurdum (in the second sense, and on themselves) by looking only at the financials and ignoring the real world aspects. How else would they have blessed liar loans? If quants and micros took the "science" part of data science seriously, may haps we wouldn't go through Great Recessions? One of the comments on the Hyndman post is on that point.

If science viewed thought as property, we'd still be cavemen.

Assume: Ass, You, Me

Some years ago, in the age of the Web 1.0, I worked for a sort-of start up (it was really owned by a long existing parent company). This was back in the days when servlets were built much as described in Jason Hunter's first edition. So, a while ago. But that's not quite the point.

Thanks to this bit of moaning on simple-talk (to which I'm considering adding some bile), I read this other bit of moaning which gave me this bit of spleen:

Perhaps more importantly, if you join or start a startup, you can knock the engineering out of the park and still end up flat fucking broke if the marketing people don't do a good job. But you're probably not going to demand that your accountants or your marketing people jump through bizarre, condescending hoops every day. You're just going to trust them to do their jobs.

Another, shorter sentence from the post, has joined the ranks of signatures. Coming to a browser near you soon.

Anyway, back to Web 1.0. This servlet implemented app was intended to be an extension to an existing app, made by some folks who bolted when the start up had gone belly up a year or so earlier. It got resurrected. Now, the original app built so-called course packs at Cornell (and Olbermann has just skewered it in worst person in the sports world, his alma mater; who knew?), which is to say articles and book chapters by professor/course. The notion was: let's make a web app which allows Joe Sixpack to build the "book" of his design. We'll have the content (or links, more often) and deals with storefront printing operations. Joe wants a "book" on Egyptian dynasties, so he picks a chapter or two from six books and five articles from journals. Sends the list to the storefront printer, and walks home with his "book".

What's wrong with this picture?

Now, this was before music publishers were bludgeoned by some Big Corp into disintermediating albums and pop music albums only have one or two songs that you want, anyway, right? The start up and the parent were teeny, tiny little fish in the publishing pond; no leverage. And most of the books were expected to be textbook types; possibly trade non-fiction but not novels. The notion was that these sorts of publishers, covetous of their copyright, would be willing to sell off bits of books, rather than the whole enchilada. Whether authors might benefit wasn't part of the discussion.

While the system got built, lo and behold, publishers refused to play nice. Why should they sell off for pennies what they could demand for dollars? It was their expectation, which turned out to be true at that time, that selling bits of books wouldn't offset the cost of slicing and dicing the books nor loss of the sale of whole books. The business model was never vetted; just assumed to work because the academic prototype worked. Never assume. Times have changed a bit. Perhaps someone will resurrect the app?

While Kevin Costner got famous for saying it, it ain't true, "if you build it, they will come". Sometimes, they just stay home and watch football.

A Giant Sick Koala

GlaxoSmithKline is a drug company that Wall Street hates. I don't follow it, except the occasional piece that appears in places I usually visit. One such piece is from the prolific Dana Blankenhorn.

The only reason I bother to mention it is his following observation:

[Dr. Moncef Slaoui, then the company's head of research and development] said that researchers should "gamble their careers" on the drugs they worked on.

This sounds good, if you don't know science, or scientists. Because scientists don't get into science for ROI. They don't get into science to "gamble their careers" on a clinical trial. They go into science to do science. They go into science to learn things, to discover things, to help mankind. What Slaoui wound up with, instead of a bunch of entrepreneurial scientists, was a bunch of backstabbing neurotics.

[my emphasis]

Sound like a recent musing? Yes, yes it does.

Prometheus Unbound [update]

The mainstream pundits, and the blogosphere too, continue to be perplexed by the refusal of open market interest rates to rise as they had hoped/wished/needed. So, it falls to moi to yet again 'splain some more.

The notion of moolah as commodity rests on the base assumption of trade in real goods: that supply and demand are functionally infinite, and that the "curves" result from individuals' (and remember, corps are people too) levels of psychic satisfaction. To put it yet another way: real demand requires only that "I" want another widget and have the moolah to pay "more" for that widget. The only limit is my satiation with said widget. On the other side, supply, Widget Corp. is willing to make widgets until Hell freezes over if enough folks want them. Availability of production inputs are assumed to be infinite in the sense that a source of supply always exists, at some price.

Now, when it comes to investment, the situation gets a bit gnarly. The MBA types, and most quants, don't distinguish between fiduciary capital and real capital. Most, likely, don't even acknowledge that the latter even matters. But, of course, it does and explains what's going on now. Financial engineering isn't real engineering; not by a long shot.

If we view moolah as an investment commodity, then the gnarl sets in. On the supply side we have retained earnings by business and non-consumption by the rest. On the demand side we have business. What's the motivation by business to take fiduciary capital (moolah, to you and me) and turn it into physical capital? One thing only (in a rational world, of course): that the plant and equipment so made will yield more profit than by not doing so. Nothing else. Financial engineering, so beloved by Wall Street and The City, doesn't count.

Note, however, the difference in the demand function betwixt consumer demand and capital demand: the consumer just wants a better hard on, but the CxO demands that the capital produce returns **not otherwise attainable**. The CxO, if s/he's rational, isn't maximizing some satisfaction function, but some very specific engineering breakthrough. In order to get that return the CxO has to know, or have a high probability, that the new plant and equipment are better than what s/he's got now.

Said plant and equipment may be something entirely new, such as a Swiss screw machine around 1870, or it could be off-the-shelf items not yet adopted by a business. Either way, unless such plant and equipment actually exist, there is no demand for the fiduciary capital. It will just sit around as retained earnings or used to buy back shares or buy bling for the CxO class. Since the MBA and quant class remain stuck in a 19th century mindset (Right Wingnuts all), they can't see that the arc of technology and resource availability has turned, at best, flat. If one considers the widgets we use today, how many implement a semantic that didn't exist before say, WWII? I'll bet a nickel that the number is 0. Capital has spent the last few decades in ever decreasing incrementalism. Ponder that for a moment. Near zero interest has not a thing to do with Big Gummints, but an utter void in the heads of the CxO class to find new, and useful, ways to turn moolah into machines.

This is the base reason so much fiduciary capital has been thrown at non-productive uses such as residential real estate. The Masters of the World are intellectually adrift. And we, as a species, know just about all there is to know about physical reality. Hell, we've found the God Particle. Once you've reached the edge of the World, there's no frontier to explore.

[update]
Sometimes ya just shouldn'ta oughta got out of bed. Left the punchline in the briar patch. So, here it is.

Widget Corp. can go buy more land, labour, and physical capital if it has the moolah. What it can't buy is any new Law of Nature (or God, depending on your -ism) or the brains to find it. Those two either exist, or they don't. These days, the chances that there remain any of the former get nearer and nearer to 0 and thus the chances to buy the latter (I wonder, would paying Newton or Einstein more moolah made any difference?) do too.

60 Seconds

That appears to be how much time and effort went into last night's "60 Minutes" piece on consumer credit card and ATM and such hacking. If you accepted the statements made, then you believe:
1) retailers and banks are doing everything humanly possible
2) Eastern European hackers are the Smartest Computer Folks in the World
3) breaches can't be stopped
4) Corps. should buy the services of that "security expert"

Here's what we actually know (and the "60 Minutes" reporter[s] would have if they did any reporting):
1) the POS breaches are due to antique versions of Windoze that Corps don't want to spend money to upgrade
2) Corps. routinely don't isolate valuable and vulnerable customer nets from the rest
3) hackers are using time honored methods to breach old versions of Windoze
4) banks are replacing OS/2 run ATMs, which are nearly bulletproof, with Windoze rather than secure certified *nix

I want to be clear with my point of view here. I think that migrating from OS/2 to Windows is the most stupid thing that can be done to an ATM Machine.

So, as with the coders who refuse to upgrade their skills to Organic Normal Form™ schemas, Corps. continue to run vulnerable systems for customer data just because it's seen as too expensive in time and money "to do the right thing". [Aside: if banks ran ATM networks as described by Allen Holub in "The Bank of Allen" series, none of these problems would occur, of course.] IOW, once again, the problem isn't tech it's politics. The restitution, if any, is a slap on the wrist. Just a cost of doing business. The restitution and fines for allowing breaches can be written off, so the taxpayer picks up most of the cost. The upgrades can still be put off. The CxO types still get their fat bonuses for saving on IT spend by not taking security seriously.

Play to theFront Row

The pundit class has been wondering, in print, how it can be that we have increasing income/wealth inequality, but, if one believes the data, a recovery from The Great Recession and general growth.

The answer, in a nutshell, is Apple. Apple has made a living by ignoring the bottom 80% (or thereabouts) and selling a limited set of models to the top 20% (or thereabouts). A small list of models means what Henry Ford (is said to have) said, "you can have it in any color you want, so long as it is black". Smaller global BoM, smaller unit cost, higher gross margin. You just have to convince your market that they only need that one model. Mo money. As other microeconomists and quants see the light; likely have for some time, what can we expect?

As these endeavors have pointedly asserted, when it comes to data, analysis and decisions based on data is a worthy exercise only when the venue under analysis obeys God's (or Nature's, depending on the -ism you subscribe to) laws. Read the news every day, and you'll find a case where the clever (but not clever enough to avoid getting nabbed, of course) have bent or changed the rules to their advantage. Data on human endeavors is not reliable when used to predict the future. The future's rules won't be today's or history's. When the players are making up the rules as they go along, relying on recent data is foolish. Data driven decision making only works when the decision makers are forced to obey a rule set out of their control. Lower animals and glaciers come to mind.

When the recommendations of data and incentive conflict, follow the incentive. The clever will, like mouses, squeeze through the tiniest crack in the wall. Build a new wall, and they'll find the new cracks. Time series won't tell you where those cracks are.

So, what does Apple have to teach us? First, that by concentrating on the High End, its market in units is limited; as concentration continues apace, the number of individuals near the top X% diminishes as the X%-ers accrue more moolah. Apple counters that fact by limiting models of any device, and thus minimizing corporate BoM. The counter is to increase the number of distinct devices. Whether the iPad (or the don't-call-it-an-iWatch) can survive is an unanswered question. Reporting today reveals geographic/economic differences in mix between iPhone 6 and 6+. Why might this be? The logical conclusion is that lower wealth populations will have more buyers willing to compromise both ends of the size factor by having just one device. The phablet only holder, while the higher wealth populations will have a more reasonable device for each end of the size factor. Should this be a surprise? Not.

What of the macro effect then? It depends. That Burberry marketing gal had an easier time of it peddling coats than she will compute devices. In general, the top X% will have more moolah to spend on bling, but they won't necessarily spend more on bling from any one XYZ, Inc. widgets. Said widgets, unlike fancy coats, may have no value beyond having one. What we should expect then (looking at the incentive, not the old data), is to find more XYZ, Incs. making different sorts of bling for the X%-ers. Samsung was reported in the last couple of days to be reducing its model count drastically. Lowering the corporate BoM to make more moolah. So, what's the next big thing in X%-er bling? I wish I knew.

But, not all is wonderful in X% land. Consider participant sports. Turns out Tiger Woods is thought by some to be key to the golf widget makers' survival. Golf can be an expensive pastime, right up there with skiing. Poor person envy? May be yes, may be no; but it doesn't matter to the analysis. I've never had any interest in either, which I can duly afford if I wanted to. And therein lies the problem with growing the moolah flow to providers of the various bits and pieces associated with them: how to engage the small population that is the X%-ers, because only they can afford the vig. What Americans play tennis anymore?

Professional team sport provides some insight into how it can be done. Players in these stadium sports get ever more tens of millions of dollars per annum to behave like teenagers. How can this happen? Again, look to the incentive. Partly, TV dumps billions of dollars into the pot. But consider the stadiums. Research (I've not got cites to hand, but they weren't hard to find when last I went looking) has shown that most are partly or fully paid for by taxpayers, both directly and indirectly. Most are in large metropolitan areas, where the number of seats is a small fraction of the market population. Since, let's say, the stadium on average needs 50,000 behinds to fill it up, then the team need only convince the increasingly wealthy 50,000 to attend out of a few million in total population. They have no need to attract the lunch bucket crowd. IOW, the Apple crowd has another way to spend disposable income. For the privilege of being special, they can afford to pay yet more. The Washington football team attacked in 2009 when The Great Recession hit. All was not well in Mercedes land (for the record, the real money in the DC area isn't made by civil servants). What's odd about such suits (the piece reports that Washington is not alone in suing) is that, according to legend, season tickets for Washington football (and many others) has a long waiting list. Hard to see where the loss is? Kind of like foreclosing on a house that's still brand spanking new. It should be no surprise that the various leagues expansions into ever smaller markets (hockey in Florida, Arizona, and North Carolina???) have led to failures, and near so. They have to have some minimum population to have the necessary 50,000 X%-ers. Some leagues (3 of 4, not NBA, near as I can find) counter with "revenue sharing" wherein the rich big city owners give to the small hick town owners, aka rich white guys' Socialism.

In general, the incentive for an economy's production under rising concentration of income/wealth is to shift to more varieties of lower volume bling. No one needs two smartphones, drug dealers and con artists possibly excepted, so some other forms of bling must be created to sop up the excess moolah. This happened before in the USofA. The most well known was called The Gilded Age, a coinage and book title from Mark Twain, which, by way of setting context, he wrote before both "Tom Sawyer" and "Huckleberry Finn".

So, the question for macros, micros, and quants: can such an economy thrive? And, of course it can. Except for the period post-WWII to the 1973 oil embargo, that's been the history of the USofA. It was only that rather short period of unspoken Socialism that is the unusual condition. The downside, depending on which side of the X% you sit, is the necessity of an increasingly motivated police state to keep the non-X% from revolting. If you read 19th century American history, it was awash in minor and major insurrections. The fact that there was ever more land to move to and pillage helped keep the lid on, until it didn't. The Civil War is the one remembered, but there were more clearly economic ones across the century. Here is another list, but both don't include Bloody Kansas, more properly the First Civil War. So, of course we can have an economy based on the top X%, but as X gets smaller, the police state must get more motivated. And to think, all those ex-Middle East armored personnel carriers and such have found their way to civilian police under a black president. The X% can't say, "if you don't like it here, move to the frontier and see if it's any better." There is no more frontier. Have a nice day.

Why Not Be Normal? [update]

There is a recent thread on simple-talk dealing with stored procedure as the repository (execution environment) for business logic. One commenter took the opportunity to assert that such a horrible mistake gave rise to NoSql; may be not entirely. And, of course, the implied inferiority of SQL databases, normal forms, and the RM paradigm by implication. Bollocks, again. NoSql is yet another failed, one would dearly hope, attempt by client-side coders to return to the thrilling days of yesteryear, when a lone ranger coder siloed all the data management in his code. And so forth.

The argument against 5NF boils down to: joins are too slow. One might add that most application developers are too dense to understand normalization, and that they prefer the employment security inherent in client-side data management. In other words, it ain't tech it's politics.

The point of these endeavors has always been that tech as it has emerged over the last half-decade or so gives us the tools to implement the RM as Dr. Codd intended. And, of course, that we'd be idiots not to seize the opportunity.

As node sizes have dipped toward 10nm, we get multi-billion transistor cpu/gpu implementations, motherboards with multiple sockets and four or more memory slots. For under $10K, one can build an awful lot of server. That same $10K will support about one month of a client-coding developer. If that doesn't conjure up an image of Jabba the Hutt on one end of the seesaw and Bruce Lee on the other, here's another reason.

Intel has released details of its next 3D NAND, terabytes are now on offer. Fast terabytes. There's just no excuse for all the mess and heartache caused by flat earth folks.


[update]
The comments remind me of the core irony of the conflict from coders toward NF data: the OO crowd always talks about two principles that they hew to, no matter what. First is DRY. The second, related, is that objects and methods must deal with as limited a scope as can be conceived. One java coder of my recent acquaintance used to say that his methods were never "more than a few lines long", as if this were the ultimate virtue. "Do one thing, and do it right" is what we are told. No bloat. No repetition. No duplication. It isn't much of stretch to see that both attitudes applied to data must yield NF schemas and the DRI which results. Yet, they insist on silos of flat-file (or xml, which amounts to the same thing) data.

My Pappy used to tell the old wheeze about the miner, the mule, and the eastern fop. A miner comes into Busted Flats to stock up on supplies. He ties up his horse and the mule to the rail. Or tries to. The mule stops dead in the muddy street. Standing on the walk by the general store is an eastern fop, just off the train. The miner pulls at the mule for a bit, then tires of the effort. He reaches into the pack, pulls out a 2 by 4 and whacks the mule upside the head.
Fop: "You can't do that!!!"
Miner: "Why not? I gotta get his attention somehow."
The mule walks up to the rail, gentle as you please.

A Dirty Little Secret [update]

The quants operate, oft times unknown to themselves, based on a dirty little secret. "What might that be", called from the Peanut Gallery?? The notion that the risk-free, inflation-free interest rate is equal to homo economicus' "rate of time preference", i.e. homo econ will lend you one unit of fiduciary capital (dollar, bolivar, ruble, whatever) for one unit of time (day, month, year, decade, whatever) on condition that homo econ gets one unit plus some vig. And that such rate is completely independent of everything in the real world but homo econ's demand for the vig.

All those wonderously complex financial engineering models, the quant ones not the reg fiddlers of course, depend on this notion of some exogenous interest rate. Taint so, alas.

The secret, of course, is that this assumption of control, or even choice, is false. The risk-free, inflation-free rate of interest is determined by technological improvement. This risk-free, inflation-free rate is the last chance opportunity cost faced by capitalists. But, here's the thing: if they're unable to conjure productive uses of physical capital, there's no other demand that homo econ can bring to bear on borrowers. Homo econ may want 10% for the use of his moolah, but no one can earn that, so his demand is ignored. And so on down the number line, to 0% where there is no productive use and time preference is irrelevant. No one wants your damn moolah!! In times, such as now, when capitalist overlords can find no way to turn fiduciary capital into physical capital, i.e. (again) they cannot conceive of new plant and equipment improvements which are more productive than existing, it matters not how much holders of units of value wish to be paid. There's no growth in the system to pay them. Full stop. Period.

Unlike usual trade with a supply and demand which are independent and infinite (that last bit is vital), capital return isn't. While a capitalist can make an infinite number of widgets, simply given a high enough price (or consumers take an infinite number given a low enough price), an infinite return on physical investment has to be the result of infinite ingenuity in technology. You have to be organically smarter, you have to know that which is better and was not known before; it's not sufficient to, somehow, "work harder" or have access to more abundant, cheaper inputs. Ah? You need a bigger brain which can divine more about the real world than is universally known, already. Ah? What happens when all that's left to discover is trivial? Cold fusion? Mr. Fusion? Ford Fusion? Movies on a portable phone?

Thus, in this state of the world, we get stories with titles thus, U.S. stocks are soaring, global economy in a ditch: Now what?. Even the business press is befuddled. They really ought to know better. It's just simple arithmetic.

And what's great for American corporations isn't always great for most Americans. Merger activity has picked up to new highs this year, and U.S. profit margins continue to hit record levels, defying forecasts that they are going to come back down to earth. [Yahoo Finance Editor-in-Chief, Aaron] Task says, "The problem for the rest of us is that these corporations... are not spending a lot of money on new development... and they aren't doing a lot of hiring or when they do hiring they are not paying people big wages."

There's a reason that Big Gummint debt, which is auctioned (not set by fiat) just to remind, is going for so little. The alternative is to sit on your moolah when you can't figure out a way to allocate to growing (organically, not just by buying your competitor) your business. Which Big Corps are doing, to a faretheewell. The opportunity cost is, basically, 0.0 (props to Bourdain). They're sitting on trillions of dollars, with nothing to spark their interest. So to speak. Well, they've invented securitized, sub-prime used car loans. Deja vu all over again.

The Wiki article is pretty much what is written in Econ 101 (and subsequent) texts. Quite obviously, it ignores the source of the vig. That source is technology, nothing more. The Dark Ages were a time of stasis because knowledge was so, not the other way 'round. Now that we know pretty much everything about how the real world works, we're near a permanent Dark Age; this isn't 1800 with the periodic table to be filled in. We're in a period, likely infinite, of ignorance, no longer a world of discovery. Most everything that marks us as modern was discovered in the first half of the 20th century or earlier (the understanding of semiconductors began in the 19th century). The technology discovery curve has flattened out; the big deal is whether a 4" smartphone is worse than a 5" smartphone. The population curve hasn't. There is no New World to pillage. And so on. Have a nice day. And, abandon the hope that compound interest will make you, or your spawn, rich just because it worked for your grandpappy. Those days are past.

[update]
To demonstrate how meager "innovation" is these days, we now know that the iWatch is just visual Bluetooth!! Some fool recently compared present day innovation to the steam engine. Gad.

Make It a Double

When I was growing up there were three categories of lushes: the crude drank bourbon, the smooth drank scotch, and the sissies drank screwdrivers (so Mom couldn't tell; she still could, of course). Seems to be still the case.

There's been a push, and some pushback, for artisinal bourbons. They don't really exist, of course. Here's the short answer as to what makes a bourbon, bourbon. Note, in particular, that aging is not a requirement. In fact, the latest fancy of lushes is white lightning, often called corn squeezins, because it's just the pure alcohol distilled from corn mash. Put that stuff in burned barrels until it gets some color (kind of like white folks wanting a tan), and you can call it bourbon. Doesn't even have to done in Kentucky, although those that are make a big deal of it.

The point being: bourbon is crude liquor made fast. You can leave it in the barrel for a year or more, but it's still just colored corn squeezins.

I bring this up, because Neil Irwin badly tries to analogize bourbon to petro-products in his piece today. For someone who presents as a quant/economist, bad piece. About the only thing they have in common is that they're both liquids. Too bad that Irwin chose such an approach, since the petromarket is totally screwed up. But bourbon production has nothing to aid in fixing it.

Bourbon:
- little physical investment; little barrier to entry
- available nearly instantly, unless you want other wise; the longer you hold the higher you can charge
- typically, but not required to be, made in Kentucky

Petro:
- lots of physical investment; massive barrier to entry
- not available, if at all, for years after initial investment; holding may, or may not, yield a higher price
- can be found nearly anywhere, that's left unexplored

What is true of petro-products, and needs to be dealt with, is that it is, for first world countries, as vital to existence as clean air and water. As such, it should be husbanded for the benefit of all, and all future generations. The cyclic nature of production, in response partly to market price (but also to more efficient, in the short term, technology), is quite real. Do we, of the current generation, have the right to burn off nearly all of it in our Hummers? The Rand-ians say so. May be we should stuff a cabbage in their mouths.

Sweet Charity, From an Unlikely Source

One of the conundra of our times: Amazon can't manage to make a consistent profit, yet its share price seems to be always rising. Well, until this year where it's been bouncing around the $340 mark. And why is this? Once again: it costs a bunch more to ship one widget by aeroplane then it does to ship that one widget by the tonne in a train. Consumers won't pay the extra if they can avoid it, so the billing price of an Amazon widget has to come down such that the price + transport is in the ballpark of brick&mortar stores. Econ 101.

Today, the good folks at UPS fired yet another shot across the bow of Amazon, et al. Be wary. Be vewwwy, vewwwy wawwwy.

Managing those costs as e-commerce continues to grow is a priority, [UPS CEO David] Abney said. E-commerce accounts for about 45 percent of UPS' business, and should reach 50 percent within five years.

"I think anybody that's delivering directly to consumers is certainly going to be looking at those costs," he said.

It's now some hours later, and this piece came through the innterTubes. Amazon realizes, yet again, that it has to eat the transport costs. For now, at least. Another way to spend years not making any money.

E-commerce professionals polled by MarketWatch were more mixed in their views. Consumer World founder Edgar Dworsky said he was "not aware of that trend," and said ordering a bottle of Tide online could prove an "expensive proposition" if the consumer were responsible for shipping costs.

Yeah, we know: lose money on each widget shifted, but make it up on volume.

ATM at Home

These endeavors began at the bottom of the Great Recession, and one of the first themes (and continues unabated) was that the use of the Home as ATM was driven by the increasing inequality of wealth/income. The right wingnuts are always complaining that growth is poor because poor folks don't spend money, but that's their point, of course. As the 99% continued to lose out on the growth rewards, they were enticed into burning the (un)earned equity in the Old Homestead. Hard numbers were hard to come by, unfortunately, so the schemers could continue to claim that the Great Recession was the result of Democrats' loosening of the mortgage process to allow their poor folk constituents to get into the game.

One Bethany McLean has talked to the perps, and comes up with the hard data.

According to the financial statements of New Century, the huge lender whose bankruptcy in early 2007 helped kick off the financial crisis, cash-out refinancings were 64.2 percent and 59.5 percent of its business in 2003 and 2004; home purchase loans made up only 25 percent to 35 percent for the two years. A New Century executive told Congress that its customers needed to "tap into their home equity to meet other financial needs, such as paying off higher-interest consumer debt, purchasing a car, paying for educational or medical expenses and a host of other personal reasons." I'll always remember seeing a bank ad blowing in the windy, bleak Chicago winter of 2009. "Let your home take you on vacation," it read.

The right wing just can't admit that it's trying to have its cake and eat it too: keep the 99% poor, but keep the economic merry-go-round spinning at breakneck speed. It can't work that way. Li just made it easier to justify, but not intellectually or arithmetically viable. That link is to a Forbes article; amazing.

The bottom line, so to speak, from McLean's research:

According to Jason Thomas, now the director of research at the Carlyle Group, only about a third of subprime mortgages that were turned into mortgage-backed securities between 2000 and 2007 were used to buy homes.

Makes one wonder what's driving the consumer sector these days? The obvious answer is that the Jobs-ian paradigm is at work: consumer oriented companies are all chasing the 20% with money. The 19th century USofA economy worked that way. Back to the future.

Talkin' 'Bout My Generation

It's been rather some time since I've explored the world of GUI generation from schema. So far as I'm concerned, it was a solved problem when middlegen (RIP) came out about a decade ago. Nor am I a PHP fan. Nevertheless, here's another shot on goal. Why do we still need so many client-side coders? Hmm???

Booooooiiiiiiinnggg

It appears to be an article of faith, among the New Age capitalists such as Jobs/Cook of Apple fame, that one need only target the few, at a high price, to be success in business (without, of course, really trying). In other words, as you learned in Econ. 101, demand is price inelastic. In fact, one might argue, that the Apple paradigm is to grossly overcharge the BoM, since if the widget's painfully expensive it must be good. Marketing has to convince the few who buy that they've gotten a good deal.

But price inelasticity isn't a given, and most often applies to things like food, water, and clean air. Necessities of further existence, in other words.

Now comes word that M$ has seen the light. Gee. Lower the price, take a smaller cut on lots more shifted, and abracadabra, more moolah. Gad.

Since the price cut was introduced on November 2, Xbox One sales in the U.S. have tripled.

Nobody, pimply faced adolescent males who don't bathe possibly excepted, needs any sort of game console. Go read a book, for crying out loud.

D'oh!!! Get me a Duff's.

Remember When M$ Was "The Borg"?

It looks like The Federation is close to winning. If only SQL Server (I do like and respect it; Windoze, not so much) can be made to run on linux?

What Are The Odds?

The initial title was going to be something like, "Marty McFly is Grounded", since the subject is Matt Martoma, and yes, it's a bit of a stretch for an allusive pun.

Mathew Martoma, a former portfolio manager at billionaire Steven A. Cohen's SAC Capital Advisors LP hedge fund, on Wednesday lost a bid to stay out of prison while he appeals his insider trading conviction.

As the saying goes, "be careful what you wish for, you just might get it." The wish, in this case, is that cracking down on white-collar financial crime will deter the self-absorbed, greedy Wall Streeters from sinning. It was TNT running "Castle" marathons that got me hooked on the first run on ABC. In the archive shows, Lockwood (a bad guy hired by the Real Bad Guy) says to Capt. Montgomery, "If you and Raglan and McCallister hadn't sinned so spectacularly, then God wouldn't have sent a punishment like me." Soon after Lockwood and Montgomery have a shoot out with both getting killed.

In the case of Wall Street and Martoma, what seems the most plausible outcome? Well, these guys are quants, more or less, so they think in terms of odds and expected value. Well, if we raise the odds of getting caught, losing the moolah, and having to wear Orange for a decade or so, then we've lowered the expected value of schemes currently in play. We, therefore, should expect that the Wall Streeters will dream up schemes which increase in value (to them, not the 99%). I'll all for the guillotine as punishment for such behavior, but that's not enough. The folks in Congress, well the past one not the next one fur shur, also need to anticipate where the wedge will go next. Crooks be clever dudes.

Chasing Unicorns

Josh Berkus, Postgres factotum, has a new post up asking whether there should be a, and what it might look like if there were a, web/PG benchmark app. I kind of noted that web performance in the normal HTTP world doesn't have much to do with RDBMS specific performance, anyway. He wasn't impressed. Not that I mind. The notion did get me to ruminating, though.

This overview is from 2010, but confirms my experience: smart code fiddling dumb data remains the paradigm. Gad.

Most of the social networks also use Sun's MySQL database management system to organize their users' messages and status updates.

They, obviously, would rather keep writing code to keep track of things because, well, they get paid to write code. So, write code they do. There is no try, there is only code or no code.

PG has long been the "Open Source database that isn't MySql and sort of relational", so why would the PG inner sanctum want to toss that all away by jiggering PG into just another sql database which is good at storing flatfiles? Josh mentions that such a benchmark would target social networking. Again, this is the ken of those who are certain that flatfile storage is the bee's knees, at which MySql 3.0 was very good; current releases can be run that way, too. I strongly suspect they are.

Here's a chronicle (or, may be, a superbly written indictment) of a social network app that started life on MongoDB. I find it interesting. The horse punches out Mongo, this time. May be the (social networking/start up/real) world is actually relational, not flat or hierarchical, after all (Date is right, in other words)? Why chase them? Not that I've been harping on it, much.

You can also see why this is dangerous. Updating a user's data means walking through all the activity streams that they appear in to change the data in all those different places. This is very error-prone, and often leads to inconsistent data and mysterious errors, particularly when dealing with deletions.

Kiddies spend too much time learning multiple languages, and not enough time reading Date and Weikum & Vossen.

I learned something from that experience: MongoDB's ideal use case is even narrower than our television data. The only thing it's good at is storing arbitrary pieces of JSON.

Just for context, the author made a reference to Romulans to some of the kiddies she was dealing with, and got nuthin'. If they don't even know about recent history... Kiddies keep making the mistakes our grand pappies made; inventing square wheels. Gad.

Schema flexibility sounds like a great idea, but the only time it's actually useful is when the structure of your data has no value.

She's nicer about it than I am: the flexibility of xml, and its -ish cousins, is just pretty pink bows on chaos, aka illusory, which the coders have to work around since a hierarchy is rigid (while adding a layer, from a typist's point of view, is simple...) and to change it requires walking all the code which deals with the data at the point of change on down (at least). They don't mind, as it gives them more code to type. They like to type code. Real flexibility comes from Organic Normal Form™ schemas; since the data is stored in an orthogonal manner, changes to the schema are transparent to any existing data and code that are disinterested. With the cheapness of white box *nix machines and prosumer SSDs (Intel has just upped the ante, by a tad), joins are basically free. High 5™s all around.

Why chase unicorns?

Really?

Really. And people continue to argue about Organic Normal Form™. Get a clue.

Worst Study in the World

Olbermann just did the "Worst Persons in the Sports World" bit, and eviscerated the "study" of short NBA refs. The show repeats a couple of times more, on the East Coast at least. A Must Watch.

Grinding the Hard Candy

Artima has this post up today. The topic is how to approach the problem of making code, basically. The poster discusses the "hardest first" versus "easiest first" paradigms. He, imagine that, falls into the hard first class, and lonely.

To me, at least, this mirrors (or is the origin of) the database as RM engine versus brain dead file dump with easier backup. After all, figuring out the Organic Normal Form™ schema is mostly a thinking exercise, not a typing exercise. And folks still wonder why applications are still garbage. They're mostly made by monkeys whaling away at keyboards looking busy and spewing thousands of lines of code per hour. Swell.

I do hope we've not reached the point where it's automatically seen as wrong to stop and think instead of diving straight into writing a test for some "simple" case.

Just A Taste

One of the continuing themes of these endeavors is that quant isn't always the best guide to the future; moreover, the data which is ground, finely, through the mill stones isn't always understood for what it is. This is particularly vexing for the macros in the crowd. The micro folks tend to have population data to hand, so aren't really doing stats as a math stat would define the effort.

Chief among these are the government (any one, by the way) economic activity data. Floyd Norris saves me the time to recite the problem(s) once again. The main point is that only the weekly UI numbers are population data. All else is sample data, of varying "quality" and source. Census does most of the actual surveying, for BLS, Commerce, and such. There are periodic censuses for economics, mostly in non-population census years for obvious reasons.

Having been a Fed quant, although not for any agency which produced public macro-economic numbers, I will attest that the grunt quants took no shit from the political appointees. The recent "scandals" that seem to come from the bureaucrats were actually perpetrated by what were known as Supergrades for decades. They're now called The Senior Executive Service, but the issue remains: they're a bit fish and a bit fowl; some of their moolah is scheduled and some arbitrary from the Suits. They sit (or sat, depending on how one looks at it) in the top three rungs of the ladder of civil service. Some argue that this change, which happened under Carter and while I was there, has weakened the Chinese Wall betwixt the grunts who try to be honest and the appointees, who aren't.

Anyway, Norris does a decent job of explaining the situation, without getting into talking about stratified random samples and such. [Aside: with the current hard on for Big Data, one might wonder whether the expertise, across the profession, for sampling might just go poooof???]

My favorite quote:

[Paul Singer, a hedge fund billionaire and top Republican donor], in his denunciation of official figures, questions adjustments that are made for the quality of goods and argues that government indexes understate the costs for the wealthy because prices of things like luxury real estate and art have increased far more than the prices of goods purchased by the middle class.

Oh, the poor little rich kid!! And such folks just don't understand why the French finally had enough and separated some heads from some bodies?? These are the same folks, of course, who bet that house prices would keep going vertical forever, and when they didn't came to the taxpayer begging. And, having gotten their billions, bitch and moan about any small attempt to keep them from desolving the economy, again. I think that's a working definition of ingrate.

As to quant not being the best avenue to understanding the direction of the road ahead, the NYT has another article today which tells you more than any time series. Both Japan and China have forced working folks into mortgages as the sole investment vehicle available. Various reasons why, but the consequence is, mostly, that mortgages aren't productive use of capital. Any return comes from the holder's ability to shift moolah from consumption to paying the mortgage. Psychic income is baloney. The notion of safety in home mortgages was true before the 1% sucked up so much of national income; mortgages were made and held locally, and so on. So olde fashioned. But we didn't get chaos. Change the rules, and you change the incentives, and you change the money flows. Kind of like a switch in a train yard. That Giant Pool of Money is still out there, and continues to grow. American corporations continue to: sit on it, buyback shares, buy each other, and otherwise spend it on non-productive (in the macro sense) allocations. So soon you forget, heh?

Home Is Where the Heart Is

A couple of interesting occurrences here in West Redneck, CT this past week. First, I got a LinkedIn message, which I ignored. Then, some auto nagger fired up. OK, so I read the message. Turns out that a recruiter was looking for a "SQL Advocate" and did I know of any?? I guess I didn't meet the req. Having a bit of time, and feeling a bit snarky, I typed up, with reduced snark, a reply.

Said reply amounted to:
- I'm the only SQL advocate (really, RM advocate, let's be honest) in this part of the world
- unless the C-suite types enforce it, the coders will obstruct, obstruct, obstruct (kind of like Republicans)
- my experience is that SQL advocates, in orgs run by either pure coders or business analysts, end up with RDBMS as passive filestore
- does your client really, really understand the implications of having a person who's going to be at odds with most of the staff
- what, exactly, does the client want to accomplish; what change is desired
The database wasn't named, but I'll guess SQL Server.

Much to my surprise, I got a message back from her. And it confirmed my concerns; she didn't know what the client wants to do, and doesn't think the client knows. I was shocked, shocked I say!! May be not so much. The recruiter showed a Boston area address, which means there could be some form of intelligent life. Her client, turns out, is not far from West Redneck, but not Hartford or New Haven. Not that either necessarily means a SQL advocate would be warmly welcomed. I took a position at CSC in Hartford a while back, mostly on the assertion (from the hiring person, no less) that "we're going to be doing heavyweight database development". Turns out, all they did was map COPYBOOKS into tables (that link is about a decade after I weathered the nonsense at CSC, the Dark Ages continue). Moreover, mainframe DB2 allows existing COBOL code to be executed as a stored proc. A yukky time for all, but past.

So, then comes a simple-talk piece on where to check the data. Naturally, I didn't keep my mouth shut. But I remain amazed that it's still necessary to slay such octogenarian dragons. Them dragons do endure.

And, it turns out that there are a few coders who recognize that having centralized control is generally a good thing. Allen Holub has been my favorite java guy since the late 90s. He seems to have fallen off the map, largely. I think I know why. Here's a bit from his Bank of Allen series:

The only recourse is to change the ROMs in every ATM in the world (since there's no telling which one Bill will use), to use 64-bit doubles instead of 32-bit floats to hold account balances, and to 32-bit longs to hold five-digit PINs. That's an enormous maintenance problem, of course.

Changing the clients is a waste of time and effort. Bad design, but good for billable hours. Do it his way, and you don't need quite so many coders, don't ya know?

Time Loves a Hero

Well they say time loves a hero
But only time will tell
If he's real, he's a legend from heaven
If he ain't he was sent here from hell
-- Barrere, Payne, Gradney (from "Time Loves A Hero", Little Feat)

Yesterday was a bit different, schedule wise, in that this is deemed to be a Windy Weekend here in West Redneck, CT. Thus my morning was devoted to moving mucho leaves from their temporary pile on the lawn to permanent recumbence elsewhere. To gird my loins, I made a quick look at r-bloggers, and found that Rob Hyndman had, unusually, posted a job advert. It is from Amazon, and provided me with great mirth as I set about to haul leaves.

His post is mostly a quote from the (unnamed) email he received from Amazon:

... we have found that applied economists compare quite favorably with the machine learning specialists and statisticians that are sometimes recruited for such roles.

The job posting itself is on the AEA site. And provides added humor.

Amazon economists will apply the frontier of economic thinking to market design, pricing, forecasting, program evaluation, online advertising and other areas. You will build econometric models, using our world class data systems, and apply economic theory to solve business problems in a fast moving environment.

Sounds a whole like the description of what those benighted knuckleheads from Wall Street were doing, doesn't it??

All well and good, you say; but where's the laugh out loud part? Well, as I set out to haul leaves I noted that Amazon had provided a job (or, at least, career track) email. Oh goody!! I penned a snarky, but lighthearted, rebuke. But why, you may wonder?

Consider that, among quants, the economists are the most incompetent. While a grad student, most of my profs were the initial wave of New Quant Economists who represented what has been wrong with econ quant for some decades since: they were all flunked out math and science Ph.D. drudges, who found that they could find repentance in the econ department. Thus, we get econ quants who still think in terms of God's Inviolate Laws running the show, rather than the reality that it's Political Economics run by social Darwinists making up the rules as they go along. The rules change without regard to data. The data changes in response to rule changes, which are blithely ignored by said quants. They all believed that a well known ratio (median house price / median income), could go straight up forever. Yeah, sure. They apply the wrong context to a field of which they've not even had 101 training. They consider it just another series of algebra exercises. Political economics ain't algebra.

So I hauled leaves, which meant that I'd missed my mid-morning stroll to get my dead trees NYT. And, if it's Friday, it must be Krugman. I sorely would miss my dose of Krugman, but I really didn't want to re-gather all them leaves. Remember, this is after I'd sent my snarky email (no, they haven't replied). Having done with hauling, I didn't expect there'd be any left, but I got the last copy. Oh joy. He, once again, rubs the mainstream and right wing cabal's collective noses in the Japanese experience.

For now, here's what you should know: Japan used to be a cautionary tale, but the rest of us have messed up so badly that it almost looks like a role model instead.

What, you must be wondering, does Amazon's love affair with econ quants got to do with Krugman and Japan? Glad you asked. It dawned on me that Amazon and econ quants were a perfect match, the totally dysfunctional marriage. Amazon's only hope to be consistently profitable is to gain monopoly status in retailing, since:

The net result of nearly two decades in business is that Amazon's trailing 12-month price-to-earnings ratio stands at an alarmingly high 550. Compare that to consistent profit earners with significant online retail operations such as Google (p/e 29), Wal-Mart Stores (2) or eBay (25), and it's easy to be confused by investors' hunger for Amazon.

Amazon, Bezos in particular, hasn't yet figured out that moving widgets by the tonne on trains is way cheaper, per widget, than by each in aeroplanes. He seems to be getting a clue, what with all these so called distribution centers he's building. They're really just brick and mortar stores, by a different name. In order to compete on price, Amazon has to eat the inflated distribution cost by setting the widget price low enough that, when delivery is added in, the total isn't way out of whack. Some folks may have been stupid enough not to do the arithmetic, but not now. The total price of that widget from Amazon has to compete with the price, ignoring the total cost of driving, at Target or WalMart or Sears.

The only way Amazon gets justifiable profit is when it gets monopoly power to pass on the inflated delivery costs. There is no other option, save bankruptcy. The numbers can only add up one way. Well, unless you're an accountant who can bend or change the rules. Clueless CEO and clueless quants seeking to be "the king of the world!" Jack, and his boat, sank.

You Can Never Be Too Thin...

or too rich.

For those still going gaga over the thinness of the iPhone 6, here's what's really going on.

5 volume (cube mm): 55,136
6 volume (cube mm): 63,843

So, Apple has much more space to work with, and not much more, other than the larger screen, to support. Piece of cake, if you ask me.

You Don't Have to Believe Me

Recent musings, motivated by continued discontinuity between real world finance time series and micros/quants insistence on predicting past the data, have been devoted to this problem. The world is not linear (Dr. McElhone), and is mostly driven by incentive modifications rather than historical data. Those that believe otherwise still think The Great Recession was caused by the Trilateral Commission attempting to impose a New World Government. Baloney, of course.

On the other hand, some academics understand that time series prediction is flaky in actual practice.

There are at least four sources of uncertainty in forecasting using time series models:
...
4. The continuation of the historical data generating process into the future.

QED

Lord of The Flies

Being stranded on an island in the Atlantic, though not all that far out, without my beloved desktop I'm forced to keep this short. I hate laptops, and this one (not mine, fur shur) is horrid. These endeavors have talked about the arc of the future, from the many perspectives of the macros, micros, and quants. Data doesn't help a bunch, if the incentive structure which generated the time series you're relying on to predict the future really isn't there any more. Much like Oakland in the view of Stein.

In today's NYT Business Upshot piece, Shiller takes on the miscreants. He's explicitly dealing with the Behavioural Economics aspects of Mr. Market, although he doesn't use the term. Rather, it's a faint feint to Ebola.

Fundamentally, stock markets are driven by popular narratives, which don't need basis in solid fact. True or not, such stories may be described as "thought viruses". When they are pernicious, they are analogous to the Ebola virus: They spread by contagion.

In the end, what's behind the recent collapse of Mr. Market's arches? Time for Dr. Scholls.

Some people say that a theory of John Maynard Keynes -- known as the "underconsumption theory" because it says people inherently underspend once they become prosperous - is taking hold.

Mark Twain wrote, arguably, the best dissection of that idea with "The Gilded Age", a decade before Keynes was born (and before "Huckleberry Finn", for what it's worth). Even if you're Daddy Warbucks, you can use only so many fast cars, loose women, and fancy food. Economists have known this, if not by explicit name, since long before Keynes.

The Wiki has a, not fawning, piece: here.

One Life to Live

It was raining cats and dogs this morning here in South Fireplug, CT (although not nearly what my beloved, yet benighted, Bermuda faces tomorrow), so I postponed my daily stroll to get the dead trees NYT and a cuppa joe until it let up. In the meantime, I spent some time with r-bloggers and came across a young pup who talked about predicting life expectancy from historical data. I considered a tongue in cheek rebuttal, but tasks interfered.

The rain let up, and there were still NYTs on the shelf. What do I find? David Leonhardt's, et al take on the issue of quants not looking out the window to see if it's raining. These endeavors have addressed this issue a few times. Time series data of some attribute may be useful in predicting the attribute's future value, IFF the entities which have the attribute are running under God's Laws over which they have no control. On the other hand, when measures are of some aspect of human activity, it's mostly a Wizard of Oz situation: some human(s) are behind the curtain bending, breaking, or re-writing the "laws" of behavior which determine the attribute. It wasn't thermodynamics which propelled house prices, but various Wizards playing various games with the rules of the game. That financial quants were all too happy to ignore the absurdity of their data, well...

Curiously, one might argue that the examples (possibly, save one or two) he gives of odds are of attributes and entities which obey God's Laws. Such odds are accurate, by definition of our understanding of God's Laws. No mention of house prices continuing to rise like smoke from a volcano.

So far as life expectancy goes, if you go and look (I've provided links more than once), you'll see that from 1900 to 2000 life expectancy at birth increased by nearly 70%, but life expectancy at 65 increased about 7% of total lifetime (less if you measure from the start of Social Security). The increase in life expectancy at birth is the number that right wingnuts use to bray, "we have to kill SS because 'we' can't afford it. You all have to buy stocks and bonds from our friends in the financial services industry." Not that they see any conflict of interest. The overhead of SS is about 1%, and protects citizens from oscillations of Mr. Market. Can't say the same thing about that 401(k); all those Wall Street fat cats got fat off "your" retirement nest egg.

So, why is predicting life expectancy from today forward based on the accumulated data over the last X years silly? Because, just as the Wizard of Oz, those added years are the result of human intervention, sporadic and specific. Increase in life expectancy, whether at birth or 65 or some other age, is not a God given gift to humanity for just being on the planet for evermore years. Doesn't work that way. Most of the increase at birth is due to greatly diminished infant and child mortality, this due mostly to public health initiatives; vaccines and anti-biotics being discovered and made widely available. That's not going to happen again. One might even speculate that, with increasing wealth concentration, what had been widely available will become more restricted. With the loonies chirping about autism and vaccines and denying same to their spawn, we are finding increased (although not epidemic levels, yet) incidence of old diseases. And so on. Humanity, despite what some quants believe, isn't Brownian motion or a random walk. No, progress exists because humans change the rules, from time to time. And sometimes those changes benefit most of us rather the the 1% and we all live a bit longer. It isn't God that has provided us with longer lives than we had in 1900, it's us. As we've reached near the limit of our ability to stave off Father Doom, and, perhaps, our willingness to make this ability available to all, we will see a leveling (if not diminishment) of life expectancy at all ages.

I Have No Interest in You, Anymore

Regular reader should recall that I've been pestering the macros, micros, and quants (especially those who're in love with time series) about The Giant Pool of Money, corporate moolah hoards, the limits of knowledge (once you know the Laws of Nature, there's little left to discover), and such. The upshot of these concerns is that real returns to physical capital, which is the only, and controlling, manifestation of compound interest must needs be declining. If there's no place to put the moolah that is productive, well...

Turns out, I'm not entirely alone. A bond pundit has weighed in, and reach just that conclusion. The piece doesn't detail the reasoning, but, really now, what else could it be? Too much moolah chasing too few real opportunities. House and car notes don't actually generate real return, only foregone consumption. That's not organic growth.

If you read German (I got tired of waiting for Google Translate to finish, sigh), this is the source interview. Not much longer than the English reference articles.

Diamonds Are a Tim's Best Friend

I wonder: will anyone be willing to supply Apple now? They clearly, based on reporting, shafted GT. Nor, for that matter based on reporting, was GT Apple's first choice. Those earlier explorations led to companies saying NO to the terms Apple demanded. Much, if not most, of Apple supply dealings are in the Far East, away from US media reporting. I suspect there are many more corporate corpses over there. Make a deal with the devil...

Apple holds at least one patent on S/G/S and S/G lamination. Given BendGate, the likelihood that a monolithic sapphire screen would survive is slim.

Apple looks to have won this battle. The war is another matter.

I Want My Gumby Phone

A few times I've mused that the phablet sized phones' days were numbered, and that flip phones (with flexible displays and/or hinges) would come back. Well, I'm no longer the only one.

Over the next couple of years, a lot will change. OLEDs, which are being sold as televisions by LG, can be made to conform to shapes. You can see the beginnings of the implications of that in Samsung's Galaxy Note Edge phablet. Thanks to new technology from companies like Kateeva, it's going to get even easier to make those flexible displays very soon. The implications are critical on two fronts (1) displays that bend, flex, fold and expand will become possible (2) screens that are nearly unbreakable will become commonplace.

I hereby claim the trademark and copyright on The Gumby Phone. Should have done that in the first place.

The Lone Ranger, Superman, and Jim

"You don't mess around with Jim."
-- Jim Croce/1972

Or Tim, who learned at Steve's knee. GTAT has filed Chap. 11. They may have been headed there anyway, given their exposure to solar, and the vicissitudes of same. But still. Betting that Apple would be a good fairy godmother? Apple did much the same with capacitive touch screens with the first iPhone, although I don't recall any Chap. 11s from that escapade. Apple has, once again, frozen competitors out of useful technology. GTAT isn't the only supplier of sapphire, but nearly the only supplier of machine tools for making it. Who said Tim was a milquetoast?