27 March 2014

Barkeep, There's Water in the Whiskey!!

Every now and again, and today was one of those nows, some sports pundit will trot out the tried and true, "X is horrible today compared to 19XX because of all that player dilution". And blah, blah, blah. Baloney.

Take baseball, please.

In 1950 there were 16 teams with 25 player rosters. My calculator tells me that there were 400 major leaguers that year.

So.

In 2010 there were 30 teams (still are, of course). That's 750 major leaguers.

Since 1950 we have: bigger, stronger, faster athletes; orthopedic surgeries; safer equipment; and so on.

In 1950 the ratio of ball players to population was: 400/151,000,000 or .0000026
In 2010 the ratio of ball players to population was: 750/308,000,000 or .0000024

Or, just by simple inspection, the number of ball players per person *is less now* than in 1950. So shove it. And, that assumes that all 750 are native born Americans. Baloney to that, too.

I wonder how many certified Sabermetricians have ever bothered to figure that out?

Baseball may be lousy today compared to what some imagine it was like in 19XX, but player dilution ain't why.

26 March 2014

Core of the Apple

In a piece about Facebook's Oculus buy, one Richard Beales states, for the first time I can recall seeing from a mainstream pundit, Apple's situation. I'll take credit for pointing this out some time ago:
... what Apple's iPhone did for smartphones: combine available technologies into a package that's both user-friendly and a revelation for those that try it.

Recall, Apple had cornered the supply of capacitive screens?
Apple has access to new component technology months or years before its rivals. This allows it to release groundbreaking products that are actually impossible to duplicate. Remember how for up to a year or so after the introduction of the iPhone, none of the would-be iPhone clones could even get a capacitive touchscreen to work as well as the iPhone's? It wasn't just the software - Apple simply has access to new components earlier, before anyone else in the world can gain access to it in mass quantities to make a consumer device. One extraordinary example of this is the aluminum machining technology used to make Apple's laptops - this remains a trade secret that Apple continues to have exclusive access to and allows them to make laptops with (for now) unsurpassed strength and lightness.

Apple, in order to start a growth spurt while iPhone goes into global replacement mode (and while carriers, at one and the same time, move to no contract, no phone, plans) has to A) figure out which of the existing platforms it can bling up and B) corner the bling-ish hardware for said device. One has to wonder whether it was really Steve who divined the various products?

25 March 2014

Not a Joe Sixpack

Joe Conway sent out an email to the PL/R group announcing a PL/R chapter. It's not very long, and it's rather expensive. The book is pre-order at Amazon for the better part of $180. Since ecology isn't of interest, I'll skip it. Fact is, Joe's documentation has always seemed sufficient. One must needs be familiar with Postgres, of course.

OTOH, it is a Good Thing to find PL/R appearing in texts. Today wildlife, tomorrow Wall Street!!

9 to 5

It is reported today that MicroSoft is ending support for Office 2003, commonly used with XP. As you read the comments, those that paid for O2003 have little use for subscribing to O365, since it doesn't do anything needed and costs moolah. Welcome to the Good Enough World (actually, happened in word processing at least 2 decades ago).

In the past, MicroSoft has changed file formats to spike others reading/writing the document files. That should be the next step. MicroSoft (not the only ones, alas) have taken the Gillette model (give away the razor and sell the blades) that further step by not giving away the razor. Makes one almost yearn for indentured servitude.

22 March 2014

Dumb Jocks

I had been mulling over writing this all up, under the title "What's a Quintillion Among Friends", but it's only a bunch of college basketball games and not worth the effort to get the number exactly right. Nah, ain't worth it. BUT...

Two things happened.
1) I saw Nate Silver bloviating, on Olbermann IIRC, and he didn't scotch the 9.2 quintillion number
and
2) I caught a bit of the intermission shows on one of the networks, IIRC it was TNT, and one of the jocks, again IIRC, Kenny Smith made the salient point

That salient point is that the 9.2 quintillion numbers results from, as Kenny (I think, as they were all screaming at each other at once) said, coin flips across all 64 teams. And that's baloney. That number is 2^63: all permutations all games. So not happening. But all the reporting repeats that number. If Silver gainsayed it, I missed it in his interview.

By sometime last night, all the ESPN and Buffett brackets had been torpedoed. Lots of double digit seeds had won their games. Not going to buy my Caribbean Island. And so the discussion ensued.

Not all permutations can occur. There are 4 sets of 16, which if you treat each as permutations of coin flips (2^15) is 32,768 outcomes. Now, if you treat the sets as independent, that would be to the fourth power, and thus about 1.1 quintillion. Again, too many possibilities, since what one wants is the distinct number of binary trees.

Stackoverflow has discussion of binary trees, this one, at least not couched in March Madness terms. What a relief. But this could easily be wrong, too, since it is a calculation of root to branch tree building, and would thus have to account for each region having 16 different starting nodes (in this upended view).

If you do Catalan numbers, for 16 seed region, pivot on 16, then multiply be 4 regions, you get 615,506,319,360. Not sure that's quite right, since it doesn't account for the Final Four? Anyway.

Too early in the morning, and only because the dogs insist they have to poo, to work out such a meaningless number.

21 March 2014

Boots and Pants

There's a new Maxwell GEICO commercial out, of which I've only seen the last few seconds, where Maxwell is sitting by the pool (beach?) in a deck chair, phone in lap. He's jiggling his trotters whilst looking dreamily at the phone and chanting "boots and pants and boots and pants". I haven't the foggiest what that means. Probably some Angry Birds replacement.

But all is not a waste of time. The bit of schtick reminded me of Apple's hiring of that Burberry marketing exec a few months back. Remember that? At the time, I wrote that persuading the 1% to have lots of Burberry coats, each appropriate to its own occasion, was a bunch easier than convincing them to have lots of iPhones, one for each occasion. Then the iWatch rumoring went into high gear.

May be not so pointless, after all. "Coats and watches and coats and watches..." A watch is mostly fashion accessory, leastly utilitarian device. One might say the something of the same for today's smartphone. What percent of its use time is devoted to making phone calls? 10%? 1%? Still and all, there's not much traction to be had convincing folks to have more than one. Hell, with the rise of the phablet, Apple's going to face (if not already) the prospect of folks having one phablet over having an iPhone and iPad.

Watches, on the other hand, can be owned multiply and still make some sense. A sport watch, a dress watch, an old clunker in the desk for when you forget your office watch. And so on. The 1% are always open to the Stuyvesant scam. Hiring a coat hawker begins to make sense. Now, that means, of course, that Apple goes the way of multiple versions of iWatch, at the same time, a tack not taken with previous devices. We'll see.

18 March 2014

Normal Is So Overrated

R-bloggers had this post linked (the title block is worth the visit). Hmmm... F# talking to R? And F# talks to SQL Server?

So, that got me to the F# post, which in turn gets one to the R Type Provider.

As mentioned a few times, it would be a Good Thing if SQL Server offered the equivalent of PL/R. I'm 100% Ubuntu these days, so it wouldn't help me directly, but a, somewhat inelegant, way to get there would be connect to both R and SQL Server in a F# session/program? I didn't find a C# equivalent (the R Provider appears to be F# specific).

Following some links landed me here (the original author), and the question was asked:
This is great. Can you use the RProvider from C#?

[ Don says - no, C# doesn't support type providers ]

Too bad.

17 March 2014

Duh??

A story out today, "ARMs are back! Reverse mortgages too! Is this housing bubble 2.0?" contains this prized morsel of wisdom:
"A typical first-time homebuyer may be unable to afford a typical home in the near term, if mortgage rates and home prices continue to rise without sufficient increases in income," writes Orawin Velz, director of economic and strategic research at Fannie Mae.

Well, Duh??? Where was s/he in 2003?? Has Mr. Housing-Market been popping those blue trapezoids again?

Santayana and Pete Seeger had something to say about all that.

16 March 2014

Confirmation and Contradiction for 2014-03-16

A wealth of confirmation over the last few weeks, but I'll resist temptation, and keep this missive to one, Jeremy Rifkin's riff on the death of capitalism.

What he misses, or at least doesn't say, is what's really driving the death spiral. And that factor is what I've mentioned a few times along the way: as capital moves to finance, rather than production, it faces ever diminishing returns. This was true in the run-up to The Great Recession, just because the tsunami of moolah had to crash somewhere, and the American buck was the safest place to land. And, within that venue, the surest and safest way to get more return than what Treasuries were offering (recall that Greenspan had already cratered rates) was the slam dunk residential mortgage market. Until it wasn't.
While economists have always welcomed a reduction in marginal cost, they never anticipated the possibility of a technological revolution that might bring those costs to near zero.

Or as Paul Graham once put it (2005; that long ago):
Like everything else in technology, the cost of starting a startup has decreased dramatically. Now it's so low that it has disappeared into the noise. The main cost of starting a Web-based startup is food and rent. Which means it doesn't cost much more to start a company than to be a total slacker.

So, we get, as I recently termed it, high tech cottage industry. The problem is that, while one can outfit a quad-core i7 with 32Gig and an SSD for about what the average 20-something slacker spends annually on a smartphone and thus be able to code most anything, the fact is that what gets made by these guys is truly trivial. (I slipped that bit of confirmation in the backdoor. Hehe.) One guy can make most of an angry bird by his own self, then hire a bunch just like hisself later on. Building a modern ERP, to replace SAP (for instance), takes a big bunch of people working industrial. All these kiddie apps? Not so much.

What Rifkin doesn't get at: the problem as been around since, at least, the music CD. They're really much cheaper, in marginal cost terms, than the LP. Nor does he address the unintended consequence of vanishing returns to real capital. Capital will continue to move to financial/fiduciary instruments, and thus to more Too Big To Fail corporations. Which brings with it the concentration of wealth and income we've seen recently. That's not going away any time soon. So, through the bathroom window, here's where the Kaufman quote came from. Without a fat middle class, we end up with a stagnant economy, much as the 19th century was. Huh? Well, in 19th century America, there was an entire continent full of resources never seen before. The development of the USofA derived not from being smarter than those gay Europeans, but by having natural resources sufficient to enable unprecedented waste and profligacy. We don't have a continent like that, anymore. There was little in the way of a middle class, and most worked to eat and produce goods for the 1%. What Rifkin gets to is that we won't have even the need for vast numbers of subsistence laborers to make stuff for the top X%. But the 47% won't be allowed to lie around in hammocks, leaching off those productive banksters.

Capital will still control, and as it continues to concentrate, its control becomes more absolute. And the finance quants will continue to ask, "what are my orders?"

15 March 2014

Big Dummies

Yet another cautionary tail from the Annals of Big Data.

Big Data, at best, becomes an exercise in descriptive stats. At worst, it's a colossal waste of time and money.

Some points:
- irreproducible research isn't of much use - that it was done "internally" by/for Google makes no difference
- Big isn't always better - in the case of Flu Trends, the Google folks (and if there were math stats involved, they should be ashamed) didn't have a clue about measurement or sampling, much less inference

What the Googlers didn't, or wouldn't, comprehend is that while the data was Big, it wasn't population data. And thus, it was sampled data (with no controls, apparently). And sampled data is subject to the constraints of frequentist (or Bayesian, if that's your bowl of porridge) inferential stats. None of that was done, of course.
Put another way, it's not uncommon to hear the argument that "computer algorithms have reached the point where we can now do X." Which is fine in and of itself, except, as the authors put it, it's often accompanied by an implicit assumption: "therefore, we no longer have to do Y." And Y, in these cases, was the scientific grunt work involved with showing a given correlation is relevant, general, driven by a mechanism we can define, and so forth.

Kiddies are so damn lazy, these days.

14 March 2014

Why Girl!! You're Stunning(ly Obvious)!!

Some of the things that come up are so stunningly obvious that I've never considered them worth typing about. But then, I'm an RM zealot. And a firm believer in the closer the better. When I was studying karate (from a Greek guy who included ancient Greek philosophy in the black belt's curriculum; OK, I guess you had to be there...), Tae Kwon Do was rather popular. I also studied Tai Chi with the Miller brothers. Don was teaching us Tai Chi push hands (that's not his site, but you should read it nevertheless) and someone made mention of Tae Kwon Do and all that kicking. Don's reply was, in essence, get inside his guns (which is to say, kicking distance) and beat the crap out of him.

Too often, coders act like Tae Kwon Do knuckleheads, flailing away with their feet at great (in martial terms) distance. Don't do that. Get close, and beat his brains in.

So, today brings (courtesy Artima) this post which, once again, takes such folks to task. It's SQL Server centric, but any reasonable RDBMS will serve. Don't forget CTE.

The sad fact remains that, modulo a bit of syntax, today's kiddie koders design applications just like grand pappies did back in the 70's: COBOL on the client to rule VSAM files on the server. Such foolishness.

06 March 2014

Robocar

Recent musings reminded me of an (apocryphal) quote from the late 1970's: "I'd rather have my chip in every Ford than in every PC." The point being, of course, at that time engine management was moving to embedded processor control from simple mechanical/pneumatic control, and there were more of those to shift than PCs (which, technically, didn't yet exist). Which got me to wondering what the state of the auto is today.

According to this (and a decade ago), the answer is
The current 7-Series BMW and S-class Mercedes boast about 100 processors apiece. A relatively low-profile Volvo still has 50 to 60 baby processors on board. Even a boring low-cost econobox has a few dozen different microprocessors in it. Your transportation appliance probably has more chips than your Internet appliance.

(This NYT article from 2010 yields the same number.)

I suppose the answer to Apple's situation: the iCar.

Hey Guys, Watch This

Apple is in trouble. Before the iPhone, it was a small time computer assembler, with little impact outside a few niche sectors. The iPhone changed all that, not least of which was morphing Apple from a computer company to a toy company. Not that anyone, Jobs possibly excepted (I don't have a cite one way or the other), understood this. Apple can continue to broaden the TAM by attacking second and third world markets, but they are second and third world incomes, and those currencies are notoriously manipulated. The right wingnuts bitch about debasing the buck, but the rest of the world still uses it as the reserve moolah of record. Apple doesn't get that protection anywhere else. Here, and Europe, Apple's getting very close to replacement mode. With no-contract plans becoming more common, even the iPhone folks will go over to the dark side and just keep the damn thing for years. Or until the battery dies. I expect to see reports that Apple batteries are, mysteriously, failing earlier and earlier. Innovation, or planned obsolescence, is the saviour of Apple?

The core reason Apple got rich off the iPhone was that it had a stranglehold on necessary components of touchscreens leading to, and for a period after, the iPhone's release. The iPhone offered features not available elsewhere. It turned out that the marketing machine of Apple convinced those with significant disposable incomes to covet those features. That was seven years ago, give or take. Time for the seven year itch.

Apple has always been a follower, not a first mover. Each of its successful products have been a bling-ified take on existing platforms. The few times Apple has gone first, it's fallen on its face. Can Apple continue to grow? Apple TV can't generate the revenue of iStuff, given that TV is held to the house, while iStuff are held to the person. The legendary iWatch is seen by most pundits as Apple's next iPhone, money spinning game and market changer.

Not likely.

There are a host of fancy watches out there. This isn't a case where Apple has (he says with trembling confidence) cornered the market on vital technology. High class/cost exercise/health watches, with digital displays, have been around for decades. The form factor isn't negotiable. Power isn't negotiable. Display, also likely, not negotiable. Most importantly, the iPhone didn't cannibalize other Apple products. It's been something of a juggling act, keeping the TAM of iPad and iPhone at arm's length. iPod clearly has taken it in the neck from iPhone. I doubt that Tim wants that to happen again. Thus, no iPhablet. So far. An iWatch which does something neat would likely chew into iPhone, even iPad. Tethered, aka Gear, isn't taking off, and standalone isn't either.

Smartwatches, per se, already exist, beyond the Samsung Gear. The Wiki includes a rather long list of existing smartwatches. Not anything like the landscape that Apple exploited with the iPhone.

As the NYT article points out, keeping enough juice in these devices is the main issue. For a watch to do what an iPhone 5S (or its cpu) does requires a 5S battery. That ain't gonna happen. The article goes on:
The biggest drawback to the Pebble Steel -- indeed, all smartwatches -- is charging. The Pebble requires charging every few days, and annoyingly uses a proprietary charging cable rather than standard Micro USB.

Well, yeah. Induction chargers have been around for years; my toothbrush runs on one, and a charger stand makes a lot of sense. Would simpler charging be enough to induce another reality distortion field around an iWatch? Not likely.

But it still comes down to: what tech monopoly can Apple depend on? Sapphire isn't much to hang your hat on. A super duper dense battery would permit, at least, long up times, but VLSI still can't (and likely, never will) put a 5S in a wrist sized package. And even if it could, Apple would be robbing Peter to pay Paul.

Apple needs to find a platform which exists, has a small but enthusiastic market, current primitive implementations, no Apple presence, and a tech moat that Apple can hide behind. I know of none. How about you?

Winner by TKO...

"In today's fights over financial reform, the advantage goes to those who hold the low ground, the underground, the dark room where a rule can be modified in ways only a small handful of experts can follow. These are the battles the banks can win."

That's from Jesse Eisinger's DealB%k column today. You should read up on it.

The point, made here on more than one occasion, is that events trump data whenever the two conflict. Unlike data from the physical sciences, data from human processes is subject to the whims of Goliath. And, unlike in that Bible story taught in Sunday School, Goliath nearly always wins.

03 March 2014

Mind the Cutlery

When I was a kid, the parents had many sayings related to child rearing. Some were intended to restrain the kids just for the sake of the parents' nerves. "What would the world be like if everyone behaved like you?" was one of those. A few were actually intended to keep the kids from doing further harm to themselves; very anti-Darwinist, if one stops to ponder. Let the dumb ones kill themselves before they can reproduce. Aye, matey!! "Don't let the kids near the sharp cutlery" was one such.

In the larger venue of quants/stats/OR/foo, there is a similar need. Not as often enforced. The financial quants were given 1-2-3 (sired by VisiCalc, and sire of Excel), and the world hasn't been as safe for civilians ever since. One no longer needed to really know the maths, anymore; anyone could run 1-2-3 and anyone could write a macro. Many admin assistants did both. Fact is, though, is that it is difficult to decide which causes more harm: the use of Excel by the London Whale (and other equally inept "business users") or the blind transference of maths into finance by the likes of Li's copula (or any of the multitude of failed math/science Ph.D.s who found fortune in Wall Street)? Which is the bigger threat? Hard to say.

Now we have R threatening to displace not only SPSS and SAS and Stata and such, but also Excel. Couple this with the increasing number of books/websites/tutorials aimed at arming the ignorant ("You don't need any maths to do this. Don't worry, I'll tell you how to do it.") finance folk (the spawn of those 1980's admin assistants) with "R skills". It's beginning to look a lot like we're creating a children's army armed with Samurai knives. Would you fly in a 787 if it were calculated by admin assistants? Very much of the global financial structure is.

A couple of postings have recently appeared via r-bloggers. One guardedly cautionary, the other not so much.

Martin gets to the point:
There are too many decent but not great R books on the market already and there is no reason for me to spend time to create another one.

Now, he's referring to ggplot2, not some esoteric branch of stats, which may or may not be relevant to some particular area of endeavor. Still, he's quite right.

On the other hand, arsalvacion reviews a Packt text (yes, same publisher that Martin turned down) which says it can teach you finance quants in R in a mere 164 pages. That's got to be handing out straight razors to three year olds, for crying out loud. Fortunately, some of the reviewers over at Amazon are a tad more circumspect.

Finance quants are hobbled (or spurred on, depending upon one's point of view) by the difference between quantitative analysis in human activities and the physical sciences. In the latter venue, God makes the rules and they don't change, although it may take a very long time to figure them out. Humans can't make new ones, or change the ones we've figured out. In the former, humans make the rules, and change them to suit the whims of those with the most control. The causes of The Great Recession were humans either fiddling with the rules, or attempting to game them. Both, every now and again.

You, too, can be a finance quant; just like learning taxidermy from those matchbook cover correspondence courses.