We'll Need a Bigger Boat

Why did/do flunked out math and physics PhD candidates go into finance/business admin/economics? The quants. Follow the money. They started in earnest in the 1970s, and became the Mongol hoards (or a school of squid?) which cratered the global economy a couple of years ago (we are past that, right?). Now that the pillaging is done with, Wall Street has turned off the money spigot, yes? No. See here. Not all that newsy, in that the previous year wasn't much different.

Between 2009 and 2011, compensation in the securities industry grew at an average annual rate of 8.7 percent, outpacing 5.3 percent for the rest of the private sector.

Note that the main data refers to 2011; we don't yet know what the numbers will be for 2012, but...

Some 48 percent of 911 Wall Street employees surveyed by eFinancialCareers.com said they felt their bonuses this year would higher than in 2011.

OK, so they've done so much better by the world's economy, they need yet more moolah. But here's the real problem:

Financial jobs accounted for nearly a quarter of all private sector wages paid in [New York City] last year, even though they accounted for just a fraction, 5.3 percent, of the city's private sector jobs.

One of the real conundra of the financial services sector is the assumption that it's driven by computers and quants and superior smarts. Put another way, in the industrial sector, wages have been falling in deference to capital. Earlier posts have discussed the fall in labor as input to production. In the finance world, not so much. With all the talk of double secret probationed HFT computers, it's the Boys in the Boiler Room who get most of the moolah. Moolah that, could, be used to buy plant and machinery for physical production. All that money just to partner savers with borrowers? My, my.

Nearly half of all revenue on Wall Street is earmarked for compensation; in 2009, Morgan Stanley, which was hit harder during the crisis than most of its rivals, found itself paying out a record 62 percent of its net revenue in compensation and benefits. That number has since come down.

I'm Melting!

Well, it really did happen, OCZ has crashed and burned. Time to muse on the effect, if any, on the thesis of this endeavor, that SSD on multi-processor/core/thread machines will take over the world. And that high NF databases will be the data store of choice.

Hmm. OCZ was never, despite its assertions, a player in the Enterprise Space, so there shouldn't be any direct effect. STEC slumped this morning pre-market when OCZ did its face plant, but has recovered. STEC does have its own CEO problem, of course, but does have a track record of producing Enterprise Space SSD that OEMs and end buyers really want. Fusion-io keeps on truckin'. The niche/private Enterprise Space continues on. The Enterprise Flash Appliance Space looks to be going well, at least according to Forbes reportage.

The loss, if it comes to that, of OCZ will most likely be felt in the Postgres/MySql world of SMB/web startups. These entities are largely populated by pimply faced boys who've never heard of Dr. Codd, or those that have, dismiss him.

In the months leading up to Peterson's "leaving" (and it's now reported that he's on the dole of OCZ for rather a while), OCZ touted its imminent release of "the first TLC" SSD. Didn't happen. Samsung's 840 (not the 840 Pro) is about to deliver, and has been tested, a bit, by AnandTech. The consumer SSD market is, it has always seemed to me, coelescing with the Big Boys, those that can survive and thrive in a race to the bottom. But then, I did predict that a while back. The boutique houses, that only assemble Other People's Parts, will either have to provide unique capabilities (STEC has it's own controller, so it doesn't count, either) or die. Darwin was right.

[Yes, if you read this earlier, the title is changed. I'd allocated that to a piece not yet completed. Too busy.]

Cheap at Twice The Price

High normal form relational databases provide the minimum byte load, and guaranteed integrity. Why have these qualities been so clearly ignored over the years? One excuse has been that normal form is too slow on conventional hard drives and uni-processors. Now that we have high quality SSD and multi-processor/core/thread machines, the technical complaints should disappear.

But my experience says that there is still an uphill slog in view. Why? Coders will remain obstructionist, since they're in self-preservation mode. Another piece today provides yet more examination of how this happens. Were humans rational, this sort of behavior wouldn't be tolerated.

It's an unfortunate reality that efficiency often goes unrewarded in the workplace. I had that feeling a lot when I was a partner in a Washington law firm. Because of my expertise, I could often answer a client's questions quickly, saving both of us time. But because my firm billed by the hour, as most law firms do, my efficiency worked against me.

Sound familiar? An unexceptional relationalist can build rings around a hoard of coders, but doing so threatens not only the hoard, but the (bureaucratic) bosses of the hoard.

...a measurement system based on hours makes no sense for knowledge workers. Their contribution should be measured by the value they create through applying their ideas and skills.

But, still, the coders circle the wagons, and insist that the olde ways are better. The bureaucrats who, ostensibly, manage them are happier with the bloat: the bigger the budget the greater the importance. Progress is wonderful.

The solution, it seems to me, has always been clear: engage in fixed price contracts, whether internal or external.

When Good News Happens to Bad People

Back in February, there was "Lies, Damn Lies, and the BLS", which you can find in any of the various incarnations of this endeavor. At the time, the numbers came out better than the pundits predicted. It happened again this week (for September), but since it is the second but last run of the numbers before the election (Election Day is Tuesday the 6th, while the numbers day is Friday the 2nd) with enough time to get all piqued up.

And did the Right Wingnuts get all piqued!!! To repeat: the data come from sample surveys, two not a single integrated survey. And if you read the fine print, as I suggested back in February, then you'll see that the estimates have plenty of room to waffle. I don't recall any articles in February that went into the details. Well, this time there is. Worth reading up. The on-line version is graphier, and therefore more useful.

As to the GOP stroking out: the BLS folks who make the decisions are Supergrades (here for explanation), who, in all likelihood, got to these positions of authority under BushII. If there's any conspiracy, it's among geeks who live in Fairfax County with a stay at home wifey and three kids in Christian School. I spent a decade working with those sorts (not at BLS, for the record).

Let's Have a Party!!

It qualifies as something of an open secret that lots of financial service (even actuarial) analysis is done in Excel. The revelation appears as snide comments in texts with some regularity.

ggplot2 has become (to my mind, anyway) the de facto graphing package in R. Recently, Hadley released an updated version, with themes getting the most attention. Today, David Smith (of Revolution Computing, and an Aussie, if memory serves) posted some comments on themes (referencing theme work done by Jeffrey Arnold), including this beaut:

or if you're an R user on the down-low, with that "classic ugly look and feel" of Microsoft Excel. (A better idea would be to just come out!)

More often than not, Americans use the phrase (in TV dramas quite noticeably) way, way out of context. One might wonder what they call Excel in Gay Paree?

Many Happy Returns

Today brings not one, but two, cautionary tales. The reason for doing quants, either for anonymous blogging or mucho dinero, is to deal with issues which can only be answered with data, not policy. I'm on record that policy trumps data every time, but with the proviso that the policy can be enforced *despite* the eventual collapse. Greenspan's crashing of interest rates was a policy which motivated The Great Recession; he almost fully admitted it after the fact. There was clear data that the collapse was in the making, but ignored by both policy makers (who wants to admit error?) and participants (who wants to be first to miss the boat?). Capital's need for real return, and the consequent need for monopoly, appears in disparate stories today.

To recap. The justification for real physical investment is to make, and sell, either more product at constant price or current/less product at lower cost. This productivity delta is the real return. The Great Recession(s) come about when fiduciary capital is placed in other fiduciary instruments, rather than physical investment. Returns on real estate, whether residential or commercial, can only come out of the income streams of the underlying entities, households or businesses. Residential housing provides no financial returns, in use; paying the vig either comes out of rising incomes (there weren't any during Bush II) or consumption shifts (the volume of moolah needed couldn't be supported by that, although some apologists asserted so). In almost all commercial cases, the same is true. One might argue that a Park Avenue address will attract more business than one in Alphabet City (do they still call it that? and is it still a slum?), in greater proportion to the rent differential; but I'll consign that to outlier status.

So, real return to real capital requires making more and better stuff. Banksters tend to ignore that. AnandTech has another Haswell piece posted today.

If all mainstream client computing moves to smartphones, and Intel doesn't take a dominant portion of the smartphone market, it will be left in the difficult position of having to support fabs that no longer run at the same capacity levels they once did. Without the volume it would become difficult to continue to support the fab business. And without the mainstream volume driving the fabs it would be difficult to continue to support the enterprise business.

There's more background in the text, but it amounts to this: Intel needs to keep its fabs running full blast to get the return on the cost of the fabs. In order to do that, it needs to produce chips which move like hotcakes. You sorta have to get it right.

On the other side of the world, we get the Chinese solar problem. The title: "Strategy of Solar Dominance Now Poses a Threat to China" in my dead trees copy, the title on-line is different.

But now China's strategy is in disarray. Though worldwide demand for solar panels and wind turbines has grown rapidly over the last five years, China's manufacturing capacity has soared even faster, creating enormous oversupply and a ferocious price war.

Trying to generate that real return? You betcha. Does it work, by default? Not hardly.

In the solar panel sector, "If one-third of them survive, that's good, and two-thirds of them die, but we don't know how that happens," said Li Junfeng, a longtime director general for energy and climate policy at the National Development and Reform Commission, the country's top economic planning agency.

We have to do something about that Ruinous Competition!!! Wind turbines? Same thing.

The Chinese government also wants to see the country's more than 20 wind turbine manufacturers, many of which are losing money, consolidate to five or six. "Wind does not need so many manufacturers," said Mr. Li, who in addition to drafting renewable energy policies is the president of the Chinese Renewable Energy Industries Association.

Capitalists continually assume that they deserve outsize returns, but every time they try it, chaos results. Will they never learn?

The modest cutbacks in production barely put a dent in China's overcapacity problem. GTM Research, a renewable energy consulting firm in Boston, estimates that Chinese companies have the ability to manufacture 50 gigawatts of solar panels this year, while the Chinese domestic market is on track to absorb only 4 to 5 gigawatts. Exports will take another 18 or 19 gigawatts.

The enormously expensive equipment in solar panel factories needs to be run around the clock, seven days a week, to cover costs.

Both Intel and the Chinese alt energy sectors are the poster children for fiduciary "investing": while fiduciary "investing" is faux, the return is largely controlled, in the short run (which is all they care about), by policy. The residential home builders made out like bandits, literally, while mortgage companies, banksters, and MacMansion buyers got the shaft. As always, one needs to follow the money. The Chinese alt energy companies (and the government) can't, or won't, find buyers for its shiny new toys. One might argue, and the government surely did, that investing in some fiduciary capital in product producing entities is better than investing only in infrastructure. Infrastructure, as MacMansions, is difficult in the same way: how does one impute (much less collect) real returns? For infrastructure, the return is explicitly societal. PhD candidates have been writing dissertations on the problem for decades. Eisenhower's "National Defense Highway System" was the earliest in my lifetime. The official name became "Dwight D. Eisenhower National System of Interstate and Defense Highways" under Bush I. What's it worth? Well, Ike wanted it because he saw the difficulty (to the Allies) caused by Germany's Autobahn; it was intended to be a network to move men and materiel during the coming wars. Just as DARPAnet was all about the military and turned into a commercial enterprise we call The Web. Who gets the return?

The Best Laid Plans

For those interested in how real world databases and operating systems get along (or not) in the post single-cpu, sorta parallel regime of today, there is a piece on lwn.net which is instructive. I had originally inserted the link, but it's not that simple. The link I used was supplied by a post on the Postgres/Performance email group. For better or worse, lwn.net is by subscription, but it appears that subscribers are free to link to content. So, go subscribe to the Performance email group (you should anyway, here), and you'll get the link in today's posts. It's got "20% performance drop on PostgreSQL 9.2 from kernel 3.5.3 to 3.6-rc5 on AMD chipsets" in/as title.

The discussion in the post revolves (that's a pun) around the inherent conflict in scheduling between an (multi-client) operating system and a (multi-client) database engine. While flagged by the PG folks, any engine which attempts its own scheduling would have gotten caught, to some degree, by the patch. Who said parallel was easy?

Here's a quote which describes the issue:

So what is different about PostgreSQL that caused it to slow down in response to this change? It seems to come down to the design of the PostgreSQL server and the fact that it does a certain amount of its own scheduling with user-space spinlocks. Carrying its own spinlock implementation does evidently yield performance benefits for the PostgreSQL project, but it also makes the system more susceptible to problems resulting from scheduler changes in the underlying system. In this case, restricting the set of CPUs on which a newly-woken process can run increases the chance that it will end up preempting another PostgreSQL process. If the new process needs a lock held by the preempted process, it will end up waiting until the preempted processes manages to run again, slowing things down. Possibly even worse is that preempting the PostgreSQL dispatcher process -- also more likely with Mike's patch -- can slow the flow of tasks to all PostgreSQL worker processes; that, too, will hurt performance.

Spinlocks have been around for at least two dogs' ages. And the description reminds me of IBM's AS/400 (now called something else) solution, and M$'s abortive (so far) effort to replicate it, winFS; in which the database engine and the operating system are merged. Those with even longer memories could be reminded of PICK.