29 July 2013

No Mas!! No Mas!! [update]

Perhaps there is something in the water; or more folks read this endeavor than I thought. With somewhat more frequency (not enough to let one get cocky, though), database folks (more or less RM devotees) are writing about the silliness of NoSql. I don't recall a NoSql Zealot (i.e., trainer/evangelist) doing a mea culpa, alas. But this post is more pointed than ever. Sound familiar?
Developers might argue that people can use NoSQL just agreeing on some common rules and information. My answer to this has always been: There is no point in having rules if they are not enforced by something or somebody. SQL has always followed a "think first" philosophy while NoSQL seems to rather fancy a "store first" approach.

Of course, the NoSql zealots, just as their COBOL brethren (whom they routinely disparage for being "old legacy") before them, argue that transactions should be handled in the application. That means some form of rudimentary TPM. And lots of silos of data that can't communicate. And so on. Who needs 40 years of experience building something so simple?

This reflects Pascal's view on xml as just data transport (and is an occasional member of the quote squad):

The fact is that in order for any data interchange to work, the parties must first agree on what data will be exchanged -- semantics -- and once they do that, there is no need to repeat the tags in each and every record/document being transmitted. Any agreed-upon delimited format will do, and the criterion here is efficiency, on which XML fares rather poorly...
-- Fabian Pascal/2005

That's getting on to a decade ago! Some people are just slow learners.

[update]
First, it's Roberto Duran.

Second, every now and again, the nice folks at Database Weekly will include a link to one of these posts in their weekly email notification. This is one of those weeks. There are two links in the NoSql heading, the other being to this piece on an interview with Date and Darwen, said interview conducted by Iggy himself. In the course of the post, is this:
The inventor of relational theory, Dr. Edgar "Ted" Codd, himself blessed eventual consistency in the very first paper on relational theory "A Relational Model of Data for Large Shared Data Banks"...

He then gives a lengthy quote. Now, one might argue and I certainly do, that in 1969 Codd was faced with a hardware situation utterly different from today. IMS, which was his target with the RM, was intimately designed to the hardware of the day and Codd sought to break that bond. While I can't cite him, I believe he would make such a statement based on the prevailing practice of batch processing. Even in 1990, accounting modules on RDBMS verticals running on *nix mini machines, still did batch updates. While Codd and acolytes make the point of the RM being divorced from implementation, that quote is clearly inconsistent with that bill of divorcement: it reflects the limited random access support of mainframe disk subsystems of the day. Today we have multi-processor/core/SSD machines that make batch updating as obsolete as high button shoes. I can't quite figure Iggy's point here either. Is he supporting Amazon's preference for "eventual consistency" (an oxymoron) by quoting Codd, or is he supporting eventual consistency in today's RDBMS engines? Don't know, but either isn't needed with today's hardware.

28 July 2013

Blindingly Obvious

Some situations are just blindingly obvious, and not worth the time to type up. The futility of NoSql datastores (I refuse to elevate them to "database") is one such. Except for single user, single use (a wordprocessor program on your unconnected PC, for example), where some application specific file structure is sufficient, in any multi-user setting the coders who go the NoSql route end up having to build a TPM from scratch. Or cadge one from some OS kiddies. Say what one wishes about DB2/Oracle/SQLServer, they've been actively developed for decades. There's thousands of person-years of experience on offer. While I use Postgres, it simply isn't up to snuff relative to DB2.

Well, someone has bothered. The piece is short on details, but the trend is clear: with QaD datastores, you get what you pay for.
For example, when installing Couchbase, you are informed that you need a system with 4 CPUs and 4 GB of RAM. It will run on less, but when your website gets a traffic boost, be prepared to allocate more resources in a hurry.

That's just absurd. You've been warned.

The allure of NoSql is the illusion that data structure doesn't matter, and/or can be easily morphed. Fact is, with data chained to the application, change in one requires change in the other, generally far more code change. As the Fram oil filter ads used to say: "pay me now, or pay me later". Or what one hears from Professor Bunsen the first day of Econ 101: "they ain't no such thing as a free lunch".

26 July 2013

Up A Creek Without A Paddle [update]

Up a creek without a paddle. Neat phrase, and describes how I feel about Kayak.com. I've sworn never to use the site. Why, engaged reader might ask? Well, I abhor those moronic commercials, particularly the one where a black guy emerges from the sweater of an Asian guy (or vice-versa, it's been a while since I've seen the ad) typing at a laptop. By what logic anyone, at Kayak or its ad agency, sees the point of these ads? They're not humorous, nor informative. Just creepy. If you search with 'kayak ads creepy', you'll find that I'm not alone in that regard.

So, why, engaged reader, do I bother typing on and on about Kayak.com? Well, there's a wealth of interesting quant and economic drivel in today's NYT (just go read it, esp. the Business section). A regular feature of the section is "Corner Office", in which the CEO/founder/doyen of some company is interviewed, and the reader is treated to all manner of superiority of said CEO/founder/doyen. And, surprise, today is Paul English's day. He *is* Kayak.com. According to Wikipedia (and you have to enter Paul M. English; just Paul English takes you straight to Willie Nelson's drummer, what a relief) he went to one of The Other UMass's, in Boston. And did a bunch of software companies in, around, and on 128.

So, while I can't stand Kayak, English's background should make me bromance him. And I did, even before checking the Wiki to fill in the details; just on the basis of his "Corner Office" interview. Do I believe the interview? Not so sure. The nasty bits are likely true, not so sure about the free rein bits; the usual dichotomy. Here's some quotes that made my heart go pit-a-pat (again, the fact of Kayak leaves me dubious that he actually believes what he says).


The most important thing I learned is something I'm still actually working on, which is how to be really blunt with feedback. It's the most difficult thing for a manager to do. But I worked hard at it, because when managers were blunt with me, it hurt a little bit, but I'm very grateful to those few managers who helped me.
Obviously, no one's doing that to him now that he's The Man; only he gets to do that. The creepy commercials clearly show he's out of his depth.


We're a little bit reckless in our decision-making -- not with the business, but the point is that we try things.
That's, at face value, a good thing. Later in the paragraph:
It may or may not have been successful, but it almost doesn't matter, because it showed that we value speed, and we value testing ideas, not talking about them.
He hasn't learned that Speed Kills. As my Pappy used to say, "engage brain before mouth". Again, what ninny takes pride in those commercials?


Then the two criteria I really look for are productivity -- which is about speed and judgment and drive -- and the second one is fun. A lot of companies have the no-jerks rule. But I have the "no neutrals" rule.
That's where he gets it wrong, and I'm going to guess that he spent his pre-Boss days as a client side coder. "Engage brain before fingers." Far too many coders type away, then spend hours in the debugger trying to figure out how to *really* write some piece of code. Show me a coder who loves his debugger, and you're looking at a naif.


The statement that made my heart go all aflutter:
We're known for having very small meetings, usually three people.
...
I just hate design by consensus. No innovation happens with 10 people in a room. It's very easy to be a critic and say why something won't work.
On further reflection, I'm of two minds about this attitude. On the one hand, I agree that the best systems, especially database-centric ones, don't require lots of cooks stirring the broth. On the other hand, this could just be code (so to speak) for "A yes man, a yes man, my kingdom for a yes man". Impossible to tell from the interview.

For yucks, I checked the job listings, and they don't really do that. Nothing about who they want. They could be using flatfiles, xml, or MySql. Who knows? My guess: flatfiles and php. Shall we? Umm. Ah, segregated in its own silo. Perl and MySql (here). Nuff said; at least they skipped php. Bad commercials, bad tech. Who'd a thunk it?


[update]
Anyway, at the bottom of the Tech page was this carrot:
Cheers,
billo, Ultra Vice President of Code

P.S., if you want to send me encouraging notes or vitriolic hate mail, I'll bet a clever person like you could guess my email address.

I left this quote out, but now it seems appropriate:
When I gave people their performance reviews, I would literally take a crinkled envelope, and I'd write five words on it.

So, I was going to send off the link to this musing, for his amusement. But I decided to see who billo was, by name. Turns out Bill O'Donnell, and he's gone from Kayak after more than 9 years. Turns out: Priceline bought Kayak a few months ago and now Priceline just pooped the bed for its quarterly. May be the Kayak architect should have stayed and the Priceline gone (the Shatner commercials are nearly as bad).

25 July 2013

Oh Rickyyyyy!

Rick Perry, juvenile IQ governor of Texas has been out company hunting in all those Blue States. He's been especially active here in New England the last few weeks. The weekly report from BLS on unemployment generally only gets a headline number. But there's more data on offer. In particular, the list and comments (from the state UI agency) of those states with more or fewer than 1,000 change.

Here's the irony of it all:

MI -11,969 Fewer layoffs in the manufacturing industries.

TX +6,001 No comment.

Yee Haa!! Of course, one week isn't the be all and end all; on the other hand, there's been plenty of reporting recently that the Texas Miracle (or whatever they're calling it) is based nearly entirely on hamburger flipping jobs. As the Keynesians know, a "job" isn't a homogeneous unit. Not even water is.

20 July 2013

The Night They Raided Minsky's

Not for the first time, the NYT editors let two of their fair haired boys (Krugman and Norris) collide. The point of collision: one Hyman Minsky. Here's the wiki, if you've never heard of him. It's a decent read.

I'll start with the Norris piece, since it relies more heavily on Minsky than Krugman's off-hand reference. Norris sets out to show that Bernanke got it wrong by not anticipating the Great Recession:
... what it got wrong: policies that allowed the dangerous imbalances to grow and bring on the crisis.

Now, that assumes that the Fed/Bernanke could have shunted the train off to a safe siding, as one routinely saw in the Perils of Pauline movies. Well, may be not. It is true that the Giant Pool of Money is still looking to beat the interest rate, and it is true that this is what has propelled the stock market. It that such a bad thing? Your typical Right Wingnut congressman says so:
[Jeb Hensarling, the Texas Republican who is chairman of the House Financial Services Committee] was upset that the "Federal Reserve has regrettably, in many ways, enabled this failed economic policy through a program of risky and unprecedented asset purchases."

Such an ingrate. Big Ben has enriched his constituents (not his district, but the banksters), and still he complains. It's worth noting that in decades past, there was an intense debate on how to fund capitalists. In the US, the typical method was through equities; while in Europe it was debt (bonds). What Hensarling is promoting is a preference for debt. Why would anyone do that? Well, it's simple: if one holds a large sum of moolah, then one likely has more money than one needs, but paranoia sets in. Buying equities leaves one subject to the vagaries of traders; one's income is in the form of capital gains, and therefore vulnerable to the whims of Mr. Market. On the other hand, holding bonds guarantees one income on the coupon (more or less guaranteed). The value of the bond will fluctuate, of course, but that doesn't affect the income stream. Barring total collapse, of course.

Norris spends much of his piece quoting Bernanke, and this is some of that:
Then Mr. Bernanke pointed to "another counterproductive doctrine: the so-called liquidationist view, that depressions perform a necessary cleansing function." That was the view pushed in the early 1930s by Andrew Mellon, the Treasury secretary, to such an extent that it angered even President Herbert Hoover, who did not, however, seem to think he could overrule the secretary. Now the comments could be read as a reproach to those, in the United States and Europe, who push for austerity above all else.

I suppose as one gets older, and Mellon was in his 70s when the Great Depression hit, enemas seem necessary; especially when medicine was primitive. We don't need no economic enemas.

Here is where Norris, otherwise a wise man, goes off into the weeds:
The intellectual framework it used simply could not cope with the idea that financial stability can itself become a destabilizing factor, as investors and bankers conclude that it is safe to take on more and more risk.

He's concluded that the financial world, up to 2003 (to pick a date for the beginning of Viagra at The Home) was somehow stable. Nothing could be less true. Income concentration was revving up. Median income was reversing from the positive trend under Clinton. Thus, the middle class looked to home equity raises (almost wholly unearned) to fund consumption. With Greenspan's cratering of interest rates and real estate appreciation that followed, the die was cast.

"Improved assessment and pricing of risk, expanded lending to households without strong collateral, more widespread securitization of loans, and the development of markets for riskier corporate debt have enhanced the ability of households and businesses to borrow funds," [three Fed economists, Karen E. Dynan, Douglas W. Elmendorf and Daniel E. Sichel] wrote. "Greater use of credit could foster a reduction in economic volatility by lessening the sensitivity of household and business spending to downturns in income and cash flow."

What those Fed economists were engaging in is cognitive dissonance: they were hailing as positive the very machinery which would ignite the Great Recession. What Bernanke and Norris ignore, if they even yet realize it, is that the search by middle class households for funds to maintain their position in the consumption distribution was what propelled the collapse. Now, if China hadn't been generating great quantities of moolah at just the moment that Greenspan chucked his hail mary pass (crashing interest rates, since Dubya wouldn't countenance fiscal policy), things would have turned out rather differently. And they likely will.

While quants (monetarists, all) look at numbers to divine the future, looking at the wrong numbers leads to wrong answers. Bernanke, and Norris, look at the wrong numbers.

The Fed chairman conceded that "one cannot look back at the Great Moderation today without asking whether the sustained economic stability of the period somehow promoted the excessive risk-taking that followed. The idea that this long period of calm lulled investors, financial firms and financial regulators into paying insufficient attention to building risks must have some truth in it."

Again, and I repeat, the calm was *an illusion*. Professional economists, nearly all, looked at the numbers which they traditionally looked at, and in the traditional manner. M1 and U3 (and not, devil be damned, U6) and so on. Few looked at the driving force of the US economy: the middle class consumer and where his/her income came from and what the dynamics of that income was/is. Look at that number, especially today, and one sees not calm but a duck: placid above the water line, and chaotic below. Some understood that median income was in the toilet since Reagan, but the data weren't part of the regular data stream, and easily ignored. Means (averages) were almost always cited, even though it is well known in both economics and quants that income (and wealth) are highly right skew, and that mean income (typically, GDP per capita) is a lousy (way too optimistic of equality) descriptor of central tendency.

At this point in the narrative, Minsky (in a ghostly fashion) enters, stage left. His voice is in the body of an Australian economist, Steve Keen. The year is 1995. For review: Clinton had begun to reverse the Reagan/Bush assault, and life was looking up for the middle class. It was a hopeful time.
Eventually, the income-earning ability of an asset would seem less important than the expected capital gains. Buyers would pay high prices and finance their purchases with ever-rising amounts of debt.

Note that this was a decade before the Giant Pool of Money had been spotted. But the key idea is the same: asset appreciation (whether real or nominal) would eventually overwhelm coupons. What isn't mentioned: the savers and borrowers weren't in the same countries; the moolah, that debt, had to come from somewhere. If Minsky/Keen were correct, then the rational actor would never offer up moolah as debt to others, but use that moolah to buy equities. That matters. There must be one or both of two conditions: asymmetric information (savers don't know) or asymmetric policy (savers can't shop). Norris references a critique of Minsky by Bernanke:
They had, Mr. Bernanke wrote, "argued for the inherent instability of the financial system but in doing so have had to depart from the assumption of rational economic behavior." In a footnote, he added, "I do not deny the possible importance of irrationality in economic life; however it seems that the best research strategy is to push the rationality postulate as far as it will go."

Again, looking at the traditional numbers in a traditional way. Yet, by now one hopes, quants/economists/policy wonks understand that what compelled the Great Recession wasn't a financial panic of the previously traditional type nor was it the product of irrationality. Everyone was rational, responding to incentives that bolstered self interest. What was irrational was establishing/distorting the incentives; this is policy, immune to quants. With Greenspan having crashed interest rates, and all that moolah looking for higher returns (at no risk), standard (or so-called, anyway) vehicles were created. Severely bending the rules of creation in order create sufficient vehicles was the problem. Boatloads of moolah from China (or manna from heaven?) had no precedent; we're still living with it, although China appears to be internalizing the positive feedback loop that used to include the US housing market. Declining median income, without depression or recession, was also without precedent. Using that Giant Pool of Money to pay for the kids braces was a no brainer. The house would generate enough moolah until the cows came home. Or is it, the chickens came home to roost? It entered on little cat feet (extra points for that allusion), one tiny step at a time over a generation. OK, enough barnyard metaphors.

What neither Minsky nor Keen appeared to know in 1995 was that the center of the middle class was being crushed; the process had been pedal to the metal since the PATCO strike, 1981. The financial numbers would make less and less sense as time went on. They make even less sense now. While the standard unemployment number seems to have settled at a barely tolerable number, the gross ratio of employed to population remains near a post WWII low. That ain't a harbinger of glory.

In sum, Bernanke, Norris, Minsky, and Keen (the latter less culpable in 1995) are missing the point. A structural change has been progressing (or regressing, depending on one's point of view) since Carter; he declared war on Federal employment before Reagan did. I was there. I had to re-apply for my position which had been officially made redundant, as did all of my colleagues. Not all Democrats are bleeding heart liberals.

That's it for Norris and Minsky. Now for Krugman.

He invokes Minsky at the end of his column, thus:
Unfortunately, these aren't ordinary times: China is hitting its Lewis point at the same time that Western economies are going through their "Minsky moment," the point when overextended private borrowers all try to pull back at the same time, and in so doing provoke a general slump. China's new woes are the last thing the rest of us needed.

That Lewis point? He begins,
All economic data are best viewed as a peculiarly boring genre of science fiction, but Chinese data are even more fictional than most.

As I've written many times: macro data is wonky, built from sampling, and all too infrequent censuses. Policy trumps data at the macro level in any case: policy is designed to reward friends and punish enemies, and data be damned. Since policy is determined by what friends demand, rather than any rational exercise, it's always correct. So far as the policy maker is concerned.

So, Krugman goes over the problem mentioned here rather a lot: the unsustainability of China's income distribution. He invokes another ancient economist W. Arthur Lewis, who proposed
... that countries in the early stages of economic development typically have a small modern sector alongside a large traditional sector containing huge amounts of "surplus labor" -- underemployed peasants making at best a marginal contribution to overall economic output.

This is the traditional American explanation, overlaid on China. I'm not sure I believe it, but that's secondary to Krugman and Lewis. Whether labor moves to cities/factories either willingly or by force, incomes that don't rise are incomes that don't sustain capital. China is an odd sort of industrial exporter. In times past, export economies were small relative to shipped-to markets. That's what made the gag work: capital relocated to small, preferably dictatorial, locations with cheap labor and sent goods to large economies with moolah. The factories of New England were moved to the more receptive South, and sold to wealthier Northeners. China, if it wished to, has no need for Western markets. Nixon went to China to exploit all those soon to be consumers. That was nonsense, of course. He went there to get access for capital to unlimited cheap labor. And it sort of worked. The problem is that he and successive Right Wingnuts have succeeded in killing the necessary other half of the process (aka, The Golden Goose), the money filled consumer. No consumer, no sales, no profit, no capital. A question of balance.

China faces a problem. Without a robust set of importing countries (which soaked up China's exports and excess savings with CDOs and such, that damn Giant Pool of Money just won't go away), its export based industrialization is in trouble. It's only hope is domestic (organic) growth. Reporting over the last year or so on the reliance on real estate, residential and commercial, for capital placement should unnerve the doyens in Beijing. As I said, they've internalized The Great Recession. If empty apartment buildings are strewn about, one might infer that capital allocation in China is seriously out of whack. No one seems to be able to put that Giant Pool of Money to productive use. Recall what's been said here about capital payback. Real returns are declining. Capital is losing its value. Oops.

China, the US, the EU all share the same problem: "rebalancing". Also known as, income redistribution. The Austrians must be turning over in their graves. Even the vampire ones that still show up on late night cable "news" programs.

In both Norris and Krugman are swipes at consumerist US middle class; but that's stupid. Without a robust middle class with most of GDP, there's no way for capital to earn a return. It's a zero sum game. US corporations continue to generate ever higher profits, and used said profits to buy up competitors, mostly, rather than make physical investment. In other words, the US division betwixt investment and consumption is arguably still not enough consumption; moolah that is "invested" isn't really. Micro (individual corporation) growth through acquisition is a wash at the macro level, at best. It's nothing to base policy on.

So, there you have it. A raid on Minsky.

I Feel for Boris

The weekly feed of Database Weekly from sqlservercentral.com has a couple of non-sqlserver specific links dealing with "data science", one of which is particularly trenchant. Here's the ending, money quote:
I don't want to put you in a bad humor, or make you fear that predictive analytics could send you to an asylum, but it's likely that your experiences with sharing the insights provided by data science within your organization will follow a trajectory similar to how Harvard University professor Lant Pritchett describes the eventual adoption of any new insight: "Crazy. Crazy. Crazy. Obvious."

This is similar to the quote attributed to Gandhi (that he actually said/wrote this is still disputed),
First they ignore you, then they laugh at you, then they fight you, then you win.

Since it's been more than 40 years from Dr. Codd's RM paper and "developers" still denigrate even SQL databases much less Organic Normal Form™ schemas, I'd be truly bummed out if the Data Science folks can get to the end of their Yellow Brick Road in less than a decade. As Boris whined, "Unfair to local 12: Villains, Thieves and Scoundrels Union". Not that, to be clear, RM advocates are any of those miscreants.

19 July 2013

A Mile Wide, and An Inch Deep

R is an amoeba like language, growing various bits and pieces without any thought. Some consider that good. Others, not so much. So, this post caught my eye. A very short Inferno, I suppose. But what really surprised me was:
Make the screen wide

options(width=160)

Boy howdy!! We've been getting wider screens for more than a decade, and shallower too for the last half decade. Yet even the kiddie koders worship at the foot of 80 columns. That's the 1890 Hollerith card, for those who've not read the history. The tech weenies give us shallow, wide screens. Use what you get, for crying out loud.

18 July 2013

Bayeing At The Moon

It's not a secret that I'm not fond of Bayes. I grew up at a time when Bayesian analysis was an emerging cult, whereas now it seems (in some areas of discourse) to be trite. The problem with Bayes is simply that priors are subjective, and can affect the results immensely. In the final analysis (so to speak), we have a bunch of data, and we wish to assess whether there's an equation (model) which closely predicts these values. The data are concrete, the model is fungible. The data tell the story, but we must be careful in our quest to find the moral.

So, Andrew Gelman stirred up a hornet's nest. Have a read; it's a gas.

In the end, it's always been my feeling that the reason Bayesian stats have become popular is that Ph.D. dissertations from the classical/frequentist arena became increasingly hard to come by. It was simply easier to gin up some Bayesian proposal. And so it was.

Thank God Almighty

As Dr. King said, we're free at last. AnandTech (and lots of other sites) are reporting on Korea and Samsung's new SSD initiatives. You should go off and read up on them. At your favourite site.

However, regular reader should note the list of quotes which open this endeavour, and that the first was the first and always will be the first. It's the most important, even though we're nearly a decade since it was stated.
While this is something relatively new, it is not on the 840 Evo, but as part of the summit today it is worth some discussion. The principle behind NVMe is simple -- command structures like IDE and AHCI were developed with mechanical hard-disks in mind. AHCI is still compatible with SSDs, but the move to more devices based on the PCIe requires an update on the command structure in order to be used with higher efficiency and lower overhead. There are currently 11 companies in the working group developing the NVMe specifications, currently at revision 1.1, including Samsung and Intel.

The first step, really, to file systems which assume flash. By the time we get there, flash may be replaced by some other form of NVM, but I don't think anybody will quibble.

15 July 2013

Hello, Joe

In some of my more somber moments, I've whined that Joe Stiglitz has been conspicuous in his absence of late. And yesterday's Times didn't arrive. Fortunately Groklaw had an off-topic link (may be not so off topic, really) to his piece on Myriad, and how Myriad and the patent regime of the US is good for the few and bad for the rest.

10 July 2013

Give Me a Minute

The June Fed (FOMC, Federal Open Market Committee, which sets the Fed Funds rate) meeting minutes (which they really aren't) have been posted (here's the wiki describing the FOMC). Given that the Fed's intentions have been roiling the economy since then, it is instructive to have a read. This is the sort of Black Swan event that gives quants the heebie jeebies: how to stuff the information into some logistic regression model, or such? Hehe. Policy always trumps data.

09 July 2013

Phasers on Obliterate!!

Recent discussions about pair programming and cloud lead me to conclude that cloud, in particular, depends on a peculiar reality. The attractiveness of cloud, from a client point of view, is that one need not store inventory of hardware (on-line or off-line) to deal with (un-)expected peaks in demand. Cloud vendors, at least apocryphally, provide instant access (and de-access) to resources. In order to do that, of course, the cloud vendors have to hold the necessary inventory against demands of hundreds or thousands of clients. If said clients are Mom & Pop Shops, who've little buying power for hardware, then cloud vendors, with the aggregate demand with which to bargain, should be able to eke out a small profit on the delta.

But what about the steady state? Where the business will, presumably, be spending most of its life? In that circumstance, how does the cloud vendor keep its head above water? Well, near as I can tell, peaks in demand from half of the clients have to be exactly in phase with troughs in demand (or that "gentlemen, phasers on obliterate!!") from the remainder; two sine waves 180 degrees out of phase. Otherwise, cloud vendors, just like their clients previously did, will continue to add inventory resources as the organizations grow. If they don't grow, then they provide the cloud vendor with declining revenue, and the cloud vendor goes looking for a bigger cloud vendor to palm off clients. A rock and a hard place. Moreover, where's superior value add of a cloud vendor? Other than a box of parts, what do they bring to the table?

08 July 2013

Who's Integrated and Who's Not?

Minyanville isn't a site I read with much frequency. There's a piece linked from Oracle's Yahoo! page that can't go unremarked. Tech press is so tech ignorant. I think it's because so much of tech press is made by freshers, who've no memory of tech history nor any evident reading of it. If something is new relative to yesterday, it must be new incarnate. Not often true.

So, in the piece, the author makes a big deal of Oracle's fully integrated offerings versus Salesforce.com's a la carte (or may be it isn't?) menu. Having been in the packaged software business for most of my adult life (not using Oracle, though), I've been on both sides of the vertical integrated package software world; as maker and buyer. What doesn't happen is that
[Intel and Oracle] both have a long history of over-serving their customers -- of selling comprehensive, high-performance products that almost never get used to their full potential.

That's not how it actually works. Yes, the vendor will have as many parts as it thinks it can sell profitably. But, no, those parts are not woven together as a single quilt. The clients do buy some core module (oft times described as some number of integrated modules, but that can't be disassembled). After that, it's a matter of adding what you need, if and when you need it. HR functions are most obvious. Most of the time the real sticking point is the semantics of the core: how does it do the accounting, how does it do the product assembly, and so forth.

Then, the author contradicts himself
Although the cloud possesses the aura of technological freshness, in many ways, it's a throwback to the days of mainframes, when computing was so expensive that sharing it was necessary. Today, the great recession has once against brought expense to the forefront, and raised difficulties for the old over-serving approach.

Sound familiar?? It should. It's been the driving theme of this endeavour from the beginning. Now, whether it makes sense to out-source such a business model isn't driven by the semantics, but by cost and security. Cloud providers can only make money (and the author asserts that they don't, yet) if they can acquire the hardware in greater bulk discount than their clients. For generic Mom & Pop clients, they likely can. For the Fortune X00, not so much. A private cloud is just the old glass house with 3270's, just longer wires. Even if cloud vendors target M&P by the thousands, being able to provision new resources in a snap of the fingers means the cloud vendor has the inventory on site. That costs money. Being able to spread the load to keep all the hardware loaded, but not overloaded, requires that usage spikes be uniformly distributed (with countervailing troughs) over some span of time and space. Also, not likely in the real world. Whether the continuing security breaches are just a flash in the pan, or a true issue remains to be seen. Given that EULAs are always in favour of the vendor, a backlash is inevitable.

The one lever that cloud has: explicit non-ownership of software. Most EULAs (without a source license), are just rental agreements, but without a lot of teeth. Whether real clients will accept rental with teeth has yet to be determined.

Moreover, the author fails to distinguish among the eras of computing:
- mainframe with terminal, where application code (COBOL/RPG/PL1 and files) is executed on mainframe (with some rudimentary editing on a dumb terminal, possibly)
- client/server, where application is partially/wholly executed on client and files stored on server (the engineering workstation on LAN)
- the *nix (and Windows, to a lesser extent) database (usually RDBMS, but not commonly relational schema) with VT-X00 terminal
- webby client/server, where the application is in a browser and is mostly executed on client; data is commonly in RDBMS but not relational
- post industrial computing, where much of "new" development is infotainment, sometimes coerced as business support; RDBMS, if used, is file image

To the extent that smartStuff is limited to using its smarts to manage its screen, we're back to mainframe semantics or *nix/database/VT-X00. As I've been arguing from the beginning. Then, the issue becomes whether we revert to 1960s COBOL (file storage) style mainframe development for applications, or move forward with RDBMS in Organic Normal Form™ as the basis. Angry Birds may make some people some money and attract the bulk of freshers for employment, but it isn't relevant to how Oracle Applications (and their presumptive replacements) are built. To replace an existing Oracle Application (using Oracle as universal surrogate for legacy applications) means having a superior (for some definition of "superior") industrial application. Oracle has been slurping up its clients (who wrote those verticals for end users) with abandon for about a decade. Does the world need yet another GL? Not if it's just a re-hash of the thousands made since 1960. So, Oracle buys its clients in the realization that new ones aren't clamouring at the door. Micro (corporate) growth with macro (economy) stagnation. And the freshers go off to make some other pointless iStuff app. Just repeating COBOL/VSAM structure won't be enough. Touch based, and therefore disaggregated data based, versions mean either leveraging ONF™ RDBMS or writing the equivalent in code/files from scratch. Today's news that Facebook is re-inventing relationalism is kind of quaint. The young-uns never know. I'll be a quid that, like nearly everything else on the net, it will soon turn into a porn dating mechanism.

06 July 2013

Superman, Up in the Sky

A while back, in some venue (may be even here), I opined that the coder cabal is the apotheosis of destroyed productivity. That building Organic Normal Form™ relational databases on a $10K Xeon machine, at 1/10 the annual cost of a cowboy coder, is where the real productivity lies. If The Suits (ever the bureaurats, looking only to fatten their org chart) would only look past the ends of their noses. Fat chance.

Anyway, the $10K number was actually kind of old and arbitrary; I haven't been on the lookout for such a machine here. But then, looky here. AnandTech does a (Brit) look at a Supermicro quad Xeon machine. The nature of the test presented doesn't have much direct bearing on RDBMS, that's true. I spent some time at the Supermicro site, but didn't see price lists. However, one of the comments allowed as how the machine in question was £16K. So, a bit more than the $10K, but still.

01 July 2013

Time For the Cattle Prod [update]

The transition to Organic Normal Form™, propelled by MultiProcessor/Core/Thread/SSD, hasn't been as seamless as I've hoped/predicted. More like Rory McIlroy's recent golf game. Ah well. But the truth will out, in truth.

Since I'm not a MS sort of guy for some time, I'm only recently aware of Hekaton. Boy howdy! Now, not by any means the first in-memory engine (or partial engine) from an industrial strength RDBMS (TimesTen/Oracle, SAP/HANA, and solidDB/IBM come to mind), but SS does serve a larger population of installs (that's speculative, since sources such as IDC charge a pretty penny for their data, although "According to Microsoft officials, 46 percent of the databases deployed worldwide are now SQL Server, and customers are running 300,000 SQL Azure databases in Windows Azure" [here]). The point, of course, is that if SS goes in-memory, the pressure to normalize goes up a tad, what with all those SMB OLTP systems (existing and under development) that benefit from minimum footprint supplemented by joins. This could be the straw and camel moment.

Here's hoping.

Here's a Microsoft Research page. While it doesn't push the normal form benefits of Hekaton, just have a look at that pix of the developers. THEY'RE OLD FARTS!!!!! Take that, Kiddie Koders. It generally helps one in making something that is truly new to know what, exactly, was the Olde Stuffe. Had the Kiddies paid attention, we might have been spared all things xml and NoSql. ISM/VSAM variants to the hilt.
"We had an 'aha' moment," Lomet recalls, "when we realized that a single table that maps page identifiers to page locations would enable both latch-free page updating and log-structured page storage on flash memory. The other highlight, of course, was when we got back performance results that were stunningly good."

Here's a hot off the presses report. Read it, and lick your chops. Note that FK constraints (but doesn't say whether FK can be defined for access path purposes; yes, there already exist RDMBS engines* that support such an option) aren't yet supported.

Or, as one of Red Gate's (my favorite SS/RDBMS site) bloggers put it:
My biggest disappointment with the initial version is that it does not support foreign key constraints or check constraints. For a database designer/developer, "let the application handle data quality" is like fingernails on a chalkboard.

Nice to know that some people vote for Abbie Normal.

So, half a loaf. But, I strongly suspect that FKs will show up soon enough; if only to have a sensible OLTP engine. After all, an RDBMS targeted for OLTP, as MicroSoft states, can't be truly serious without that.

Hmm. SSD to support buffer pools? I have to finally admit defeat and run Windows?? While I've always had a hankering for SS, I just can't abide Windows. Scylla and Charybdis.

[update]
* Informational constraints tell the database manager what rules the data conforms to, but the rules are not enforced by the database manager. However, this information can be used by the DB2® optimizer and could result in better performance of SQL queries.

here