Regular readers know that I submitted to the Real Cringely a prediction. He's only up to 6 so far, thus, I don't know whether he's deigned to accept it. When he made the announcement, he only said that any outside predictions would be credited, not that the author(s) would get advance notification of "winning".
While we wait, and given the recent spate of SSD and processor news, a new idea has been worming around my cerebrum. This post is just to establish originality, not the full blown patent. I could well abandon the idea. It's just one of those thought experiments.
My submitted prediction was that Oracle would leap frog the other RDBMS vendors by pushing the BCNF approach in order to win over the IBM/COBOL mainframe application crowd to the Oracle/SSD machines. The magic bullet approach.
But what if a vendor, may be Oracle may be not, took the vision a step further? What direction would such a step take? How about writing the *engine itself* to multi-core/processor + SSD machines? How would such an engine differ from today's versions? Well, at minimum it would have an optimizer and execution unit which are parallelized. Some (most?) engines are parallel only to the extent of being able to execute multiple queries at once, one per thread (generally). What is not common is parallel execution of a query into the datastore.
With rising processor/core/thread counts, blasting a query through many threads begins to make sense, if the datastore can respond fast enough. The existing model of concurrency, whether the locker model or MVCC, seeks to minimize the time that a given row is locked. Row level locking developed to make this possible, and MVCC developed to make it "irrelevant". Both approaches are based on the concept of a conflict serializable schedule (Weikum & Vossen, pg. 92 et seq). In practice, the engine does what the COBOL/VSAM coder used to do: iterate over a bunch of "records" doing stuff one at a time. The RDBMS presents to the client an "all at once" facade, as Dr. Codd demanded, but there's just a really fast squirrel spinning the cage.
But matrix operations are what Dr. Codd really meant; one can view the relational algebra as essentially linear algebra for a limited domain. What if a RDMBS vendor looked at current, and near future, machines rather than past machines? Current databases are still based on uniprocessor slow disk technology. The emerging machines have been obvious for five years, anyway. How far along could, Oracle for example, be toward an engine that *demands* X number of cores and Y amount of SSD primary storage? Could you build a database engine from scratch in five years? Abso-freaking-lutely. OS/360 was written in less time, and in assembler. So, yes, this could be the year. My guess: Microsoft.
06 January 2011
Losing My Religion
I don't know what OCZ is supposed to be an acronym for (don't end sentences preposition with), but 'Oly Crap Zoroaster will do. They're at CES, and here's some info from our friends at AnandTech. Note that the thing screams even with compressed data. Hmmm.
Later in the article is the toss-away:
"On the other end of the spectrum, OCZ presented an even bigger (physically) drive: the IBIS XL. Now this isn't going to be productized, but it's simply something to test the waters with. The IBIS XL fits into a standard 5.25" drive by and starts at 4TB."
As I've mentioned a few times, Zsolt at storagesearch makes the case that sooner, not later, SSD will even take over the petabyte world of storage. I've always thought that a bit daft. Now, I'm not so sure it's daft.
Gimme a B! Gimme a C! Gimme an N! Gimme an F! What's it spell??? The future. (This is where I toss the cute blonde cheerleader over my shoulder.)
Later in the article is the toss-away:
"On the other end of the spectrum, OCZ presented an even bigger (physically) drive: the IBIS XL. Now this isn't going to be productized, but it's simply something to test the waters with. The IBIS XL fits into a standard 5.25" drive by and starts at 4TB."
As I've mentioned a few times, Zsolt at storagesearch makes the case that sooner, not later, SSD will even take over the petabyte world of storage. I've always thought that a bit daft. Now, I'm not so sure it's daft.
Gimme a B! Gimme a C! Gimme an N! Gimme an F! What's it spell??? The future. (This is where I toss the cute blonde cheerleader over my shoulder.)
05 January 2011
Is That Oz Up Ahead?
Is that an Emerald City over the horizon? Are we almost to Oz? Yes, yes we are.
The last week has brought a host of news, which taken together, indicate that the near future is nearly here. If I keep stepping half-way to the wall, do I ever get to the wall? Yes, yes you do, to within any delta you wish to name.
First, there was the Windows on ARM announcement.
Next, we have two reports from CES, courtesy AnandTech: the next Tegra, and its successor.
The future is clearer: it will be a pixelated VT-220 connected to a wireless network and thence to a relational database stored in BCNF on SSD. Such applications (not ported, in the common way, from COBOL) will run rings around all that legacy file based stuff. With sufficient bandwidth, and a persistent connection (your phone is, right?), it's back to the future. No, I don't believe that the phone/web (or web/phone if you prefer) paradigm is the winner. The flexibility and accuracy of the persistent connection paradigm will win.
For those who think that the web is really great progress over what came before, you need to know the simple history. I'll start with the consolidated mainframe world of the late-60's. There were mainframes and terminals (3270 is the archetype), over a disconnected connection. There were local edit programs written into the 3270. The transfer was block mode, meaning that all keyboard activity was between the user's fingers and the local edit program. Hitting Send (what we now call Carriage Return/Enter) sent off the edited screen to the mainframe app code.
I've just described a browser/html application.
Then, along came Unix and the VT-100 (there were later, more capable VT's, the VT-220 in particular). While connected by a wire, just as a 3270, this wire is always on. Ah. As a result, database engines and application code residing on the server see each keystroke on the VT-220. In fact, it was common to have 4GL's resident with the RDBMS in what was often referred to as "client/server in a box": the database engine being the server in its patch of memory, and a patch of memory being the application code for each client connection. Blindingly efficient, and allowed the database and the application code to edit/check *character by character* as the user progressed; if that sounds a bit like AJAX, well, yes it is. Not quite as cheap in memory, but this was the early 90's and later, when memory began to get cheap. Supporting 1,000's of terminals (or PC's running a terminal emulator) was not uncommon.
Then, along came the www, and young-uns thought this was new and fabulous. Negatory.
With the increasing density of chips we see an artefact of Moore's Law not often (except por moi) remarked: look at block a diagram of recent multicore chips. Most of the real estate is dedicated to various caches, oddly it seems, rather than using all those transistors to execute native instructions in hardware, the trend has been to emulating, for example, X86 instructions in a "hidden" RISC machine. Not that this is new; the 360/30 was widely believed to use a PDP-11 to run the instruction set. IBM did acknowledge that most of the 360 series emulated the instruction set; only the top end machines executed in hardware. The upshot is simple: few, in any, normal client machines have need for the compute power on tap.
So, we see the rise of ARM and MIPS (you did buy ARMH when I told you to, yes?) running minimalist machines at low power and low cost. With a persistent connection to a persistent datastore, let's dance like its 1995. And we will. You can't fool Mother Nature, and She's saying: keep the data and its management in one secure place, and let the kiddies make pretty, pretty pictures on their phones and pads. Just don't let them mess with the data.
Just stay away from the poppies. You heard me. Don't go into the field.
The last week has brought a host of news, which taken together, indicate that the near future is nearly here. If I keep stepping half-way to the wall, do I ever get to the wall? Yes, yes you do, to within any delta you wish to name.
First, there was the Windows on ARM announcement.
Next, we have two reports from CES, courtesy AnandTech: the next Tegra, and its successor.
The future is clearer: it will be a pixelated VT-220 connected to a wireless network and thence to a relational database stored in BCNF on SSD. Such applications (not ported, in the common way, from COBOL) will run rings around all that legacy file based stuff. With sufficient bandwidth, and a persistent connection (your phone is, right?), it's back to the future. No, I don't believe that the phone/web (or web/phone if you prefer) paradigm is the winner. The flexibility and accuracy of the persistent connection paradigm will win.
For those who think that the web is really great progress over what came before, you need to know the simple history. I'll start with the consolidated mainframe world of the late-60's. There were mainframes and terminals (3270 is the archetype), over a disconnected connection. There were local edit programs written into the 3270. The transfer was block mode, meaning that all keyboard activity was between the user's fingers and the local edit program. Hitting Send (what we now call Carriage Return/Enter) sent off the edited screen to the mainframe app code.
I've just described a browser/html application.
Then, along came Unix and the VT-100 (there were later, more capable VT's, the VT-220 in particular). While connected by a wire, just as a 3270, this wire is always on. Ah. As a result, database engines and application code residing on the server see each keystroke on the VT-220. In fact, it was common to have 4GL's resident with the RDBMS in what was often referred to as "client/server in a box": the database engine being the server in its patch of memory, and a patch of memory being the application code for each client connection. Blindingly efficient, and allowed the database and the application code to edit/check *character by character* as the user progressed; if that sounds a bit like AJAX, well, yes it is. Not quite as cheap in memory, but this was the early 90's and later, when memory began to get cheap. Supporting 1,000's of terminals (or PC's running a terminal emulator) was not uncommon.
Then, along came the www, and young-uns thought this was new and fabulous. Negatory.
With the increasing density of chips we see an artefact of Moore's Law not often (except por moi) remarked: look at block a diagram of recent multicore chips. Most of the real estate is dedicated to various caches, oddly it seems, rather than using all those transistors to execute native instructions in hardware, the trend has been to emulating, for example, X86 instructions in a "hidden" RISC machine. Not that this is new; the 360/30 was widely believed to use a PDP-11 to run the instruction set. IBM did acknowledge that most of the 360 series emulated the instruction set; only the top end machines executed in hardware. The upshot is simple: few, in any, normal client machines have need for the compute power on tap.
So, we see the rise of ARM and MIPS (you did buy ARMH when I told you to, yes?) running minimalist machines at low power and low cost. With a persistent connection to a persistent datastore, let's dance like its 1995. And we will. You can't fool Mother Nature, and She's saying: keep the data and its management in one secure place, and let the kiddies make pretty, pretty pictures on their phones and pads. Just don't let them mess with the data.
Just stay away from the poppies. You heard me. Don't go into the field.
28 December 2010
Scotty's Wisdom
Message boards can actually be useful to the exercise of figuring out where an industry is going. STEC is the principal publicly traded Enterprise SSD vendor, so it is the public bellwether with respect to "Enterprise SSD". They've been segueing from SLC dominant to MLC dominant product mix, which ends up being a topic of discussion, especially recently. A thread is running now about the qualification of an MLC version of STEC gold standard "Enterprise SSD" (ZeusIOPS). I was moved to contribute the following:
"I canna change the laws of physics"
That will be true in the 23rd century and is true now. The number of erase cycles of MLC is fixed by the NAND tech, controller IP can only work around it, usually by over-provisioning (no matter what a controller vendor says). Whether STEC's controller IP is smarter (enough, aka, at the right price) is not a given. As controllers get more convoluted, to handle the decreasing erase cycles (what? you didn't know that the cycle count is going down? well, it is, as the result of feature size reduction), SLC will end up being viable. Cheaper controllers, amortized SLC fabs.
If STEC (or any vendor) can guarantee X years before failure, then the OEMs will just make that the replacement cycle. It would be interesting to see (I've not) the failure distribution functions of HDD and SSD (both SLC/MLC and STEC/Others). Failure isn't the issue, all devices fail. What matters is the predictability of failure. The best to have is a step function: you know that you have until X (hours, writes, bytes, whatever), so you replace at X - delta, and factor that into the TCO equation.
I think the failure function (in particular, whether and to what extent it differs from HDD) of SSD does matter, a lot. Consumer/prosumer HDD still show an infant mortality spike. Since they're cheap, and commonly RAIDed, shredding a dead one and slotting in a replacement isn't a big deal. Not so much for SSD, given the cost.
I found this paper, but I'm not a member. If any reader is, let us know. The precis' does have the magic words, though: Gamma and Weibull, so I gather the authors at least know the fundamentals of math stat analysis. If only there were an equivalent for SSD. It's generally assumed that SSDs are less failure prone, since they aren't mechanical; but they are, at the micro level. Unlike a HDD, which writes by flipping the flux capacitor (!!), the SSD write process involves physical changes in the NAND structure; which is why they "wear out". Duh. So, knowing the failure function of SSD (and knowing the FF for NAND is likely sufficient, to an approximation) will make the decision between HDD and SSD more rational. If it turns out that the FF for SSD moves the TCO below equivalent HDD storage (taking into account short stroking and the like to reach equivalent throughput), SSD as primary store becomes a value proposition with legs. Why the SSD and storage vendors aren't pumping out White Papers is a puzzlement? May be their claims are a tad grandiose?
The ultimate win will happen when MRAM (or similar) reaches mainstream. Very Cool.
"I canna change the laws of physics"
That will be true in the 23rd century and is true now. The number of erase cycles of MLC is fixed by the NAND tech, controller IP can only work around it, usually by over-provisioning (no matter what a controller vendor says). Whether STEC's controller IP is smarter (enough, aka, at the right price) is not a given. As controllers get more convoluted, to handle the decreasing erase cycles (what? you didn't know that the cycle count is going down? well, it is, as the result of feature size reduction), SLC will end up being viable. Cheaper controllers, amortized SLC fabs.
If STEC (or any vendor) can guarantee X years before failure, then the OEMs will just make that the replacement cycle. It would be interesting to see (I've not) the failure distribution functions of HDD and SSD (both SLC/MLC and STEC/Others). Failure isn't the issue, all devices fail. What matters is the predictability of failure. The best to have is a step function: you know that you have until X (hours, writes, bytes, whatever), so you replace at X - delta, and factor that into the TCO equation.
I think the failure function (in particular, whether and to what extent it differs from HDD) of SSD does matter, a lot. Consumer/prosumer HDD still show an infant mortality spike. Since they're cheap, and commonly RAIDed, shredding a dead one and slotting in a replacement isn't a big deal. Not so much for SSD, given the cost.
I found this paper, but I'm not a member. If any reader is, let us know. The precis' does have the magic words, though: Gamma and Weibull, so I gather the authors at least know the fundamentals of math stat analysis. If only there were an equivalent for SSD. It's generally assumed that SSDs are less failure prone, since they aren't mechanical; but they are, at the micro level. Unlike a HDD, which writes by flipping the flux capacitor (!!), the SSD write process involves physical changes in the NAND structure; which is why they "wear out". Duh. So, knowing the failure function of SSD (and knowing the FF for NAND is likely sufficient, to an approximation) will make the decision between HDD and SSD more rational. If it turns out that the FF for SSD moves the TCO below equivalent HDD storage (taking into account short stroking and the like to reach equivalent throughput), SSD as primary store becomes a value proposition with legs. Why the SSD and storage vendors aren't pumping out White Papers is a puzzlement? May be their claims are a tad grandiose?
The ultimate win will happen when MRAM (or similar) reaches mainstream. Very Cool.
23 December 2010
Mr. Fusion Powered Database
I keep track of my various database interests with a gaggle of sites and blogs. For PostgreSQL, I've signed up for the Performance sublist. It's mostly about fixing things in parts of the engine in response to questions like: "my query is slower in PG than it is in SQL Server; how come?", and such.
A thread started today that's of interest to this endeavor. It started out with one person wondering why a Fusion-io drive runs so fast, but PG doesn't run any faster. Then another chimed in to say he was setting up PG with Fusion-io drives. Looks to be an interesting discussion. Now, if only my employer would let me buy some of those Fusion-io drives! BCNF to the rescue.
Here's the list.
The thread is titled: concurrent IO in Postgres?
A thread started today that's of interest to this endeavor. It started out with one person wondering why a Fusion-io drive runs so fast, but PG doesn't run any faster. Then another chimed in to say he was setting up PG with Fusion-io drives. Looks to be an interesting discussion. Now, if only my employer would let me buy some of those Fusion-io drives! BCNF to the rescue.
Here's the list.
The thread is titled: concurrent IO in Postgres?
22 December 2010
Django Played Jazz
The PostgreSQL site has been linking to this blog a bit recently, and he's refreshing. I'm going to spend some time looking into it. It could be there's some intelligent life out there after all.
Here's the start of today's entry, if you didn't slide off immediately:
Don't retrieve a whole row just to get the primary key you had anyway. Don't iterate in the app; let the database server do the iteration for you.
And he signs off with this (my heart went pit-a-pat):
but far better is to make the database do all the work
It is shocking how many coders still insist on their for loops in code. I mean, Dr. Codd made that obsolete, in the sense of providing an abstract declarative data model, in 1969/70 (the year depends on whether you were inside or outside IBM then). In a few years, Ingress and Oracle were live. I've concluded that MySql, PHP, java, and the web generally is what motivated the regression to COBOL/VSAM paradigms (that is, data is just a lump of bytes which can only be accessed through bespoke source code). One didn't need to know much, and frankly most webbies didn't and don't, about data to build some snappy web site that just saves and moves gossip. I suppose that's OK for most of the juvenilia that passes for web stuff, but not for the grown ups.
Here's the start of today's entry, if you didn't slide off immediately:
Don't retrieve a whole row just to get the primary key you had anyway. Don't iterate in the app; let the database server do the iteration for you.
And he signs off with this (my heart went pit-a-pat):
but far better is to make the database do all the work
It is shocking how many coders still insist on their for loops in code. I mean, Dr. Codd made that obsolete, in the sense of providing an abstract declarative data model, in 1969/70 (the year depends on whether you were inside or outside IBM then). In a few years, Ingress and Oracle were live. I've concluded that MySql, PHP, java, and the web generally is what motivated the regression to COBOL/VSAM paradigms (that is, data is just a lump of bytes which can only be accessed through bespoke source code). One didn't need to know much, and frankly most webbies didn't and don't, about data to build some snappy web site that just saves and moves gossip. I suppose that's OK for most of the juvenilia that passes for web stuff, but not for the grown ups.
16 December 2010
An Once of Prevention
Andy Lester is a Perl coder, and I loath Perl, so there has to be a good reason for me to mention him. And that reason is a posting of his, linked from Artima, which contains the following:
This person was one of those programmers who tried for the premature optimization of saving some typing. He forgot that typing is the least of our concerns when programming. He forgot that programmer thinking time costs many orders of magnitude more than programmer typing time, and that the time spent debugging can dwarf the amount of time spent creating code.
Now, when I was young and impressionable, the notion that a developer is paid to think, and not to type, was widely accepted. But I've certainly noted that in recent years, java perhaps the culprit, lots o typing is now the metric. LOC rules, even if most are worthless. Moreover, development by debugging is also normative. Ick.
What might this have to do with the point of this endeavor, you may be asking? Simply that declarative data is so much lazier than typing, and that a BCNF schema is easy to modify (since there aren't covariances to worry about). It does require some forethought, what Spolsky calls BDUF (you should look it up, if it's foreign), but that forethought isn't carved in stone, rather a strategic battle plan which permits many tactics. The "Agile" meme appears to have eaten its children, in that its zealots really, really do believe that all projects can be built from daily hacks by masses of coders. Ick; double Ick.
This person was one of those programmers who tried for the premature optimization of saving some typing. He forgot that typing is the least of our concerns when programming. He forgot that programmer thinking time costs many orders of magnitude more than programmer typing time, and that the time spent debugging can dwarf the amount of time spent creating code.
Now, when I was young and impressionable, the notion that a developer is paid to think, and not to type, was widely accepted. But I've certainly noted that in recent years, java perhaps the culprit, lots o typing is now the metric. LOC rules, even if most are worthless. Moreover, development by debugging is also normative. Ick.
What might this have to do with the point of this endeavor, you may be asking? Simply that declarative data is so much lazier than typing, and that a BCNF schema is easy to modify (since there aren't covariances to worry about). It does require some forethought, what Spolsky calls BDUF (you should look it up, if it's foreign), but that forethought isn't carved in stone, rather a strategic battle plan which permits many tactics. The "Agile" meme appears to have eaten its children, in that its zealots really, really do believe that all projects can be built from daily hacks by masses of coders. Ick; double Ick.
Subscribe to:
Posts (Atom)