In the olden days, when I was a child, good parents instilled good values in their children. One was to not make fun of idiots; being an idiot was not the idiot's fault. Or, so the thought was. We know different now. Many idiots are so because they've chosen the path of ignorance, devolving further into idiocy. We see this in politics and IT, specifically the nexus with RDBMS.
I've been following the LedgerSMB postings (those that appear on the Postgres site, anyway), because the structure is similar to that taken by xTuple. Both, to greater or lesser extent, espouse integrity in the database, as opposed to client application code. Neither prevents editing in client screens, but both assert that the constraints, of record, are in the database; and can be duplicated in the client.
One of the fascinating aspects of this approach will be, soon, the picture when the likes of WebSocket is routinely available. Data centralization is always the future. When the IBM PC was first released, it was as a low powered programming work station; the "user" would use it to write "non-professional" code to suit his (not many hers at the time) needs. Scientists and engineers were the market, and IBM estimated 2,500 per year would sell. Then, along came 1-2-3, and the computer appliance was born; only now have I finally been vindicated in making that description. Soon, Netware showed up, to connect all those independent PCs to printers and file servers. The kodder kiddies really are convinced that The Cloud is something new. Some of them assert that it's really different this time; that this time it's about the cheapest hardware possible appearing like manna from heaven on demand. Foolishness. They'll find out, and another few hundred billion dollars will be wasted by the Fortune X00 chasing this as was when they chased J2EE applications.
From the very beginning, Dr. Codd had to cram the RM down the throats of heel dragging coders, who willfully ignore the fact that logic in code is just data compares made complicated. Do the logic with data where it lives. But, NO, that threatens the rice bowl of all them coders.
So, Chris Travers (of, and perhaps all of, LedgerSMB) has been posting about the application structure.
He published this post recently.
This led to the following post.
And that led to a supportive post from Joe Abbate.
Chris was too kind. Just as Ron Paul shouldn't be taken seriously, neither should Tony Marston. Neither has a clue, only zealotry. Joe, to his credit, took up this quote: "The database has always been a dumb data store, with all the business logic held separately within the application." His response was also too kind.
Marston has clearly never read, or if he did hasn't a clue what he read, Codd or Date. His statement is only true of coders back to AutoCoder and COBOL, not databases. As with the COBOL/VSAM/IMS folks that gave Dr. Codd the finger, Marston is just another guy who wants to keep writing lots of code, no matter how dumb that is. Prior to Dr. Codd's RM/RDBMS, there was IDMS and IMS; network and hierarchical database respectively. Both implemented logic in the database structure. Moreover, in 1968/9, IBM released CICS, likely the longest lived TPM (transaction processing monitor). A TPM does the read/write control for external code, largely COBOL with the IBM mainframes of the time. So, Marston is wrong in toto. Marston embodies the latter day Goths, who wish to take us back to the Dark Ages of their crude ancestors. A lot like Ron Paul, come to think of it.
The entire purpose of TPMs, pre-relational databases (IDMS, IMS, PICK), and RBDMS is to put data control with the data where it belongs. That is a threat to the horde of coders. Too bad.
[update]
For the record, I went back to see how the thread had gone, and couldn't keep my mouth shut. You have been warned.
03 January 2012
01 January 2012
Back to the Future
As those who've been following this adventure since the beginning must know, one part of the journey down The Yellow Brick Road involves the nature of application execution. I've been saying for many years, since long before I ever put fingers to blog, that the inevitable result of "progress" in application design is to replicate, albeit with pretty pixels, the world of *nix databases and the VT-100. The reason is simple: this paradigm provides the greatest control with the greatest ease of use.
The continuing problem with actually getting there is the disconnected nature of http. "Server push", as it's often called, hasn't been functional. So, imagine my surprise to see this Eckel post on Artima. Here's the Wikipedia entry, too. The upshot of WebSocket is a terminal on a long wire to the database. The winners in this world will be those who recognize that the high normal form database, with small bits of data per UI screen, will be the winners. This paradigm supports centrally edited data, to any screen. In other words, client agnostic.
In due time, perhaps with my consulting advice, organizations will realize what WebSocket means. There's that old joke: "Doctor, it hurts when I do this." "Don't do that." The easiest approach to dealing with Big Data is to not build stupid flatfile datastores. *nix databases, of high normal form, on multi-core SSD machines allow data to be sent to clients in whatever form needed. Smaller is better, of course, but massive joined rows can be sent. Not that one would want to, of course.
The continuing problem with actually getting there is the disconnected nature of http. "Server push", as it's often called, hasn't been functional. So, imagine my surprise to see this Eckel post on Artima. Here's the Wikipedia entry, too. The upshot of WebSocket is a terminal on a long wire to the database. The winners in this world will be those who recognize that the high normal form database, with small bits of data per UI screen, will be the winners. This paradigm supports centrally edited data, to any screen. In other words, client agnostic.
In due time, perhaps with my consulting advice, organizations will realize what WebSocket means. There's that old joke: "Doctor, it hurts when I do this." "Don't do that." The easiest approach to dealing with Big Data is to not build stupid flatfile datastores. *nix databases, of high normal form, on multi-core SSD machines allow data to be sent to clients in whatever form needed. Smaller is better, of course, but massive joined rows can be sent. Not that one would want to, of course.
24 December 2011
Head In The Cloud (or Somewhere)
Came across this posting whilst surfing, with the following:
"For example, one large package deliverer sees a 400% increase in its network traffic, database needs, and computing power over a single 45 day window. Utilizing a service such as Azure will allow them to pay for that excess capacity only in the 45 days it is needed, not the other 320 days a year when it is not utilized, but must be maintained, upgraded, and licensed."
Why folks persist in believing that there is a Free Lunch up in The Cloud??? The notion that seasonal/repetitive demand spikes for IT resources are uniformly distributed over a time period (day/week/year) is just silly. Yes, as a loss leader, a Cloud Vendor may choose to not add a Load Factor Penalty. In the beginning. But, that's not sustainable, because demand spikes aren't uniformly distributed. Cloud vendors will make the clients pay for all that idle storage and cpu (and a nice profit on the idle riches), you betcha. That package deliverer experienced the same spike as all of the retail chain. And all of the energy vendors. And so on. It's Econ. 101, folks.
"For example, one large package deliverer sees a 400% increase in its network traffic, database needs, and computing power over a single 45 day window. Utilizing a service such as Azure will allow them to pay for that excess capacity only in the 45 days it is needed, not the other 320 days a year when it is not utilized, but must be maintained, upgraded, and licensed."
Why folks persist in believing that there is a Free Lunch up in The Cloud??? The notion that seasonal/repetitive demand spikes for IT resources are uniformly distributed over a time period (day/week/year) is just silly. Yes, as a loss leader, a Cloud Vendor may choose to not add a Load Factor Penalty. In the beginning. But, that's not sustainable, because demand spikes aren't uniformly distributed. Cloud vendors will make the clients pay for all that idle storage and cpu (and a nice profit on the idle riches), you betcha. That package deliverer experienced the same spike as all of the retail chain. And all of the energy vendors. And so on. It's Econ. 101, folks.
21 December 2011
Icarus
The sky is falling. Oracle reported down, a bit, but, in particular, didn't report above expectations. Larry has been sly for a long time, in setting guidance low enough that bettering it is a piece of cake. Not this time. As I type, it's down 15%, and news is that much of the tech sector is getting the flu.
The knee jerk reaction: mortgage the farm and buy Oracle stock. IIIIIIIIIIII'm not so sure this time. Here's why.
Oracle didn't get quite the *new* software sales and *new* hardware sales. The latter is, by all accounts, due to customers waiting on the new machines during the quarter. The former is more speculative. The reports are vague. My take: given the aggressive pricing of Oracle RDMBS, ditto for MySql (yes, it's GPL, but Oracle blasted its support prices into the sky in the past year), and putting the screws to java adopters; folks are looking for a safer port.
On the RDBMS side, Postgres gets ever closer to Oracle, if you're not a Fortune X00 company (and even if you are, and building apps off the mission critical axis). Mainstream pundits are crying that "Da Cloud, boss, Da Cloud" is putting Oracle in an untenable position. It is said that cloud providers use dirt cheap components, soft and hard, and Oracle's RDBMS and Sun-ish machines are just too expensive Up There. As if being cheap were the best way to make money!? "Cheap goods sold dear" is an aphorism that's been around forever. The Cloud is shaping up that way, and if so, I'd avoid Fortune X00 companies that chose to put *my data* Up There. In this nascent era of Cloud, too many stories of wandering data to suit me.
What's really stupid about Larry's ploy: the Oracle RDBMS is built on an engine (the piece that actually does all the inserting and updating) which uses Multiversion Concurrency Control (MVCC, as it is known) which is better suited to the asynchronous nature of the Web/Cloud than the locker paradigm that most other (notably, not Postgres) RDBMS have been using for decades. They've been backing in, so to speak, MVCC support recently, but none is a true MVCC database. In other words, Larry has the proper mousetrap for the setting, but has managed to offend his customers. But, that's Larry's way.
Reports say that Oracle claims the shortfall is due to last minute non-signings. If so, then this is an aberrant glitch. Given that Fortune X00 companies are sitting on, by some accounts, more than $3 trillion, there's no macro reason to not buy new IT. Unless you're a Fat Man yearning for Famine.
The knee jerk reaction: mortgage the farm and buy Oracle stock. IIIIIIIIIIII'm not so sure this time. Here's why.
Oracle didn't get quite the *new* software sales and *new* hardware sales. The latter is, by all accounts, due to customers waiting on the new machines during the quarter. The former is more speculative. The reports are vague. My take: given the aggressive pricing of Oracle RDMBS, ditto for MySql (yes, it's GPL, but Oracle blasted its support prices into the sky in the past year), and putting the screws to java adopters; folks are looking for a safer port.
On the RDBMS side, Postgres gets ever closer to Oracle, if you're not a Fortune X00 company (and even if you are, and building apps off the mission critical axis). Mainstream pundits are crying that "Da Cloud, boss, Da Cloud" is putting Oracle in an untenable position. It is said that cloud providers use dirt cheap components, soft and hard, and Oracle's RDBMS and Sun-ish machines are just too expensive Up There. As if being cheap were the best way to make money!? "Cheap goods sold dear" is an aphorism that's been around forever. The Cloud is shaping up that way, and if so, I'd avoid Fortune X00 companies that chose to put *my data* Up There. In this nascent era of Cloud, too many stories of wandering data to suit me.
What's really stupid about Larry's ploy: the Oracle RDBMS is built on an engine (the piece that actually does all the inserting and updating) which uses Multiversion Concurrency Control (MVCC, as it is known) which is better suited to the asynchronous nature of the Web/Cloud than the locker paradigm that most other (notably, not Postgres) RDBMS have been using for decades. They've been backing in, so to speak, MVCC support recently, but none is a true MVCC database. In other words, Larry has the proper mousetrap for the setting, but has managed to offend his customers. But, that's Larry's way.
Reports say that Oracle claims the shortfall is due to last minute non-signings. If so, then this is an aberrant glitch. Given that Fortune X00 companies are sitting on, by some accounts, more than $3 trillion, there's no macro reason to not buy new IT. Unless you're a Fat Man yearning for Famine.
20 December 2011
A Warren-ted Search
One of the points "for further research" as I used to say when I was an academic, in the Triage exercise was using social media to measure outcomes. R has a library, twitteR, (yes, R folks tend to capitalize the letter at every opportunity), which retrieves some data. I was at first disinterested, since I don't have a twitter account. Thankfully, twits can be gotten without being a twitterer. Since Elizabeth Warren's campaign is just over the border, and sort of important in the grand scheme of things, I've been exploring.
Here's the entirety of the R code (as seen in an Rstudio session) needed to return the twits (1,500 is the max, which will prove troublesome when the battle is fully engaged):
The lines starting with > is the R code. The lines starting with [x] are the output. Here we have 9 twits.
Now, what do we do with the text? For that, I'll send you off to this presentation which came up in my R/twitter search (and is the source of what you've seen here), conducted in Boston. Missed it, dang. With slide 11, is the explanation of how one might parse the twits looking for positive/negative response. By the way, even if you're not the least bit interested in such nonsense, visit slide 29.
As I mentioned in Triage and follow-ups, getting the outcomes data is the largest piece of the work. Simply being able to "guarantee" the accuracy of twitter (or any other uncontrolled source) data, given the restriction on returned twits and such, will require some level of data sophistication; which your average Apparatchik likely doesn't care about. The goal, I'll mention again, isn't to emulate Chris Farley's Matt Foley and pump up a candidate no matter what the data say, but to find the candidate out of many most likely to win given some help. Whether Triage would be useful to a single candidate; well, that depends on the inner strength of the candidate.
Here's the entirety of the R code (as seen in an Rstudio session) needed to return the twits (1,500 is the max, which will prove troublesome when the battle is fully engaged):
The lines starting with > is the R code. The lines starting with [x] are the output. Here we have 9 twits.
Now, what do we do with the text? For that, I'll send you off to this presentation which came up in my R/twitter search (and is the source of what you've seen here), conducted in Boston. Missed it, dang. With slide 11, is the explanation of how one might parse the twits looking for positive/negative response. By the way, even if you're not the least bit interested in such nonsense, visit slide 29.
As I mentioned in Triage and follow-ups, getting the outcomes data is the largest piece of the work. Simply being able to "guarantee" the accuracy of twitter (or any other uncontrolled source) data, given the restriction on returned twits and such, will require some level of data sophistication; which your average Apparatchik likely doesn't care about. The goal, I'll mention again, isn't to emulate Chris Farley's Matt Foley and pump up a candidate no matter what the data say, but to find the candidate out of many most likely to win given some help. Whether Triage would be useful to a single candidate; well, that depends on the inner strength of the candidate.
19 December 2011
Ya Can't Get Theah From Heah
I'm working my way through Wilkinson's graphics book, and right there, on page 404 (could there possibly be a better page for this?) is this:
"The price paid for this efficiency is the loss of flexibility. We cannot examine relations not represented in the hierarchies. Navigation through OLAPs is quite difficult, which is why so many graphical models for representing them have been proposed."
Now, this is a graphics coding guy; he created SYSTAT. He's not, so far as I know, an RM guy. But he gets it.
"The price paid for this efficiency is the loss of flexibility. We cannot examine relations not represented in the hierarchies. Navigation through OLAPs is quite difficult, which is why so many graphical models for representing them have been proposed."
Now, this is a graphics coding guy; he created SYSTAT. He's not, so far as I know, an RM guy. But he gets it.
16 December 2011
Lies, Damn Lies, and Statistics
The Other-R folks have posted a recent entry which references an EMC paper (here if you follow the breadcrumbs) on the state of Data Analysis and Business Intelligence, from the point of view of practitioners. The blog post makes some useful points, but misses some.
I'm referring to the graph in the original post, which is on page 3 (in my screen) of the EMC paper.
What this graph tells me, mostly, is that BI folks are still tied to MS, Excel in particular. Data analysts, not so much; although they'll be tied to corporate policy in such venues.
A few words about each.
Data Storage: SQL Server is tops, which means that most folks, in both camps, are tied to corporate group level machines, not the Big Iron. It's been that way for decades; the analysts have to extract from the Big Iron, and crunch on their own PCs. The categories Other SQL, Netzza, and Greenplum leave room for the Triage with PL/R approach, since the latter are explicitly Postgres and Other SQL is likely as much Postgres as MySql (yuck!). The category is, possibly, misleading if one jumps to the conclusion that companies are MS centric with their data.
Data Management: No real surprise here. Excel is the tool of choice. Way back when I was teaching PC software courses, 1-2-3 was the spreadsheet of choice and all data went through it, and Excel inherited the mindset that a spreadsheet was sophisticated analysis. It is a bit unnerving to realize that so much of what corporations decide is supported by such drek. Note: the BI folks, in the past executive assistants and "secretaries", still use spreadsheets a lot. The Data folks, the other way round. There is small comfort in that. The presence of BASH (or Korn or ...) and AWK (Python and Perl too, but not quite so much; each has bespoke language I/O in the mix) is interesting, in that it means that a fair amount of data is clear text ASCII files. Think about that for a second.
Data Analysis: Clearly, the Data folks use stat packs while the BI folks mostly don't. SAS and SPSS and Stata leading says that the EMC client base is largely large corporate, which isn't a surprise. What is a surprise is the absence of Excel. On the other hand, in the original paper is this (next by each the graph): "While most BI professionals do their *analysis* and data processing in Excel, data science professionals are using SQL, advanced statistical packages...", which corresponds to my experience (emphasis mine).
Data Visualization: The absence of R is suspect, as any R user would understand.
And, finally, this has nothing to do with Big Data, in any case. BD is just another attempt to money-spin by those with an agenda. Janert, in his book "Data Analysis...", makes clear that BD isn't worth the trouble (my inference). The point being that population data, which is what BD offers, is just descriptive stats, and smart data folks aren't interested in descriptive stats. Sports fans, well yeah.
I'm referring to the graph in the original post, which is on page 3 (in my screen) of the EMC paper.
What this graph tells me, mostly, is that BI folks are still tied to MS, Excel in particular. Data analysts, not so much; although they'll be tied to corporate policy in such venues.
A few words about each.
Data Storage: SQL Server is tops, which means that most folks, in both camps, are tied to corporate group level machines, not the Big Iron. It's been that way for decades; the analysts have to extract from the Big Iron, and crunch on their own PCs. The categories Other SQL, Netzza, and Greenplum leave room for the Triage with PL/R approach, since the latter are explicitly Postgres and Other SQL is likely as much Postgres as MySql (yuck!). The category is, possibly, misleading if one jumps to the conclusion that companies are MS centric with their data.
Data Management: No real surprise here. Excel is the tool of choice. Way back when I was teaching PC software courses, 1-2-3 was the spreadsheet of choice and all data went through it, and Excel inherited the mindset that a spreadsheet was sophisticated analysis. It is a bit unnerving to realize that so much of what corporations decide is supported by such drek. Note: the BI folks, in the past executive assistants and "secretaries", still use spreadsheets a lot. The Data folks, the other way round. There is small comfort in that. The presence of BASH (or Korn or ...) and AWK (Python and Perl too, but not quite so much; each has bespoke language I/O in the mix) is interesting, in that it means that a fair amount of data is clear text ASCII files. Think about that for a second.
Data Analysis: Clearly, the Data folks use stat packs while the BI folks mostly don't. SAS and SPSS and Stata leading says that the EMC client base is largely large corporate, which isn't a surprise. What is a surprise is the absence of Excel. On the other hand, in the original paper is this (next by each the graph): "While most BI professionals do their *analysis* and data processing in Excel, data science professionals are using SQL, advanced statistical packages...", which corresponds to my experience (emphasis mine).
Data Visualization: The absence of R is suspect, as any R user would understand.
And, finally, this has nothing to do with Big Data, in any case. BD is just another attempt to money-spin by those with an agenda. Janert, in his book "Data Analysis...", makes clear that BD isn't worth the trouble (my inference). The point being that population data, which is what BD offers, is just descriptive stats, and smart data folks aren't interested in descriptive stats. Sports fans, well yeah.
Subscribe to:
Posts (Atom)