I thought I would take some time this weekend to stage the ultimate test: the cross-join. Now, the tables I have at my disposal for this test, with enough rows to make it interesting, are Personnel and Dependants. While one would not expect to meaningfully cross-join such tables (excepting data for certain primitive societies/religions), they do serve the direct purpose.
So, for those not familiar, the cross-join (old syntax):
Select count(*) from personnel, dependants
I chose count() simply to remove the screen painting cost from the exercise; I merely want to measure the data cost. There are 1,200,240,012 synthesized rows.
I ran the query against the SSD database, and the HDD database (well, sort of).
The timing for SSD: 452.87 seconds, or about 8 minutes.
The timing for HDD: well, it never finished.
I initially ran both with 5 bufferpools, in order to force hard I/O in both cases. The SSD tables ran just fine. When I ran the HDD tables, it eventually errored out with a bufferpool exhaustion error. So, I increased the bufferpools for the HDD database to 100, and let 'er rip. 3 hours (about) later it errored out with a divide error.
A, somewhat, more fair test might be the cost of a range query between the two structures, that is a PersonnelFlatCross with the billion plus rows versus the normalized tables. If I can get DB2 to load the table, I'll give it a try.
IFF AI, as currently implemented, can intuit Special Relativity from just a clock tower and a moving tram will I take AI seriously.
Infinite granularity yields infinite complexity.
The Kleptocracy of Muskistan Devours You
The world is not linear.
-- Dr. McElhone/1974
[T]he massive push to relocate federal jobs outside the Beltway comes with the risk of kneecapping agencies through the loss of experienced workers and bogging them down in logistical minutia.
-- Curt Devine/2024 [that's the point: make the Damn Gummint incompetent, then destroy it because it's incompetent]
Scientists aren't vocal enough about science. There are large groups of people who think scientists are all frauds and who don't
believe in science, and they're being cultured by some of our far right-wing politicians, religious leaders and community leaders. -- Dr. McElhone/1974
[T]he massive push to relocate federal jobs outside the Beltway comes with the risk of kneecapping agencies through the loss of experienced workers and bogging them down in logistical minutia.
-- Curt Devine/2024 [that's the point: make the Damn Gummint incompetent, then destroy it because it's incompetent]
-- Drew Weissman/2022
To date, Microsoft is stating that organizations testing In-Memory OLTP have
seen transaction speeds improve by up to 30 times compared to past
performance, with the best performance gains achieved when the business
logic resides in the database and not in the applications.
-- Jonathan Watts/2015 [my emphasis]
-- Jonathan Watts/2015 [my emphasis]
I have had to explain and re-explain and re-explain and re-explain, you know, how relational databases work, what is an eigenvector,
what is dimensionality reduction.
-- Christopher Wylie/2018
... but Flash-based storage has such a different performance profile from rotating media, that I suspect that it will end up having
a large impact on filesystem design. Right now, most filesystems tend to be designed with the latencies of rotating media in mind.
-- Linus Torvalds/2007
-- Chris Date/2009
This Week's thought
[Scotland's HPV] vaccination program began in 2008, and so far the women who were vaccinated as 12- or 13-year-olds have shown not one single case of cervical cancer. And the rate in women who were vaccinated at later ages was 80% lower than the unvaccinated. Cervical cancer is almost completely preventable through such vaccination programs, and the anti-vaccine activists who have come out against them (such as the dangerous fool nominated to be secretary of HHS) can go to hell. -- Derek Lowe/2024 [the morons will be running the asylum, if we let them]
See you next week in a brand new show©Heckle and Jeckle
In a time of SSD, multi-core/processor, two terabyte memory and Optane App Direct Mode machines, there is no reason not to
build from BCNF data. Time to do what Dr. Codd demonstrated. Technology has finally caught up with the maths.
About
- Robert Young
- Been doing databases, both relational and others, as well as operations research since Hector was a Pup.
Shameless Plug
I know a little something about SQL, DB2 (preferably LUW, z/OS if the money's right), Oracle, SQL Server, Postgres, and database design in general. Mentoring/teaching/consulting on a short term basis, too. And a bit of stat analysis if it's R.
gnuoytr at rcn dot com.
gnuoytr at rcn dot com.
No comments:
Post a Comment