20 June 2018

Paht Country

"My girl left me, my dog died, and my truck won't start." Or something like that is the essence of country music. Now we can add, "I's so fat,I cain't see my dick no more."

The reality of the fat country sluggard is true; let's go to the videotape. Not that anyone paying attention should be surprised. Orange Julius Caesar rode white grievance and white entitlement to his throne. Most of his voters were so conked out on opiates that they likely didn't know what, exactly, Orange Julius Caesar was up to. They just knew he'd put those darkies in their place; while he picks their pockets, to be sure. Make America White Agin. May be enough will croak before the next election? Thinning the herd; just what Stephen Miller wants. Just not his herd, naturally.
Country folk are being hit harder by the U.S. obesity epidemic than city dwellers, two new government studies show. Nearly 40 percent of rural American men and almost half of rural women are now statistically obese, U.S. Centers for Disease Control and Prevention researchers reported Tuesday.
...
For example, people who had a college degree were less likely to be obese or severely obese.

So, here's the question: how much longer will the heads of CDC and the unit(s) doing the research still be employed? I bet they're gone by the end of the week.

Not On My Watch

The outrage over asylum seekers' treatment by Orange Julius Caesar and his xenophobic cabal is genuine. But the wailing that "this isn't America" and such is nonsense. For most of its history, my country has been xenophobic and racist. Moreover, fish belly white folks, who came from countries considered lower than England and Scotland, the Irish as one example, got what Orange Julius Caesar is dishing out today. (This is source.)
Many native-born Americans claimed that "their incessant childbearing [would] ensure an Irish political takeover of American cities [and that] Catholicism would become the reigning faith of the hitherto Protestant nation."

Sound just the bit familiar? Do you want Muslims and brown people taking your country away from you? This is not new. If you don't read the wiki article through, that was in the mid-1800s during the Irish potato failure. For much of the 1800s, job postings carried, not EOE, but NINA. No Irish Need Apply. Orange Julius Caesar is just channeling age old xenophobia.

Site Launch

What's intelligence? What's insight? I'd say a close enough definition is the ability to see a real world truth before there's measurement. This was how Einstein figured out special relativity; observing a bell tower from a tram. Some refer to this process as intuition. Others as real intelligence, vis-a-vis the artificial version.

Which brings me to a David Leonhardt from the last week. One of the bones I've had to pick with the Right, who know they're lying, is their assertion that small business was, is, and will be the savior of American economy. Orange Julius Caesar went on a tirade at a small business conference yesterday. All one need do, over the last three or four decades is see what has happened in one's town, city, or state. The Big get Bigger and the Small get crushed. As Leonhardt says, mainstream punditry, and analytics as well it seems, hasn't been measuring. So he set out to do so. Good on him. I already knew it, of course. But it's heartwarming to see my insight validated.
Today, companies with at least 10,000 workers employ more people than companies with fewer than 50 workers.

There's a, as the Brits put it, knock-on effect of behemothing (that's now a word) of American business. In times past, economists and other analysts have made the point that the blue collar middle class emerged due to the confluence (or symbiosis) of big business and big unions. Auto workers made a middle class living because The Big 3 had oligopoly power, and the unions had the legal clout to force a sharing of that power in a form of enforced profit-sharing. The workers got a share of the vig. So, between the Right Wing destruction of unions and off-shoring of jobs (no, China didn't steal them, American corps. sent them), the blue collar middle class no longer exists. And those knuckleheads are convinced that a big city con man will make it all better? Here's the first paper that came up in a simple search.

You really want to MAGA? Bring back strong unions. With more market power held by large corporations, there's more 'rents' to share with workers. "Vote for George O'Brian. Get poor Charlie off the MTA!"

Ah, Truth

One of my many pet peeves with the right wing set is that they claim the increase in life expectancy since the beginning of Social Security is proof positive that SS is unsustainable. Just look at how many years have been added!!!!!! Well, as has been reported here a number of times, life expectancy increase, measured from any time before about 1960 (and any place), is not due to people of 60 or so living all those many years longer. Au contraire! It's been that young folks, from birth, now make it through young adulthood standing up. Vaccines and sulfa drugs and cleaner water and a host of other public health measures mean that an American baby in 2018 is better off than an American baby born in 1932. But not better than most OECD countries.

Here is a fellow thinker.
Life expectancy is calculated directly from death rates. And mathematically speaking, changes in infant mortality have a much greater impact on life expectancy than do changes in death rates in any other year.

Or, put more starkly:
More realistically, if death rates in the first year went up to 250 out 1,000 (which would be around the worst current day level, but well within historical ranges), life expectancy comes down from 79 to around 50, despite death rates at all other ages staying the same as in 2015 France.

17 June 2018

Thought For The Day - 17 June 2018

Just finished Bourdain's next but last (so far, CNN hasn't said how many season 12 episodes were finished and if so would be broadcast) episode, and did some innterTubes looking about, and came across a CNN piece where an anonymous CNNer allowed has how they should continue the show with a replacement reporter (or whatever Bourdain was; he denied being a journalist). I vote for Paul Theroux. I doubt he would do it, but he's as close to Bourdain as I've encountered.

Other nominations welcome.

13 June 2018

Neo Barnum

There are times, sometimes too many, when quant looks sheepish. This is one of those days. I have a few rules of thumb. One is that biopharma from the Boston-Cambridge axis is legit, while that of the San Diego version is garbage. I don't recall an instance where the latter has failed. But here's a case of the former proving false.
The biotech industry's Arctic ice-cube salesman Cristoph Westphal managed to raise $86 million Wednesday night in an initial public offering to develop a treatment for leg cramps made from food ingredients commonly found in your spice drawer and refrigerator.

That's one guy's assessment from three years ago. Here's today's results
... announced that the Company is ending its ongoing Phase 2 clinical trial investigations of FLX-787 in amyotrophic lateral sclerosis (ALS) and Charcot-Marie-Tooth (CMT) due to oral tolerability concerns observed in both studies, in a subset of patients being treated, with the oral disintegrating tablet formulation at 30 mg, taken three times a day.

What was it Barnum said...? It should be noted that cinnamon, by folklore, is a cure for diabetes. And my PCP swears by magnesium, not potassium, for cramps. Works for me.

09 June 2018

Phil and Tony

In the early 1960s, Bobby Zimmerman left Lake Wobegon, became Bob Dylan and was annointed the angry young man singer-songwriter doing protest songs. In due time, we all figured out that he was just another pop singer. About the same time, a real protest singer, in the folk tradition, emerged. His name was Phil Ochs. In 1976, age 35, he hanged himself in the bathroom of his sister's house. His records, when he could get them made, didn't sell. Live gigs had dried up. He despaired of making a living. You can get his 'greatest hits' from Amazon. And you should.

Friday Anthony Bourdain did the same, only in a five-star hotel in Kaysersberg, France while recording an episode of "Parts Unknown" with Eric Ripert, one of his best friends. He had his crew with him. He was not isolated. He was wildly successful. But he left for parts unknown. Regular readers recall that I've said his show the best thing on the TeeVee, so it's unnerving to consider TeeVee without him.

You'll find all manner of tributes, including one I missed on CNN last night; I didn't know it was on. It'll be run Sunday at 10pm. You should watch. The scheduled episode will go on at 9pm. Whether the remaining completed episodes will be shown, I don't yet know. It is a bit unsettling watching the 'favorite episodes' as I write this. Still, it would be a shame if CNN withholds completed episodes.

JOOQ

I can't say I know much about JOOQ. Seen it referenced here and there, but this piece just appeared, and I am heartened to see someone in the java world make a strong case for 'database first, code last'. There is hope for Western Civilization.
The real "truth" of your database schema, and the "sovereignty" over it, resides with your database. The database is the only place where the schema is defined, and all clients have a copy of the database schema, not vice versa. The data is in your database, not in your client, so it makes perfect sense to enforce the schema and its integrity in the database, right where the data is.

The only point missing from the article: an Organic Normal Form™ schema will be, to some small epsilon, immune to schema update modification problems, since tables/columns in normal forms are, more or less, orthogonal. They can be, if one wishes, fully orthogonal, and when so, there exist no side-effects. The re-generated client code will be restricted to the manifestation of such tables. Easy peasy.

Going database first does demand that rogue coders aren't welcome in development. Database first development demands careful thought aforehand, an approach not welcomed by the coding community. Coders want to keep making mistakes, finding them in the debugger, patching. Rinse repeat. There is a better way.

And, there isn't any such thing as impedence mismatch. That's been dealt with here more than once. This is spectacularly true in the java world of actionObject/dataObject. It's a witch hunt.

04 June 2018

A Hop, A Skip, and A Jump - part the fifth

Yet another carrot for the stew, a paper from last year directly discussing RDBMS on PM. Of all the references so far in this series, this one is *a must read*. Really. Not just because it validates every notion flowing from my noggin. No, not at all.
In this tutorial, we provide an outline on how to build a new DBMS given the changes to hardware landscape due to NVM.

Yum. And it's from what was Carnegie Tech (which accepted me, but I ended up going elsewhere), one of the first Computer Science curriculums!
In the case of memory-oriented DBMSs (e.g., VoltDB, MemSQL), they contain components for overcoming the volatility of DRAM. Such components may be unnecessary in a system with byte-addressable NVM with fast random access.

Exactly why the announced kludges for *nix/Windows aren't the end. The OS/400 turns out to be the winner. In some form.
Peloton is an open-source HTAP DBMS that we are building that is designed from the groundup to use NVM. Our intended audience are developers, researchers, and practitioners with knowledge of DBMS internals. They do not need any in-depth background or experience with NVM.

Here we see another step closer to the OS/400 model:
To cope with these shorter NVM latencies, Microsoft and Linux are adding support for direct access storage (DAX) in Windows Server 2016 [48] and Linux 4.7 [1], respectively. With DAX, a DBMS directly allocates and uses NVM without an intervening filesystem. This requires only one copy between the file and the user buffers, thus improving the file I/O performance by an order of magnitude compared to block-oriented filesystems.
[my bold]

Recall an earlier part of this series' mention that transaction semantics will be different with PM? Make it so:
The write-ahead logging (WAL) protocol supports efficient transaction processing when memory is volatile and durable storage cannot support fast random writes [58, 39, 33]. But this assumption causes unnecessary performance degradations in a DBMS with NVM storage [14]. Consider a transaction that inserts a tuple into a table. A DBMS first records the tuple's contents in the log, and it later propagates the change to the database. With NVM, a DBMS can employ a logging protocol that avoids this unnecessary data duplication. The reason why NVM enables a better logging protocol than WAL is two-fold. The write throughput of NVM is more than an order of magnitude higher than that of an SSD or HDD. Further, the gap between sequential and random write throughput of NVM is smaller than that in SSD and HDD. Hence, a DBMS can flush changes directly to the database in NVM during regular transaction processing [15, 14, 12, 64, 40, 62, 80].

The Future is Now.

A Hop, A Skip, and A Jump - part the fourth

During my near awake status, it occurred to me that the I5 System (nee, AS/400) would be the perfect candidate for PM. The machine after all, doesn't use a file system in the *nix/Windows sense, but an object store (in the pure data sense). It's machine designed, however well applications exploit it, to be Relational.

Turns out, some folks already have. And not just with the Optane announcement.
But key here is that when the object gets created, it is also created in a system-wide address space that IBM i calls Single Level Store (SLS). In the context of this article on persistent storage, it happens that SLS is also persistent; if you restart the OS — say after a power failure — the objects remain with the same address. Said differently, every byte of data out on your hard or solid-state drive has a corresponding location-independent high-level address used by the OS. If your program wants to reference that byte in whatever object, your program just uses the byte's address, no matter how many times the OS was restarted.

Some folks consider IBM to be a hidebound dinosaur. And, to some extent that's right. But here's a bit of history of the I5 OS:
IBM's design of the single-level storage was originally conceived and pioneered by Frank Soltis in the late 1970s as a way to build a transitional implementation to computers with 100% solid state memory. The thinking at the time was that disk drives would become obsolete, and would be replaced entirely with some form of solid state memory. IBM i was designed to be independent of the form of hardware memory used for secondary storage. This has not come to be, however, because while solid state memory has become exponentially cheaper, disk drives have also become similarly cheaper; thus, the price ratio in favour of disk drives continues: very much higher capacities than solid state memory, very much slower to access, and much less expensive.
[my bold]

A few years ahead of his time.

03 June 2018

A Hop, A Skip, and A Jump - part the third

Some more spelunking. Turns out that linux and Windows have existing mechanisms to leverage persistent memory under the covers of block-mode IO; i.e. mmap and DAX. Such might be sufficient to get by in the short term. Here's Intel's report.
Both Windows* and Linux* support memory-mapped files, a feature which has been around for a long time, but is not commonly used.

Interestingly, the BDFL of Progress database said, around 1990, that if your database didn't use mmap, get another database. That long ago.

Intel also has a FAQ, which has some interesting tidbits.

02 June 2018

A Hop, A Skip, and A Jump - part the second

Do you remember that Date, at least by his last version (re-gifted previous ones along the way so I can't vouch for them) of "Introduction..." included a section called "ACID dropping" in chapter 16? I wasn't convinced way back in 2003 when the edition was published. With hardware prior to Optane, updates are a serial process, thus the need for ACID.

With what's now being named "Persistent Memory", of which Optane is Intel's version, we have to re-think transaction semantics. Let's assume, for the sake of argument, that 4TB Optane machines with a slot or two of DRAM and an SSD for OS and applications storage (herein, Godzilla) are the norm in Server World. Who are the losers? I'll argue that ACID and MVCC are the most obvious candidates for being no longer relevant.

With HDD as permanent storage, the hop, skip, jump process is required to process updates. ACID is the mechanism to manage concurrency in a locking protocol (MVCC still locks, just late rather than early). With Godzilla as the platform for RDBMS systems, Codd's "all at once" is natural; no hop, skip, jump. Will it be faster than hop, skip, jump in all cases? I've no idea. We don't yet have on-the-record speeds for Optane. But my suspicion is that most of the code in relational/SQL engines goes away, particularly for schemas that are in Organic Normal Form™. Explicit locking, in the engine, won't be needed, since the OS takes care of memory locks, and that's all there are. Since updates become visible essentially immediately, the locking avoidance of MVCC is, also, not needed.

Godzilla won't be birthed for a while. *nix/Windows and datastore based applications will need to be modified, or more likely forked, to support/exploit Persistent Memory. File system no longer matters, objects (in the older sense) are all that exist. Search can, and should, be relational! No more silly one-way to the bottom hierarchies. And, that's not far fetched, since such machines existed in the past (not sure if the current progeny are the same), called AS/400, itself a development from earlier machines. The OS included an embedded RM-ish object engine, which got renamed DB2/400 for a while. In one incarnation of my employment, an AS/400 hosted application lost to one on an early RS/6000 and Progress. Worked out OK, since that company is still running the code.
Unlike the "everything is a file" feature of Unix and its derivatives, on IBM i everything is an object (with built-in persistence and garbage collection).
Remember, the hierarchy file system was concocted to support a server/terminal hosted word processor: Unix. Since Bell, as most any corporation, was built as a top-down org chart, so too was its word processor. All before Codd had released his first version, and when the state of the art datastore was IBM's mainframe IMS. Yes, the iconic hierarchical "database".

31 May 2018

A Hop, A skip, and A Jump

Free at last, Free at last, Thank God almighty we are free at last.
-- Martin Luther King

OK. So what does such a quote have to do with a site, titularly at least, devoted to the RM, stats, and data generally? Well, this new report on Intel's Optane DIMM.

[On a side note, the earlier piece on Optane SSD has a comment from the author to the point that the byte addressability of Optane is of no value for files. That's true for fixed sector non-mainframe HDD, and later SSD, but not true for CKD DASD:
IBM CKD subsystems initially operated synchronously with the system channel and can process information in the gaps between the various fields, thereby achieving higher performance by avoiding the redundant transfer of information to the host. Both synchronous and asynchronous operations are supported on later subsystems.
]

Current SQL engines do their work with a hop, skip, and a jump. Optane, with proper motherboard, chipsets, and (heavily?) re-written applications can go from cpu to datastore. We could, then, do what Dr. Codd said: "all at once". Yum.

The current paradigm is from cpu registers to on-chip cache to memory to SSD/HDD; more or less a hop, a skip, and a jump. Now, for SQL databases (or any serious datastore), what generally happens is that the engine writes to the trans log (more or less synchronously), which is at some later time flushed to the "permanent" datastore on SSD/HDD. A system/application crash is supposed to only lose open transactions; all commited trans exist in durable storage either in the trans log or named tables. Wouldn't it be cool if transactions commit to durable storage immediately?

As the piece says:
Intel has been laying the groundwork for application-level persistent memory support for years through their open-source Persistent Memory Development Kit (PMDK) project, known until recently as NVM Library. This project implements the NIA NVM Programming Model, an industry standard for the abstract interface between applications and operating systems that provide access to persistent memory. The PMDK project currently includes libraries to support several usage models, such as a transactional object store or log storage. These libraries build on top of existing DAX capabilities in Windows and Linux for direct memory-mapped access to files residing on persistent memory devices.

Application re-write may be non-trivial:
Intel will be offering remote access to systems equipped with Optane DC Persistent Memory so that developers can prepare their software to make full use of the new memory. Intel is currently taking applications for access to this program. The preview systems will feature 192GB of DRAM and 1TB of Optane Persistent Memory, plus SATA and NVMe SSDs. The preview program will run from June through August. Participants will be required to keep their findings secret until Intel gives permission for publication.

But for the RM world, Optane offers the ability to do what Dr. Codd said, "all at once".

One might ask, in a semantic sense, how a SQL engine on such a machine might do its work. Let's chew on that.

First, what about the divide between trans log and named tables? Do we still need a trans log? The blockchain example, which is a database-less trans log, suggests we do. For audit purposes, it is needed, too. Does the trans log still need be the bottleneck to the named tables? Not with such Optane storage. The engine would just do immediate writes to the log and the table(s); that's the duration of the transaction.

Second, do we still need buffers? May be, but may be not. The purpose of buffers is to mediate between fast memory and slow disk, now SSD, but still slower. According to the report, 512GB DIMMs will be available. Current boards go to eight slots, which works out to 4TB. How many applications need more than that? Google and Amazon and Facebook. Commercial applications? Not so much. Taking into account the data reduction side-effect of 3 or 4 or 5 normal form, may be most would be happy with that much storage.

Third, typical applications don't keep all data in hot storage even now, so 4TB looks to cover many, if not most, use cases.

Fourth, some DIMM would be normal DRAM to hold OS and application code and code's invariant data; this appears to be supported according to the report and comment stream. It would certainly make sense to do this.

So, how would a SQL engine work with such a machine? The engine and its data would reside in DRAM, while the tables and (active) log in Optane. By active log, we mean that the engine would flush completed transactions to SSD as needed; in reality, only long running transactions (you don't write such monsters, do you?) would be in the active log. There would be no need for memory buffers, but the notion of locks needs to be considered. Current practice whether locking or MVCC relies on memory buffers to "improve" performance. Simple, single row, update would be written directly to table and log. Normal memory locking would suffice. Multi-row/table update? Umm. In the MVCC case, it depends on multiple images of rows to segregate transactions, but would there still be any point to MVCC semantics with Optane storage? For now, I'd say not. Since update (could/should) happens "all at once", collision doesn't occur; each client sees the current state of each row all the time, modulo duration of actual write. Isolation amounts to memory locks. Cool.

Years ago, diligent reader will remember, I had occasion to work with TI-990 machines. One ran a Ryan McFarland COBOL application on a TI machine while the other ran the chip version on a custom board/OS/language (ah, those were the days). What was (nearly?) unique about the 990 architecture was that it was registerless, sort of. All code and data resided in memory, and there were three cpu registers, one of which was a pointer to the current application's base address in memory. A context switch required only resetting these registers. Each application retained its context in situ. I wonder whether TI will resurrect it?
The TI-990 had a unique concept that registers are stored in memory and are referred to through a hard register called the Workspace Pointer. The concept behind the workspace is that main memory was based on the new semiconductor RAM chips that TI had developed and ran at the same speed as the CPU. This meant that it didn't matter if the "registers" were real registers in the CPU or represented in memory. When the Workspace Pointer is loaded with a memory address, that address is the origin of the "registers".

Whether Optane has much value-add for most applications, I don't know. But for heavy multi-user/single datastore applications such as SQL engines, yeah man!!

This, of course, is merely first-pass thought experiment. More to come.

25 May 2018

The Object of My Affection

Return with us now to the thrilling days of yesteryear. If you're old enough, you remember that as the intro to "Lone Ranger" episodes. One might also see that as the devolution of application development. So, here goes.

Recall the Holub Mantra (my description), this "Bank of Allen" tale. The signature observation:
The maintenance manager is happily sleeping in the back office instead of running around changing ROMs.

Holub is talking about regular application development, independent of datastore. But the principle applies in spades to databased applications.

The development of the relational model was driven by maths, specifically predicate logic. It was not, as many infer, data reduction; which does come as part of the package. But the whole point is data integrity. The primary way to ensure data integrity is to limit the number of hands who can mess with it. As Holub points out, if one models an ATM network based on object capabilities, the datastore is the controller of the data. That is how it must be. In time, Holub gave in to the innterTubes meme, alas. That might have been defensible back before highspeed connectivity, but not now.

If there is just one point of control, then there's just that one point of modification and maintenance. That's why the manager doesn't have to run around changing ROMs. But the driving point of object oriented design and programming is that an object encompasses both data and function, aka methods. What has happened, especially in the java world, was the re-invention of the language, called either javBOL or COBava. Same old procedure/data protocol from the 50s. Swell.

Every now and again, one sees a crack in the wall of reactionaries. Here's a recent example. The author doesn't reach the logical end of the argument, but does show more insight than usual.
Too often, though, business logic is built and added late in the process, forcing it into whatever nooks and crannies are available. While this duct-tape approach sometimes works, it makes the resulting system difficult to maintain when the business logic is spread across many moving parts in the data architecture.

The standard response from the COBava folks is, "it's impossible to design an application in the first place, so let's just build it like a Frankenstein monster." Or, parts is parts. Remember, all code logic comes down to data compares. Period.

22 May 2018

Your Tax Dollars at Work

Remember all that braying about how corporations would use offshore profit to add more workers if/when they brought it back? You do remember? Did you believe it? You shouldn't.
16:17 CW Curtiss-Wright to utilize $50 million in repatriated foreign cash to expand 2018 share repurchase program (133.92 +2.03)

Board of Directors has authorized an additional $50 million for share repurchases in 2018, to begin immediately via a 10b5-1 program, by using cash that the Company has begun repatriating from its foreign subsidiaries.

Earlier this month, the Company repurchased more than $12 million in shares opportunistically through a supplemental share repurchase program and has ~$36 million remaining under the current share repurchase authorization. Since early 2014, the Company has repurchased more than 7 million shares for an aggregate purchase price of nearly $550 million

19 May 2018

Mask of The Red Death

Have you noticed that the "unmasking" meme has fallen away from the Orange Julius Caesar cabal's whining? Why might that be? Reporting that the FBI had an informant talking to Page, Papadopoulos, and Clovis very early in the investigation has created the spy meme. Hmmm. May be they were warned that such talk will only speed up the inevitable end.

Last night's MSNBC nightly feed included Maddow's refreshing of the Nixon Enemies List, in the context of Orange Julius Caesar's secret attempt to coerce the PO into doubling(!) Amazon's delivery charge. That shouldn't be surprising. What I noted was Maddow's use of video and audio from the Watergate investigation. The tapes proved that Nixon, et al, really did subvert the law. The outing of the Amazon/PO corruption was offered as today's version. But I disagree.

From the moment that Comey told us that the FBI had an open investigation into Trump/Russia, I inferred the jig was up. Maddow came this close to closing the circle. Have I mentioned in the course of these missives that I spent some time working for Jack Anderson? Well, yes. And you can find the articles I worked on if you look hard enough. This was after Anderson was co-opted by Reagan, but he had long since quit doing much reporting or writing, leaving that to the staff. I got to meet some spooks. That was fun.

My time in DC convinces me that the intelligence community, the NSA in particular, has the Trump Tapes. It's perfectly legal for NSA/CIA/FBI to sweep up Americans who engage with foreigners in treasonous activities. Doesn't matter where their feet are at the time. The cabal knows what it did, and was counting on not winning to keep their activities from public view. Thus the FISA nonsense. They just wanted the money. Just as Jared extorted from Qatar. In due time, the community will release the tapes. Whether Orange Julius Caesar spends time in prison is the only question.

07 May 2018

The Book I Didn't Get Around to Writing

On more than one occasion, I've bemoaned the silliness of Big Data. So have some others, every now and again. This morning's email brought a spiel from Amazon, with some books I might like. There is one: "Big Data, Big Dope".
While others have written about the dangers of Big Data, Stephen Few reveals the deceit that belies its illusory nature. If "data is the new oil," Big Data is the new snake oil. It isn't real. It's a marketing campaign that has distracted us for years from the real and important work of deriving value from data.

Exactly. OTOH, I worry that BLS/Census survey efforts have been corrupted by Orange Julius Caesar's minions in the agencies. Remember, it was the political appointees who made the case for WMD, not the grunts doing the work. When something is too good to be true, it likely is. One doesn't need all the population to make appropriate data-driven decisions. But it is easier to put a thumb on the scales when sampling is involved. It was bad stratification that led Crooked Hillary to believe she had it in the bag.

06 May 2018

You Must Read This

Finally, a more famous than I data nerd tells the truth.
Instead of learning, I was spending 2-3 hours a day mindlessly scrolling through Facebook looking at cat videos and political hate speech. I was also posting family pictures of our adventures and eagerly awaiting the dopamine reward of getting likes from friends and family. My attention span was getting shorter and shorter by the day.

Now, lo those many decades ago, one of my graduate profs noted that you'll know you're working a profession when you spend 80% of your time thinking. And the greatest asset you acquire while a grad student is your library. Not so much these days.

28 April 2018

What's a Blit?

It was many years ago today that Dr. McElhone came to say a new word (to me, anyway): blit. What's a blit, you might ask? Well, I'll tell you: five pounds of shit in a four pound sack. Being from Texas and Nevada, it was sack and not bag. Kind of the same vein as, "Girl, you're as dumb as a sack of hair!" So, what's up with blit and hair? The death spiral of capitalism.

"Robert, you're out of your bleeing mind!"

Could be. But follow along with me. More than once, these pages have described the problem of heavily capitalized production, namely the strait jacket of amortization and the need for stable, if not growing, demand. As more cost is claimed by capital, the more difficult it is to manage average cost, and thus price (modulo customer discrimination). Ya gotta pay the vig. In the Olde Days, as demand softened you just fired employees. Average cost dropped to more closely match market clearing price. Capital heavy production makes that anywhere from difficult to impossible, since they ain't no bodies to fire.

"But Robert, that never actually happens!" Thanks to all the news that's fit to print, we get a case study.
The Headline:
For Cancer Centers, Proton Therapy's Promise Is Undercut by Lagging Demand

Can't get more on point than that. Proton therapy is a five pound cost in a four pound sack of demand. The shit drips over the top. Yuck.
But about 30 years after the Food and Drug Administration first approved proton therapy for limited uses, doctors often hesitate to prescribe it and insurers often will not cover it.

That means there simply may not be enough business to go around.

Or, as has been argued in these pages more than once: if only the 1% have healthcare, even they won't be able to afford it. Another bone oft picked here: Say's Law, aka supply creates it's own demand. Laffer being the modern zealot of that knucklehead idea. Were it true, we'd never have recessions, much less panics and depressions.
At Indiana, he added, "we began to see that simply having a proton center didn't mean people would come."

What's even worse is that such machines seem to be no better than regular treatment.
But its pinpoint precision has not been shown to be more effective against breast, prostate and other common cancers. One recent study of lung-cancer patients found no significant difference in outcomes between people receiving proton therapy and those getting a focused kind of traditional radiation, which is much less expensive.

Remember 2005, when all the smart people in the room told us that skyrocketing house prices were the new normal and that there had never been, and could never be, a nationwide stall in home sales? Remember?

26 April 2018

Old Guys Finish First

Well, now we know, again, that youth ain't all that neat. Jim Keller is now at Intel. Hardware guru is a fair description. At 58, he's had the time to see and do a bit of work. Be still my heart.

We Save Lives

The constant refrain from the most profitable industry (not counting the FIRE parasites), "we need sky-high prices because we save lives", is, once again, belied. Here's the story.
Those who received pumps (785) survived a median of 67 months, compared with 44 months for patients who did not get them (1,583). The result was especially striking, Dr. D'Angelica said, because patients who got pumps were sicker to begin with.

May be there'll be enough backlash to throw the critters out of the swamp.

25 April 2018

Skew You Buddy

A recurring theme in these endeavors: data analysis is helpful only when there's a stable underlying data generation process. Among other things, this makes stat as a tool for macroeconomic policy analysis fraught with danger. The members of that branch of study have been arguing over the structure of analysis. The quants against the historicals. On the whole, historicals have been right. On the other hand, there's the old saw, "economists have accurately predicted 10 of the last 5 recessions!" As an example, here's an interesting post. I'll leave it to the reader to decide whether the effort is Type III error.

An even more egregious error happens when median/mean is relied on. It's also worth noting that one of the few shorts was explicit: they didn't believe an MoA existed. Ouch.
This time around, Celldex had comparison data that strongly suggested its lead candidate, glemba, could provide a survival benefit over chemo for patients with a difficult-to-treat form of breast cancer. Unfortunately, the data came from a subgroup contained within an otherwise failed trial. In retrospect, it looks like several outliers resulted in a lot of wasted resources.
[my bold]

Outliers, on one side of the median/mean, will skew you dead. Not to mention: subgroup post-hoc "analysis" is the Very Big Red Flag biostats. On often wonders why a legit math stat would get within a barge pole's length of such companies. But they do. I guess the money's not bad.

It is all too common for small bios to go all in, given that they have, in general, small pipelines. If I had a nickel for every one of these that generated traffic like, "but there's one guy still alive after 5 years taking WhizzBang250", I'd be sipping tequila on a Caribbean island. ImmunoOncology is one of the bigger deals these days, but there're even spontaneous remissions in cancer and always have been.

With PhARMA constantly braying that it costs $X billion to get "a drug to market" so drugs have to be really expensive to pay for all that, it's worth pondering whether the moolah is funnelled into R&D or mgt. pockets?

20 April 2018

The Asymptote of Progress - part the ninth

Yesterday morning was going to be a Carnac day, in which a missive stating the prediction that FAA/NTSB would demand stepped up inspection of high-cycle engines was to be offered. Never got around to it and, naturally, such was reported later in the day. Which brings us to today's reporting where we get some fruitful quotes about jet engines.
Like the engine on the Southwest jet, two others — one used on the Boeing 787 Dreamliner and another on some Boeing 767s — developed cracks. On Tuesday, the same day as the engine failure on the Southwest plane, the Federal Aviation Administration said Boeing 787 Dreamliners powered by Rolls-Royce engines could no longer be flown on ultralong, over-water flights.

The engines are produced by three different manufacturers, but the fact that all three have developed safety issues is prompting questions about the engines' design, operation and their inspection procedures.

The worry is that the flaws are part of a trend as manufacturers push to develop ever more powerful and complex machines.

"We've gotten smarter," said Richard Giannotti, an aerospace engineer. "We can design things to a very low margin with a lot of reliability data to back it up. But when we get to the ragged edge, it doesn't take much for things to go wrong."

Sound like turbofans have reached the wall? Along with Li-ion batteries?

16 April 2018

The Asymptote of Progress - part the eighth

This hot off the presses (7:56):
VALE Vale S.A. reports Iron ore production reached 82.0 Mt in 1Q18, 4.2 Mt and 11.4 Mt lower than in 1Q17 and 4Q17, respectively (13.12 )
Iron ore production reached 82.0 Mt in 1Q18, 4.2 Mt and 11.4 Mt lower than in 1Q17 and 4Q17, respectively, mainly due to management's decision in 2Q17 to reduce production of lower grade ore, reinforcing the positioning of Vale as a premium producer and resulting in higher price realization and better margins since 1Q17. The more intense rain season also affected production in 1Q18.

So, let's make America Great Again with imported iron ore, since we haven't enough in the ground worth digging for. Such a Wonderful Idea.

15 April 2018

The Deplorables - part the second

Yet another series, with unknown length, impelled by the Trumpistas. I'm going to guess that Bevin hasn't the faintest idea that he's indicting his state's redneck deplorables. And he's implicitly demanding that public school teachers be domestic violence cops, in addition to gun-totin', rootin-tootin eraser cleaners. Hillary is lookin righter by the day.
I guarantee you somewhere in Kentucky today a child was sexually assaulted that was left at home because there was nobody there to watch them. I guarantee you somewhere today a child was physically harmed or ingested poison because they were home alone because a single parent didn't have any money to take care of them.

13 April 2018

The Case for Public Goods

Since Orange Julius Caesar is the illegitimate love child of Ayn Rand and Benito Mussolini, we've seen (I'm talkin Pruitt, Ryan, et al) the rise of Social Darwinism to never attained heights. Which brings us to this "study".
When curing a disease with gene therapy is bad business

The answer, naturally, is that such science has no business being a business. Especially in the cited cases where the "cures" are for truly rare diseases, and not the faux types which Pharma is jumping into with both feet.

So, where's the moolah going to come from?
Novartis's projections that the treatment could bring in several billion a year in revenue also suggest that the company might charge unheard-of prices, perhaps $2.25 million according to Wall Street bankers.

Society as a whole, through whatever damn gummint insurance exists and "taxing" the privately insured, should shift mountains of money for some basic science?? Time to wake up and smell the coffee. There are public goods, and there's no benefit to piling on gobs of empty profit to the price. The science will out, no matter.

09 April 2018

The End of The World as We Know It - part the second

In today's news
According to Mintel's research, 43 percent of adults eat cereal as a snack at home. Of people who eat cereal, 30 percent choose cereal that tastes good regardless of how nutritious it is.

Stupid is as stupid does. Or as Mrs. Darcy says at the end of that 'Law and Order' episode,
Girl, you're as dumb as a sack of hair.

R.U.R. - part the second

The first installment in what is now another series of data analysis made this point:
The standard whipping boy of the lunatic right, who are clearly conflicted, is the unionized auto worker. On the one hand, they've spent decades demonizing such folks, just the Rust Belt angry white folks who're now out of work. But, now of course, Kim Jong-Don claims that he's their savior. Baloney, of course. Here's a detailed analysis of the way the world really was, at the time of the auto rescue.
A final note on all this: Labor costs only account for about 10 percent of the cost of producing a vehicle.

Today we get the news that Der Trumper Bankster, aka John Cryan CEO du Deutsche Bank, is out the door. Couldn't happen to a nicer guy. But, he's not the only one likely to get FIREd in the coming times. Last fall, we heard this:
AI has, however, piled pressure on costs by automating mundane and repetitive tasks. The implication is that employees like the accountants singled out by Cryan as "doing work like robots" are living on borrowed time.

Which raises that nasty existential question yet again: if high-value, high-wage, high-education FIRE jobs are on the bonfire, how is it that education is the key to prosperity? Will we really need millions of new EEs in the coming years? And if so, can your average MBA actually earn the lowly BSEE? My wild ass guess? Not in a million years. It's that Pareto gut punch, yet again.

08 April 2018

The Asymptote of Progress - part the seventh

Another note of asymptote existence. Anandtech just reviewed the latest SSD from Western Digital. For the first time in a while, they implemented their own controller. One of the comments complained that the review didn't reveal the entire design/implementation of said controller. Billy Tallis replied (in part):
We asked repeatedly, and all we could get was that it isn't RISC-V. But every other NVMe controller used in consumer SSDs uses Cortex-R, and there's no reason to suspect WD is doing anything different. There aren't many alternatives.

Which comment points to, at least, two asymptotic factors:
1) CPU availability (not the whole controllers, of which there are legion) boils down to something ARM or RISC-V
and
2) The designers assert that new principles are becoming rare in instruction set design, as the most successful designs of the last forty years have become increasingly similar. Of those that failed, most did so because their sponsoring companies failed commercially, not because the instruction sets were poor technically. So, a well-designed open instruction set designed using well-established principles should attract long-term support by many vendors.

That second is from the wiki article on RISC-V. This is the quote's link. That old Zeno problem, stepping halfway to the wall. Or as stated in recent missives, CPU is a manifestation of maths, and in the end there will be one "best" resolution to a maths problem. You really should read the whole thing. One, therefore, might wonder how much microarchitecture "evolution" is pointless wheel spinning?
If you are one of the few hardware or software developers out there who still think that instruction set architectures, reduced (RISC) or complex (CISC), have any significant effect on the power, energy or performance of your processor-based designs, forget it.
Not to say that engineering of chip production has hit the wall (it is close, though). (And an amusing stackoverflow discussion.)

The commoditization of even CPU and OS might lead one to ask the next question. What becomes the high-value/high-wage occupations? Do we really need to repeat the fiasco of FIRE burning down the global economy? But if real science/engineering becomes unnecessary (in the creation and production of consumer goods; cosmology need not apply), how should we educate all those little grey cells (yes, that's not original, and you should look it up) for what sorts of work? The soft questions are always the most difficult to answer.

And, it turns out, that RISC-V machines are being made. Not all is peaches and cream, naturally. A short tour of the innterTubes will dig up criticism. But CPU was, is, and always will be a manifestation of maths, and as every other type of maths, there's one best way to do it. Guido says so.

06 April 2018

Parts is Parts

"Parts is parts"
-- Wendy's ad making fun of Mickey D

I've been waiting for some time for the primetime pundits to figure something out. Which is that much of China's "exports" to the USofA isn't visible as such. So today, (I hate) Neil Irwin and Paul Krugman both point to the real issue. Which is that red-blooded, God fearing American corporations use Chinese labor and capital to a significant extent. That iPhone is a Chinese "import", even as Apple is an "American" company.

Irwin:
When Apple assembles an iPhone in Zhengzhou and sells it in Shanghai, that doesn't count as international trade, though the profits accrue to the benefit of a California-based company. The Chinese government has any number of tools to try to weaken that business if it wishes. It could decide that phones made by a foreign company are a national security threat, or shut down plants because of minor regulatory problems.

Krugman:
China is the Great Assembler: it's where components from other countries, like Japan and South Korea, are put together into consumer products for the U.S. market. So a lot of what we import from China is really produced elsewhere.

Just as American capital fled New England for the Rebel South, and then Mexico; they now decamp to China. The impact on American labor is not subtle, which fact played into Orange Julius Caesar's paranoid mind (note to self: some praise paranoia), and he lied on it to the Throne. As previously noted, the USofA no longer has the raw materials to make primary steel or aluminum in quantity. The salvation of redundant coal miners and steel workers is socialism. Taxing, which is what tariffs do (just read up Smoot-Hawley), just makes things worse for everyone. The right wingnuts, naturally, cry "but Pareto says you can't redistribute!!". There are only two choices; 1) let them perish or 2) support them. Socialism makes the support explicit and efficient. Tariffs and other market meddling only rewards the 1%, mostly.

Here's some arithmetic about the issue (2011):
[T]he total wholesale value of the iPhone — for the 3G model it was about $180 — goes on the Chinese import side of the trade ledger. As a result, says economist Rob Feenstra of the University of California, Davis, "The U.S. trade deficit with China tends to be exaggerated."
...
In a much-talked-about paper, Chinese economist Yuqing Xing took a stab at the figure. "If you look at the manufacturing costs, China's contribution is $6.50."
He says that figure represents the actual cost of assembling each iPhone. And he concludes that if that number were used instead of the entire wholesale cost, the U.S. trade deficit with China would shrink by roughly $2 billion.

So, we know that the net Chinese imports, and by net we mean product fully created within China with Chinese materials and capital and labor, is far less than the $375 billion (yes, Orange Julius Caesar brays $500 billion) that even the mainstream pundits state.

The moral of the story: figures don't lie by liars figure.

03 April 2018

The Left Behind - part the second

Every now and again, I get the feeling that Krugman reads me as much as I read him. He gets paid to do so. Alas, I don't get paid on either end of the process. Nevertheless, today brings a timely example discussing the left behind phenomenon. Not surprisingly, he reaches much the same conclusion, and offers up some useful quotes from others.
On the economic side, some parts of America, mainly big coastal cities, have been getting much richer, but other parts have been left behind. On the political side, the thriving regions by and large voted for Hillary Clinton, while the lagging regions voted for Donald Trump.

See, he even says "left behind". I should have copyrighted that. Likely would have been sued by that nutball Christian cult, though.

The most interesting part of the text is taken from recent academic writings (I know; that makes it Fake News to the rednecks) on regional economics.

He references a 2012 book "The New Geography of Jobs" which I haven't known,
As a result, these regions are experiencing a virtuous circle of growth: Their knowledge-intensive industries prosper, drawing in even more educated workers, which reinforces their advantage.

And at the same time, regions that started with a poorly educated work force are in a downward spiral, both because they're stuck with the wrong industries and because they're experiencing what amounts to a brain drain.

Crooked Hillary got roasted, of course, when she stated the obvious: her voters represent 65% of US GDP. Now you know the details.

The most astounding finding: from the socialism of FDR and subsequent Democratic administrations which saw the poor Red states move upwards, we've had them fall very much back near where they were (in a relative measure) prior to FDR thanks to their voters' insistence on installing right wingnut governments at the local, state, and Federal level. At some point you just have to say, "well, you want to live in shit, so be it".

Once again, with feeling: since Reagan (that's 38 years and counting) the Right has controlled
House: 20
Senate: 22 (Cheney effect)
White House: 20 (if Orange Julius Caesar finishes, which is not quaranteed)
Supremes: 38

To put it more succinctly:
Reagan had: 2 out of 3 for 6 years
Poppy Bush had: 1 out 3 for 4 years
Clinton had: 2 out of 3 for 2 years
Dubya had: 3 out of 3 for 6 years (Cheney gave control)
Obambi had: 2 out of 3 for 2 years
Orange Julius Caesar has: 3 out of 3 for 2 years, so far

In that period, only Dubya and Orange Julius Caesar have had Congress for more than a single session. And the Right has had the Supremes for the whole time. Those left behind have done it to themselves. Whenever they've made some progress, it was the Democrats that gave it to them. Low informatin ingrates, all.

31 March 2018

Thought for the day - 31 March 2018

Am I the last person on earth to be sick and tired of **everything** being sold as 'curated'? It doesn't even mean anything.

30 March 2018

The Left Behind [update]

The Orange Julius Caesar zealots have a world of grievances, primary among them that the coastal Blue States get all the breaks while they live in backwoods poverty. The reason, which they steadfastly refuse to admit, is that they have, from the beginning of the USofA, voted for fascist governments in their states. Which governments certainly treat education as a burden on the taxpayers. Since such states have radical upward income distributions, the rich get their own private schools while the rest get the crumbs.

Well, another Red state is beginning to understand that teachers are an enemy of the state. So to speak.
Shoring up the pension system has been a priority of [Republican governor Matt] Bevin's since taking office in 2015. But his efforts have met with difficulties. Thousands of teachers and public workers marched recently at the Capitol, and lawmakers balked under the pressure. Frustrated, Bevin called opponents of his plan "selfish" and "willfully ill-informed." His comments earned rebukes from Republican legislative leaders and galvanized the opposition.

As is usual with rightwing bait and switch, Kentucky didn't bother to put money into the pension fund along the way, too much taxpayer money of course, so now it has an actuarial shortfall of $41 billion. Or so. Keep 'em stupid and breeding.

[update]
Well, it turns out that most folks (although I'll guess that 'most' skews from Blue areas) understand that teaching is a short-changed occupation.
but Democrats (77 percent) and independents (69 percent) are more likely to say this than Republicans (54 percent).

28 March 2018

Sheeple

It should be getting clearer to civilians that all those "free" apps out there on the innterTubes aren't for them. They aren't the clients, but the product. Throughout history, the 1% have sought to keep the 99% in check. Orwell's greatest dissembling was his re-inforcing the notion that Big Brother-ism would be brought by the damn gummint. Orange Julius Caesar has said that he couldn't even collude with the rest of his campaign, much less the Russians. It appears that Manafort and Gates sure could.
Documents filed by special counsel prosecutors reveal that former Trump deputy campaign chairman Richard Gates was knowingly working with an individual with ties to Russian intelligence during the campaign, CBS News Paula Reid reports. In a late Tuesday court filing late, the special counsel alleged that this unnamed person worked for one of former Trump campaign chairman Paul Manafort's companies and was in touch with Gates in September and October 2016.

Big Brother is the 1% and corporations. And this was demonstrated years before most humans currently on the planet were born. In 1957, Vance Packard exposed the power of Madison Avenue. The book isn't a comedy. The social network cabal of today is advertising of Packard's day on giga-steroids. Cambridge Analytica is just the top of that totem pole. So far.
Packard explores the use of consumer motivational research and other psychological techniques, including depth psychology and subliminal tactics, by advertisers to manipulate expectations and induce desire for products, particularly in the American postwar era. He identified eight "compelling needs" that advertisers promise products will fulfill.

Just the sort of psychographic analytics that Cambridge Analytica claims to peddle.

Doesn't that sound a bit like "Make America White Again"? Of course it does. And documented 60 years ago.
Oh, when will they ever learn?
Oh, when will they ever learn?

You want to be a quant? Check your morals at the door. Just do it.

26 March 2018

The Sad Truth

One doesn't often see the pharma press actually be fully honest. Turns out that Merrimack Pharmaceuticals recently upped the sample size of a trial. The share dipped significantly; relative to 1Y high, nearly crashed. Here's an explanation:
The company plans to increase the number of participants to 100 from 80 citing rapid enrollment and "robust clinical interest." Frequently, however, a sponsor increases enrollment to improve the chances of a statistically valid result.

Ooo! Did baby bump his head on the floor?

22 March 2018

Here Come Da Judge

More gaga news.
In addition to the new dual-rank 64 GB RDIMM module, Samsung is set to develop quad-ranked 128 GB RDIMMs and octal-ranked 256 GB LRDIMMs. Today's servers running AMD's EPYC or Intel's Xeon Scalable M-suffixed processors feature 12 or 16 memory slots - if the processors were capable of fitting all 256 GB modules, this could lead up to 4 TB per socket. This should be a massive advantage for applications like in-memory databases, virtual desktop infrastructure, and so on.
[my bold]

Such a machine, though costing a bit more than a generic white-box, could handle 99.44% of R use case data sets without the onerous cost of SAS/SPSS. I suspect such a machine could pay for itself in software license cost avoidance in a year or two?

21 March 2018

Iron Fisted Rule

Regular reader may recall a recent missive offering data on iron and aluminum ore reserves here in the USofA. It is not a comforting picture for those eager to return to the 19th century's industrial model. Today, Eduardo Porter reports on further research regarding tariffs on steel and aluminum vis-a-vis their use. The main point being: how many jobs are affected by tariffs in steel making and steel using industry. Not surprising, the numbers still say that the USofA workers (net, all of them) take it in the neck as do consumers of USofA manufactures.
For every job in Tupelo producing steel or aluminum, there are 200 jobs in industries that consume them that could be put at risk as tariffs push up the prices of these metals, according to research from Jacob Whiton and Mark Muro of the Brookings Institution.
Take a look at those colorful maps: the redneck knuckleheads, once again, voted for suicide. They don't need no education.

This finding is not an outlier, by any means. Total employment in steel and aluminum has been dropping like a rock since the end of WWII, when we gobbled up vast quantities of the Mesabi Range ore
While the Mesabi Range had single-handedly supplied the iron for steel during World War II, it essentially dug its own grave. The Range totaled output of over 188 million tons of ore during the course of the war, and exhausted itself of natural hematite until the process of making taconite into iron was discovered into the 50's and 60's. The Mesabi Range lived like a star, dimly lighting the industry, and going supernova when it was most needed, provided for the country and gave everything it had until it had nothing left. In being the key for us winning the war, the Mesabi Range gave its life force, becoming just another casualty of the war.

The USGS has a report on the levels of mining, production, and reserves. It agrees with other data from that previous missive.

Stupid people make stupid choices. That fact has always, and will always, be true.

16 March 2018

Shame

There are times when one couldn't say it better (scroll to 3:11 PM - 14 Mar 2018):
17 people die in a high school and a month later there's a nationwide protest to get the attention of lawmakers. One dog dies on an airplane and there's a bill in the Senate rectifying it within 48 hours.

As pointed out here many times, this is of a piece with HRC's "deplorables" and 'the states I won are 65% of GDP', IOW, the left behind knuckleheads are in that situation just because they insist on electing fascist governments. The coastal states mostly do otherwise. You elect government that treats you like chattel, you'll be left behind. And it's your fault.

HRC has been excoriated for telling the truth, even by pundit Democrats. The truth hurts some times.

14 March 2018

Jobs R Us

Here's the headline: "Toys "R" Us liquidation could cost tens of thousands of jobs"
Toys "R" Us is expected to start court proceedings to liquidate as soon as Thursday. That's a first legal step in moving to close all of its 850 brick-and-mortar stores in the U.S. -- and to lay off up to 33,000 workers.

So, let's see if Orange Julius Caesar thinks they're worth saving; the way coal miners and steel workers are. Ya think may be not????

13 March 2018

Predestination - part the second

Ben Casselman's at it again. As regular reader may recall, these missives have pointed out that macro and micro data are very much brothers from different mothers. Micro data is, within a teeny epsilon, population while macro is samples. Today's piece is filled with quotes from important pundits, which make the points seen here.

First
"We have trouble measuring any of these things," said Tara Sinclair, an economist at George Washington University and for the job-search site Indeed. "This is definitely one of those situations where you can feed in the data and get out whatever response you're looking for."

IOW, the wiggle room in the data is sufficiently large that one can make any argument, plausible or not. Just take a look at the technical section of the Monthly Report to see the admission that the numbers are of Gumby.
BLS analyses are generally conducted at the 90-percent level of confidence.

For example, the confidence interval for the monthly change in total nonfarm
employment from the establishment survey is on the order of plus or minus 115,000.

Let that warm and fuzzy sink in. OK, back to Casselman's quotes.

Second
Despite the familiarity of the average hourly earnings figure, however, economists say that it is among the least reliable indicators, especially in the short run.

Third
Most government statistics, for instance, have failed to adapt to the rise of the so-called gig economy and other trends that are changing the relationship between companies and workers. The hourly earnings measure in the monthly jobs report excludes Uber drivers and similar contractors.

Now, does that bit of missing data mean; A) higher earning folks are being missed or B) lower earning folks are being missed? The answer, naturally, is B. IOW, the data/report are, to an unknown degree, rosy.

Fourth
Moreover, Americans are increasingly refusing to respond to government surveys. The response rate to the monthly Current Population Survey, the data source that underlies the unemployment rate and many other key statistics, has eroded in recent years. Of the households that do respond, about a third refuse to provide information about their earnings, a rate much higher than for most other questions. Similar problems have affected other government and private-sector surveys.

Without accurate macro data, all we get is demogogic policies. Demagogues prefer to ignore data, but when the data is known to be wonky, pushback gets tougher.

08 March 2018

Blind Pundits

Yet another case where Red/Blue is being misconstrued, and over an issue of some significance. Here's a report on 'electric' cars. With this assertion:
Out of the top 10 states for new and used hybrid, plug-in hybrid and all-electric cars, only two -- Arizona and Utah -- voted for Donald Trump in the 2016 presidential election, according to an analysis of 2017 auto sales from from research site iSeeCars.com.

I don't doubt that for a second. But it would be helpful to readers seeking to understand WHY that is true to explore the motivation. As they say in the real estate business, it's location, location, location.

At no time in the text is it offered that electric/micro cars only make sense in urban areas. If you have to drive 100 miles to the grocery store, and the only charging station would be the one in your garage (if you actually bought an 'electric' car), you're not likely to buy one. States which, in toto, vote Blue are ones where cities overwhelm empty counties. Red, the other way round. Why won't mainstream pundits figure this out?? I guess it's too sophisticated to actually think about a subject, and look at the data.

03 March 2018

Yes Ore No

To paraphrase Orange Julius Caesar, "if you don't have iron ore and bauxite, you don't have steel or aluminum". And, one might add, to turn bauxite into aluminum eats prodigious amounts of electricity.
CSIRO calculate that the embodied energy (all the energy used to make the material) for aluminium is 211 GJ per tonne, compared to 22.7 GJ per tonne for steel.

Not for nothing, our economy would rather burn all that electricity on bitcoins. Make sense to you?

As to base ore endowment, the USofA has nearly no bauxite and has depleted iron ore to about the noise level. Without imports, we don't have industry.

Here's iron reserves. And, bauxite; you're not missing something, the USofA isn't listed.
However, since 1981, none of the bauxite mined in the US was used to make metallic aluminium. US bauxite is instead used for abrasives, high-temperature refractory materials, and as a high-strength proppant for hydraulic fracturing of oil and gas wells.

Besides announcing tariffs while "unglued", there really, really isn't viable US supply. Moron.

We Don't Need No Education

The West Virginia teachers remain on strike. About time that someone in the Empty States takes a stand. From the beginning of these endeavors, it's been asserted that there's a reason they have been "left behind", and it hasn't been a conspiracy among the elite Eastern intellectuals and nattering nabobs of negativism. It has been the Right Wingnuts elected by the residents of the Empty States who treat said residents as indentured servants. And those folks accept the situation blindly.

It should come as no surprise that West Virginia ranks at the bottom, topping only a few other Empty Red States. Keep 'em stupid and breeding. Works OK, for the Marie Antoinette Class when your economy is largely agrarian/extractive. Not so much in the 21st century. Also not surprising that Orange Julius Caesar promised these morons that he'd send the country back to the 19th century. And they bought it. Stupid is as stupid does.

It's not a coincidence that the prosperous coastal Blue States are closer to socialist than social Darwinist. The West Virginia teachers are figuring it out.

01 March 2018

The Asymptote of Progress - part the sixth

As mentioned a few times, there's an advantage to having a NYT press pass: you get great quotes from intelligent folks. And so it is today with some reporting from Binyamin Appelbaum. The piece is, superficially, about the cost of mining bitcoin and follows on previous reporting from many sources. But it's really about the dirth of productive investing venues. In other words, the Asymptote of Progress©.
While Bitcoin mining may not be labor intensive, it diverts time, energy and capital from other, more productive activities that economists say could fuel faster growth.

"It appears that much of our evolving digital infrastructure is devoted to activities, like the proliferation of cybercoins, that are worse than frivolous," said James McAndrews, the former head of research at the Federal Reserve Bank of New York.

Where is the next steam engine??

The driving notion behind bitcoin is the blockchain, naturally. And, naturally, it's spiffy spanking new. Well, not so much. Databases have been based on, and some are explicitly as datastores, append-only data files. Here's a famous one. Current RDBMS keep their data in immutable transaction logs. As you might expect, that's just a blockchain. Zebra skips the nicety of writing out to user-facing tables/files, and just keeps the log. Blockchain is nothing more. The use of cryptokeys as pointers is a minor tweek.

As mentioned here a few times. The Asymptote of Progress© forces the Marie Antoinette Class into odd places. On the one hand, there's risk averse Treasuries. On the other, all manner of MeToo Corps and crazy shit.

The problem with keeping only the translog as the datastore is that processing is a royal pain.
But Bitcoin remains so hard to use that a major Bitcoin conference in January had to stop accepting Bitcoin. It is, in practice, a speculative investment, like gold. And Tyler Cowen, an economist at George Mason University, said mining gold was a better use of resources, because even if it lost value, it could be used to fill teeth.

Oops!!!

28 February 2018

wHo NkeW

One of the most interesting uses of R popped up on "R-bloggers" today. It's a test of reading. In R? Yes. Moreover, the earliest cite in the article is 1981, and text makes reference to research done in the 70s. Who Knew?

An earlier missive here asked what sense it makes to build computers that are more "brain like". Text recognition has been of interest for some time, and highlights the (nearly?) unique capabilities of the human brain. Not so much for processing Big Data, alas.

24 February 2018

The Asymptote of Progress - part the fifth

If there are any stragglers among the readership who deny the reality of the Asymptote of Progress©, here are a couple of links from comments to a piece on Anandtech.

The first is from 2017, and is generalist. And is, on the whole, optimistic.

The second from this year, on the other hand, is decidedly not.
In fact, depending on what aspect of Moore's Law you look at, it ended a long time ago (power scaling), or recently (cost per transistor scaling) or still has several nodes to go (technical barriers).

Oops!!

And, in conclusion, it gets worse:
Moore's Law is, indeed, ending. But that is only one part of the problem:

MOSFETs have run out of steam, with newest nodes requiring more money for very limited improvements
FETs of all the weird types are not going to save us
Von Neumann compute is reaching its limit

Read that last point. Let it sink in. Offer up a superior alternative. This is his:
On a longer term still, we will end up moving away from von Neumann architectures to something more "brain like."

Of course, we don't even have a definition of how the brain actually does what we call computing. Show me a brain that computes Big Data? It's one thing to conjure up the words to "War and Peace" in a "brain like" brain, and quite another to run multiple regression on a million row data frame in a "brain like" brain. Why would a neuron based compute engine be successful at such a task? All that nonsense about only having a hammer seeing the world as nails. Any answer welcome.

22 February 2018

I Still Hate Neil Irwin - part the tenth

It's the nature of quants to be lazy, despite all the brow sweat they brag about. This laziness leads them to seek out easy data. Which, in turn, leads them corporations, i.e. the micro sector, who collect as much data as they want. And, since corporations are mostly rolling in dough, fat salaries to be had. So long as the quant doesn't find the requirement to pander undesirable. Hog heaven for the quant. The problem is that, as any macro quant will tell you: 99.44% of micro performance is the result of the macro environment. Do the CxO class, and their quant servants, really think Mr. Market's spectacular rise over the last years is due to their brilliance? The CxO class has had, at most, an itsy bitsy teeny weeny yellow polka-dot effect; almost entirely M&A to destroy competition.

So, today is another reason to hate Neil Irwin!! As stated here more than once, Say is crap. If it were true, there'd never be recessions or depressions because business would ramp production (and employment and wage incomes) in the face of falling demand. I dare anyone to cite any company that did so. You won't find one. They follow the Lemming Law: each attempts to conserve cash and profits by cutting production and employment. Thus constructing a downward spiral of global contraction. What the individual sees as rational through his own eyes, leads to a mess when everyone does it.
The latest wrinkle is that the researchers now believe that productivity growth depends not just on the supply side of the economy — what companies produce and what technologies they use to do it — but also significantly on the demand side.
...
[C]ompanies need to increase production to match demand for their goods, and a shortage, either of workers or of materials, forces them to think creatively about how to do so.

As Homer says, "D'oh!!". And, it's worth noting, McKinsey has an extreme rightwing reputation in the consulting world.
McDonald comments that apart from slashing cost by slashing headcount (increasing employee churn in order to keep costs low), there have been many instances where the recommendations that have come from McKinsey seem to focus on cold blooded cost cutting at the expense of long term staff loyalty, creating a culture of fear and short-termism in the organisational culture.

It is kind of amazing that McKinsey and Irwin seem to ask, "What would Jesus do?". Again, the repeat: go read Baumol, as well. FIRE has eaten GDP, and measuring productivity using tools created for a manufacturing economy will, by definition, show lousy productivity as GDP skews more toward services. Which is not to assert that some other measure of productivity would both be honest and at much higher levels than we've actually seen. After all, productivity is supposed to be a measure of labor output. A few years back, GDP's definition was fiddled to add IP related activity. No one seems to ask the fundamental question: is there any productivity benefit to a mobile phone that runs "Candy Crush" 10% faster???

Here's a long-ish article making these points.
A problem with GDP even when it is being asked to do nothing more than measure production is that it is a relic of a period dominated by manufacturing.

Just like I said. Of course.

21 February 2018

A Brother From Another Mother

We all now know, beyond doubt that Dear Leader is an illiterate moron. With the brain of chicken, as Connie used to say when she did something silly. Orange Julius Caesar does nothing else. So, today brings us some stark contrast.
"How did you end up here, Marilynne? What happened? Was it libraries?"

In 2015, President Obama sought out the novelist Marilynne Robinson, the author of the acclaimed Gilead trilogy, for a wide-ranging conversation. They spoke with the ease and deft mutual flattery of friends, touching on democracy and education, their Midwestern roots and love of literature. Obama's question lay at the crux of their discussion.

Is there any way to imagine Carrot Top even considering books???

Here's the quote that made me stop.
As a child, her teachers told her, "You have to live with your mind your whole life"; they taught her the value of building it, making it worthy. It was, she has said, the most important lesson she ever received.

Since at least high school, I've always answered various existential questions with, "you live between your ears". A point of view that doesn't drive one to keep up with the Joneses, naturally. All these years later, it's more than a tad comforting to find it in print.

An ironic way to close the piece
"Slander" is the story of Robinson's strained relationship with her mother. "With a little difficulty we finally reached an accommodation, an adult friendship," she writes. "Then she started watching Fox News." Her mother and her fellow retirees began to share "salacious dread over coffee cake," fretting over the rumored "war against Christmas." "My mother lived out the end of her fortunate life in a state of bitterness and panic, never having had the slightest brush with any experience that would confirm her in these emotions, except, of course, Fox News," Robinson writes. The essay brings all the abstractions home, makes real and painful the cost of ceding an independence of mind.

19 February 2018

Thought For the Day - 19 February 2018

17 of my classmates are gone. That's 17 futures, 17 children, and 17 friends stolen. But you're right, it always has to be about you. How silly of me to forget. #neveragain
@Aly_Sheehy
Very sad that the FBI missed all of the many signals sent out by the Florida school shooter. This is not acceptable. They are spending too much time trying to prove Russian collusion with the Trump campaign - there is no collusion. Get back to the basics and make us all proud!
-- Trump/2018

18 February 2018

Shooting Fish in a Barrel

N. Gregory Mankiw is a Marie Antoinette Class quant type, with a very public platform. Recently he expounded on free trade. Much of it is baloney, but this bit
To be sure, expanding trade hurts some people in the short run, especially those in import-competing sectors who have to find new jobs. That fact may call for a robust safety net and effective retraining. But it does not undermine the conclusion that free trade raises average living standards.
[my bold]
deserves shooting. Like a fish in a barrel.

I wonder how he feels about all those coal miners? Not to mention that, being a card carrying economist he knows better, citing "average living standards". Given the continuing income skew towards the 1%, and documented destruction of the remaining 99%, that cite rather then median income (or, as even some Trumpians cite, vis-a-vis 1970s income) which has barely budged since the OPEC oil embargo sent the economy into the ditch. IOW, any data analysis of known skewed distribution, income or wealth or widget weight using mean rather than median (or some other skew reducing/removing measure) is politics, not analysis.

14 February 2018

Two For the Seesaw

Regular reader likely realizes that I don't often just say, "go read this". Mostly because the mainstream pundocracy hasn't caught up yet. Eduardo Porter, on the other hand, does so more than the others. Well, not counting Krugman, naturally.

Today Porter takes on Mr. Market. As has been the case with the rest of the pundit class, he avoids asking the basic question, "where have all the dollars gone?" But he does deal with the obvious question of overall stock prices.
What I contend is that if the American economy behaved in the way that most economists say market economies should, stocks would in all likelihood be cheaper.

Why?
The Standard & Poor's 500-stock index increased 8 percent per year from 1970 to 2015, on average.
...
What makes this particularly puzzling for scholars reared on the classical models of competitive economies is that all this happened despite a persistent decline in real interest rates.

The money quote, buried in the middle
The ratio of the capital stock — the value of factories, machines and such — to the nation's economic output has actually declined a little since the 1970s.

So where have all those dollars gone? Well, to Treasuries and other fiduciaries. Capitalists were, are, and always will be risk averse. Little to no rise in physical investment in a decade of low rates, yet all that idle moolah keeps buying Treasuries, and knocking down the opportunity cost of real investment. And they still won't bite. The obvious reason is that, intuitively, they understand that only massive capital (e.g. Fab 8) can eke out a teeny increment in progress. So they toss in the towel and buy Treasuries. In due time Orange Julius Caesar will demand that Treasury cease the auction and sell at named interest rate. Chaos follows.

12 February 2018

Bad Vibrations

The recent fibrillations of Mr. Market have been the grist for countless pundits. Rather than attempt to precis all of them, I'll just note this one from Jeff Sommer
"What we have seen in the last week or two is minuscule compared with the amount of real risk that is coming in the months and years ahead," said Salil Mehta, an independent statistician with deep experience in troubled markets and their consequences. He was the director of research and analytics for the federal Pension Benefit Guaranty Corporation and for the Treasury's Troubled Asset Relief Program, which was set up to help stabilize the financial system in the 2008 crisis.

Is that a grand oops? Perhaps so. What Mr. Sommer, and his correspondents, miss (and such media pundits do so frequently) is that Mr. Market can only go on a Jabba The Hut binge if he's fed lots of moolah, increasingly. Where does the moolah come from? As well, so far as stocks are concerned, it's the opportunity cost (and imputed risk aversion) of Treasuries. But opportunity cost is a two-say street. Many/most pundits take as an article of faith that "government borrowing steals from private investment". That's never been true, of course. There are, fundamentally, three things to be done with idle moolah. You can stuff your mattress. You can buy Treasuries. You can buy corporate instruments. From left to right, increasing (universally assumed) risk. If you want to make more moolah, you go corporate. If you want safety, you go gummint. And so it went after the Great Recession. Why? If you read up the recent piece on Fab 8, you can see that returns on corporate instruments is limited by science/engineering progress. Ever more Bongo Bucks to spit out chips. The only way that can generate financial returns is if there's an expanding market for those chips. Remember that the 300mm wafer was dead meat more than a decade ago? Still around.
If the foundry can keep its wafers-per-hour production rate on 450mm wafers close to its 300mm wafer production rate, it can produce vastly more chips per hour. This helps reduce costs, provided that the semiconductor economy is healthy and the foundry utilization rate is high.

Hasn't happened. So, Treasuries and private physical investment are competitors, but the former is driven by psychology (mostly) while the the latter is driven by physics (mostly). The added risk, as the Extreme Tech piece tells us, is that cheaper unit fixed cost leads to capital returns if, and only if, output can be sold at high enough price. As we've seen in the Treasuries market, all that moolah chasing a, more or less, fixed supply drove up prices; and down imputed return. Now that more and more corporates find themselves in the tech cul-de-sac of the asymptote of progress, returns there still have risk, but lower intrinsic return. Unless, naturally, floor sweeping wage earners do earn enough to want and pay for the iPhone X. Good luck with that.

And, by the way, here's another quote from the Fab 8 story:
Growing up is realizing that fabs are sophisticated sinkholes of ever-increasing capital expenditure in the eternal quest to outsmart physics, so much so that where there used to be dozens of foundries and IDMs jostling at the cutting edge, there are now just four: TSMC, Intel, Samsung, and GlobalFoundries. Of them, only TSMC and GlobalFoundries are pure-play foundries, only taking orders from fabless chip designers rather than fabbing their own designs.

Don't screw with Mother Nature.

09 February 2018

Where Have All The Dollars Gone?

The recent fibrillations in the market, at least on a post hoc analysis basis, have been driven by a smidgen of increase in wages for the 99%. In retaliation, the Marie Antoinette Class has burned down the asset markets. Spite. The thing is: where have all the dollars gone? The inflation in the asset markets could/can only happen if there's excess moolah chasing the instruments. Right? Treasuries represent the lowest risk opportunity cost to the MAC, and the MAC has shown over the last decade or so that they'd rather hold Treasuries than expand the real economy. Spite. One might even harbor the notion that the MAC would prefer the "return" from catastrophic deflation (held moolah buying a bit more each day as prices plunge) over building out new physical capital.

But, you say, what if there's a sharply diminishing venue for new physical capital? Fab 8 is instructive (see recent missives). What if the notion that new knowledge of the scientific/engineering kind is slowing? What if Treasuries aren't the opportunity cost, but it's really the other way round? What if the MAC look at their world from the other direction? What if they say, "I'm not going to dump Treasuries and make real investment until the real return difference is way higher than it is today"?

If so, then the turbulence of recent days is just pique at the fact of a bit more in the pockets of the 99%. It won't last just because the Era of Low Returns is here to stay. All that idle moolah will, soon enough, go back to chasing Treasuries. As the punch line of the adage of the frog and scorpion says, "It's just my nature".

08 February 2018

Brave New World - part the second

Despite the Pollyanna spewing from some folks, here's the future:
The operations engineer would tell us that this was the most automated fab in the world, where no one moves or touches the wafers at all. GlobalFoundries had built Fab 8 from the ground-up to be highly automated, and the 'vehicles' were the key part.

As one version of these escapades puts it, "It's the distribution, stupid". If capital continues to displace labor, who's going to have any moolah to buy the neato products???? Riddle me that, Batman.

05 February 2018

Dee Feat is in Dee Flation - Part the Thirty Third [update]

So, what's going on with the 1,000 point (or so) drop in the DOW? All together now: "it's the 1%!" Recall all the braying about TARP and QE and such leading to ruinous inflation? Well, as reported here many times, ruinous inflation in stocks and bonds has been going on since 2009; March of that year.

The 1%, being utterly risk averse, mostly have plowed their idle moolah into Treasuries, and some into private instruments. Enough to drive DOW over 26,000. But now that there's just a whif of wage growth in the 99%, they bail. Figures. So, yes, that's really all there is to it. They've stuffed their mattresses, again. But not for long. Economies are caught on the leveling part of the asymptote of progress; there is no steam engine to invest in. You want to live on 10% vig? Hop in your time machine to 1900.

[update]
Well, well, well. Today the estimable Andrew Ross Sorkin weighs in:
Sometimes you can have too much of a good thing. Don't forget what set off the plunge on Friday: better-than-expected job growth numbers.

A day late, a dollar short!