19 October 2018

Thought For The Day - 19 October 2018

OK. What are the odds that the latest 'caravan' was instigated by the The Manchurian President brigade? The perfect October Surprise. The Honduras nasties get a few bucks and rid of some malcontents. The Manchurian President gets his 'issue'. 99.44%.

18 October 2018

A Feature, Not a Bug

Just go read this piece. Calls bullshit. Just as any intelligent analyst did at the outset. Shift $1.5 trillion from the many to the few, blow up the deficit, and round up the usual suspects.
One way to think about it is from the perspective of a small-business owner. Let's say you run your own bakery. You sell bread for $4 a loaf. Today, you sold 90 loaves, for $360 in revenue. You expect that, because it's a busier day at the bakery tomorrow, you'll sell 100 loaves then, earning $400. But you"d like to sell even more than that, so you lower the price to $3 a loaf to encourage additional purchases.

Congratulations! You sell 125 loaves. Your revenue goes up, to $375. That's more than you brought in the day before. Your price cut, though, has not "paid for itself" - because you ended up bringing in less revenue than you would have otherwise.

In other words, you brought in more money than the day before. But it's less than you would have made if you hadn't cut the price.

It will be so much better for the country when SS, Medicare, and Medicaid are eliminated. Right?

16 October 2018

Good Quant, Bad Quant

What is your estimate of the amount of quant analysis or data analysis or data science that does neither descriptive nor inferential statistics? My guess is 99.44%. Sports stats being the most irritating. They're just numbers, fur cryin out loud. Which is not to say that decisions can't be bolstered by mere numbers. It happens a lot. One recent day in the life of The Failing New York Times brought forth some good quant and some bad quant. And no statistics to be found.

First, we get a story about a cranky investor, on the heels of the tenth anniversary of Lehman bullet to the back of the head.
Mr. Rasmussen, founding partner of Verdad Capital in Boston, has written an article and two reports that make a case against investments in private equity, venture capital and private real estate, and he has piles of data to back up his argument.

If you do read the article, there's no analysis beyond stating the obvious: buy small and cheap and sell out expensive. Nice work if you can get it. And no p-values to argue about.

Second, is an article Serena's meltdown, if it were such, and whether her (and others) claim that girls are treated more harshly by officials than boys. Bad quant in this one.

15 October 2018

Our Grey Matter as The RM [updates]

Some times I find the experts off in the weeds. Kind of like The Manchurian President does toward any area of endeavor. By now, if you're not living under a rock, you've heard of AI. Self driving cars. Individualized innterTubes adverts. And so on. Exactly what AI is, isn't clearly defined. By coincidence, a NYT crossword blogger of all folks, generated a very off topic discussion of 'what is intelligence, anyway' a few days ago.

So, today the regular NYT prints this profile. Here's where I go off the rails. Hawkins asserts that it's necessary to figure out exactly how the human brain works in order to make AI. As any discussion/argument over the issue of defining 'intelligence' tells us, humans can't even decide where accumulated knowledge ends and intelligence begins. Which is to say: AI has no need to work the same way as the human brain, only be able to solve problems correctly. Just as there are, at least, four distinct ways to store 'structured' data, there's more than one way to skin the problem solving cat.

That said, here's an intriguing quote:
As Mr. Hawkins looked at that cup, he decided that cortical columns did not just capture sensations. They captured the location of those sensations. They captured the world in three dimensions rather than two. Everything was seen in relation to what was around it.

Does three dimensional mean 3NF?? Well, not necessarily, but you get the idea.

Once again, a dude has figured out that the world isn't flat or hierarchical, it's relational. BooYah!! And it gets better:
"When the brain builds a model of the world, everything has a location relative to everything else," Mr. Hawkins said. "That is how it understands everything."

So, the moral of the story: intelligence is just a RDBMS made out of little grey cells. Now, should it be built on locking transactions, or MVCC? I can live with either.

[updates]
Well, that didn't take long. The NYT has an additional section today in the dead trees version (on-line is a bit fractured); subject being AI. Who knew? Recommended. If only because one of the interviewees has this to say:
What remains the biggest misconception about A.I.?
People confuse intelligence with consciousness; they expect A.I. to have consciousness, which is a total mistake. Intelligence is the ability to solve problems; consciousness is the ability to feel things -- pain, hate, love, pleasure.

Sound a bit familiar?

And this bit of discomfort:
International Business Machines Corp.'s Watson Health head, Deborah DiSanzo, is leaving her role, Stat News reported on Friday. The high-profile departure, which a company spokesperson confirmed to Stat, comes as the famed computer system's usefulness in health care, including in treating cancer, has been criticized as overhyped, including in reports by Stat News and The Wall Street Journal.

Oops.

13 October 2018

God's Punishment

This just in.
Michael -- a powerful Category 4 storm -- shattered late-season records. According to Colorado State University hurricane expert Philip Klotzbach, Hurricane Michael is the strongest storm on record to make U.S. landfall in October.
...
Just two weeks ago, scientists at the National Oceanic and Atmospheric Administration's Geophysical Fluid Dynamics Laboratory in Princeton, New Jersey, published a study using a state-of-the-art hurricane climate model. They found that rising sea surface temperatures due to an increase in greenhouse gas emissions will likely lead to a greater number of major hurricanes in the future. That is a consistent finding in most hurricane climate research.

Everybody says that's baloney. It's God's punishment for electing liberals.

12 October 2018

I Bet You Didn't Know This

One of the most disturbing reports in quite a long time. I didn't know it. I bet you didn't either.
Native Americans did not obtain the right to vote in the United States until 1965, when the Voting Rights Act was passed, prohibiting voter restriction based on race.

Thought for the Day - 12 October 2018 [update]

The Fed is playing with fire, i.e. disaster. For the first time, The Manchurian President may be right in his yelling at The Fed. I doubt his yelling is motivated by anything more than self interest. He has a lot of debt in his Failing New York Businesses.

As has been posited in these missives since the Great Recession, economic health, i.e. widespread growth, is driven by increasing demand for the stuff (tangible and intangible) that we make. It's a long establish fact that the marginal propensity to consume goes down as income goes up. There're only so many bananas that one person will buy in a given time period. Giving Mo Money to those with lots o Money doesn't do the job. It makes them happier, naturally, but does nothing to promote growth. Business doesn't make Mo Stuff in the face of static demand. Much less falling demand.

Which brings us to this line from briefing.com today - 8:06AM:
Starbucks says executing a $5 bln accelerated share repurchase program as part of previously announced plan to return $25 bln to shareholders (54.86 )

What do you think will happen to that $5 billion?? Will it be spent buying a new car?? A new house?? Nah. It will be plowed back into the market, in some form. It might go directly to Treasuries, and push up prices and down yields. It might go (indirectly pushing up Treasuries) into some other share, pushing up its price and down its return. Some might end up in the hands of Joe Sixpack Retail Plunger who might buy an extra case of Bud each week, but that'd be the shrinking minority.

No matter how you spin it, liquidity is being pushed into the hands of, mostly, folks with more money than they know how to spend on consumer product already. So, they'll try to spend it on assets. And so goes up the price of assets, and down the various types of interest rate. Most of the recovery from the Great Recession has been driven by unwarranted increases in asset prices. This should be no surprise, as most of the recovery effort, intentional or accidental, has been directed at the 1%. Joe Sixpack hasn't seen much benefit. The financial pundits, and the elites who make the decisions, won't tell us the truth. It might just make enough folks mad enough to see that The Manchurian President has been scamming the Populist President label from the beginning.

By pushing short term rates higher, in the face of stagnant returns on long term real capital in the private sector, the Fed is shooting us all in the foot. Rate inversion, here we come. You won't enjoy the ride.

[update]
Here's some more reporting I just finished reading with my morning greasy spoon java:
When companies have more cash than they believe they can use productively, they typically return it to shareholders either with cash payments - known as dividends - or by repurchasing shares in the market. Buybacks raise demand, putting upward pressure on share prices.

And, as before, rising share prices drives percent return, aka interest rate, down. The Fed can do nothing to make the CxO class smarter at finding new, profitable, ways to build physical capital. Monetarism can go only so far.

11 October 2018

Database Option

This just published at AnandTech:
Q: Are you working with any database providers (Microsoft, PostgreSQL, etc.) to help them take full advantage of Optane's distinctive characteristics (latency/mixed workload perf/low queue depth perf)? Very broadly speaking, how do you currently expect Optane DIMMs' bandwidth and latency to compare to DDR4? Do you anticipate it being possible to create a RAID of Optane DIMMs? Is there anything else that you want to say about Optane and database performance?

A: Databases are a great fit for Optane technology. Yes, we are working with all the major database companies. Here is an example of our work with the Developers of MySQL. You can try it yourself here.

We are building a developer community for optimizing software for Optane SSDs with our partners at Packet. You can join this community on Slack. Details are here.

Similarly, we are growing a developer community for optimizing software applications to take advantage of Persistence, aligned to the Intel Optane DC Persistent Memory DIMMs. More info here.
[my bold]

Wahoo!!!

09 October 2018

Yes Ore No - part the third

In answer to a diminishing number of requests, I offer up another data point in the argument over tariffs. Studious reader will recall "Yes Ore No - part the second" which contains this perfect gem of logic:
Now, you can't have steel, or aluminum, without the source ores. It's not a case of 'where there's a will there's a way'. Yeah gotta have ore.

Putting up tariffs is done for one or two reasons (mayhaps both); 1) to finance the government (here's the data) or 2) to generate excess profit to fledgling domestic industry. As stated, steel isn't fledgling, and aluminum is dead as a door nail. Steel is nearly so, based on known reserves of iron ore. We used to have bunches, but burned up much of it building ships and tanks during WWII. Moreover, to make specialty steels, other minerals are needed.

Which brings us to today's NYT missive about Finnish (mostly, stainless) steel. It's a case where there's no possibility of making such steel with American resources, and the amount of money that tariffs will bring is a spit in the ocean of government budget. Stupid is as stupid does.
Perhaps most important, there are no active chromium mines in North America.

You can't make stainless without chrome. Nickel and molybdenum are commonly added as well.

05 October 2018

He Did It [update]

One of my guilty pleasures is the "Law and Order" shows. Just the Original and "Criminal Intent", though. There must be thousands of actors in the NYC metro area, since they've got to have gone through that many. Some actors are considered "exotic looking". Bette Davis when she was younger. Among today's performers, Penny Balfour is the tops.

In the L&W episode "Hubris", the bad guy is played by Tim Guinee, also border line exotic. Certainly smarmy this time around. During the trial portion of the episode, Guinee is self-lawyer while Balfour is jury foreman. He flirts openly with her. He gets acquitted. She pushes the jury to do so.

With a couple of minutes to go, she comes to see McCoy. Admits that she steered the jury. And that he dumped her right after the trial.
Gibbons (Balfour): "He did it, didn't he?"
Carmichael; "Yeah, he did."

The episode ends with the cops called to her apartment, with Morriston (Guinee) dead on the kitchen floor with a knife in his neck. Seems he attempted to garrot her while she was making dinner, with null effect.

Kavanaugh did it, and listening to Collins dance around the truth for an hour...

[update]
A while back, I heard something about cum laude graduations from Yale around the time Kavanaugh did. Basically, it was a Lake Wobegon sort of place. So, here's the data:
Last year [1987], 46.4 percent of all seniors graduated from Yale with general honors - 11 percent graduated summa cum laude, 11.5 percent magna cum laude and 23.9 percent cum laude. In 1986, 50.5 percent of the graduating class received honors.

How special they were.

01 October 2018

Day and Night

More than once over the course of the last few years, these missives have asserted that the best show on the TeeVee is 'Anthony Bourdain, Parts Unknown'. That was said long before he died in June. As more information emerged about the circumstances, it seemed to me that the most likely situation was a Carradine. After the first 2 episodes of this final set, I'm not so sure. At the time of the suicide, he was in France with Eric Ripert recording an episode, and was later reported to show no signs of despair or other darkness. Ripert, by all accounts, was one of his closest friends. Perhaps first among many.

It was a bit eerie watching the remaining episodes of the then current season being broadcast. He was there in full view, but not. Then, CNN announced that episodes for season 12 in production were in various forms of completion, and would be shown. The season would be 1 fully complete episode, 4 episodes with sufficient location recording, but no post-production narration, and 2 'tribute' episodes. This could be difficult.

Sunday before last was the complete episode, Kenya, with W. Kamau Bell as tag-along sidekick. The majority of episodes over the seasons are just Bourdain going somewhere, meeting up with foodies, musicians, politicians, academics, chefs, cooks, and so on. The crunchier places require some number of fixers to keep things going, if only navigating language. This was a standard 'Parts Unknown', not to be missed, but known to be the last. Ever. What comes next?

Spain came next, last night. Again, a sidekick in the form of Jose Andres. And a good thing, too. Sort of. This episode proves what made 'Parts Unknown' so extraordinary: Anthony Bourdain's view of the human condition, spoken with heart and feeling and experience. Most episodes have Bourdain interacting with local folk for extended periods of time. I never took double stopwatches to time how much of Bourdain we got in situ, and how much post-production narration. I'm pretty sure it's more post, but not by a wide margin. I'll guess that the intent was for Bourdain to fill in the history of Andres, Asturias, the food, the people in post, because we hear very little of him in the episode. We hear Andres. I can't be sure, but I suspect having to do the post for the episode was not an easy experience. There's a short bit about a miners' strike that's just the sort of incident Bourdain would chew on. Loudly, mouth wide open. But we got only a bit of Andres and a local musician telling us about it.

We do get some tighter head shots, which struck me as out of the norm. His hair is silver, and his face is lined like a newly harrowed field. He looks tired. I don't know whether these episodes are in chron order. Which is to say, whether we'll see him change as he approaches his end. Will there be clues to his state of mind, and so on? Will we see him just a few days or weeks before France? Will we be told how these episodes mesh with France? Some of us really, really need to know. The Kenya episode had a few seconds of a wall GUI map of a game preserve, with incident locations and dates, that were February of this year. Just a handful of months left.

The remaining 3 episodes may be painful, if there's not a sidekick or two to fill the void. I'll watch, but I'll likely be angry at the bastard for doing what he did.

30 September 2018

Join Me For Drinks

Some interesting news (well, not news to some):
For the first time since 2005, the U.S. Army missed its recruiting goal this year, falling short by about 6,500 soldiers, despite pouring an extra $200 million into bonuses and approving some additional waivers for bad conduct or health issues.

Those who've been paying attention know that the military has been lowering the bar to recruitment for decades. Fatter, dumber, and crazier after all these years:
Only about 30 percent of 17- to 24-year-olds meet the physical, mental and moral requirements for the military, and only one in eight are interested in serving.

Only about 30 percent of Americans are the The Manchurian President base. That means 70% aren't thin enough, smart enough, and sane enough. Now we know where the base comes from. Coincidence?

Last October, Army said it would meet its goal:
The Army will reach its goal of 80,000 new soldiers without compromising quality, predicted Maj. Gen. Jeffrey Snow, who leads its recruiting command.

Well, no.

Turns out, not too surprisingly, that the problem has been reported for some time. Here's 2006:
Replacing a gunner who'd scored Category IV on the aptitude test (ranking in the 10-30 percentile) with one who'd scored Category IIIA (50-64 percentile) improved the chances of hitting targets by 34 percent.

Is this stat sig?? Not reported, but I'll guess that it's close to population data, so it's just numbers.

Which gets me to wondering: have the Cats been re-defined over time? That is, is a Cat III or IV less intelligent now than in the past? Has there been a double whammy here? The wiki has some history.
The ASVAB was first introduced in 1968 and was adopted by all branches of the military in 1976. It underwent a major revision in 2002. In 2004, the test's percentile rank scoring system was renormalized, to ensure that a score of 50% really did represent doing better than exactly 50% of the test takers.

Was 'renormalized' just another way to cheat dumber kids into the Service? Well, here's contemporary pushback against that accusation.
"We don't want people to think we are lowering the standard of quality coming into the Air Force," Ms. Strickland said. "It appears that way because 36 is lower than 40, but it will still be the same quality of applicants we are accepting into the Air Force today."

That newspeak for: "we've found that all these kids are dumber than in years past, so we've lowered the raw score to meet the percentile we've previously used." The assertion made by DoD is that kids got smarter from 1980 to 2004, so raw scores needed to be lowered to meet the median (IQ?) that existed in 1980.
"Potential servicemembers in the last 20 years, simply put, have gotten smarter," Ms. Strickland said. "So the scores have shifted."
Do you believe that? Make you feel better?

We hear all about, esp. in Military recruiting adverts, how the "new Army/Air Force/Navy" is more tech savvy. Which means more tech savvy personnel. One meme is that video gaming prowess is now a plus. Yes, it's true.

Here's some other data. It's from the wiki, but as usual is heavily linked. It shows that, even after boosting the raw scores with a renormalization, math and verbal SAT scores have been and continue to decline. Is it likely that college aspirational students would be in decline, while HS/GED students are ascendant? I thought so.

24 September 2018

Show Me The Data!!

Just a short note, to tell y'all to go do some reading. As most of you are aware, the fish oil vs. heart disease story has been public for at least years, if not decades. The wiki page on the Mediterranean Diet alludes to fish oil benefit from the 1960s. Exactly when fish oil capsules took off, sometime later. (A bit of surfing revealed "cod liver oil"!! Of course. My Mama didn't force it on me, but yeah, it's been widely consumed nearly forever. According to the wiki, specifically to children as a source of vitamin D.)

What type, and possible mixes, of fish oil has been debated since then. One company, Amarin, has been developing a single oil type, EPA. FDA gave them clearance some time ago to sell it, but to a limited population with skyrocket triglycerides. FDA wouldn't label it for general cardiovascular disease reduction. Amarin was pissed. FDA demanded a large population, long term study; by their nature, CV studies have to take a long time and mostly are post-approval for that reason. We're talking thousands of people for years.

Not the least reason: all the studies on near-similar fish oils have failed. Ummm. The arguing, on a host of venues, between the believers and dis-believers was something not often seen. The MoA of EPA in general CV prevention has never been rock solid; why FDA wouldn't label in the first place, and why pundits were, mostly, predicting the trial would fail. Badly, most said.

The data has been known to be nearly ready for some months, so the bickering intensified, waiting for the shoe to drop. It did this morning. Boy howdy. Not only did it meet the primary endpoint, but with a p-value<.001. Now that's stat sig!! Here's Amarin's PR. There'll be a whole lot bickering for a while. I expect the dis-believers will look to dredge up some contrary conclusion as the data is revealed. Ain't quant fun? To add to the mix, Amarin is one of those USofA corporations that fled to Ireland (in name only, of course; the facilities and work remained stateside, naturally).

23 September 2018

Live Long and Procreate

During the fiasco called Kavanaugh one of the Left pundits (I didn't take notes, but this isn't a unique assertion) allowed has how the USofA is the last modern country which puts its supreme court judges on the bench for the rest of their lives. He also allowed that an 18 year term has been broached, and he supported such. The argument is simply that a life term in 1800 spanned far fewer years than today. That's not disputable. But what is disputable is how much? The pundit stated 36 as the life expectancy then, while now it's around 75. Lifetime appointment is nuts!!!! And so on.

But, naturally, it's a somewhat bogus argument. Life expectancy at birth didn't change by big numbers until the 20th century. Childhood vaccinations, sulfa drugs, antibiotics, and such account for why folks live longer. And most of the increase comes before 21 or so. Here's a succinct table.

So, yes lifetime Supreme Court appointment is nonsense, but not because middle aged white men live a whole lot longer now than in 1800. Another table which indicates about 8 years more now for 50 year olds, a reasonable guesstimate.

So yes, Kavanaugh is a horrid candidate (and should be impeached for lying at his 2004 and 2006 hearings), but he'd not be around as much longer as the "life expectancy has risen 40 years since the Constitution" assertion. Don't use stupid numbers to make your case.

14 September 2018

A Good Wheel

Lots of times there's no reason to reinvent the wheel. The Microsoft/R folks have gone through the Maria deaths report, and provide good evidence that The Manchurian President is a Tillerson. In spades.

The process used by the George Washington researchers is easily defended by anyone who is not innumerate. Go have a read.

13 September 2018

Big Lies - part the first

If you're going to lie, might as well make it a big one. Today is unemployment reporting day, and once again the reported number of new claims is the lowest since 1969. There will be a tweet in due time taking credit for this. Which isn't even true, in the time series sense. But consider the base number for 1969 and 2018: the size of the labor force. The BLS has such data, surprise! Here's the picture.

If you look real close, you can see that the labor force has doubled. Not an apple to apple comparison.

11 September 2018

Papa Doc Makes a House Call

Another skunk in the outhouse. Papadopoulos, convicted, is now setting up interference for The Manchurian President. Too bad the mainstream pundits don't see it. So I'll tell what Papa Doc is doing.

First, the denial/no recollection of the statement to the Australian diplomat about the Hillary emails.
Second, the assertion that Session not only didn't push back on the Trump/Putin meeting, but encouraged it.

The first gag is designed to impugn the onset of the FBI counter-intelligence investigation. Which led to the current mess. Trump gets to claim it's all a fabricated witch hunt.

The second gag is designed to provide an excuse for canning Jolly Jeff, putting in DeVos (or any other bag o flesh that's been confirmed) in as AG and then can Rosenstein, Mueller, and anyone else up on Russia.

Now you know. It'll be a while until the usual suspects figure it out.

The Man of The Book

Two meaningful events today. Both remind us of The Manchurian President: he bragged that his building was now the tallest after the towers were downed, and Woodward's book goes on sale. It's all about The Donald. Sad.

It seems that we can interpret Woodward's book title in two ways. And even at the same time.

As a complete sentence - Fear Trump in the White House. Which means exactly how it reads.

or

As a definition - Fear: Trump in the White House. Which means exactly how it reads.

10 September 2018

Thought For The Day - 10 September 2018

The Manchurian President responded to Obambi's stump speaking with a complaint, as one might expect. The recovery from the Great Recession was the weakest in history. And such. This sounds an awful lot like a motorcyclist, who rides without a helmet, and who runs into a bridge abutment after drinking all night. The Obamacare folks put him back together, but he continues to complain that his face isn't exactly as pretty as it was before. Some ingrates have all the luck.

09 September 2018

Horton Hears a Who

With all the hubbub over the Anonymous op-ed in the Failing New York Times, the most agita (from my reading) is that the Times risks significant loss of credibility, and perhaps faces an existential threat from Dear Leader and minions, if it turns out that Anonymous is the second assistant deputy director of toilet paper in the Small Business Administration. Or some such.

What all those hand wringers appear to ignore is the critical sentence:
I work for the President.

Now, it is possible that the Editors at the Times would allow Anonymous to make that statement, knowing that s/he is really the second assistant deputy director of toilet paper in the Small Business Administration, but I find that impossible to believe. The Editors preamble describes Anonymous as "a senior official in the Trump administration". That's not the second assistant deputy director of toilet paper in the Small Business Administration, either. Those two statements combined have to mean at least Cabinet member or White House denizen. Some people are scared shitless by this fucking moron, as Tillerson has it.

07 September 2018

Protect The Rich. NOW!!

Yet another ode to protecting the rich from the poor huddled masses. Football owners, who on the whole get stadiums paid for by taxpayers to begin with, are being discriminated against by the states' tax policies. Move to Red states where the governments don't provide education, healthcare, clean water, clean air to the bulk of their citizens. Poor football owners.

Here's the tale from Fox Sports; not the left-wing rabble you might expect to talk about the issue.

It's worth noting that moving high capital sports to Red states doesn't work. We already know how that ends up. NASCAR has been failing for some time, especially since the Great Recession. It's just that good ole boys just can't pay the freight to attend. And watching roundy-round cars until one crashes isn't as much fun since there aren't so many wrecks and few die; any more.

03 September 2018

Quants' Hubris - part the fourth

The more things change, the more they stay the same. The perpetual mantra of the financial quant, horrid experience to the contrary.

And so back to the main question - can quants actually predict the future? Quants depend on the past, as manifest in their data, being the same as the future. For natural phenomena, that's true. The problem for quants in that space is knowing what God's Rules really are. Black plague is carried by rat fleas, not a punishment for impiety.
Because they did not understand the biology of the disease, many people believed that the Black Death was a kind of divine punishment - retribution for sins against God such as greed, blasphemy, heresy, fornication and worldliness.

But, in time, a scientific explanation emerged:
The most authoritative account at the time came from the medical faculty in Paris in a report to the king of France that blamed the heavens, in the form of a conjunction of three planets in 1345 that caused a "great pestilence in the air". This report became the first and most widely circulated of a series of plague tracts that sought to give advice to sufferers. That the plague was caused by bad air became the most widely accepted theory.
[the wiki]

Well, may be not so much science. Or, as Mellencamp has sung, "people believe what they want to believe when it makes no sense at all."

We saw just this happen when the quants enabled the Great Recession; believing that house prices could rise, everywhere, like a rocket forever. After all, the data proved it. What the data didn't prove was why so many wonky mortgages could be made. The answer to that question was simple, if not widely expressed: the Powers That Be had changed the rules for making a mortgage. The old data was under the old rules. Not the new mortgages. Oops.

Recent reporting brings us another example of data lagging reality. And, perhaps, just as unpleasant results. You should read it up, if you only follow Fox News. It's worth noting that fracking is nothing really new. Tertiary extraction, which has been used for decades, works by the same principle: force out trapped oil with high pressure fluid. What's new, to me, in this article is this:
A key reason for the terrible financial results is that fracked oil wells show a steep decline rate: The amount of oil they produce in the second year is drastically smaller than the amount produced in the first year. According to an economist at the Kansas City Federal Reserve, production in the average well in the Bakken — a key area for fracking shale in North Dakota — declines 69 percent in its first year and more than 85 percent in its first three years.

IOW, fracked oil is not sustainable, either in terms of output or financially. Which leads to the second question: if these well go dry as fast as a high school lineman sucking up a milkshake, how is output sustained? A good question, it seems to me.
Because the industry has such a voracious need for capital, and capital costs money, fracking could not have taken off so dramatically were it not for record low interest rates after the 2008 financial crisis. In other words, the Federal Reserve is responsible for the fracking boom.

The problem with QE, not so much TARP, was that it didn't make any requirement for sensible use of the capital. So, sense went out the window. The article ends thus
But rhetoric doesn't produce profits, and most things that are economically unsustainable, from money-losing dot-coms to subprime mortgages, eventually come to a bitter end.

Another voice in the choir of quant hubris and the asymptote of progress. If welfare queens had been given the moolah, and went out and bought crack and Cadillacs, ya think the billionaire donor class might have yelled and screamed? Ya think?

30 August 2018

The Oracle Pooped

Yes, the Oracles pooped in FL a couple of days ago, which means there's been time for the pundits (and pollsters?) to do the usual post mortem to explain how they missed Gillum. It appears to be along the same line as 2016/Trump: ineffective stratification. Long time reader knows that one of the mantra of this quant is 'stratification is critical and it is haaahhd'. Getting to the moon was easier.

Given the typical size of such samples, ~1000, accuracy depends of either of two factors: the population is largely homogeneous with respect to the issue being measured, or strata/subsamples have to reflect both defining characteristics of subpopulations and their weights. Miss either, much less both, and your results don't hit the bullseye. The Manchurian President was, inadvertently, right. The pollsters didn't oversample the cities, but rather treated ex-urban areas in those three states as homogeneous to the cities. And, before 2016, the data on which the stratification was based that was an accurate criterion. Some people, voters, were fooled by lying. Some people, pollsters, were fooled by historical data.

It's worth mentioning that The Great Recession happened just because the Smartest Guys in the Room believed that explosive growth in house buying and thence prices was not only sustainable, but an organic growth. If something is too good to be true, it is. In the housing case, the rules of the game were changed, mostly by the Smartest Guys in the Room to their advantage. They were the piper and they played the tune. The rest of us paid the price.

So, with all that, here's one self-congratulatory missive.
A Change Research poll released last week showed Andrew Gillum winning the Florida Democratic primary with 33% of the vote — making it the only pollster to predict Gillum's stunning victory last night.

And, in a footnote, they tell us how they did it (well, not with trumpets blaring, naturally):
Post-stratification was done on age, gender, ethnicity, region, and 2016 Democratic primary vote.

Know your voter.

27 August 2018

The Asymptote of Progress - part the tenth [update]

Boy howdy! Yet more reporting on the next node of semiconductors. Another halfway step to the wall. Their best bit is a comment:
A few years ago, an ex-Intel buffin held a talk on chipmaking at the end of Moore's law. He predicted it would come at 2016-2020 and around 10nm.

That's really it then. We'll get another node at 7nm (eventually) but after that? Slow, incremental improvement.

Few people understand how big of a deal this is, but think about this. All of us grew up in a world where the power and speed of computers and electronics would double roughly every two years.

This exponential growth was a virtual certainty and it fed the biggest prosperity engine in human history. There isn't many fields and businesses that didn't in some way benefit from it.

And that's almost over. There's a few years left where it almost seems like nothing has changed, but it has.

Some of us have children. Those children won't live in a world with electronics doubling in speed and capability every few years.

The next generation of video games for them will look pretty much like the last one, unless you'll know where to look.

For them, it'll be nothing but slow, incremental improvement year after year: A 2% improvement here. A 1% improvement there. The way it was before the transistor.

The scariest thing is, what will happen to the economy. Trillions of dollars and billions of man hours have been spent in the shadow of Moore's law the past decades. What will happen, once there isn't a reason to get a new TV every 4 years? If computers only get a few percent faster every year, will computers become a thing you buy once a decade?
-- V900
[my emphasis]

The other half of the problem is that ole amortization boogie man. The article does recognize that bit of horror:
Development of leading-edge process technologies is extremely expensive. Every new node requires billions of dollars in investments. Those costs are eventually amortized over each chip the company makes, so to keep increasing R&D costs from driving up chip prices, foundries need to produce more chips.

And, naturally, chip makers need customers who make things with the chips that raw civilians buy with their incomes. More concentration of incomes to the 1% leaves an ever diminishing pile for the rest to buy things with said cheeps. The frog is beginning to feel the boil.

[update]
Just got back from my FNYT and coffee to find this new comment:
"Some of us have children. Those children won’t live in a world with electronics doubling in speed and capability every few years."

Oh....the.....horror.....
-- Manch

23 August 2018

Stop Pickin On Me

[apologies to Warren Zevon]

I went home with a waitress the way I always do
How was I to know she was with the russians, too?

I was gambling in havana, I took a little risk
Send lawyers, guns, and money
Dad, get me out of this, hiyah!

An innocent bystander
Somehow I got stuck between a rock and a hard place
And I'm down on my luck
Yes, I'm down on my luck
Well, I'm down on my luck

I'm hiding in [Mar-a-Lago], I'm a desperate man
Send lawyers, guns, and money
The shit has hit the fan

Send lawyers, guns, and money
Send lawyers, guns, and money
Send lawyers, guns, and money, hiyah!
Send lawyers, guns, and money, ow!

Given the reporting out today, yeah, the shit has hit the fan.

We Don't Need No Education - part the third

The right wing's efforts in Social Darwinism hit another home run. Cheer for the hole in one. It seems that teaching is the profession that only the lame want to enter.
[F]ewer college students are studying education. Enrollments dropped by 35 percent between 2009 and 2014, according to the Learning Policy Institute.

So, plenty of excuse to put an end to that pesky free public education. If you can't afford it, you must not need it and you can't have it.

22 August 2018

Another Guy's Rant - part the first

Every now and again, one comes across a rant with lots more words than one's own on the same subject. One just bows in humility. It should come as no surprise that NoSQL is one of my pet peeves. Grant Fritchey goes the extra mile. Bless his heart.

I Remember Mama

These missives have mentioned my forays with the TI-990 mini in two of its forms back in the late 80s and early 90s. The signature characteristic of the machine was its (nearly) registerless design, using main memory for both 'registers' and 'memory'. The motivation for the design was that, at the time, memory speeds and cpu speeds were near enough that a greatly simplified cpu had a cost/benefit edge over conventional designs. Or so the TI engineers thought. Outside of a few verticals, the 990 didn't make much of a splash. Notably, none of the mainframe companies pursued the idea.

Well, may be a bit of that 990 magic could be back. We have this reporting on nanotube memory. It's non-volatile, fast, and dense. A registerless machine, with a single level of storage, makes abundant sense. We'll just have to see.

It's also notable that the company has been around for nearly two decades attempting to get the stuff made. As ARM, it doesn't manufacture. The founders are ex-Harvard, although the Big Cheese isn't a techy. Hmmm??

Thought For The Day - 21 August 2018

Today The Manchurian President now knows what it's like to be on the business end of a double barrel 8-gauge shotgun loaded with deer slugs. Yes, there are such, though not in current production.

20 August 2018

Yes Ore No - part the second

There's a few line story in the news today. It seems Wilbur Ross will be braying at Century Aluminum in Kentucky on Wednesday; a smelter is re-opening idled lines. The Manchurian President wants credit. Recall the first entry in what is now a series: the USofA has little iron ore and virtually no bauxite (in global resource terms).
IF YOU DON'T HAVE STEEL, YOU DON'T HAVE A COUNTRY!
-- The Manchurian President/2018

Of course, that's pure nonsense. In the financialized economy of the 1%, national steel and aluminum are completely irrelevant.

Now, you can't have steel, or aluminum, without the source ores. It's not a case of 'where there's a will there's a way'. Yeah gotta have ore. And for aluminum, lots o electrons.
Alcoa, formerly the Aluminum Company of America, and another American company, Century Aluminum, have opened factories like this in Iceland, and closed factories in the United States, for a simple reason: Electricity is much cheaper here.
...
Bauxite may be mined and refined in Jamaica, shipped to northern Quebec for smelting, then hammered into car parts in Alcoa, Tenn.
[here]

Cheap electricity is what supports smelting aluminum; it requires prodigious amounts of leetle electrons to turn ore into metal. That Kentucky smelter exists because of the TVA.
In Washington state, for instance, the smelters that used to operate near the hydroelectric power plants along the Columbia River have been priced out by the power-chugging server farms of tech companies such as Microsoft.
[here]
and
Aluminum has been called "solid electricity" because of the amount used to make it (13,500 to 17,000 kWh per ton). That's why most of the aluminum smelters were built where the Tennessee Valley Authority and the Bonneville Power Authority could deliver cheap hydroelectric power.
[here]

So, a huge Pyrrhic victory for jingoism. Swell.

17 August 2018

Thought For The Day - 17 August 2018

As if one needed yet more reporting on Turkey basting and the US Buck's supremacy. Not to mention all those trillions looking for hefty returns at low risk, we get some briefing.com words(7:55):
Looking at other markets, the Turkish lira is down 4.2% against the U.S. dollar this morning, on track to end its three-day rebound, but still remains a ways above its all-time low, which it hit on Monday. Meanwhile, U.S. Treasuries are higher, pushing yields into the red, with the benchmark 10-yr yield down two basis points at 2.85%.

Or, as we say (or so I'm told) in the Navy, "Dive motherfucker!! Dive!!!" Yeah, there's no advantage to being the world's reserve currency.

15 August 2018

New Gold - Part the Sixth

The Manchurian President continues braying that the USofA is the victim of bad the international monetary and trade deals. Day after day. While these missives have explained why it is that the US Buck being the world's reserve currency actually gives us the royal flush in spades hand. The Manchurian President is either too stupid to figure that out, or he's just lying to cover his ass. Blame the global elites and their Democratic enablers. And so on.

Well, today brings yet more reporting that the USofA is gutting lots of the world.
A crumbling currency inflicts many damaging consequences: Companies that borrowed in dollars — the global reserve currency — have to come up with steadily more money in their local currencies to repay U.S. dollar debts. Having to do so also raises risks for the banks that lent to them.

That's the kind of squeeze that ignited the catastrophic 1997-1998 Asian financial crisis. A currency disaster in Thailand infected the entire region, and East Asian economies absorbed devastating damage.
[my emphasis]

We're the victims. Yeah, sure.

And, how might other countries manage this:
And emerging-market countries learned lessons from the debacle two decades ago. Many piled up reserves to fend off speculative assaults on their currencies. At the start of 1997, emerging-market countries' reserves amounted to barely 6 percent of their economic output. Now, they equal nearly 18 percent, according to the Institute of International Finance, a banking trade group.

Guess what?? All the US Buck reserves came from our current account trade deficit. No trade deficit, no expanding New Gold stock, no expanding world market, world market deflation. You don't want to live in that kind of world. Arithmetic doesn't lie. Yummy.

13 August 2018

New Gold - Part the Fifth

Here we go again. There are upsides and downsides to the US Buck being the world's Reserve Currency, aka New Gold. One of the side-effects, in both directions, is on domestic interest rates. Many's the time these missives have called out the repercussions of rate inversion. Well, here's some more reporting on where we are.
It has been a long bull run for both stocks and bonds, and borrower defaults have been at historically low levels for years. As has the spread -- the difference between the yields -- of Treasury-backed securities and riskier bonds. But as interest rates continue to rise, and some companies and other borrowers fail to meet their debt obligations, defaults will inevitably increase along with the spreads.

The implication, of course, is that longer term private debt has been written at artificially low rates, and thus high prices. Again, there's nearly $2 trillion in idle money just on corporate balance sheets (see recent missives). Not to mention the 1% and .1%. All seeking 10% and no risk.

So, that reporting was a few days ago. Today we hear another siren, sounding in Turkey, of all places. Much the same as China over the last decade or so, lots of real estate building. As the saying goes, there's only so much time and so much land. You'll never get the former back once it's gone, and there's no more of the latter. Easy money in real estate.
Mr. Lee noticed that Turkish banks were borrowing in dollars to make other loans to fast-growing Turkish companies. He also saw that, over all, Turkey's economy was growing more reliant on financing from foreign investors. It struck him as similar to what had happened to Thailand in the years before the Asian financial crisis in 1997.

Gold giveth and gold taketh away. As with the Great Recession, lots of idle money, unable to find productive homes in physical capital, chases real estate.

09 August 2018

Blues for Donald in D# Major

The most important news out of Tuesday, not widely discussed that I've seen, is that right-to-work got drowned by a blue tsunami in Missouri, the 'Show Me State'. It seems that lots o folks there have figured out that The Manchurian President has been lying all the time. As Rachel Maddow tells us, "watch what they do, not what they say".

The Manchurian President keeps taking from the many poor and giving to the few rich. "Vote for George O'Brien, get Charley off the MTA".

Real growth can only happen if aggregate demand grows, and that has never happened while income and wealth concentrate as they have over the last few decades. Rich people don't buy/spend as much of their moolah, per dollar, as the poor. They chase instruments, and they've had a big appetite for high-return/low-risk such; not that such actually exist, naturally. Which gave us the Great Recession. What will be next? I don't have a firm answer. May be soon.

Along that line, we have this reporting that Treasuries are inching toward 3%, and the pundits are losing their minds. Let's see. Warren is sitting on $116 billion. How much is corporate America sitting on? $1.9 trillion as of the end of 2017. The Smartest Guys in the Room can't figure out how to make real investment. In due time, not much IMHO, they'll go back to chasing Treasuries and driving up price once again. Rates will collapse.

Why? Well, my frenemy Neil Irwin reveals why he's sorta, kinda Luddite.
I think most digital technology is more mature than we think. People have been using personal computers since the 1980s, and smartphones for more than a decade. And the thing about a mature product is that improvements tend to be marginal rather than transformational.

What these missives have been naming, 'The Asymptote of Progress' for a long while. Marginal 'improvements' don't generate significant growth or return, naturally. The latest iPhone doesn't have the impact of the steam engine. That pesky asymptote gets flatter by the day.

02 August 2018

Fool's Goal

Are you old enough to remember pets.com? Cute Lamb Chop-ish sock puppet spokescritter. Lots o fun to watch the adverts. Didn't last all that long.
It began operations in February 1999 and liquidated in November 2000.
[the wiki]

Before I looked it up just now, I hadn't realized just how short its life was. Poor doggy. Got hit by bus.

Today's missive is about a similar ploy: dinner over the internet. There's in the neighborhood of half a dozen such outfits. Today's report is for (7:09)Blue Apron:
Reports Q2 (Jun) loss of $0.17 per share, in-line with the Capital IQ Consensus of ($0.17); revenues fell 24.6% year/year to $179.6 mln vs the $188.51 mln Capital IQ Consensus.

Customers decreased 24% year-over-year and decreased 9% quarter-over-quarter as the Company progresses through the aforementioned transition period. Average Revenue per Customer was $250 in the second quarter of 2018 compared to $251 in the second quarter of 2017, and $250 in the first quarter of 2018.

Going the way of the sockpuppet doggy? Only time will tell.

26 July 2018

Words Have Meaning?

Them Cheatin Canadians

Well, I suppose the truth had to come out eventually. Turns out the seat of Tech Growth isn't where you might expect, if you drink The Manchurian President's Flavor Aid. It's Toronto. Those wily Canadians are using currency manipulation and open border to drain our precious bodily fluids. We must attack before it's too late. Once a friend, now an enemy.
The capital of Canada's Ontario province created 28,900 tech jobs last year, up 14 percent from 2016. Toronto's more than 241,000 tech employees represent a 52 percent increase during the last five years, according to CBRE.

25 July 2018

The Doctor Is In

Recall the mentioning of that old saw, Dr. Copper? Well, a major American copper corporation had this to say (8:09):
Despite the recent decline in copper prices associated with the uncertain impact on the global economy of recent international trade actions, we remain positive on the outlook for copper prices given limitations on supply and the important role of copper in the global economy. To date, we have not experienced a decline in demand for our products, but will be prepared to adjust our plans if necessary to respond to market conditions.

I guess all is well. "What you're seeing and what you're reading is not what's happening."

Here's what's happening to the good Doctor. A bit of whistling past the graveyard. Ya think.

22 July 2018

Thought For The Day - 22 July 2018

'60 Minutes' is in re-run mode, so we've just seen the piece on Hubble. I'm a sucker for such science. One might wonder whether The Manchurian President gives a hearty shit? Probably not. Real Americans don't care about real science.

18 July 2018

Thought For The Day - 18 July 2018

Dear Dear Leader:

It's not about how kissy face you are with Vlad the Impaler, or other dictators; it's about doing what's best for the USofA. Sucking up to Vlad may get you your Trump Tower Moscow, but that's not in the USofA best interest. And, what's this about the SS blocking subpoenas for Jared?? So, I guess you've decided that The Family is above the Law?

16 July 2018

Thought For The Day - 16 July 2018

Well, given the events of today, now would be the right time to apply to the Small Business Administration for a fat loan for my start up, The White Boy Brown Shirt and Jackboot Co. Ya think?

10 July 2018

Quant's Hubris - part the third

Once again, we find quants being utterly, and arrogantly, innumerate. Here is a piece telling quants how to build a "database". It's not, of course. Just a flat file your Grand Pappy would be happy with. Gad. We're into serious civil regression.
Commandment 1: all your data shall fit into one single dataframe

And it goes on to be worse. Sigh.

09 July 2018

Quant's Hubris - part the second

"Regular reader, by now, knows one of the mantras: "the bane of the capitalist is capital". Meaning, of course, that using capital to get rich by substituting machines (hardware and/or software) for people reaches a tipping point. Not just the wall of diminishing returns, but the abyss of unremitting sunk cost. That's just a fancy way of saying that automation methods become more expensive, as unit cost."

Thus begins Part the First (not then known to be yet another serial) of the capitalist's dilemma. Capital, the real physical kind at least, makes its value-add either (or both) by replacing labor while keeping output static, or by increasing output with the same labor. The problem for the capitalist is that as automation (not to mention robotics) moves ahead, labor is gradually (or instantly) removed, thus removing yet more folks from earning and no further benefit from labor substitution. The only way for the capitalist to win when the labor component reaches that tipping point, is to push more output with all those magnificent machines. But can that be done as labor continues to be ejected? Fewer folks earning, whatever happens to GDP, inevitably leads to diminishing demand. Macro, anyway. The reason there's a fraction of the coal miners today vis-a-vis 1940 is not the EPA or other effete bureaucrats, but massive changes in how coal is mined. More machines, less underground extraction (aka, hill topping), poof! There went the jobs. The Kenyan President had nothing to do with it.

Quite the same thing has happened in general manufacturing. First, New England manufacturers departed for the labor antagonistic South. Still aiming to sell to the remaining higher wage North, of course. Thence to Mexico and Central America and Caribbean. And thence to Asia, China as poster child.

There's been a paper making the rounds of the innterTubes for a while now, and makes another appearance today. The in-your-face number is $8.46 as the China specific value-add to an iPhone7. Naturally, one can go find whining that such a number is way too low, but that's not the main point of this missive.

The main point is that amount of direct labor in making an iPhone is teeny, no matter the dollar value. Most of the value in such things is delivered by automated processes; almost no humans touch semi-conductor in production these days. Robots, even. And, of course, China didn't steal American jobs, Steve Jobs and his friends sent them away. And, of course, Apple's source isn't even a Chinese (the big one on the continent), but from that little island called Taiwan.

The authors, without resorting to irony (I would, fur shur), say that Foxconn got about $3,000,000,000 (or may be $4,000,000,000) to make a plant in Wisconsin. Welfare queen. Taxpayers shouldn't pay for sports facilities, either, but that's another show. If you want a rundown, here's one.
To land the massive Foxconn factory, Gov. Scott Walker has committed the state to paying more than eight times as much per job as Wisconsin will provide under similar job creation deals struck last year, a Milwaukee Journal Sentinel analysis has found.

So, the next logical question: does capital hang around after such incentives end? Way back in 2012, the Failing New York Times published an extensive piece on corporate extortion. Worth the read.

A more caustic review, and newer, is here.

What's being discovered, but only occasionally acknowledged, is that the farm-to-factory paradigm of the 20th century no longer holds. Back then, factories absorbed displaced farm hands since skill level wasn't an impediment. Today, we've two problems: 1) new jobs are a fraction of those eliminated, and 2) new jobs simply can't be done by those displaced.

The answer, of course, is socialism, wherein the net winners compensate the net losers. GDP grows, demand is maintained, and technology can continue. If technology reaches a point where it serves only to eliminate macro-demand, it's game over. Reducing price through cost reduction of automation is meaningful only to those who remain with income sufficient to afford the new, lower price. As income concentrates, the number of demand units, aka people, diminishes. Jay Leno may have hundreds of old cars, but that doesn't do much for Ford or GM.

06 July 2018

Let The Battle Begin

It should come as no surprise that I remain a Bayes skeptic. Imagine my amusement (of a macabre nature, naturally) reading this review of Biogen's latest Alzheimer's trial.
Study 201 used adaptive randomisation to enrol 856 subjects with mild Alzheimer's disease across five dose levels. It used a complex Bayesian statistical model to yield a faster result than traditional designs.

You should read the article through. There are issues, in the author's view, beyond the use of Bayes. One might infer that some or all of these other issues are, to steal a word, co-linear with the choice to leverage Bayes.

Alzheimer's is a dread, particularly for those of us past 40 or so. Finding clinical trial failure supportive is perverse, to say the least. On the other hand, sponsors shouldn't (and should really be prevented from) "torture" data to defend a compound. And in this kind of case where the history, even of this compound, with the target is universally negative. What's worse, Biogen isn't some pump&dump nano-cap San Diego bio, either.

Go to your corners and come out fighting!

02 July 2018

RC and A MoonPie

One of Dr. McElhone's more fun adages went, sort of: "breakfast is an RC Cola and a MoonPie". At the time, I don't think I'd ever seen a MoonPie, New England and all that. Just to confirm, I consulted The Wiki, happily discovering I remembered correctly. The little grey cells aren't all done.
There is a custom for eating Moon pies with RC Cola, although the origin of this is unknown.

He also had a long held antipathy for pie charts. That's more common these days. Thanks to the folks at MicroSoft we get an R version of a different kind of "pie" chart. Not only is it a really effective way to present data, but look closely. Orange Julius Caesar is gulling you. Again.

01 July 2018

I Still Hate Neil Irwin - part the eleventh

This time, just go read the piece. As Crooked Hillary said, her votes came from the counties that produces 65% of GDP. In other words, well fed, well educated, healthy folks do better. As pointed out here many times, the pool of white grievance is the result of unfed, uneducated, unhealthy fish belly folks voting in governments which grind them under the boot heel. And, of course, they blame Democrats. Low information voting at its finest.

Just go read the piece. I will note that Virginia, led by the DC suburbs, is throwing off the burden of Dixie. West Virginia, not so much.
Individual proposals aside, experts haven't formed a consensus on how to make economically moribund places feel more like economically dynamic ones. But it is clearer than ever that this divergence explains much of what ails the United States' economy, and just maybe its politics, too.

Quant's Hubris

Regular reader, by now, knows one of the mantras: "the bane of the capitalist is capital". Meaning, of course, that using capital to get rich by substituting machines (hardware and/or software) for people reaches a tipping point. Not just the wall of diminishing returns, but the abyss of unremitting sunk cost. That's just a fancy way of saying that automation methods become more expensive, as unit cost. And it's not just cheap Chinese hands beat robots; it's that robots aren't necessarily adept. We all do remember the smartest guys in the room? Will Musk end up a latter day Lay? We'll see what happens.

Well, Tesla is an exploding data point (oddly, the NYT version???).
Ron Harbour, a partner at the consulting firm Oliver Wyman, noted that in his annual survey rating auto plants worldwide, the most efficient ones use a lot of manual labour. "The most automated ones are at the bottom of the list," he said.

The money quote:
Two assembly lines inside the plant already exist to handle at least some of those tasks, but they have proved troublesome and perform the work more slowly than Musk had hoped, in part because Tesla used robots for tasks that are better left to humans.

Tesla engineering executives acknowledge that the company overestimated the rate at which it could produce cars and designed a production system that proved to be too complicated — a problem that Musk lamented at the company's June shareholder meeting.

"One of the biggest mistakes we made was trying to automate things that are super easy for a person to do but super hard for a robot to do," he said. "And when you see it, it looks super dumb. And you are like, wow! Why did we do that?"

As the title says, the quant's hubris.

Here's the other thing to consider. We've built a global economy on petroleum just because we've found gobs and gobs of it. In order to replace the internal combustion engine with some sort of electric (battery?) vehicle, we must needs have an equivalent supply of some other substance. Lithium? World reserves of it are speculative, but
And if that happens, the 365-year supply would be less than a 17-year supply (13.5 million tons of reserves divided by 800,000 = 16.9 years).

And, of course, that assumes we could economically mine, refine, and produce such batteries. Good luck with that.

27 June 2018

Don't Let the Big Chunks Get Stuck

Over the years there have been commercial zero gravity rides. Whether NASA or otherwise, the name vomit comet applies. I imagine it's quite unsettling. Tender readers will recall the numerous times these missives have pointed out the coming rate inversion (that missive nearly two years ago) will have dire consequences. At one point, I offered up that when it happens it will snap your head off. Blow big chunks. Or something like that.

At long last, the mainstream sees fit to write about it. Took ya long enough, boy.

The Money Quote:
If enough investors begin to grow concerned about a recession, they will most likely put more and more money into the safety of long-term government bonds. That buying binge would likely help flatten, or invert, the yield curve.

Then people will write articles about the curve's sending a stronger signal on recession. And that could, in turn, drive even more people to buy into long-term bonds. Rinse. Repeat.

The fact remains, that American capitalists long ago ran out of new ideas how to spend all those trillions of Uncle Sugar Bucks. Risk aversion is just the polite way of saying they're greedy idiots. They've pissed away the additional trillion they got from Orange Julius Caesar on share buybacks, M&A, and dividends. None of which improve productivity, wages, or employment. Fact is, historically, M&A in particular has been a job destruction exercise. MAWA!!

25 June 2018

Splainin Trump

What follows is, so far as I know, new. It is, too, what Dr. McElhone described as intuitively obvious to even the most casual observer. I have expected that one of the mainstream pundits would get around to offering it up real soon. Real soon hasn't happened, so here we go.

What we know so far is that Orange Julius Caesar and his followers are motivated by white paranoia and white grievance. What we don't know is why it works. Certainly the message is 'Make America White Again' before we're no longer the one race majority. This is what appeals to the lower class white folk, whose only solace in their poverty is the certainty that black and brown folk could be treated even more poorly; there is 'an other' identified to be below them on the totem pole of society. But what motivates the Koch cabal? Ain't no black folk coming to take their billions. They don't need such reassurance. What gives with them, and pretenders to their throne?

Well, it seems pretty obvious. Consider how government is run. Citizens pay taxes of various sorts, and government spends such funds on public goods. Now, when income is more or less equal, each citizen pays more or less the same amount into the kitty. A progressive income tax, which we allegedly have, weighs the wealthy more than the poor out of a sense of fairness. No one, or group, has cause to complain. It is shared sacrifice for shared benefit. Prior to the income tax, most Federal revenue was from sin taxes and tariffs, the latter being in effect a national sales tax. Some assert that the income tax (by way of amendment) coinciding with prohibition wasn't a coincidence, rather an offset to revenue source.

Which brings us to now. What we know is that income and wealth continue to concentrate. We need only recall Romney's 47% admission. It turns out that among the 47% are some quite well off citizens. But the effect remains: as income becomes more concentrated, those at the top see little reason to pay taxes for services they expect they'll never need. After all, they've got plenty of money. In order to keep the government running as it has been, requires $X. But with fewer citizens having sufficient income to be subject to the tax, those privileged ones must needs contribute more to meet the $X requirement. And they just don't want to do that. So Orange Julius Caesar lowered their taxes overwhelmingly. In due time, he'll call for raising taxes and cutting benefits on those of the 47% who aren't well off. "The American taxpayer can't afford it!!"

This is the driver for the 'Donor Class' that pushes Orange Julius Caesar and Trumpism: revenge of the privileged class.

20 June 2018

Phat Country

"My girl left me, my dog died, and my truck won't start." Or something like that is the essence of country music. Now we can add, "I's so fat,I cain't see my dick no more."

The reality of the fat country sluggard is true; let's go to the videotape. Not that anyone paying attention should be surprised. Orange Julius Caesar rode white grievance and white entitlement to his throne. Most of his voters were so conked out on opiates that they likely didn't know what, exactly, Orange Julius Caesar was up to. They just knew he'd put those darkies in their place; while he picks their pockets, to be sure. Make America White Agin. May be enough will croak before the next election? Thinning the herd; just what Stephen Miller wants. Just not his herd, naturally.
Country folk are being hit harder by the U.S. obesity epidemic than city dwellers, two new government studies show. Nearly 40 percent of rural American men and almost half of rural women are now statistically obese, U.S. Centers for Disease Control and Prevention researchers reported Tuesday.
...
For example, people who had a college degree were less likely to be obese or severely obese.

So, here's the question: how much longer will the heads of CDC and the unit(s) doing the research still be employed? I bet they're gone by the end of the week.

Not On My Watch

The outrage over asylum seekers' treatment by Orange Julius Caesar and his xenophobic cabal is genuine. But the wailing that "this isn't America" and such is nonsense. For most of its history, my country has been xenophobic and racist. Moreover, fish belly white folks, who came from countries considered lower than England and Scotland, the Irish as one example, got what Orange Julius Caesar is dishing out today. (This is source.)
Many native-born Americans claimed that "their incessant childbearing [would] ensure an Irish political takeover of American cities [and that] Catholicism would become the reigning faith of the hitherto Protestant nation."

Sound just the bit familiar? Do you want Muslims and brown people taking your country away from you? This is not new. If you don't read the wiki article through, that was in the mid-1800s during the Irish potato failure. For much of the 1800s, job postings carried, not EOE, but NINA. No Irish Need Apply. Orange Julius Caesar is just channeling age old xenophobia.

Site Launch

What's intelligence? What's insight? I'd say a close enough definition is the ability to see a real world truth before there's measurement. This was how Einstein figured out special relativity; observing a bell tower from a tram. Some refer to this process as intuition. Others as real intelligence, vis-a-vis the artificial version.

Which brings me to a David Leonhardt from the last week. One of the bones I've had to pick with the Right, who know they're lying, is their assertion that small business was, is, and will be the savior of American economy. Orange Julius Caesar went on a tirade at a small business conference yesterday. All one need do, over the last three or four decades is see what has happened in one's town, city, or state. The Big get Bigger and the Small get crushed. As Leonhardt says, mainstream punditry, and analytics as well it seems, hasn't been measuring. So he set out to do so. Good on him. I already knew it, of course. But it's heartwarming to see my insight validated.
Today, companies with at least 10,000 workers employ more people than companies with fewer than 50 workers.

There's a, as the Brits put it, knock-on effect of behemothing (that's now a word) of American business. In times past, economists and other analysts have made the point that the blue collar middle class emerged due to the confluence (or symbiosis) of big business and big unions. Auto workers made a middle class living because The Big 3 had oligopoly power, and the unions had the legal clout to force a sharing of that power in a form of enforced profit-sharing. The workers got a share of the vig. So, between the Right Wing destruction of unions and off-shoring of jobs (no, China didn't steal them, American corps. sent them), the blue collar middle class no longer exists. And those knuckleheads are convinced that a big city con man will make it all better? Here's the first paper that came up in a simple search.

You really want to MAGA? Bring back strong unions. With more market power held by large corporations, there's more 'rents' to share with workers. "Vote for George O'Brian. Get poor Charlie off the MTA!"

Ah, Truth

One of my many pet peeves with the right wing set is that they claim the increase in life expectancy since the beginning of Social Security is proof positive that SS is unsustainable. Just look at how many years have been added!!!!!! Well, as has been reported here a number of times, life expectancy increase, measured from any time before about 1960 (and any place), is not due to people of 60 or so living all those many years longer. Au contraire! It's been that young folks, from birth, now make it through young adulthood standing up. Vaccines and sulfa drugs and cleaner water and a host of other public health measures mean that an American baby in 2018 is better off than an American baby born in 1932. But not better than most OECD countries.

Here is a fellow thinker.
Life expectancy is calculated directly from death rates. And mathematically speaking, changes in infant mortality have a much greater impact on life expectancy than do changes in death rates in any other year.

Or, put more starkly:
More realistically, if death rates in the first year went up to 250 out 1,000 (which would be around the worst current day level, but well within historical ranges), life expectancy comes down from 79 to around 50, despite death rates at all other ages staying the same as in 2015 France.

17 June 2018

Thought For The Day - 17 June 2018

Just finished Bourdain's next but last (so far, CNN hasn't said how many season 12 episodes were finished and if so would be broadcast) episode, and did some innterTubes looking about, and came across a CNN piece where an anonymous CNNer allowed has how they should continue the show with a replacement reporter (or whatever Bourdain was; he denied being a journalist). I vote for Paul Theroux. I doubt he would do it, but he's as close to Bourdain as I've encountered.

Other nominations welcome.

13 June 2018

Neo Barnum

There are times, sometimes too many, when quant looks sheepish. This is one of those days. I have a few rules of thumb. One is that biopharma from the Boston-Cambridge axis is legit, while that of the San Diego version is garbage. I don't recall an instance where the latter has failed. But here's a case of the former proving false.
The biotech industry's Arctic ice-cube salesman Cristoph Westphal managed to raise $86 million Wednesday night in an initial public offering to develop a treatment for leg cramps made from food ingredients commonly found in your spice drawer and refrigerator.

That's one guy's assessment from three years ago. Here's today's results
... announced that the Company is ending its ongoing Phase 2 clinical trial investigations of FLX-787 in amyotrophic lateral sclerosis (ALS) and Charcot-Marie-Tooth (CMT) due to oral tolerability concerns observed in both studies, in a subset of patients being treated, with the oral disintegrating tablet formulation at 30 mg, taken three times a day.

What was it Barnum said...? It should be noted that cinnamon, by folklore, is a cure for diabetes. And my PCP swears by magnesium, not potassium, for cramps. Works for me.

09 June 2018

Phil and Tony

In the early 1960s, Bobby Zimmerman left Lake Wobegon, became Bob Dylan and was annointed the angry young man singer-songwriter doing protest songs. In due time, we all figured out that he was just another pop singer. About the same time, a real protest singer, in the folk tradition, emerged. His name was Phil Ochs. In 1976, age 35, he hanged himself in the bathroom of his sister's house. His records, when he could get them made, didn't sell. Live gigs had dried up. He despaired of making a living. You can get his 'greatest hits' from Amazon. And you should.

Friday Anthony Bourdain did the same, only in a five-star hotel in Kaysersberg, France while recording an episode of "Parts Unknown" with Eric Ripert, one of his best friends. He had his crew with him. He was not isolated. He was wildly successful. But he left for parts unknown. Regular readers recall that I've said his show the best thing on the TeeVee, so it's unnerving to consider TeeVee without him.

You'll find all manner of tributes, including one I missed on CNN last night; I didn't know it was on. It'll be run Sunday at 10pm. You should watch. The scheduled episode will go on at 9pm. Whether the remaining completed episodes will be shown, I don't yet know. It is a bit unsettling watching the 'favorite episodes' as I write this. Still, it would be a shame if CNN withholds completed episodes.

JOOQ

I can't say I know much about JOOQ. Seen it referenced here and there, but this piece just appeared, and I am heartened to see someone in the java world make a strong case for 'database first, code last'. There is hope for Western Civilization.
The real "truth" of your database schema, and the "sovereignty" over it, resides with your database. The database is the only place where the schema is defined, and all clients have a copy of the database schema, not vice versa. The data is in your database, not in your client, so it makes perfect sense to enforce the schema and its integrity in the database, right where the data is.

The only point missing from the article: an Organic Normal Form™ schema will be, to some small epsilon, immune to schema update modification problems, since tables/columns in normal forms are, more or less, orthogonal. They can be, if one wishes, fully orthogonal, and when so, there exist no side-effects. The re-generated client code will be restricted to the manifestation of such tables. Easy peasy.

Going database first does demand that rogue coders aren't welcome in development. Database first development demands careful thought aforehand, an approach not welcomed by the coding community. Coders want to keep making mistakes, finding them in the debugger, patching. Rinse repeat. There is a better way.

And, there isn't any such thing as impedence mismatch. That's been dealt with here more than once. This is spectacularly true in the java world of actionObject/dataObject. It's a witch hunt.

04 June 2018

A Hop, A Skip, and A Jump - part the fifth

Yet another carrot for the stew, a paper from last year directly discussing RDBMS on PM. Of all the references so far in this series, this one is *a must read*. Really. Not just because it validates every notion flowing from my noggin. No, not at all.
In this tutorial, we provide an outline on how to build a new DBMS given the changes to hardware landscape due to NVM.

Yum. And it's from what was Carnegie Tech (which accepted me, but I ended up going elsewhere), one of the first Computer Science curriculums!
In the case of memory-oriented DBMSs (e.g., VoltDB, MemSQL), they contain components for overcoming the volatility of DRAM. Such components may be unnecessary in a system with byte-addressable NVM with fast random access.

Exactly why the announced kludges for *nix/Windows aren't the end. The OS/400 turns out to be the winner. In some form.
Peloton is an open-source HTAP DBMS that we are building that is designed from the groundup to use NVM. Our intended audience are developers, researchers, and practitioners with knowledge of DBMS internals. They do not need any in-depth background or experience with NVM.

Here we see another step closer to the OS/400 model:
To cope with these shorter NVM latencies, Microsoft and Linux are adding support for direct access storage (DAX) in Windows Server 2016 [48] and Linux 4.7 [1], respectively. With DAX, a DBMS directly allocates and uses NVM without an intervening filesystem. This requires only one copy between the file and the user buffers, thus improving the file I/O performance by an order of magnitude compared to block-oriented filesystems.
[my bold]

Recall an earlier part of this series' mention that transaction semantics will be different with PM? Make it so:
The write-ahead logging (WAL) protocol supports efficient transaction processing when memory is volatile and durable storage cannot support fast random writes [58, 39, 33]. But this assumption causes unnecessary performance degradations in a DBMS with NVM storage [14]. Consider a transaction that inserts a tuple into a table. A DBMS first records the tuple's contents in the log, and it later propagates the change to the database. With NVM, a DBMS can employ a logging protocol that avoids this unnecessary data duplication. The reason why NVM enables a better logging protocol than WAL is two-fold. The write throughput of NVM is more than an order of magnitude higher than that of an SSD or HDD. Further, the gap between sequential and random write throughput of NVM is smaller than that in SSD and HDD. Hence, a DBMS can flush changes directly to the database in NVM during regular transaction processing [15, 14, 12, 64, 40, 62, 80].

The Future is Now.

A Hop, A Skip, and A Jump - part the fourth

During my near awake status, it occurred to me that the I5 System (nee, AS/400) would be the perfect candidate for PM. The machine after all, doesn't use a file system in the *nix/Windows sense, but an object store (in the pure data sense). It's machine designed, however well applications exploit it, to be Relational.

Turns out, some folks already have. And not just with the Optane announcement.
But key here is that when the object gets created, it is also created in a system-wide address space that IBM i calls Single Level Store (SLS). In the context of this article on persistent storage, it happens that SLS is also persistent; if you restart the OS — say after a power failure — the objects remain with the same address. Said differently, every byte of data out on your hard or solid-state drive has a corresponding location-independent high-level address used by the OS. If your program wants to reference that byte in whatever object, your program just uses the byte's address, no matter how many times the OS was restarted.

Some folks consider IBM to be a hidebound dinosaur. And, to some extent that's right. But here's a bit of history of the I5 OS:
IBM's design of the single-level storage was originally conceived and pioneered by Frank Soltis in the late 1970s as a way to build a transitional implementation to computers with 100% solid state memory. The thinking at the time was that disk drives would become obsolete, and would be replaced entirely with some form of solid state memory. IBM i was designed to be independent of the form of hardware memory used for secondary storage. This has not come to be, however, because while solid state memory has become exponentially cheaper, disk drives have also become similarly cheaper; thus, the price ratio in favour of disk drives continues: very much higher capacities than solid state memory, very much slower to access, and much less expensive.
[my bold]

A few years ahead of his time.

03 June 2018

A Hop, A Skip, and A Jump - part the third

Some more spelunking. Turns out that linux and Windows have existing mechanisms to leverage persistent memory under the covers of block-mode IO; i.e. mmap and DAX. Such might be sufficient to get by in the short term. Here's Intel's report.
Both Windows* and Linux* support memory-mapped files, a feature which has been around for a long time, but is not commonly used.

Interestingly, the BDFL of Progress database said, around 1990, that if your database didn't use mmap, get another database. That long ago.

Intel also has a FAQ, which has some interesting tidbits.

02 June 2018

A Hop, A Skip, and A Jump - part the second

Do you remember that Date, at least by his last version (re-gifted previous ones along the way so I can't vouch for them) of "Introduction..." included a section called "ACID dropping" in chapter 16? I wasn't convinced way back in 2003 when the edition was published. With hardware prior to Optane, updates are a serial process, thus the need for ACID.

With what's now being named "Persistent Memory", of which Optane is Intel's version, we have to re-think transaction semantics. Let's assume, for the sake of argument, that 4TB Optane machines with a slot or two of DRAM and an SSD for OS and applications storage (herein, Godzilla) are the norm in Server World. Who are the losers? I'll argue that ACID and MVCC are the most obvious candidates for being no longer relevant.

With HDD as permanent storage, the hop, skip, jump process is required to process updates. ACID is the mechanism to manage concurrency in a locking protocol (MVCC still locks, just late rather than early). With Godzilla as the platform for RDBMS systems, Codd's "all at once" is natural; no hop, skip, jump. Will it be faster than hop, skip, jump in all cases? I've no idea. We don't yet have on-the-record speeds for Optane. But my suspicion is that most of the code in relational/SQL engines goes away, particularly for schemas that are in Organic Normal Form™. Explicit locking, in the engine, won't be needed, since the OS takes care of memory locks, and that's all there are. Since updates become visible essentially immediately, the locking avoidance of MVCC is, also, not needed.

Godzilla won't be birthed for a while. *nix/Windows and datastore based applications will need to be modified, or more likely forked, to support/exploit Persistent Memory. File system no longer matters, objects (in the older sense) are all that exist. Search can, and should, be relational! No more silly one-way to the bottom hierarchies. And, that's not far fetched, since such machines existed in the past (not sure if the current progeny are the same), called AS/400, itself a development from earlier machines. The OS included an embedded RM-ish object engine, which got renamed DB2/400 for a while. In one incarnation of my employment, an AS/400 hosted application lost to one on an early RS/6000 and Progress. Worked out OK, since that company is still running the code.
Unlike the "everything is a file" feature of Unix and its derivatives, on IBM i everything is an object (with built-in persistence and garbage collection).
Remember, the hierarchy file system was concocted to support a server/terminal hosted word processor: Unix. Since Bell, as most any corporation, was built as a top-down org chart, so too was its word processor. All before Codd had released his first version, and when the state of the art datastore was IBM's mainframe IMS. Yes, the iconic hierarchical "database".

31 May 2018

A Hop, A skip, and A Jump

Free at last, Free at last, Thank God almighty we are free at last.
-- Martin Luther King

OK. So what does such a quote have to do with a site, titularly at least, devoted to the RM, stats, and data generally? Well, this new report on Intel's Optane DIMM.

[On a side note, the earlier piece on Optane SSD has a comment from the author to the point that the byte addressability of Optane is of no value for files. That's true for fixed sector non-mainframe HDD, and later SSD, but not true for CKD DASD:
IBM CKD subsystems initially operated synchronously with the system channel and can process information in the gaps between the various fields, thereby achieving higher performance by avoiding the redundant transfer of information to the host. Both synchronous and asynchronous operations are supported on later subsystems.
]

Current SQL engines do their work with a hop, skip, and a jump. Optane, with proper motherboard, chipsets, and (heavily?) re-written applications can go from cpu to datastore. We could, then, do what Dr. Codd said: "all at once". Yum.

The current paradigm is from cpu registers to on-chip cache to memory to SSD/HDD; more or less a hop, a skip, and a jump. Now, for SQL databases (or any serious datastore), what generally happens is that the engine writes to the trans log (more or less synchronously), which is at some later time flushed to the "permanent" datastore on SSD/HDD. A system/application crash is supposed to only lose open transactions; all commited trans exist in durable storage either in the trans log or named tables. Wouldn't it be cool if transactions commit to durable storage immediately?

As the piece says:
Intel has been laying the groundwork for application-level persistent memory support for years through their open-source Persistent Memory Development Kit (PMDK) project, known until recently as NVM Library. This project implements the NIA NVM Programming Model, an industry standard for the abstract interface between applications and operating systems that provide access to persistent memory. The PMDK project currently includes libraries to support several usage models, such as a transactional object store or log storage. These libraries build on top of existing DAX capabilities in Windows and Linux for direct memory-mapped access to files residing on persistent memory devices.

Application re-write may be non-trivial:
Intel will be offering remote access to systems equipped with Optane DC Persistent Memory so that developers can prepare their software to make full use of the new memory. Intel is currently taking applications for access to this program. The preview systems will feature 192GB of DRAM and 1TB of Optane Persistent Memory, plus SATA and NVMe SSDs. The preview program will run from June through August. Participants will be required to keep their findings secret until Intel gives permission for publication.

But for the RM world, Optane offers the ability to do what Dr. Codd said, "all at once".

One might ask, in a semantic sense, how a SQL engine on such a machine might do its work. Let's chew on that.

First, what about the divide between trans log and named tables? Do we still need a trans log? The blockchain example, which is a database-less trans log, suggests we do. For audit purposes, it is needed, too. Does the trans log still need be the bottleneck to the named tables? Not with such Optane storage. The engine would just do immediate writes to the log and the table(s); that's the duration of the transaction.

Second, do we still need buffers? May be, but may be not. The purpose of buffers is to mediate between fast memory and slow disk, now SSD, but still slower. According to the report, 512GB DIMMs will be available. Current boards go to eight slots, which works out to 4TB. How many applications need more than that? Google and Amazon and Facebook. Commercial applications? Not so much. Taking into account the data reduction side-effect of 3 or 4 or 5 normal form, may be most would be happy with that much storage.

Third, typical applications don't keep all data in hot storage even now, so 4TB looks to cover many, if not most, use cases.

Fourth, some DIMM would be normal DRAM to hold OS and application code and code's invariant data; this appears to be supported according to the report and comment stream. It would certainly make sense to do this.

So, how would a SQL engine work with such a machine? The engine and its data would reside in DRAM, while the tables and (active) log in Optane. By active log, we mean that the engine would flush completed transactions to SSD as needed; in reality, only long running transactions (you don't write such monsters, do you?) would be in the active log. There would be no need for memory buffers, but the notion of locks needs to be considered. Current practice whether locking or MVCC relies on memory buffers to "improve" performance. Simple, single row, update would be written directly to table and log. Normal memory locking would suffice. Multi-row/table update? Umm. In the MVCC case, it depends on multiple images of rows to segregate transactions, but would there still be any point to MVCC semantics with Optane storage? For now, I'd say not. Since update (could/should) happens "all at once", collision doesn't occur; each client sees the current state of each row all the time, modulo duration of actual write. Isolation amounts to memory locks. Cool.

Years ago, diligent reader will remember, I had occasion to work with TI-990 machines. One ran a Ryan McFarland COBOL application on a TI machine while the other ran the chip version on a custom board/OS/language (ah, those were the days). What was (nearly?) unique about the 990 architecture was that it was registerless, sort of. All code and data resided in memory, and there were three cpu registers, one of which was a pointer to the current application's base address in memory. A context switch required only resetting these registers. Each application retained its context in situ. I wonder whether TI will resurrect it?
The TI-990 had a unique concept that registers are stored in memory and are referred to through a hard register called the Workspace Pointer. The concept behind the workspace is that main memory was based on the new semiconductor RAM chips that TI had developed and ran at the same speed as the CPU. This meant that it didn't matter if the "registers" were real registers in the CPU or represented in memory. When the Workspace Pointer is loaded with a memory address, that address is the origin of the "registers".

Whether Optane has much value-add for most applications, I don't know. But for heavy multi-user/single datastore applications such as SQL engines, yeah man!!

This, of course, is merely first-pass thought experiment. More to come.