Theranos' AACC Presentation: Lack of Data Proves Doubters Were Right

Yesterday afternoon, Elizabeth Holmes, embattled CEO of Theranos, took to the stage at AACC to finally reveal the science and data behind their products and tests. She started with a heartfelt apology for the mistakes the company had made, for the misleading information and marketing both to patients and investors, and proceeded to finally show the actual data and methodology of their tests, warts and all, and then pleaded with the community to help the company move forward and realise the initial promise of Theranos, of multiple, accurate, rapid tests from a single drop of blood. The crowd rose to their feet at the end, and applauded her honesty and humility.

Or at least that's what happened in an alternate reality where shame over fraud and scientific honesty are foremost in the minds of startup CEOs. Instead, what we were treated to was a marketing presentation for a future Theranos product, no apology, no real data, and evasion of any substantive questions. If you have 90 minutes to spare, watch the whole thing, it's a masterclass in abusing the platform given by a professional organisation to do marketing and try to give the company a veneer of credibility to those not versed in the subject.

They didn't just fail to reach the low bar set for them - they didn't even attempt to reach it. To list out what was wrong with this presentation would take a book to cover fully, and there are multiple articles detailed issues with it such as the Wall Street Journal, Wired, and the New York Times. I'll try and cover the main points below - for those of you looking for a history of Theranos, there are some summaries here, here, here, and here.

As someone who has actively organised and managed scientific conferences and talks, I'm stunned by what AACC leadership allowed Theranos to get away with. Theranos committed multiple major faux-pas, and Holmes' celebrity status allowed her to get away with things no other presenter would have been. In a scientific conference the goal is not just to present data, but to withstand the scrutiny of your peers and as such the question component is just as important. The talk was set for 90 minutes, split into 45 minutes presentation and 45 minutes questions. Holmes gave a very polished and well rehearsed talk for an hour - clearly and deliberately done to cut 15 minutes of time from questions. This is an old tactic, commonly used by presenters who fear questions, and one an experienced chair/moderator knows to look for. The session chair was left with the choice of cutting them short and leaving out the component everyone thought they wanted to see, the promised (though never really delivered) final section on actual results comparison, or allowing Holmes to cut into the question time. They went for the latter option, a difficult call to make and one I can understand them doing.

Holmes also committed one of the worst crimes - not presenting what was claimed in the abstract, and instead giving a marketing talk for an upcoming company product. In the Wired article linked above, Stephen Master, professor of clinical and lab pathology at Weill Cornell Medical College. “This sounded like a talk from a manufacturer.” Conferences have commercial presentation sections for that type of talk, and anyone who gives a marketing talk in a scientific slot will never be invited back. AACC leadership should have made comment on this after the talk, stated it was unacceptable, and in being silent they are effectively endorsing this approach.

Her talk was slick and obviously well rehearsed, and covered things in a way that would be ideal for fundraising, but inappropriate for a scientific talk. She used buzzwords that CEOs seemingly have to say, without apparent understanding of their meaning. "Inflection Point" is one of them, which is code to a VC that "we're about to have massive sales, real soon, honest" but everyone else knows is just cover for poor performance to date. For example, Holmes uses it at 42m40s correctly to describe a test function, but tries to pretend at 56m20s that sales of this future miniLab will soon boom. Meredith Perry, Founder and CEO of uBeam that similarly has been questioned about the validity of its technology, said in September 2015 "We’re at a massive inflection point. We are about to head into a completely new phase of growth.” but nearly a year later no sales, no demonstrations, no products. Beware the CEO who promises an inflection point, it's nearly always a false promise.

Instead of answering the many questions scientists had, Theranos presented their future 'miniLab' device, a 'cloud connected' unit that may play well with VCs but fell flat with doctors and scientists who actually use the equipment. Geoffrey Baird, an associate professor in the department of laboratory medicine at the University of Washington, commented in John Carreyrou's Wall Street Journal article:

Every piece of technology they presented has been known for many years, and exists in other platforms largely in the same configuration, or in some cases in much more compact form in competitor’s platforms.

So even the future (non-existent) product they are trying to distract us with does not even move the current state-of-the-art forward in any way. In the initial questions following the talk, all done in comfy chairs and a collegial atmosphere I have never seen at a post-presentation question session, one of the moderators called Holmes on the pedestrian nature of the equipment and the departure from the world-changing claims Theranos had raised $700 million with. Masters noted that the original claim was "Many broad tests, 70 tests from a couple of drops of blood - this falls far short of that... I can buy a point of care instrument today that does a finger stick lipid panel." and received much applause from the audience. Holmes then did what was standard for the rest of the question session, talk a lot, but fail to actually answer the question asked.

Holmes clearly had memorized the line "What we wanted to do here today was..." followed by a few options such as "begin engaging with the community." or "present our future plans to the audience." No-one corrected her that this differed from the abstract of her conference talk, but you could tell people were unhappy with those non-answers.

This continued with all questions designed to elicit information such as "Why present data for potassium with venous not capillary blood?" (a key question about their methods) the reply was "What we wanted to show was..." followed by no answer. When asked about false positives and concordance rate, the reply was that their technique was "Very good" and then no answer, even when asked a second time. It was clear no real answers would be forthcoming. 

Whenever pressed to answer questions on previous methodology based on their Edison systems (which were never mentioned once in the talk despite being the main company product to date), Holmes gave the most frustrating answer of all:

In the appropriate forum, we’ll address those. But today we’re hoping to be able to engage on a scientific exchange on this platform.

A canned answer, giving no indication of when that data would be forthcoming or what that appropriate forum would be - ridiculous given that there probably is no better place than the AACC, and that audience, to address them. The real data everyone wanted to see, the topic everyone wanted to cover, was dismissed in those two sentences, and not a peep from AACC leadership or moderators on that. 

As Wired magazine points out, Theranos just 'pivoted' - that's when a startup company realises their past approach and technology just can't do what it was claimed, and try to shift to another product or market in an attempt to save the company. It's clear from the reactions that no-one was impressed by the miniLab equipment, and that had this been what was pitched to investors, there would have been no $700 million raise. As one of the moderators, Master, said in Wired:

Does this live up to the hype? The answer to that is no.

Finally, no-one raised the most obvious question which should have been:

"Since you (Holmes) have been banned by the CMS from running a company doing blood tests, and you have made it clear that you will not leave the CEO position, how will the Theranos be able to offer this test and equipment?"

That in itself would have been worth the non-answer answer that would have come.

So in summary, Theranos avoided answering any of the questions scientists wanted answered, have abandoned their old technology, and shifted to a new unproven technology that has no advantages over existing available equipment. They truly live in another reality if they think this will be enough to save the company.

Equally importantly though, AACC allowed its prestigious name to be used by someone being investigated by both the SEC and FDA for fraud, to not present real data but instead a marketing talk and to avoid substantive questions, and to fail to press even on the soft-ball questions it asked. Holmes may not have the capability to show remorse, but the AACC should be ashamed for what it has enabled here.