A comparison of COVID-19 Vaccine Candidates in the United States (September 2020)

Short summary / Key points:

Credit: JPMorgan Chase & Co
  • US Government has signed contracts to buy 800 million doses of vaccines for its citizens:
    • Pfizer & BNTX’s ‘BNT162’ vaccine
    • Astrazeneca’s ‘AZD1222’ vaccine
    • Moderna’s ‘mRNA-1273’ vaccine
    • GlaxoSmithKline and Sanofi’s vaccine (as of yet no name)
    • Novavax’s ‘NVX-CoV2373’ vaccine
    • Johnson and Johnson’s ‘Ad26.COV2.S’ vaccine
  • Several of these contracts include the option to acquire hundreds of millions more
  • Several trials will be finishing in October, with approval and rollout possible as soon as November
  • Healthcare professionals, high risk groups, and essential employees will be among first recipients in 2020
  • There are some differences appearing among these candidates which I will go into below

 

Pfizer/BioNTech’s BNT162b2:

“Immunogenicity of BNT162b1 and BNT162b2. Participants in groups of 15 were vaccinated with the indicated dose levels of either BNT162b vaccine candidate (n=12) or with placebo (n=3) on Days 1 and 21. Reponses in placebo recipients for each of the dosing groups are combined (P). The 28-day blood collection is 7 days after the second vaccination. Sera were obtained before vaccination (Day 1) and 21, and 28 days after the first vaccination. Day 35 is shown only for the 10 µg and 30 µg BNT162b1 (18–55 years) groups. Human COVID-19 convalescent sera (HCS, n=38) were obtained at least 14 days after polymerase chain reaction confirmed diagnosis and at a time when the donors were asymptomatic. b. 50% SARS-CoV-2–neutralizing geometric mean titers (GMTs). LLOQ is 20. Each data point represents a serum sample, and each vertical bar represents a geometric mean with 95% CI. The numbers above the bars are the GMCs or GMTs for the group. Arrows indicate timing of vaccination (blood draws were conducted prior to vaccination on vaccination days).”

BNT162b2 is Prizer/BioNTech’s most promising candidate (and most advanced, in terms of regulatory approval). Pfizer has began Phase III testing in the US with their 30μg dose (see below). They are on track for regulatory review by the end of October, with approval meaning the vaccine could roll out as early as November. They, along with Moderna, are among the most advanced (in terms of regulatory approval) US candidates.

Promising Phase I data has been reported, with the 10μg and 30μg doses providing  SARS-CoV-2 neutralization titers that were higher than those found in serum of recovered COVID-19 patients. They are moving on to Phase III testing with the 30μg dose, according to a July 27th press release1. The vaccine will involve 2 doses, 21 days apart2.

Some of the team’s discussion is included here:

“Previously reported data from vaccination of 18–55 year old adults with 10 μg or 30 μg of BNT162b1 suggested that it could be a promising COVID-19 vaccine candidate.2,5 Consistent with our strategy to evaluate several RNA vaccine candidates and make a data-driven decision to advance the candidate with the best safety and immunogenicity profile, we compared clinical data obtained after vaccination with BNT162b1,2,5 which encodes the RBD, or with BNT162b2, which expresses the full-length spike. The data set presented here guided our decision to advance BNT162b2 at the 30-μg dose level into the Phase 2/3, global safety and efficacy evaluation in participants 18–85 years of age.”3.

Astrazeneca’s AZD1222:

Astrazeneca’s most advanced candidate is the ChAdOx1 nCoV-19 vaccine (also called AZD1222). On August 15th, Astrazeneca published phase I/II data in The Lancet. 4

35/35 participants had neutralizing antibody responses (measured by PNRT) against nCoV-2019 after one dose. After a booster dose, it remained a 100% rate of neutralizing antibody response. This trial, to the best of my knowledge, did not publish quantifiable titers in experimental subjects compared to those in convalescent plasma of recovered COVID patients. They did publish a figure which graphically represented it (see right).

“Between April 23 and May 21, 2020, 1077 participants were enrolled and assigned to receive either ChAdOx1 nCoV-19 (n=543) or MenACWY (n=534), ten of whom were enrolled in the non-randomised ChAdOx1 nCoV-19 prime-boost group. Local and systemic reactions were more common in the ChAdOx1 nCoV-19 group and many were reduced by use of prophylactic paracetamol, including pain, feeling feverish, chills, muscle ache, headache, and malaise (all p<0·05). There were no serious adverse events related to ChAdOx1 nCoV-19. In the ChAdOx1 nCoV-19 group, spike-specific T-cell responses peaked on day 14 (median 856 spot-forming cells per million peripheral blood mononuclear cells, IQR 493–1802; n=43). Anti-spike IgG responses rose by day 28 (median 157 ELISA units [EU], 96–317; n=127), and were boosted following a second dose (639 EU, 360–792; n=10). Neutralising antibody responses against SARS-CoV-2 were detected in 32 (91%) of 35 participants after a single dose when measured in MNA80 and in 35 (100%) participants when measured in PRNT50. After a booster dose, all participants had neutralising activity (nine of nine in MNA80 at day 42 and ten of ten in Marburg VN on day 56). Neutralising antibody responses correlated strongly with antibody levels measured by ELISA (R²=0·67 by Marburg VN; p<0·001).”  5

On August 31st, The NIH announced that the AZN1222 phase III clinical trial in the United States had begun and will involve 2 doses 28 days apart, with the placebo being a saline injection (as opposed to a meningitis vaccine, like the UK trials). Their goal is to enroll approximately 30,000 patients.

 

On September 9th, AZN announced a halt in the study to investigate an adverse event in a participant6. Previously, results were expected in November.

Moderna’s mRNA-1273:

Moderna’s age-stratified data, as presented to the Advisory Committee on Immunization Practices on August 26th. Key points include the comparability of titers across all age groups, as well as median titers for 56+ y/o individuals being above those in convalescent sera.

Moderna’s most advanced candidate (in terms of regulatory review) is the mRNA-1273 vaccine. Moderna’s phase I trial for this candidate involved doses of 25μg, 100μg, and 250μg, in two doses, 28 days apart. The results of this trial were described on July 14 in a NEJM article7.

According to these results, as of day 43, the 25μg dose and 100μg dose provided neutralizing antibody titers (measured by PNRT) that were 2.1x and 4.1x those found in serum of recovered COVID-19 individuals, respectively. The PNRT of the 250μg dose was not evaluated, for reasons I honestly couldn’t find. It’s possible the data on those individuals just wasn’t ready yet.

As of July 27th, Moderna is moving forward to phase III testing of approximately 30,000 individuals at its 100μg dose8. Their vaccine involves 2 doses, 28 days apart. Like Pfizer/BioNTech, this vaccine is on track for review by late October, with possible rollout in November.

On August 26th, Moderna updated the ACIP with age-stratified date from its phase I trials. The data was positive and supported immunity in all age groups, most importantly in elderly individuals (an absolutely critical need of these vaccines is the ability to build immunity in the elderly). Their slides are available here. A screenshot from one of their slides is above and to the right.

Moderna’s vaccine is notably more stable than the Pfizer/BioNTech vaccine. The latter requires transportation at -70 degrees C. Moderna’s requires -20 degrees C. Moderna’s can also be refrigerated for 7 days after being removed from its -20 degree C vessel (compared to 24 hours of stability for Pfizer’s). This difference is significant when considering distribution in healthcare systems with less advanced infrastructure (this includes both rural US areas as well as developing nations).

Novavax’s NVX-CoV2373:

Vaccine responses compared favorably with
HCS in patients with clinically significant disease

Novavax released phase I data in early August and is moving toward phase III trials at the end of September to the best of my knowledge (“end of September” is what I was told by an employee at one of their clinical sites).

Here is a slide deck presented to stakeholders on August 4th. Here is the pre-print of their early results, published August 6th.

Early Novavax trials were investigating the vaccine’s performance as well as how Novavax’s proprietary adjuvant, Matrix-M1, affects response. 

Their 5μg dose + 50μg adjuvant dose (5+M; 5+M column, right) developed neutralizing titers at levels roughly 4x those found in convalescent serum. I’m unsure what method they used to quantify this.

Based on information here it appears they’re moving forward with their 5μg/ 50μg adjuvant in 2 doses, 21 days apart.

GSK/Sanofi:

They are behind other manufacturers, just recently launching their phase I/II trials in the US (as of September 8th). On this timeline, phase III trials will begin at the end of 2020, with regulatory approval possible in the “first half of 2021”9.

Johnson & Johnson subsidiary Janssen’s ‘Ad26.COV2-S’:

This candidate published nonhuman primate (Rhesus monkey) data at the end of July10. Phase I/II trials are currently underway and pending regulatory approval, they may begin phase III trials by the end of September11. Their ClinicalTrials.gov page lists the phase III trial as already being underway, but I believe this is a mistake.

Inovio’s INO-4800:

I will update this with more info when  I have time. As of now, approaching p3 testing. No publication of p1/2 results, although a brief summary of data has been available to investors. Vaccine has displayed both cell-mediated and humoral responses. See image on right.

INO-4800 notably has a long shelf life… doesn’t require refrigeration. A consequence of the fact that it is a DNA vaccine. Makes it attractive for developing healthcare systems or rural use in developed systems.

 

 

I will update this article with more info when I have time.

 

Ulysses’ Gaze – Sarajevo, Ithaca, and the Cyclical Nature of the Film

This is the 2nd half of an essay I wrote for an upper division Classics course at Ohio State University. With the scarcity of recent writing on this film, I wanted to give others a chance to stumble on it. I change it much from the PDF I submitted so sorry for any formatting errors…


The Siege of Sarajevo was a 1,425-day long siege that is among the most brutal attacks on a civilian population in recent memory. For background, there were three prominent ethnic groups in Bosnia and Herzegovina; Croats, Serbs, and Bosniaks. Ethnic tensions were at a breaking point. After the early 1990’s breakup of Yugoslavia, nationalism was at a peak. In a 1991 census, “44% of the population considered themselves Muslim (Bosniak), 32.5% Serb and 17% Croat, with 6% describing themselves as Yugoslav” (Klemencic 2004). The three ethnic groups each had a vision for their state’s future. An independence referendum was voted on completely by ethnic lines. As war began, Serbs fought to keep territories from claiming independence, and Croats and Bosniaks fought together against them. Eventually war broke out between Bosniaks and Croats as well.

 

The Serbs began an ethnic cleansing of Bosniaks, which Serbia and some others still deny to this day (several prominent Serb leaders – both Bosnian Serbs and Serbians – were found guilty of genocide in UN war tribunals). By April 1992, the rebel Serbs controlled 70% of the country and began shelling the Bosnian capital of Sarajevo (Carroll 2009). This war torn city has become of the most identifiable parts of the Bosnian War.

 

Though it may be only a few paragraphs in history textbooks on our side of the Atlantic, it was a World War-esque Hell in Bosnia. In September 1992 (in the 5th month of the siege which would last 41 months more), William Pfaff wrote;

 

“They will not be conquered because a large modern industrial city of 350,000 people cannot be taken other than by a street-by-street infantry and tank assault, which is entirely beyond the abilities of the Serb militia. The actual resistance of Sarajevo`s people consists mainly of getting up each morning, going to whatever work they are doing, doing it, carrying on, finding something to eat, avoiding the snipers, and sleeping fitfully through the nighttime bombardments.” (Pfaff 1992)

A boy plays on a tank in Grbavica, a district in Sarajevo. ODD ANDERSEN/AFP/Getty Images

Serb forces fired an average of 300 shells per day into the city. Snipers were described as shooting “anything that moved”. This ‘anything’ very much included children. Thousands died in the siege, many (although not all) by the tactical, calculated bullets of snipers.

Various accounts of their actions were brought up in UN war tribunals, however I did not find any information on snipers being indicted for war crimes. In a quest to avoid biased sources (and regarding the Bosnian War, there is no shortage of these), I found YouTube footage from a 2012 UN International Criminal Tribunal for the former Yugoslavia (ICTY) prosecution, where prosecutor Dermot Groome lays out a case against former Bosnian Serb general Ratko Mladic. I realize that a prosecutor is, by nature, a biased party, but I am drawn to the legitimacy of the UN. Here, he described the sniper fire:

 

“Mladic’s use of snipers in the context of the attack on the civilian population was not at all like the use of snipers in armed conflict. It was a strategy of shooting civilians from a hiding spot, giving them no warning or reasonable prospect of taking cover. It was about creating insecurity, about creating terror.” (ICTY 2012)

 

Groome also presents the observations of a volunteer firefighter in Sarajevo:

 

“The thing I noticed about certain attacks was that Serb shooters would go after the youngest in the family… in a crowd of girls, it seems the most attractive would be shot. It seems there was something very personal, almost grudge attacks, doing whatever would cause the most pain to survivors.” (ICTY 2012)

Full quote from volunteer firefighter at 6:35 in part 5/5 (side note – it appears that video quality was not a priority of the UN war tribunals)

 

All in all, I have probably spent enough time delving into the horrors of the Siege of Sarajevo for the purposes of this essay. In short – it was Hell.

 

Knowing what we do about the Siege, it makes sense now that a foggy day was such a special occasion for Sarajevans in the final scenes of Ulysses’ Gaze. Under the blanket of fog, the civilians were shrouded from sniper fire. I found this quote[1] from the archive curator, Ivo Levy, especially powerful:

 

“Footsteps and voices? […] the fog… I sensed it. In this city, the fog is man’s best friend. Does it sound strange? It’s because it’s the only time the city gets back to normal. Almost like it used to be. The snipers have zero visibility. Foggy days are festive days here, so let’s celebrate!”

 

The curator hears music in the background, and excitedly says;

 

“Music… oh yes. A youth orchestra… Serbs, Croats, [Bosniak] Muslims… they come out when there is a ceasefire. They go from place to place and make music in the city. How about it, shall we go out too?”

 

After spending several paragraphs above discussing the atrocities of the Siege, this scene is especially meaningful. In a bloody, genocidal war, this youth orchestra is symbolic and idealistic, representing a group of the three ethnic groups coming together in the time of peace. It represents how things could be, or maybe how they used to be.

 

A arrives at Sarajevo after escaping Calypso’s island, notably wearing another man’s clothes. However, A’s time in Sarajevo does not end as happily as Odysseus’ journey ended in Ithaca. As he and Levy enjoy the foggy day in Sarajevo, they run into Naomi, the daughter of Levy (played by Morgenstern, the same actress who plays essentially every woman A interacts with). They dance, as she likely represents a Penelope figure. Naomi, A, and Levy begin to walk down the river, but a car of soldiers rolls up off-screen. Levy tells A to stay back as he follows his daughter to investigate. Following a Classical Greek tradition of off-screen deaths, Levy and his daughter are gunned down by soldiers behind the wall of fog. A finds their bodies and wails in a long shot where the camera backs away just enough for him to be completely obscured by the fog.

 

In a great stroke of luck, I was able to find a PDF [2] of ‘Interviews’, 195 pages of combined Theo Angelopolous interviews translated and published as a book. Here, I found the transcript of an Israeli radio show where Angelopolous described his fascination with Saravejo:

 

“­­My Interest in politics and the Balkans is very easy to explain. Look at the history of this century and you will notice that its first momentous event took place in Sarajevo, and now, as we approach the end of the century, we are again in Sarajevo. This proves to what extent we have all failed. Living in the Balkans, I am naturally much closer to the events, and much more concerned than the rest of Europe. I wanted to shoot in Sarajevo, but couldn’t. Everything was lined up for us to go there. We were all ready to go, waiting for our plane in Ancona, when the plane that left before us was turned back because the bombing had started again.” (Interviews 2001)

 

The 20th century history of Bosnia began and ended with tragedies in Sarajevo. To some degree, the entire 20th century history of Europe is bookended by these tragedies. In 1914, Archduke Franz Ferdinand was assassinated in Sarajevo (Backhouse 2018). At the time of the film’s release in 1995, Sarajevo was in the midst of its siege.

 

This argument about the cyclical nature of the film originally came from Marinos Pourgouris, a Greek academic, and his chapter in “Mythistory and Narratives of the Nation in the Balkans”. The famous journey of Odysseus had a beginning and an end in Ithaca, and his return brings a sense of order back to Ithaca. In Ulysses’ Gaze, references are made to the cyclical nature of A’s journey. In the beginning of the film, as he has returned to Greece from America, he states, “my end is my beginning”. [3]

 

Closing:

 

This film is far from a typical Hollywood film, the typical type of film I engage with. Angelopolous himself has expressed a great deal of disdain for where film is heading and how Hollywood has changed cinema in Greece:

 

We know that European cinema is not doing very well lately; less tickets are sold. The theatres today are no longer that privileged place of encounter between the creative artist and his audience. There is a small elitist minority still looking for that encounter, but the vast majority is favoring the American movies, which, as far as I am concerned, are not films but just images printed on celluloid.”

 

As I discussed on page 2 (note – this refers to part of the essay that I did not upload), the reviews for this film were highly polarized. Considering the raving endorsements and scathing critiques of Ulysses’ Gaze, I think this film is a great example of how a film can be received in such profoundly different ways based on the lived experiences of different people. Long spans of it still don’t make sense to me, but having learned more about the history of the region, I can appreciate the depth of the film. As someone who generally doesn’t understand deep, symbolic art and film, it was very exciting to take a deep dive into some of this film’s scenes as they related to Balkan history and Homer’s Odyssey.

 

A Few Random Angelopolous Quotes (curated by Dan Fainaru)

 

“In any case, Greeks are a nation of emigrants. At the turn of the century, half of them went to America. There are one and a half million Greeks in the U.S. There are already 300,000 in Germany. They are everywhere, and instead of contributing to Greek economy at home, they are working for others. The Americans are coming into Greece now, claiming they wish to industrialize the country, but of course they will do it only if it is profitable for them. And Greece, for many, is now the fifty-first state of the Union.”

 

Question: You are implying Greece is a Third World country.

Angelopolous: That is the way things are. The Third World is not limited to Africa and Latin America. If you ask me, it includes Greece and Turkey too. We do not belong to the West, we are not part of Eastern Europe-we live at the crossroads of modern civilization. However, we happen to occupy a strategic point in the Middle East; therefore, we are important to American politics. Had it not been the case, their attitude towards us would have been completely different.

 

Q: You mentioned that Ulysses’ Gaze is, among other things, a love story. But is it really about love or the impossibility of loving? At one point, your protagonist says, “I am crying because I cannot love you.”

A : That phrase is taken from Homer’s Odyssey. Ulysses remained seven years on Calypso’s island, but he would often go down to the sea and cry. For he could not love Calypso; he was always thinking of Penelope. He wanted to love her, but couldn’t. As a matter of fact, at the end of my film, the hero meets once again his first love. It’s a film about firsts-first love, first look, the initial emotions that will always be the most important in one’s life.

 

Footnotes:

[1]: timestamp 2:24:27 in this YouTube version of the film (all timestamps in this essay refer to this video)

[2]: Available from Amazon here – https://www.amazon.com/dp/1578062160

[3]: timestamp 13:08

 

 

References:

 

Andersen, Odd. A Young Boy Plays 22 April 1996 on a Tank. April 22, 1996. Accessed December 5, 2018. https://www.gettyimages.co.uk/detail/news-photo/young-boy-plays-22-april1996-on-a-tank-in-the-sarajevo-news-photo/134244637.

Getty Images states “Photo credit should read ODD ANDERSEN/AFP/Getty Images”

 

Angelopoulos, Theo. Greece, 1997. Accessed November 29, 2018. https://www.youtube.com/watch?v=hO3b-bHmu1Q.

(Ulysses’ Gaze available on YouTube here)

 

Alberó, Pere. “A Gaze by Ulysses towards the Balkans.” Quaderns De La Mediterrània 23 (2016): 115-23. Accessed November 27, 2018. https://www.iemed.org/observatori/arees-danalisi/arxius-adjunts/quaderns-de-la-mediterrania/qm23/A Gaze by Ulysses towards the Balkans_Pere_Albero_QM23.pdf.

 

Chouinard, Alain. “Historical Argument, Involuntary Memory, and the Subversion of Balkanist Discourse within Theo Angelopoulos’ Ulysses’ Gaze.” OffScreen. February 2016. Accessed November 28, 2018. https://offscreen.com/view/balkanist-discourse-ulysses-gaze#fn-5-a.

 

Carroll, Chris. “Serbs Face the Future.” National Geographic. July 2009. Accessed December 05, 2018. https://www.nationalgeographic.com/magazine/2009/07/serbs/.

 

Ebert, Roger. “Ulysses’ Gaze Movie Review & Film Summary (1997).” RogerEbert.com. April 18, 1997. Accessed November 27, 2018. https://www.rogerebert.com/reviews/ulysses-gaze-1997.

 

Fainaru, Dan, ed. Theo Angelopoulos : Interviews. Jackson, MS: University Press of Mississippi, 2001.

 

Howe, Desson. “Ulysses’ Gaze.” The Washington Post. September 16, 1997. Accessed November 27, 2018. http://www.washingtonpost.com/wp-srv/style/longterm/movies/videos/ulyssesgaze.htm?noredirect=on.

 

Klemencic, Matjaz, Matjaž Klemenčič, and Mitja Žagar. The former Yugoslavia’s diverse peoples: a reference sourcebook. Abc-clio, 2004.

 

Pfaff, William. “LIFE, DEATH AND SURREALISM IN THE HELL THAT IS SARAJEVO.” Chicago Tribune. September 27, 1992. Accessed December 09, 2018. https://www.chicagotribune.com/news/ct-xpm-1992-09-27-9203280117-story.html.

 

Pourgouris, Marinos. Mythistory and Narratives of the Nation in the Balkans. Edited by Tatjana Aleksić. Newcastle: Cambridge Scholars Publishing, 2007.

Chapter 9 covers Ulysses’ Gaze

 

“Prosecution Opening Statements – Mladić (Part 5/5) – 16 May 2012.” YouTube, International Criminal Tribunal for the Former Yugoslavia (ICTY), 18 May 2012, youtu.be/rQ2SfiPJ-cI.

 

Stratton, David. “Ulysses’ Gaze Review.” SBS News. December 31, 2008. Accessed November 28, 2018. https://www.sbs.com.au/movies/review/ulysses-gaze-review.

 

[1] timestamp 2:24:27 in this YouTube version of the film (all timestamps in this essay refer to this video)

[2] A PDF obtained in a perfectly legally manner, certainly not from a virus-ridden Russian website

[3] Timestamp 13:08

Predicting MCAT Scores Using 3rd Party Practice Exam Scores

(3/26/18 – this is an unfinished post that I’ve put up so the data can be of use to others while I add to it / revise it. Also for best experience view on a desktop, the graphs are butchered on mobile)

The AAMC only has 3 scored practice exams released, leaving prospective test takers in a tough situation – to know where they stand, they have to take an AAMC scored exam. But those exams are a precious resource, as they’re the closest thing we can get to the real MCAT. If their score isn’t where they want it to be, they’re left with only 2 AAMC scored exams. Test takers of the past had 10 AAMC exams, but the post-2015 MCAT renders those practice tests worthless. Here I attempt to predict MCAT scores using 3rd party practice exam scores and user-submitted data from Reddit.

Data:

The data has been taken from this user submitted score spreadsheet, comprised of 844 user-submitted scores. It’s comprised of users of the MCAT subreddit and the Student Doctor Network forums. I only included individuals who took the MCAT between January and September 2017.

There is a tremendous amount of self-reporting bias in this data, which I’ll touch on at the end. Impossible scores were thrown out (one user reported a ‘406’ on NextStep Exam 1, which, like the real MCAT, is scored 472-528). I also excluded data from one other user who reported a 472 on the real exam after reporting 505, 504, 509, and 509 on NextStep Exams 1-4, and a 509 and 510 on AAMC #1 and #2. I don’t know if it’s possible to drop the ball that badly on test day so I’m calling it a fluke. These were the only two scores excluded.

Kaplan (n=309):

The short story is – Kaplan’s scores are heavily, heavily, deflated, but still have predictive power. As an extremely crude conversion, you can add 10 points to your Kaplan score to get your AAMC score. This becomes less predictive at the upper and lower extremes. Kaplan and NextStep had the strongest correlation to actual MCAT scores, though this isn’t necessarily saying they’re the best practice material. It does mean that their scaling is the most accurate.

NextStep (n=354):

NextStep exams were slightly less deflated than Kaplan, but they had a similarly tight distribution (r2 of .536). An extremely crude conversion factor would be to add 7 to your NextStep average to estimate your actual score. NextStep seems to pride themselves in giving accurate scaled scores, which makes me wonder why theirs are still so deflated still. There is no ‘crude conversion’ for NextStep scores, as it is less deflated for average scores (498-510) and heavily deflated for high scores (510+).

 

 

The Princeton Review (n=190):

Princeton Review’s exams are absurdly deflated. The average person who scores a 503 on a TPR exam gets a 518 on the real exam. Princeton Review’s exams had the worst correlation to actual MCAT scores (this graph is an unfinished stand-in for one I’ll post later but the data should be the same).

 

Notes:

So why are these tests so deflated? Part of it is my skewed data set (see next section). But I think it’s primarily because the test prep companies prefer someone scoring 505 on their practice exams and a 515 on the actual MCAT, rather than the opposite. Their “100% money back guarantees” rely on you outperforming your practice test score. Kaplan’s “Higher Score Guarantee” program will only redeem if your actual MCAT score is below your diagnostic score. They’ve built a ~10 point cushion into the scaling of their practice exams to ensure this won’t be redeemed often. Kaplan is the most popular MCAT prep company, and they have a treasure trove of student data that could be used to give accurate scores, if they desired.

Princeton Review offers a similar program, but the baseline score can be either your previous actual MCAT score (if taken within 90 days of the start of the review course) or the Princeton Review diagnostic exam taken at the beginning of your course. If you aren’t a retaker, and the latter is used, there is almost no chance you fail to beat that score.

I don’t think it’s a coincidence that NextStep, a prep company without a full-refund ‘Better Score Guarantee’, doesn’t deflate their practice test scores as heavily. They do have a guarantee for their tutoring, but it only redeems a free 2-hour tutoring session.

Personally, I was dejected when I got a 500 on my first Princeton Review practice exam. In reality, most people who score that do fine on the actual exam. Had I not worked with this data, I would have thought I was on track for a 500 on the real exam – a score that has a dreadful mean acceptance rate of 22.3%, per the AAMC [1]. Princeton Review at no point notes how deflated their scores are, and many students probably don’t realize this. I now understand it’s their way of protecting their revenue while still offering a money-back score guarantee.

When I have more time, I’d like to add more test prep companies (Altaius and ExamKrackers), and compare individual sections to actual MCAT section scores to see if any companies have a notably higher correlation on one subsection.

Pitfalls:

Among the many shortcomings of this data, the most damning is the fact that the 3rd party test prep companies can change their scaling algorithms anytime without warning (and they do). NextStep has stated via email that they are constantly fine-tuning their algorithm, with a major overhaul in January 2017. Kaplan appears to do the same (as discussed above, I think NextStep may actually desire to be accurate in their score reporting and I imagine they adjust more frequently than other companies).

Then there all the biases that played into the data I had:

  •       I am betting that users visiting the MCAT subreddit and other pre-medical internet forums are more dedicated than the average test taker.
  •       Massive self-reporting bias – people are more likely to submit their scores if they’re impressive (the average score in my data was an absurdly high 515).
  •       These practice tests are expensive, and the demographics of these test takers are almost certainly skewed towards rich folk.
  •       3rd party exam scores were averaged and no consideration was given to how many exams they took. An individual who took 10 Kaplan exams and got 510 each time was treated exactly the same as an individual who took 1 Kaplan exam and got a 510, even though I suspect the person who took 10 exams is better prepared to score well on the actual exam. I’ll look into this one when I have more time but I’m guessing it’s less important than the scores themselves.

All in all, this is an extremely imperfect science with many pitfalls, but it was able to predict my score within two points, and serves as a guide for those wondering where they stand without any AAMC exam scores in hand.

 

Footnotes:

1: Applicants with MCAT scores between 498 and 501 had an acceptance rate of 22.3% (1,241 / 5,571) for 2016-2017 and 2017-2018, from AAMC’s Table A-23.