I was interested in the percent of U.S. Doctors who have been vaccinated against COVID-19 and the results I found online were surprising. While the American Medical Association (AMA) reports “96% physicians have been fully vaccinated for COVID-19”. The Association of American Physicians and Surgeons (AAPS), reported nearly 60 percent said they were not fully vaccinated against COVID. Why are these numbers so different? As a former AP Statistics teacher, I wanted to find out.
The AMA Survey (96% Vaccinated)
According to the AMA report
AMA developed the survey questionnaire and WebMD programmed and fielded the 5–7 minute survey through their physician panel
Sample size – N=301
My biggest takeaway here is it appears the AMA used Medscape Market Research (part of the WebMD brand) to get a random sample of physicians.
I want to highlight here, the AMA did NOT use their own 240,000 members for this survey but rather used an outside group to provide a random sampling of physicians.
The AAPS Survey (40% Vaccinated)
According to the AAPS blog post
Of the 700 physicians responding to an internet survey
Based on the above quote and the fact their survey was built with Online Form Builder Wufoo (which is evident by the posted survey results), it appears the AAPS emailed their 5,000 members and asked them to complete this survey and 700 members responded.
This methodology is flawed to the point the results are meaningless.
- they are not using a random sample of physicians, they are surveying their own members (and the AAPS is not a representative group, they are an extremely politically conservative group, see the Association of American Physicians and Surgeons (AAPS) Wikipedia article)
- even within the AAPS members, they are NOT using a random sample – they are asking all members and allowing them to choose whether or not to answer the question – only 700 chose to participate (see the Self-selection bias Wikipedia article)
An Analogy
Imagine you wanted to know, in the opinion of students in your area, which high school is the best.
The AMA used a third-party company to assemble 301 randomly selected students from different high schools in your area and asked them their opinion. 96% of these students had the same response.
The AAPS went to one local high school (San Dimas High School) and handed out a survey to the 5,000 students in that school. Of those 5,000 surveys only 700 were handed in – of those, 60% said San Dimas High School was the best.
Deeper Dive on 96% Vaccinated
While 96% of the respondents in the AMA survey are vaccinated, since we didn’t ask all physicians we don’t know this answer is correct. Fortunately, the field of statistics has tools to help us better understand our data. Using the “Wilson” score interval, the survey size of 301, and assuming the physicians surveyed were a truely random sample – we can say with 99% certainty that the true percent of Doctors who are vaccinated is between 92% and 98%.
Summary
The AMA physicians survey, which showed 96% of physicians are vaccinated against COVID-19 is the most honest data point we have.
The AAPS methodology is flawed to the point of being meaningless.
