I found an interesting and critical study titled “Oral and Maxillofacial Surgery Residents Have Poor Understanding of Biostatistics,” by Al M. Best and Daniel M. Laskin appearing in the J Oral Maxillofac Surg vol. 71, pp. 227-234, 2013. The article discuss how the three main oral and maxillofacial surgery (OMS) journals have low quality evidence and how some of the articles can’t be taken at face value. Hence, the purpose of this study was to evaluate residents’ understanding of biostatistics and the interpretation of research results to see if the curricula of training programs should be changed to better prepare clinicians to practice evidence-based dentistry.
In the study a 6 question survey was developed for OMS residents and compared to results for a similar survey of medical residents. Further, questions looking into demographic characteristics, the methodologic courses residents may have taken previously, their attitudes about statistics, and their confidence about interpreting and assessing statistical concepts was assessed. The survey was administrated online and sent to all OMS residents in the United States in 2009.
Even so only 112 participated in the survey. From these participants approximately 53% of the residents in OMS had taken a course in epidemiology, 49% had taken a biostatistics course, and 65% had taken a course in evidence-based dentistry. 11 of the participants had never taken any such courses before their residency. Furthermore, 14% of OMS residents reported that they do not regularly read journals. The mean score of the 6 questions was 38% correct.
The authors state
“Two keys to the highest level of evidence in a study are blinding and controls. Seventy-nine percent of OMS residents correctly identified that this avoids observer and subject bias. However, only 46% correctly identified that a P > .05 (not statistically significant) indicates at least a 1-in-20 chance occurrence.”
The specific question testing the p-value is stated in the original article (as are all the questions).
“In a placebo-controlled trial of the use of a chlorhexidine rinse to prevent alveolar osteitis (dry socket) after third molar extraction, 8% of patients receiving the treatment had dry socket and 9% of patients receiving the placebo had dry socket. In reporting this finding, the authors stated that P > .05.”
The correct answer is reported as
“The probability is greater than 1 in 20 that a difference this large could occur by chance alone.”
Interpreting a p-value is one of the most important statistical concepts. So it clearly is concerning that this question was incorrect more than half of the time by the respondents.
The authors also present the results from attitudes about statistics and compare results to medical residents. The authors state near the conclusion
“Although medical residents’ knowledge of biostatistics was low, OMS residents’ knowledge was significantly worse (51% vs 38% correct, t = 5.8, P< 0001). Despite all of the differences between the 2 groups of residents, neither group was well versed in statistical methodology.”
The result of this study and the medical resident’s study was that biostatistics need to be taught to residents. I personally have found the textbook by Zar titled Biostatistical Analysis to be the most helpful biostatistics text I have looked at so far (I have explored several).