In resource limited settings accurately tracking causes of death is hampered by a large proportion of deaths occurring outside of hospital settings and in the absence of formal medical attention. Comprehensive data on causes of death is vital for the effective management of public health burdens and reduction of premature child and adult mortality. In countries without robust healthcare infrastructures and lacking complete registration systems, an indirect method of measurement was developed, namely verbal autopsy. This process requires trained interviewers to take down narratives and ask standard questions to someone familiar with a recently deceased individual, in order to gather information on the demographics of the deceased and the signs and symptoms experienced. Verbal autopsies have then traditionally been physician-certified, however new computational algorithms have led to the development of computer-coded verbal autopsy (CCVA). In a collection of articles on verbal autopsy in BMC Medicine, as part of its Medicine for Global Health series, the merits and limitations of these two methods are discussed, highlighting conflicting schools of thought. Here Prabhat Jha from the University of Toronto, Canada, and Alan Lopez, co-Editor-in-Chief of Population Health Metrics, from the University of Melbourne, Australia, tackle the debate around computer-coded versus physician-certified verbal autopsy (PCVA).
There are conflicting reports on the accuracy of physician-certified verbal autopsy (PCVA) versus computer-coded verbal autopsy (CCVA). Where do you stand?
PJ: Physician coding remains preferable. It is a more transparent and trusted assignment method; and indeed physician diagnosis remains the standard for medical care globally. As such, there is little basis to argue for replacing current physician coding by CCVA.
AL: Where I stand is very much determined by the strength of evidence about the comparative performance of different approaches to analysing verbal autopsy questionnaires. Scientific advances, or even opinions, should not be based on what we would like to be the case, but rather on rigorous, reproducible, compelling and comparable science that enables us to measure just how well various diagnostic approaches perform when we know, with reasonable certainty, what the true cause of death is. We also need to assess comparative performance on a large number of deaths covering all, or most, diseases and injuries of public health importance. The only study (Population Health Metrics Research Consortium Mexico (PHMRC) study) that has done so, and that uses strict, invariant, a priori criteria to establish the cause of death, has convincingly demonstrated that selected automated diagnostic algorithms are consistently more accurate than physicians in diagnosing causes of death from responses on verbal autopsy questionnaires. This science was unequivocal. I believe in following the evidence.
How confident are you in the data to support this viewpoint?
PJ: Our study of 24,000 deaths was twice as large as the earlier study and run on five different settings. But the conclusions were similar: CCVA could estimate the leading physician diagnosis only about half of the time. More work is needed to test combinations of CCVA methods, however.
AL: The PHMRC study was a large, ambitious undertaking involving systematically assessing the quality of medical records of tens of thousands of deaths in countries ranging from very poor, to middle income. Inevitably, there will be some cases included in the study where residual uncertainty about the true cause of death remains, for any of a number of reasons associated with collecting and compiling data according to strict scientific protocols in different cultures. This is, in my view, only likely to account for a small fraction of cases. Given the efforts by the study team to ensure that rigorous scientific principles were followed in all sites to collect the reference cause of death dataset, including multiple data consistency checks, the PHMRC dataset is by far the most scientifically-valid basis for assessing comparative performance of verbal autopsy methods.
If CCVA was favoured over PCVA, what do you think the implications would be for monitoring cause of death?
PJ: In theory, CCVA should make coding faster. However, physician coding can be much faster. In India we have been able to recruit and certify 300 physicians by using electronic training and online certification. Recently, we double coded over 40,000 records within two months using just about 125 of these doctors. Technology now makes it much faster to code deaths, and eliminates the main delay which is conversion of paper records from the field to electronic records for physicians.
AL: Enormous, and obvious. If the emerging evidence on the ease and applicability of direct recording of responses to VA questionnaires using mobile devices can become common practice (and I see relatively few reasons why not), then application of validated automated diagnostic algorithms to these responses can yield information immediately on cause of death patterns in populations that is demonstrably more reliable, faster, efficient, and cheaper than asking physicians to do so, with the added advantage that it does not distract physicians from providing essential health care to communities.
If CCVA was favoured over PCVA, what do you think the implications would be for measuring progress in meeting development targets?
AL: Also quite substantial, to the extent that these development targets are directly dependent on knowing causes of death reliably in populations. This is certainly the case for Millennium Development Goal 5 (MDG5, maternal mortality) and MDG6 (HIV,TB, malaria), for example, and will be equally applicable to any new development goals in the post-2015 era that require an accurate understanding of cause of death patterns in communities. It is simply not realistic to expect that full and accurate medical certification of deaths, as is generally the case in more developed countries, will be attained in most developing countries in our lifetime. Hence the need for immediate, and widespread use of verbal autopsy to monitor causes of death in many countries.
PJ: The post 2015 Millennium Development Goals should adopt a new simple goal: ensure that all countries have reliable, representative cause of death (COD) data, either through national COD surveys repeated regularly, or the slower pathway of strengthening routine vital registration with COD certification.
It has been said that a ‘data revolution’ will aid countries in tracking national development targets. Is this the way forward? How can PCVA or CCVA help contribute to this?
PJ: Information technology can help move along tracking of cause of deaths (CODs), and innovations such as CCVA should be further tested. But the most important revolution should be in more countries doing what India, Afghanistan, Mozambique and others have done – take a random sample of deaths and interview them to obtain their causes of death.
In 1869, the Sanitary Commission of India wrote: “for sanitary purposes it is indispensable to know the relative mortality in small and, as far as possible, well-defined tracts to ascertain the death rates in each of these communities; to see how far this arises from preventable causes; and to apply the remedies”. This still applies!
AL: I think it depends on your interpretation of what a ‘data revolution’ might entail. Without a massive increase in understanding by governments of the importance of good quality, timely data to guide and inform public policy, and a dramatic change in the priority given by donors to reliably tracking national health progress (or indeed, aid effectiveness), it is difficult to see how current practice and the current attitude of neglect towards strengthening vital registration systems in most countries can possibly meet national development needs. Something has to change, and change quickly, with enlightened and competent global leadership. To the extent that national development strategies depend on knowing how quickly major causes of death are declining, or not, countries will require reliable, cost effective, timely, and representative population-level information on who dies of what, and at what age. The only feasible strategy currently available to do that is CCVA, either implemented in routine civil registration systems that capture all deaths, or in sample registration systems, such as those operating in India, China, Indonesia and Tanzania.
Is there consensus on the verbal autopsy questionnaire itself?
PJ: The World Health Organization has put together various instruments and done a decent job that is helpful to countries.
AL: I don’t think so, nor should we be surprised that there isn’t. Defining the set of questions that are likely to yield the most useful information to diagnose a minimum list of causes of death, itself variously determined, is a new challenge for the scientific community. Basing these questions on clinical judgement alone is a good start, but not sufficient. Formal analytical methods including item reduction approaches can contribute critical intelligence to our understanding of changes in the performance of VA when certain questions are dropped (or added). If the goal is to settle on a parsimonious set of questions that have the highest likelihood of yielding accurate information on the cause pattern of mortality in populations, for a defined set of causes of global health significance, then we need a combination of these two inputs. There is substantial knowledge and expertise among those who have implemented verbal autopsy methods that could usefully inform this process.
In terms of engendering confidence in verbal autopsy as a legitimate data collection mechanism, how do you think CCVA versus PCVA contributes to this?
PJ: More scientific scrutiny on these important instruments will always improve them. The main debate should not be about CCVA versus PCVA but direct, country-based national cause of death (COD) surveys versus reliance on indirect, econometric models of CODs. We’ve invested heavily in the latter, and it’s now time for a lot more effort in country based national COD surveys and systems.
AL: I am not sure that the community (e.g. governments, donors) that we need to convince about the merits of scaling up verbal autopsy are currently much influenced by the comparative performance of CCVA versus PCVA. Rather, they are more likely to ask whether verbal autopsy is a legitimate means of data collection on causes of death compared to the traditional, and costly practice of physician certification of deaths, the vast majority of which occur in hospitals. While that might reasonably be the aim of all countries, not only is it unlikely to happen over the next few decades, but it is also a questionable standard. Recent evidence has suggested that physicians in even the most advanced hospitals in upper middle income countries do not do appreciably better than the best automated diagnostic methods in correctly diagnosing the cause of death of their patients. If that evidence can be replicated in different settings, then this, in my view, provides a much more convincing platform to promote the widespread adoption of verbal autopsy as a legitimate data collection mechanism than the comparative performance of different diagnostic approaches.
Read more about the collection of verbal autopsy articles in this BioMed Central blog.
Complete list of series articles:
Complete list of verbal autopsy articles: