The last article looked at the utility of the different FDA databases (Investigating the Performance of FDA Databases in Terms of Reporting Medical Device Problems 1 - The utility of the FDA databases). This article focused on the Manufacturer and User Facility Device Experience (MAUDE) database by studying three cases, namely, haemodialysis (1633 reports), defibrillator (1222 reports), and nebuliser (1210 reports) using the content analysis method.
There are several reasons for focusing on these medical devices. Firstly, they are all prevalent medical devices, more so in recent years. The USA, UK, Finland, Asia and Australia all have active groups, committees and reports encouraging the dialysis devices area, especially to increase growth in home dialysis devices. At the same time, defibrillators are becoming prevalent in public places, such as railway stations, airports, and shopping centres, to help improve the chances of survival in an emergency. By 2010, in the USA, all jurisdictions had enacted laws for public access to defibrillators. Nebuliser devices are considered one of the best common devices used to deliver medications to patients with respiratory disease in clinical settings and the home environment. Secondly, they were chosen based on their varied complexity. Haemodialysis is a programmable device that is more difficult to use and design. They are heavily interactive and require users to program settings, which places a high demand on users. Conversely, a nebuliser is a simple device, which is easy to use and design. Compared with haemodialysis machines and nebulisers, the complexity of defibrillators is medium. Thirdly, because of the different complexity, the training for using these devices also varies accordingly. For haemodialysis users, it may take a long time and much effort to learn how to operate, especially for home haemodialysis users. But for nebulisers and defibrillators, there are much easier to learn how to operate. Finally, the different sizes of these three medical devices were considered. Haemodialysis requires big devices, defibrillators are medium, and nebulisers are comparably small.
Firstly, how the MAUDE database works should be understood, especially who can report adverse events to the FDA. The FDA uses Medical Device Reporting (MDR) to monitor device performance, detect potential device-related safety issues, and contribute to benefit-risk assessments of these devices. Reports for the MAUDE are submitted to the MDR department of the FDA, these reports are then housed in the MAUDE database. The 21 CFR 803 (Medical device reporting regulation) provides the regulations for adverse events reporting. There are nine MDR guidance documents available for supporting reporters to the FDA properly. Figure 1 shows the mandatory reporters, which include manufacturers, importers, and device user facilities. Manufacturers must report events which include death, serious injury, or device malfunction to the FDA. Importers submit a report in which an event involves death or serious injury to the FDA, and they also need to send malfunction reports to the manufacturers. If device user facilities submit a report, and the event involved death, they must submit the report to the FDA or manufacturers. If the event involved serious injury, they must also pass the report on to the manufacturers. Voluntary reporters include healthcare professionals, consumers, and patients.
From Figure 1, we can see that if mandatory reporters and voluntary reporters submit reports for one event, there will be few records in the database. There is no cross-correlation or compilation of data within MAUDE, but individual reports of an event can be related by an Event Key. There are six main types of FDA reporting forms. The FDA Form 3417 is the baseline reporting form for manufacturers. The FDA Form 3381 is the annual certification reporting form for manufacturers and user facilities. The FDA 3419 Form is the semi-annual reporting form for user facilities. The FDA Form 3500A is the mandatory reporting form for manufacturers (5-day reporting, and 30-day reporting), importers, and user facilities (death and serious injury reporting). The FDA Form 3500 is the general voluntary reporting form for healthcare professionals, consumers, and patients. The FDA Form 3500B is the voluntary reporting form that was designed specifically for consumers. The contents of these forms are similar, and each form includes five main sections: the 'patient information' (such as ‘age’, and ‘medical conditions’); the 'adverse event' or 'product problem' (such as outcome of the ‘adverse event’, ‘event description’, ‘relevant tests’ and ‘laboratory data’, and ‘other relevant history’); the 'suspect product' (such as ‘product name’, ‘dose and route used’, and ‘diagnosis for use’); the 'suspect medical device' (such as ‘name of company’, ‘label strength’, and ‘number of model’); and the 'reporter information' (such as ‘name and address’). MedWatch is the FDA safety information and adverse event reporting program for the reporting adverse events, but online reporting forms are not intended to fulfil the mandatory reporting requirements. Each reporter should provide proper and complete information when filling in a report form, even if sometimes the same event may be reported by different reporters separately.
Submitted mandatory and voluntary reports are analysed by FDA staff who may be healthcare clinicians, engineers, or scientists. The analysis is considered from several aspects, which include if an event was due to an inherent risk of a device; if an event relates to human factors; if a device had adequate labelling; and if the design of a device needs to be changed. The follow-up actions of the FDA include requesting additional information from reporters; conducting an investigation of events; and issuing a public health warning.
After understanding how the MAUDE database works, the utility of the report contents was investigated. There are two main parts to a standard report. In the first part, there are three content groups: basic the event information, the ‘manufacturer narrative’, and the ‘event description’. In the second part, there is basic factual information about the device and the event. This study only focused on the first part and the term ‘report contents’ refers solely to the information in the first part.
1. Inconsistent Information
Inconsistent information generally stems from two aspects of the report’s contents. The first common aspect is when information does not match between the ‘event description’ and the ‘manufacturer narrative’. The second common aspect is between the ‘event description’ and the ‘patient outcome’. Figure 2 shows, the ‘patient outcome’ is ‘other’, but in fact, the patient has died.
Figure 2 Example of a Report with Inconsistent Information
Although there is some information from different sections that does not match, this problem was infrequent: there were 56 of 4065 reports (1.4%) that had some form of inconsistent information. A relevant point is that in many reports, even when the event resulted from direct interaction with the device, manufacturers tend to put an emphasis on there being nothing wrong with the device itself.
2. Unclear Information about the ‘Device Problem’
There are three main pieces of unclear information found in the reports: unclear information on device categories; unclear information on the causes of adverse events; and unclear report organisation.
In each report, devices are categorised depending on their device problems. There is a specific and clear name for each category of the device problem such as the ‘device alarm system issue’, the ‘computer software issue’, and the ‘misconnection’. In this study, these specific and clear categories were summarised as ‘Clear Device Problems’ (see Figure 3). However, often there is no clear or definite problem for a category, and it is reported with a name such as ‘Unknown’, ‘No Information’ and ‘No Code’. In this study, these no clear or definite problem for categories were summarised as ‘unclear categories’. All five unclear categories can be found in Figure 3.
The categories ‘no information’, ‘no known device problem’, ‘no code’, ‘unknown’, and ‘other’ were counted for each of the three medical devices: haemodialysis (1633 reports), defibrillator (1222 reports), and nebuliser (1210 reports) from 2004 to 2014. Figure 3 shows the how the device problem category names are distributed for each of the three devices. We can see that there was a total 36% of occurrences without a clear device problem in the haemodialysis reports; 18% in defibrillator reports; and 5% in nebuliser reports. It was clear that was the large difference between each of the three devices, likely related to the complexity of the conditions they treat as well as the complexity of the devices.
Reports with the unclear categories often provide unclear information about the cause of an event. From the investigation of highlighting and noting key sentences, six causes of events were adapted from the language used in the reports (see Tables 1, 2, and 3). Ten reports with these six different causes were chosen for four post-doctoral participants. These ten reports had been highlighted and noted by the author, and the participants were asked to check whether the highlighted areas covered all the information about the cause of an event and whether the notes clearly described the causes. After that, the six different causes were counted for all five unclear medical device problem categories.
The causes and results are shown in Tables 1, 2, and 3. We can see that in the reports with unclear device problems, the majority of manufacturers do not attribute the event to being caused by their devices. We can also see that some reports do not provide relevant information on devices. This is worth investigating in this research: do reports, in genera, l provide comprehensive information about an event? It will be discussed in the next article: Investigating the Performance of FDA Databases in Terms of Reporting Medical Device Problems 3 - The utility of the MAUDE report contents.
3. Incomplete Information in the ‘Event Description’ and the ‘Manufacturer Narrative’
From the previous subsection, we know that there are many reports which fail to provide complete information about adverse events. This results in two situations: the report was incomplete but it still provides some useful information and can be considered useful or the report provided some information but not enough to be considered useful. Figure 4 shows two different reports in which there is minimal information and they can be considered invalid, as users would not be able to obtain any useful or detailed information.
Figure 4 Examples of Invalid Reports
To deeply investigate the comprehensiveness of the ‘event description’ and the ‘manufacturer narrative’ in the MAUDE database, report contents were grouped by analysing individual report to find common descriptions, the details of which will be discussed in the next article: Investigating the Performance of FDA Databases in Terms of Reporting Medical Device Problems 3 - How do the MAUDE database reports capture information regarding medical device errors.