Search Article 
  
 Advanced search 
  
 
About us - Editorial board - Browse articles - Submit article - Instructions - Subscribe - Advertise - Contacts - Login 
  Users Online: 385 Home Print this page Email this page Small font sizeDefault font sizeIncrease font size   



 
Browse articles
ORIGINAL ARTICLE
J Clin Imaging Sci 2016,  6:19

Effect of picture archiving and communication system image manipulation on the agreement of chest radiograph interpretation in the neonatal intensive care unit


1 Department of Diagnostic Imaging, University of Toronto, Toronto, Canada
2 Department of Diagnostic Radiology, Queen's University, Kingston, Ontario, Canada
3 Department of Anesthesiology and Perioperative Medicine, School of Nursing, Queen's University, Kingston, Ontario, Canada
4 Department of Pediatrics and Diagnostic Radiology, Queen's University, Kingston, Ontario, Canada

Date of Submission10-Mar-2016
Date of Acceptance22-Apr-2016
Date of Web Publication19-May-2016

Correspondence Address:
Donald Soboleski
Department of Diagnostic Radiology, Queen's University, Kingston, ON K7L 2V7
Canada
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/2156-7514.182730

Rights and Permissions
   Abstract 

Objective: Variability in image interpretation has been attributed to differences in the interpreters' knowledge base, experience level, and access to the clinical scenario. Picture archiving and communication system (PACS) has allowed the user to manipulate the images while developing their impression of the radiograph. The aim of this study was to determine the agreement of chest radiograph (CXR) impressions among radiologists and neonatologists and help determine the effect of image manipulation with PACS on report impression. Materials and Methods: Prospective cohort study included 60 patients from the Neonatal Intensive Care Unit undergoing CXRs. Three radiologists and three neonatologists reviewed two consecutive frontal CXRs of each patient. Each physician was allowed manipulation of images as needed to provide a decision of "improved," "unchanged," or "disease progression" lung disease for each patient. Each physician repeated the process once more; this time, they were not allowed to individually manipulate the images, but an independent radiologist presets the image brightness and contrast to best optimize the CXR appearance. Percent agreement and opposing reporting views were calculated between all six physicians for each of the two methods (allowing and not allowing image manipulation). Results: One hundred percent agreement in image impression between all six observers was only seen in 5% of cases when allowing image manipulation; 100% agreement was seen in 13% of the cases when there was no manipulation of the images. Conclusion: Agreement in CXR interpretation is poor; the ability to manipulate the images on PACS results in a decrease in agreement in the interpretation of these studies. New methods to standardize image appearance and allow improved comparison with previous studies should be sought to improve clinician agreement in interpretation consistency and advance patient care.

Keywords: Agreement and consistency in interpretation, chest radiograph, manipulation of images, Neonatal Intensive Care Unit, picture archiving and communication system


How to cite this article:
Castro DA, Naqvi AA, Vandenkerkhof E, Flavin MP, Manson D, Soboleski D. Effect of picture archiving and communication system image manipulation on the agreement of chest radiograph interpretation in the neonatal intensive care unit. J Clin Imaging Sci 2016;6:19

How to cite this URL:
Castro DA, Naqvi AA, Vandenkerkhof E, Flavin MP, Manson D, Soboleski D. Effect of picture archiving and communication system image manipulation on the agreement of chest radiograph interpretation in the neonatal intensive care unit. J Clin Imaging Sci [serial online] 2016 [cited 2017 Sep 23];6:19. Available from: http://www.clinicalimagingscience.org/text.asp?2016/6/1/19/182730


   Introduction Top


Chest radiographs (CXRs) are routinely utilized to access changes in status of newborns and infants in the neonatal intensive care setting and often lead to changes in management. [1],[2] Differences in image interpretation have been attributed to multiple factors including differences in the interpreters' knowledge base, experience level, and access to the clinical scenario. Inconsistent radiograph image quality due to patient movement, exposure setting alterations, and low lung volumes are a few of the many variables limiting assessment, and many researchers have demonstrated significant inter-observer variability in interpretation. [3],[4],[5],[6],[7] The advent of picture archiving and communication system (PACS) has allowed the user to manipulate the images by altering the brightness (window level) and/or contrast (window width) of each radiograph while developing their impression. This manipulation will vary the appearance of the radiograph, and in effect, each user will be basing their impression on their own uniquely obtained image. This resulting image will be different from the image that their colleagues will generate to help them come to their impression. The purpose of this study is to determine the agreement of CXR impressions among radiologists and neonatologists and help determine the effect of image manipulation with PACS on reported diagnosis.


   Materials and Methods Top


The study was approved by the university and affiliated teaching hospitals research ethics board (REB), and patient informed consent was waived by the REB.

Patient population

This prospective cohort study included 60 nonconsecutive patients from the Neonatal Intensive Care Unit (NICU) (age range: 1 day to 3 months; 34 females, 26 males; gestational age range: 26-32 weeks) who had undergone CXRs as part of their routine care during a 3-month period in 2011. Patients included in the study had CXRs performed on variable days during their stay in the NICU. All patients had an underlying history of surfactant deficiency disease with follow-up radiographs ordered due to a concern of disease progression respiratory status. Exclusion criteria included infants who did not undergo two CXRs during their NICU stay. Images in which the CXR was combined with the abdomen were excluded from the study as well. The radiographs were obtained using 60 kVp and 1.5 mAs, the standard technique in the department.

Intervention

The sixty sets of frontal CXRs, each consisting of one previous and one recent examination, totaling 120 CXRs, were randomly placed as acquired into two identical viewers on PACS (GE Centricity 3.2, GE Healthcare IT, Barrington, Illinois, USA) used in the imaging department. The normally scanned requisition for the study was not included and each set of radiographs was sequentially numbered from1A/B to 60A/B. The previous or earlier study was placed to the right of the most recent study as per the department's usual routine. Three radiologists with a median length of expertise of 14 years (range: 5-25 years) along with three Neonatal Intensive Care Unit (NICU) physicians with a median length of expertise of 16 years (range: 2-25 years), each reviewed all 60 sets of frontal chest images in each of the two viewers. Each interpreter was instructed to provide a solitary impression of overall "improved," "no change," or "disease progression" lung disease for each of the 60 cases in each viewer. Each interpreter reviewed the 60 sets of cases in one viewer in their usual manner with manual windowing/manipulation as judged needed. Each interpreter (neonatologist or radiologist) reviewed the 60 sets of cases within the second viewer after an independent person (a radiologist) performed image windowing and contrast adjustment to best optimize the CXR appearance and to best match appearance with the previous study performed on that patient. The interpreters were not allowed to independently manipulate the appearance of the CXRs in this second viewer to come to their impression of whether the recent examination was improved, disease progression, or unchanged. Three interpreters began their review of the radiographs in the first viewer and the other three interpreters began with the second viewer. The time span between full review of the radiographs in one viewer and the completion of assessment of the radiographs in the other viewer ranged from 1 to 12 days.

Statistical analysis

Percent agreement was calculated between all six observers and separately for radiologists and neonatologists, for each of the two methods used in determining their impressions (allowing image manipulation/windowing and not allowing image manipulation/windowing). Similarly, the percent reporting opposing views, i. e., improved versus disease progression and vice versa, was calculated. Statistical significance was assessed using Chi-square test; the magnitude of the difference in agreement and opposition between manipulated and nonmanipulated impressions was assessed using relative risk (RR) and 95% confidence intervals (CI). The within observer kappa and the percent agreement, opposing, and unchanged impressions between manipulated and nonmanipulated methods were calculated. Statistical significance was set at <0.05. SPSS version 21 (IBM Corp. Released 2012. IBM SPSS Statistics for Windows, Version 21.0. Armonk, NY: IBM Corp.) and Microsoft Excel were used to conduct the analysis.


   Results Top


Overall, there was more agreement in CXR interpretations when images could not be manipulated. One hundred percent agreement in image impression (whether thought to be improved, disease progression or unchanged) between all six observers (three radiologists, three neonatologists) was only seen in 3 of the 60 cases (5%) when allowing image manipulation while 100% agreement was seen in 8 of the 60 cases (13%) when there was no manipulation of the images allowed (Chi-square = 2.5, P = 0.11) [Table 1]. Hence, the clinicians were 2.9 times more likely (95% CI 0.7, 14.8) to agree on image impressions when no manipulation of the images was permitted. There was 100% agreement between all three radiologists in 21 of the 60 cases (35%) when there was independent manipulation and 25 of the 60 cases (42%) without image manipulation (RR = 1.3, 95% CI 0.6-3.0). There was 100% agreement between all three NICU physicians in 8/60 (13%) of cases with independent manipulation and 18/60 (30%) of the cases when no manipulation was allowed (RR = 2.8, 95% CI 1.0-7.8). Overall, there were fewer opposing radiograph impressions when image manipulation was not allowed as compared to when manipulation/windowing was permitted [Table 1]. There were opposing image impressions (improved versus disease progression, vice versa) between all six interpreters in 37/60 (62%) of manipulated images and 29/60 (48%) of cases with no manipulation of images, resulting in a 40% decrease in opposing impressions when images could not be manipulated (RR = 0.6, 95% CI 0.3-1.3). Radiologists were less likely to report opposing impressions between the two methods compared to the neonatologists. Among the three radiologists, there were opposing impressions in 17/60 (28%) with manipulation and 12/60 (20%) with no manipulation of images (RR = 0.6, 95% CI 0.3-1.6). With the NICU physicians, there were opposing impressions in 24/60 (40%) with manipulation and 11/60 (18%) when no manipulation was allowed (RR = 0.3, 95% CI 0.1-0.8).
Table 1: Assessment of sixty sets of frontal chest radiographs without versus with the ability of to manually manipulate the images

Click here to view


Radiologists were 3.5 times more likely than neonatologists to have 100% agreement on radiograph impressions when they had the ability to manipulate the image (95% CI 1.3-9.7) [Table 2]. Radiologists were also 1.7 times more likely to have 100% agreement on image interpretation when manipulation was not allowed (RR = 1.7, 95% CI 0.7-3.8) and 0.6 times less likely to have opposing impressions when manipulation was permitted (RR = 0.6, 95% CI 0.3-1.4). However, in some instances, the results were not statistically significant, primarily due to the small number of clinicians involved.
Table 2: Assessment of 60 sets of frontal chest radiographs with and without the ability to manually manipulate the images: Radiologists versus neonatologists

Click here to view



   Discussion Top


CXR interpretation is a routine essential requirement in the daily care of patients, particularly important in the NICU setting. Changes in the radiographic appearance result in a change in management in a significant percentage of patients. [8],[9],[10],[11],[12],[13] Multiple studies have demonstrated high inter-observer variability in the interpretation of radiographs. This has led to potential solutions such as double and triple reading of studies which are both costly and time-consuming. [6],[7],[14],[15],[16] More recent research has shown even less inter-observer agreement on digital compared to analog studies. [4] Numerous factors can contribute to poor image quality, such as patient and cassette positioning, exposure techniques, body habitus, and motion, and the radiographic images obtained are often suboptimal. [5],[16] These variations in image quality and appearance often result in the interpreting clinician attempting to optimize the appearance by manipulating/windowing the image on PACS. The effect of this independent, individualized ability to manipulate the radiographic images has not been well documented.

Our study demonstrates an adverse effect in regards to the agreement in image interpretation when manipulation on PACS system is performed. The poorer inter-rater agreement was more significant among the NICU physicians, possibly due to less experience in image manipulation as compared to the radiologists who perform these maneuvers on a daily basis. The study also demonstrates overall poor agreement in image interpretation which has been shown in the medical literature. [3],[6],[7],[14],[15],[16] Particularly, concerning is the percentage of opposing radiograph impressions which could result in significant changes to patient care and outcome. The ability to manipulate the appearance of the CXR images on PACS led to an increase in opposing interpretations in our study and raised the question of whether clinicians' access to image windowing/manipulation is of benefit to the patient. Access to PACS and the ability to manipulate radiographic images independently by health care providers have increased in most hospital and patient care systems lately. This has been found to be very beneficial in regards to access data and patient radiographs; however, the potential disadvantages have not been elucidated. Further studies to demonstrate the impact on patient care and hospital costs are needed to help guide future endeavors by researchers, hospitals, and industry to improve our consistency in image interpretation. Novel and improved methods are needed to optimize image quality and allow for more reliable and consistent interpretation and comparison of radiographs.

Limitations

A pitfall of the study is the lack of a gold standard in the diagnosis of the radiograph abnormality. The study was designed to assess agreement between interpreters; therefore, accuracy per se was not deemed pertinent. All interpreters were specialists in their fields who routinely accessed the PACS as part of their daily routine of caring for patients. The inter-observer variability was thus regarded as a limitation of the test itself and not related to radiologist/neonatologist expertise. The dictating of the 60 sets of CXRs may have allowed for some patient recognition, and potential bias, although the lack of patient identification, the similar appearance of these types of studies and the alternating use of viewers are thought to have minimized this problem.


   Conclusion Top


Agreement in CXR interpretation is poor, with conflicting impressions common with both neonatologists and radiologists. The ability to manipulate the images on PACS has resulted in a decrease in agreement in the interpretation of these studies. New methods to standardize image appearance and allow improved comparison with previous studies should be sought to improve clinician agreement in interpretation consistency and advance patient care.

Acknowledgments

We would like to acknowledge Dr. J. Flood, M. Clarke, A. Acker and S. Salahudeen, who provided insight and expertise that greatly assisted our study. We would also like to acknowledge Wilma Hopman for her statistical and editorial expertise.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

 
   References Top

1.
Lakhal K, Serveaux-Delous M, Lefrant JY, Capdevila X, Jaber S; AzuRéa Network for the RadioDay Study Group. Chest radiographs in 104 French ICUs: Current prescription strategies and clinical value (the RadioDay study). Intensive Care Med 2012;38:1787-99.  Back to cited text no. 1
    
2.
Graat ME, Choi G, Wolthuis EK, Korevaar JC, Spronk PE, Stoker J, et al. The clinical value of daily routine chest radiographs in a mixed medical-surgical intensive care unit is low. Crit Care 2006;10:R11.  Back to cited text no. 2
    
3.
Meade MO, Cook RJ, Guyatt GH, Groll R, Kachura JR, Bedard M, et al. Interobserver variation in interpreting chest radiographs for the diagnosis of acute respiratory distress syndrome. Am J Respir Crit Care Med 2000;161:85-90.  Back to cited text no. 3
    
4.
Rubenfeld GD, Caldwell E, Granton J, Hudson LD, Matthay MA. Interobserver variability in applying a radiographic definition for ARDS. Chest 1999;116:1347-53.  Back to cited text no. 4
    
5.
Tudor GR, Finlay D, Taub N. An assessment of inter-observer agreement and accuracy when reporting plain radiographs. Clin Radiol 1997;52:235-8.  Back to cited text no. 5
    
6.
Cleveland RH, Schluchter M, Wood BP, Berdon WE, Boechat MI, Easley KA, et al. Chest radiographic data acquisition and quality assurance in multicenter studies. Pediatr Radiol 1997;27:880-7.  Back to cited text no. 6
    
7.
Bloomfield FH, Teele RL, Voss M, Knight DB, Harding JE. Inter- and intra-observer variability in the assessment of atelectasis and consolidation in neonatal chest radiographs. Pediatr Radiol 1999;29:459-62.  Back to cited text no. 7
    
8.
Hauser GJ, Pollack MM, Sivit CJ, Taylor GA, Bulas DI, Guion CJ. Routine chest radiographs in pediatric intensive care: A prospective study. Pediatrics 1989;83:465-70.  Back to cited text no. 8
    
9.
Chahine-Malus N, Stewart T, Lapinsky SE, Marras T, Dancey D, Leung R, et al. Utility of routine chest radiographs in a medical-surgical intensive care unit: A quality assurance survey. Crit Care 2001;5:271-5.  Back to cited text no. 9
    
10.
Horst HM, Fagan B, Beute GH. Chest radiographs in surgical intensive care patients: A valuable "routine". Henry Ford Hosp Med J 1986;34:84-6.  Back to cited text no. 10
    
11.
Greenbaum DM, Marschall KE. The value of routine daily chest x-rays in intubated patients in the medical intensive care unit. Crit Care Med 1982;10:29-30.  Back to cited text no. 11
    
12.
Henschke CI, Pasternack GS, Schroeder S, Hart KK, Herman PG. Bedside chest radiography: Diagnostic efficacy. Radiology 1983;149:23-6.  Back to cited text no. 12
    
13.
Bekemeyer WB, Crapo RO, Calhoon S, Cannon CY, Clayton PD. Efficacy of chest radiography in a respiratory intensive care unit. A prospective study. Chest 1985;88:691-6.  Back to cited text no. 13
    
14.
Ioos V, Galbois A, Chalumeau-Lemoine L, Guidet B, Maury E, Hejblum G. An integrated approach for prescribing fewer chest x-rays in the ICU. Ann Intensive Care 2011;1:4.  Back to cited text no. 14
    
15.
Bhagwanjee S, Muckart DJ. Routine daily chest radiography is not indicated for ventilated patients in a surgical ICU. Intensive Care Med 1996;22:1335-8.  Back to cited text no. 15
    
16.
Lichtenstein D, Goldstein I, Mourgeon E, Cluzel P, Grenier P, Rouby JJ. Comparative diagnostic performances of auscultation, chest radiography, and lung ultrasonography in acute respiratory distress syndrome. Anesthesiology 2004;100:9-15.  Back to cited text no. 16
    

 
   Authors Top


Donald Soboleski



 
 
    Tables

  [Table 1], [Table 2]



 

Top
Previous article   Next article
 
  Search
 
    Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
    Access Statistics
    Email Alert *
    Add to My List *
* Registration required (free)  

 
  In this article
    Abstract
   Introduction
    Materials and Me...
   Results
   Discussion
   Conclusion
    References
    Authors
    Article Tables

 Article Access Statistics
    Viewed2707    
    Printed23    
    Emailed0    
    PDF Downloaded121    
    Comments [Add]    

Recommend this journal