1Medical Clinic I, University RWTH Aachen, Pauwelsstrasse 30, 52074, Aachen, Germany
2Clinic Johannes Gutenberg, University Mainz, Mainz, Germany
3Academic Hospital Dijkzigt, Rotterdam, The Netherlands
4University Charite, Berlin, Germany
5Medical University of Lodz, Bieganski Hospital, Lodz, Poland
6Deutsches Herzzentrum, Munich, Germany
7Hopital du Haut Leveque, Pessac Cedex, France
8University Bonn, Bonn, Germany
9Bracco Diagnostics Inc., Princeton, NJ, USA
10John Radcliffe Hospital, Oxford, UK
11Cliniques Universitaires Saint-Luc, Brussels, Belgium
Received 16 July 2004; revised 22 October 2004; accepted 18 November 2004; online publish-ahead-of-print 17 December 2004.
* Corresponding author: Tel: +49 2418088468; fax: +49 2418082303. E-mail address: rhoffmann{at}ukaachen.de
See page 534 for the editorial comment on this article (doi:10.1093/eurheartj/ehi142)
![]() |
Abstract |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Methods and results In 120 patients, with evenly distributed EF-groups (>55, 3555, <35%), cineventriculography, unenhanced echocardiography with second harmonic imaging, and contrast echocardiography at low mechanical index with iv administration of SonoVue® were performed. In addition, cardiac MRI at 1.5 T using a steady-state free precession sequence was performed in a subset of 55 patients. On-site, and two blinded off-site assessments were performed for unenhanced and contrast echocardiography, cineventriculography, and MRI according to pre-defined standards. Intra-class correlation coefficients (ICCs) were determined to assess inter-observer reliability between all three readers (i.e. one on-site and two off-site). EF was 56.2±18.3% by cineventriculography, 54.1±12.9% by MRI, 50.9±15.3% by unenhanced echocardiography, and 54.6±16.8% by contrast echocardiography. Correlation on EF between cineventriculography and echocardiography increased from 0.72 with unenhanced echocardiography to 0.83 with contrast echocardiography (P<0.05). Similarly, correlation on EF between MRI and echocardiography increased from 0.60 with unenhanced echocardiography to 0.77 with contrast echocardiography (P<0.05). The inter-observer reliability ICC was 0.91 (95% CI 0.880.94) in contrast echocardiography, followed by cardiac MRI (0.86; 95% CI 0.800.92), cineventriculography (0.80; 95% CI 0.740.85), and unenhanced echocardiography (0.79; 95% CI 0.740.85).
Conclusions Unenhanced echocardiography resulted in slight underestimation of EF and only moderate correlation compared with cineventriculography and MRI. Contrast echocardiography resulted in more accurate EF and significantly improved correlation with cineventriculography and MRI. Contrast echocardiography significantly improved inter-observer agreement on EF compared with unenhanced echocardiography. Inter-observer reliability on EF using contrast echocardiography reaches a level comparable to MRI and is better than those obtained by cineventriculography.
Key Words: Cineventriculography Contrast echocardiography Echocardiography Left ventricular function Magnetic resonance imaging
![]() |
Introduction |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Although the most frequently used modality in clinical practice, echocardiography has gained little acceptance in clinical trials due to its moderate reproducibility and accuracy to define LVEF. Poor acoustic windows and inadequate discrimination of the endocardial border are the main reasons for compromises in reproducibility and accuracy besides geometric assumptions resulting from the two-dimensional approach. In single-centre studies, contrast echocardiography has been shown to allow improved assessment of LV volumes and EF, especially in patients with difficult imaging conditions.912 Recent innovations in contrast-specific ultrasound techniques have further enabled improvements in visualization of the LV endocardial border above the level already shown in previous trials with the use of contrast-enhanced ultrasound imaging.
The objective of this multi-centre study was to define the agreement among different imaging techniques on LV volumes and EF using optimized and state-of-the-art technology for each of the different methods. Cineventriculography and cardiac MRI were used as reference methods for comparison with unenhanced and contrast-enhanced echocardiography. Acquisition of cardiac images was performed at eight sites. Blinded on-site and off-site reading using experienced independent core laboratories was performed for each imaging technique according to well-defined standards. Thus, the results of this study reflect the settings of large multi-centre studies requiring accurate determination of LV function, with implemented uniform and pre-defined image acquisition and image evaluation standards.
![]() |
Methods |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
To provide uniform and interpretable image datasets, recommendations on the performance of image acquisition were prospectively defined for all imaging modalities and provided to all participating institutions. Adherence to the pre-defined imaging protocols was monitored during the enrolment period of this multi-centre trial.
Each of the imaging techniques used to define LV function was assessed by on-site readers (OnR) as well as two off-site readers (OffR) unaware of the results of the other imaging techniques. For a uniform evaluation of LV function within each imaging modality, the evaluation procedures were prospectively defined and provided as guidelines both to the OnR at the study sites and to the unaffiliated blinded OffR at independent experienced core laboratories (see Appendix).
The research protocol was approved by the local institutional ethics committees. All patients gave written informed consent to participate in the study.
Patients
One hundred and twenty patients in sinus rhythm were enrolled with equal contribution at eight European centres experienced in the applied imaging techniques. Patients were enrolled at each centre by an independent physician after performance of cineventriculography to achieve an even distribution within three pre-defined EF-groups (>55, 3555, <35% by visual assessment of cineventriculography). Interpretable cineventriculography with availability of at least two consecutive non-extra-systolic cardiac cycles during ventriculographic contrast administration was a prerequisite for inclusion into the study.
Echocardiography
Two-dimensional (2-D) echocardiography was performed with a commercially available ultrasound scanner (SONOS 5500, Transducer S3, Software Version B2.X, Philips, Andover, MA, USA) using tissue harmonic imaging for unenhanced, and contrast- specific imaging for contrast-enhanced, echocardiography. Prior to patient enrolment, written recommendations were provided for the uniform use of equipment pre-sets, imaging conventions, imaging sequence, and annotations. The pre-defined identical pre-sets were digitally provided to each study centre and stored on their equipment. For unenhanced imaging, second harmonic imaging [mechanical index (MI) 1.6, gain 50%, compression 70%] was used, whereas for contrast-specific imaging a low MI of 0.3 was pre-selected (gain 60%, compression 15%). Optimization of imaging conditions for endocardial border definition was performed for each patient by modulation of transmit power, gain, focus, and dynamic range, as required. Apical four-chamber and two-chamber views were acquired without and with contrast-enhancement. The patients were investigated in the left lateral recumbent position and five consecutive cardiac cycles of each view were acquired during breath-hold and digitally stored. Great care was taken to avoid apical foreshortening and to maximize the length from base to apex.
For contrast-enhanced assessment of LV function, a 20-gauge catheter was introduced into the right antecubital vein. SonoVue® (Bracco Imaging, SPA, Milan, Italy) was administered with a starting infusion rate of 1 mL/min and subsequent adjustment in order to reach homogenous LV cavity opacification without attenuation. Additional bolus injections were administered if required to achieve sufficient contrast saturation. SonoVue® is a commercially available ultrasound contrast agent consisting of sulfur hexafluoride microbubbles stabilized by a phospholipid monolayer shell.
Analysis of unenhanced, as well as contrast-enhanced, echocardiograms was performed by one OnR and two OffR. OffR were independent, not affiliated to the study centres, and blinded to patient profile as well as to the results of the other imaging techniques. Analysis of unenhanced and enhanced echocardiograms was performed in sequence. After finalization of unenhanced image evaluation, the image and database for unenhanced images were locked, and subsequent separate evaluation of contrast-enhanced images was performed.
Analysis of echocardiograms was performed according to well-defined standards and after formal training. End-diastolic and end-systolic LV volumes and EF were determined by manual tracing of end-systolic (smallest LV shape) and end-diastolic endocardial borders (largest LV shape) using apical four-chamber and two-chamber views, employing Simpson's method for biplane assessment. Analyses were performed using an off-line workstation (EnConcert, Philips, Andover, MA, USA). As for cineventriculography and MRI, and according to the recommendations of the American Society of Echocardiography,13 the tracings were performed with the papillary muscles and trabeculations allocated to the LV cavity. The mitral annulus was to be traced as deeply as possible.
Cineventriculography
Scanners allowing an image resolution of at least 512x512 pixels were applied. Standard biplane cineventriculography was performed using a 30° right anterior oblique (RAO) projection and a 60° left anterior oblique (LAO) projection with injection of at least 30 mL of contrast medium at a flow rate of 1214 mL/s using 5F to 7F pigtail catheters in 100 patients. In 20 patients, only monoplane cineventriculography using the RAO projection was obtained. Frame rate was set at 30 Hz. Semi-automatic border tracking was used to define the end-diastolic image, based on the frame with the largest ventricular silhouette, and the end-systolic image, based on the frame with the smallest ventricular silhouette. The image calibration was performed with the use of a metal ball with a diameter of 5.0 cm, with identical positions of the X-ray tubes. Prior to patient enrolment, the adequacy of image projections, contrast medium flow, volume calibration, and image storage to pre-defined written recommendations were confirmed, to ensure quality and consistency of image data.
Analysis of cineventriculography was performed by one OnR and two independent blinded OffR, not affiliated to the participating study centres, and unaware of patient profile, and the results of the other imaging techniques. LV end-diastolic and end-systolic volumes were determined using biplane Simpson's method for all patients with biplane cineventriculography (n=100 patients), according to well-defined standards and after formal training for biplane analyses, using the CAAS II software with LV biplane analysis module (Pie Medical, Maastricht, The Netherlands).
Magnetic resonance imaging
ECG-triggered MRI investigations at a field strength of 1.5 T during breath-hold were performed for cardiac function assessment at five of the participating centres with on-site MRI facilities. A special volume-adapted surface coil was used. Four-chamber, two-chamber, and three-chamber as well as short-axis (SAX) views with a slice thickness of 10 mm were acquired in the baso-apical direction with a temporal resolution of 50 ms.
Analysis of MRI images was performed by one OnR and two OffR, unaffiliated with any of the study centres. Readers were blinded to patient profile as well as to the results of the other imaging techniques. Evaluations were performed according to well-defined standards and after formal training, using the MASS II software (Medis, Leyden, The Netherlands). Endocardial border tracings were performed for each short-axis slice separately at end-diastole and end-systole to derive LV volumes and EF. The definition of most basal slice required continuously visible myocardium including its transition into the LV outflow tract. The last apical short-axis slice was the one in which LV cavity could be visualized during end-systole.
Statistics
Statistical analysis was performed using the SPSS and SAS software packages. As pre-defined in the protocol, LV volumes and EF were summarized (mean±SD) for all imaging techniques for OffR 1. For inter-method comparisons, the differences between echocardiography and cineventriculography, or MRI, in the assessment of LV volumes and EF were summarized and tested using the Student's paired t-test. The limits of agreement (defined as ±2SD from the mean difference) between echocardiographic and cineventriculographic or MRI measurements of global LV function were compared using Bland and Altman analysis.14 The correlation between echocardiography and cineventriculography/MRI in the assessment of EF was calculated. Pearson's correlation coefficients between unenhanced echocardiography and contrast echocardiography compared with cineventriculography and MRI in the assessment of EF and the correlation between cineventriculography and MRI were tested using the single sample test of correlation coefficients.15
Inter-observer variability in determination of EF
The inter-observer variability among the three readers (OffR 1, OffR 2, and OnR) within each imaging modality was estimated using an intra-class correlation coefficient (ICC). The ICC assesses rating reliability by comparing the variability of different ratings of the same subject with the total variation across all ratings and all subjects. The ICC and its confidence interval were calculated using mean squares from the ANOVA model.16,17 The inter-observer variability in the assessment of EF between two readers was determined by percentage of error. The percentage of error was calculated using the formula:
![]() |
The mean percentage of error and its 95% confidence interval were calculated for each pair of readers within each imaging modality. Values of P0.05 (two-sided) were considered to indicate statistical significance. The primary objective of this study was inter-method comparison, in the assessment of EF, between unenhanced and contrast-enhanced echocardiography, and cineventriculography. Inter-method correlations were performed to support the primary objective. For the primary objective, the comparison was prospectively planned for OffR 1 only. No multiplicity adjustment was therefore required for the primary objective.
![]() |
Results |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The SonoVue® infusion rate to achieve optimal image quality (Figure 1) was 1.35±0.44 mL/min. After receiving the contrast agent, a total of two non-serious adverse events of mild intensity were reported in two subjects. In one patient, single ventricular extra-systoles were observed during contrast imaging. Another patient reported malaise 2 h after echocardiography with transient decrease in blood pressure. The event was attributed to ß-blocker treatment, which was initiated after the echocardiography. Both events resolved spontaneously without any sequel.
|
LV volumes and EF
Table 1 displays end-diastolic and end-systolic volumes as well as EF from the four different imaging techniques as determined by OffR 1 for each technique. There were no relevant differences in LV volumes and EF as defined by echocardiography for the subgroup of 55 patients with MRI and echocardiography data available, when compared with the whole study population. Compared with cineventriculography and MRI, LV end-systolic and end-diastolic volumes were underestimated by both unenhanced and contrast-enhanced echocardiography (Table 1). This difference was significantly smaller for contrast-enhanced echocardiography than for unenhanced echocardiography (Table 2).
|
|
|
|
The correlation between EF defined by MRI and echocardiography significantly increased from 0.60 to 0.77 (OffR1) and from 0.57 to 0.75 (OffR2) after administration of contrast. This was accompanied by smaller limits of agreement (Table 3).
Agreement between MRI and cineventriculography in the determination of EF
For 55 patients, both MRI and cineventriculography were available. The mean difference between EF defined by biplane cineventriculography (OffR1) and MRI (OffR1) was 5.8%. The correlation coefficient for the inter-method comparison based on MR OffR1 was 0.72 vs. cineventriculography OffR1 (Table 3).
Inter-observer variability in determination of EF
Inter-observer variability was expressed by the ICC between all three readers (i.e. OnR, OffR1, and OffR2). The best ICC was found for contrast-enhanced echocardiography (0.91; 95% CI 0.880.94), followed by cardiac MRI (0.86; 95% CI 0.800.92). ICC were lower for cineventriculography (0.80; 95% CI 0.740.85) and unenhanced echocardiography (0.79; 95% CI 0.740.85). The mean percentage of error between pairs of readers (i.e. OnR and/or OffRs) was in the range of 915% for cineventriculography (Table 4). It was also high using unenhanced echocardiography. The percentage of error on the EF between the OnR and the OffR of MRI was in the range of 78%. Using contrast-enhanced echocardiography, the percentage of error could be significantly (P<0.001) reduced with much smaller confidence intervals compared with unenhanced echocardiograms (Figure 3, Table 4). Furthermore, the percentage of error in determination of EF was significantly (P<0.001) lower between the two OffR using contrast-enhanced echocardiography compared with cineventriculography. The inter-observer variability on contrast-enhanced echocardiography was comparable to those obtained for MRI (Figure 3).
|
|
![]() |
Discussion |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Several published studies have compared the utility of different methods such as cineventriculography, echocardiography, and MRI to define LV volumes and EF.4,7,8,1821 In most of these studies, the comparison between the different methods was performed within the same centre and often by a single observer. A major advantage of the present study in comparison with previous single-centre studies is its multi-centre design with acquisition of imaging data at different sites and subsequent off-site reading by independent blinded core centres. Thus, the study setting reflects the situation encountered in multi-centre trials that require an accurate and reliable assessment of the LV function for either therapeutic or prognostic purposes.
Echocardiography is widely used in clinical practice to define LV function but is considerably disadvantaged by difficulties in defining endocardial contours in patients with limited image quality, and by the reliance on geometric assumptions. Previous studies have indicated that microbubble administration improves endocardial border definition and reader confidence in wall motion assessment.912,22 For the first time within a multi-centre study, the direct comparison of inter-method agreement and reader reliability for LV function assessments is provided between four imaging modalities, taking on-site evaluations and independent off-site reads for all imaging modalities into account. In addition, there are no multi-centre data referring to the impact of improved visualization of the LV cavity by contrast enhancement on the inter-method agreement and reader reliability in context with other imaging modalities, using latest stage technology.
LV volumes
LV volumes were significantly underestimated using unenhanced echocardiography with state-of-the-art harmonic imaging compared with cineventriculography and MRI. Underestimation of LV volumes by up to 50% using echocardiography in comparison with MRI and cineventriculography has been reported.1921 This can be attributed to the inability to visualize the endocardial border contours, the foreshortening of the left ventricle by tangential cuts resulting in difficulty defining the real LV apex by 2-D echocardiography and the exclusion of trabecular structures from the LV cavity. In addition, all imaging modalities relying on biplane acquisition (i.e. 2-D echocardiography and cineventriculography) require assumptions on ventricle geometry for volume calculations, as no full volume datasets are acquired.23 Contrast enhancement resulted in significantly higher volumes and better correlation and agreement with the reference methods. Better agreement between echocardiography using microbubble enhancement and reference methods has been demonstrated in small single-centre studies.911 However, in contrast to some of these previous studies, which have reported an almost complete equivalence of LV volumes defined by MRI and contrast echocardiography, we found a persistent underestimation of volumes by contrast echocardiography in spite of the use of a modern generation contrast agent in combination with contrast-specific imaging. This underestimation can be reasonably explained by the persisting difficulty in defining the real apex with 2-D echocardiography and the need for assumptions on LV geometry. A combination of contrast echocardiography with 3-D echocardiographic techniques should further reduce this limitation.23 Of note are the differences between biplane cineventriculography and MRI in the assessment of LV volumes with overestimation of systolic and diastolic volumes by cineventriculography in comparison with MRI.
EF
Unenhanced echocardiography resulted in an only moderate agreement with cineventriculography on EF while contrast application increased the correlation and improved the limits of agreement with cineventriculography. Similarly, contrast enhancement increased the correlation and reduced the limits of agreements when compared with MRI. It did not reduce, however, the mean difference between echocardiography and MRI. Interestingly, the mean difference between cineventriculography and MRI on EF was at a level similar to that between echocardiography and MRI.
The mean differences between echocardiography and both cineventriculography and MRI were comparatively small and comparable with the differences observed between MRI and cineventriculography. The maximum mean difference was observed for unenhanced echocardiography compared with cineventriculography with an underestimation of EF by 5.3%.
The limits of agreement between echocardiography and cineventriculography or MRI decreased significantly by the use of contrast enhancement.
Inter-observer variability on the determination of EF
For situations in which serial follow-up of LV function is clinically relevant, the reliability of EF determination is crucial to clinical decision making. Cardiac MRI has been commended for its high accuracy and reproducibility allowing the reduction of sample sizes compared with 2-D echocardiography.24,25
There was a remarkable improvement in inter-observer reliability for contrast-enhanced echocardiography over unenhanced echocardiography on the determination of EF over all readers (i.e. OnR and two OffRs), as expressed by the ICCs. Likewise, when inter-observer variability was assessed in pair-wise comparisons between OffR and/or OnR, significant improvements in the mean percentage of error were demonstrated with the administration of ultrasound contrast, and the inter-observer variability for contrast echocardiography reaches the same level as that of MRI and is better than that of cineventriculography. Data on the inter-observer variability have been reported for echocardiography, MRI, and cineventriculography.10,11,2427 In most reports, only readers of the same centres participated in the studies. In addition, there have been no data allowing a direct comparison on the inter-observer variability of unenhanced and contrast-enhanced echocardiography with other methods. Of note is the large inter-observer variability measured for cineventriculography. This finding was consistent between OnR and OffR, as well as between OffR. Thus, although cineventriculography has been used in multiple therapeutic and prognostic trials to calculate EF, it has important limitations compared with modern echocardiographic techniques with contrast enhancement, or with MRI.
The low inter-observer variability of contrast echocardiography indicates that it may be a very valid method for studies requiring serial assessment of LV systolic function, especially if accurate determination of absolute LV volumes is less important. This is likely to allow detection of relevant changes in LV function more reliably and with smaller sample sizes, as has been shown for MRI.24,25
Study limitations
It is impossible to blind observers to the presence of contrast agents on echocardiographic images, and this may potentially induce bias. However, observers were totally blinded to the patients' identity and to each patient's other results. Training of OffR was similar for all imaging techniques. Evaluations of unenhanced and contrast-enhanced echocardiography were performed separately but in sequential order, reflecting clinical practice more appropriately compared with a fully randomized presentation.
MRI was performed only at five centres allowing only 55 patients to be recruited. Thus, the number of patients in whom all four imaging techniques were obtained was limited. This reflects the limited number of centres able to perform all applied imaging modalities. However, there were no differences in patient characteristics, LV volumes, and EF defined by cineventriculography between all patients and the subgroup with MRI. Similarly, inter-method agreement levels between echocardiographic techniques and cineventriculography as well as inter-observer variability on reading of echocardiography and cineventriculography for the subgroup of 55 patients with available MRI data were similar to the total study population.
The most basal slice evaluated in the MRI dataset required continuously visible myocardium. The applied analysis method is widely used and well-accepted. However, it should be noted that there is no general consensus on the best method to define LV volumes by MRI. The inclusion of a more basal segment in MRI would have resulted in larger volumes.
![]() |
Conclusions |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
Appendix |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Core laboratories
Echocardiography
John Radcliffe Hospital, Oxford, UK: Harald Becher, MD. Clinic Johannes Gutenberg University Mainz, Germany: Stephan von Bardeleben.
Cineangiography
University Clinic Munich, Germany: Hans-Ullrich Stempfle, MD.
University Charite, Berlin, Germany: Wolfgang Boecksch.
Cardiac MRI
CIRCLE (Cardiovascular Imaging, Research, Core Lab and Education), Berlin, Germany.
Radiology Department, Johannes Gutenberg University Mainz, Germany.
![]() |
Acknowledgements |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
References |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Related articles in EHJ: