10
Validation of the French version of the Hospital Survey on Patient Safety Culture questionnaire P. OCCELLI 1 , J-L. QUENON 1 , M. KRET 1 , S. DOMECQ 1 , F. DELAPERCHE 1 , O. CLAVERIE 2 , B. CASTETS-FONTAINE 2 , R. AMALBERTI 3 , Y. AUROY 4 , P. PARNEIX 5 AND P. MICHEL 1 1 Comité de coordination de lévaluation clinique et de la qualité en Aquitaine, Pessac, France, 2 Laboratoire des problèmes sociaux et de laction collective département de sociologie, Université Victor Segalen, Bordeaux, France, 3 Institut de médecine aérospatiale du service de santé des armées, Brétigny sur Orge, and Haute autorité de santé, Saint-Denis, France, 4 Institut de médecine aérospatiale du service de santé des armées, Brétigny sur Orge, and Hôpital dinstruction des armées du Val de Grace, Paris, France, and 5 Centre de coordination de la lutte contre les infections nosocomiales Sud-Ouest, Bordeaux, France Address reprint requests to: Jean-Luc Quenon, CCECQA, Hôpital Xavier Arnozan, 33604 Pessac cedex, France. Tel: +33-05-57-65-61-44; Fax: +33-05-57-65-61-36; E-mail: [email protected] Accepted for publication 27 May 2013 Abstract Objective. To assess the psychometric properties of the French version of the Hospital Surveyon Patient Safety Culture ques- tionnaire (HSOPSC) and study the hierarchical structure of the measured dimensions. Design. Cross-sectional survey of the safety culture. Setting. 18 acute care units of seven hospitals in South-western France. Participants. Full- and part-time healthcare providers who worked in the units. Interventions. None. Main outcome measures. Item responses measured with 5-point agreement or frequency scales. Data analyses. A principal component analysis was used to identify the emerging components. Two structural equation modeling methods [LInear Structural RELations (LISREL) and Partial Least Square (PLS)] were used to verify the model and to study the relative importance of the dimensions. Internal consistency of the retained dimensions was studied. A testretest was performed to assess reproducibility of the items. Results. Overall response rate was 77% (n = 401). A structure in 40 items grouped in 10 dimensions was proposed. The LISREL approach showed acceptable data t of the proposed structure. The PLS approach indicated that three dimensions had the most impact on the safety culture: Supervisor/manager expectations & actions promoting safety’‘Organizational learningcontinuous improvementand Overall perceptions of safety. Internal consistency was above 0.70 for six dimensions. Reproducibility was considered good for four items. Conclusions. The French HSOPSC questionnaire showed acceptable psychometric properties. Classication of the dimensions should guide future development of safety culture improving action plans. Keywords: safety culture, safety climate, safety management, validation studies, psychometrics Introduction In France, as in other Western countries, adverse events are frequent, serious and preventable in more than one-third of cases [1]. These events have their roots in system failures, care processes and working conditions that do not promote safety [2]. Developing a safety culturebased on a systemic view of the determinants of safety could be a leverage to improve patient safety in health care [3]. There are many denitions of the safety culture [4]. One denition comes from the nuclear power industry and has been adapted to healthcare: the safety culture of an organiza- tion is the product of individual and group values, attitudes, perceptions, competencies, and patterns of behavior that International Journal for Quality in Health Care vol. 25 no. 4 © The Author 2013. Published by Oxford University Press in association with the International Society for Quality in Health Care; all rights reserved 459 International Journal for Quality in Health Care 2013; Volume 25, Number 4: pp. 459468 10.1093/intqhc/mzt047 Advance Access Publication: 5 July 2013 at Aston University on January 17, 2014 http://intqhc.oxfordjournals.org/ Downloaded from

Validation of the French version of the Hospital Survey on Patient Safety Culture questionnaire

  • Upload
    p

  • View
    216

  • Download
    1

Embed Size (px)

Citation preview

Page 1: Validation of the French version of the Hospital Survey on Patient Safety Culture questionnaire

Validation of the French version of the

Hospital Survey on Patient Safety Culture

questionnaire

P. OCCELLI1, J-L. QUENON1, M. KRET1, S. DOMECQ1, F. DELAPERCHE1, O. CLAVERIE2,B. CASTETS-FONTAINE2, R. AMALBERTI3, Y. AUROY4, P. PARNEIX5 AND P. MICHEL1

1Comité de coordination de l’évaluation clinique et de la qualité en Aquitaine, Pessac, France, 2Laboratoire des problèmes sociaux et de l’actioncollective département de sociologie, Université Victor Segalen, Bordeaux, France, 3Institut de médecine aérospatiale du service de santé desarmées, Brétigny sur Orge, and Haute autorité de santé, Saint-Denis, France, 4Institut de médecine aérospatiale du service de santé des armées,Brétigny sur Orge, and Hôpital d’instruction des armées du Val de Grace, Paris, France, and 5Centre de coordination de la lutte contre lesinfections nosocomiales Sud-Ouest, Bordeaux, France

Address reprint requests to: Jean-Luc Quenon, CCECQA, Hôpital Xavier Arnozan, 33604 Pessac cedex, France. Tel: +33-05-57-65-61-44;Fax: +33-05-57-65-61-36; E-mail: [email protected]

Accepted for publication 27 May 2013

Abstract

Objective. To assess the psychometric properties of the French version of the Hospital Survey on Patient Safety Culture ques-tionnaire (HSOPSC) and study the hierarchical structure of the measured dimensions.

Design. Cross-sectional survey of the safety culture.

Setting. 18 acute care units of seven hospitals in South-western France.

Participants. Full- and part-time healthcare providers who worked in the units.

Interventions.None.

Main outcome measures. Item responses measured with 5-point agreement or frequency scales.

Data analyses. A principal component analysis was used to identify the emerging components. Two structural equation modelingmethods [LInear Structural RELations (LISREL) and Partial Least Square (PLS)] were used to verify the model and to study therelative importance of the dimensions. Internal consistency of the retained dimensions was studied. A test–retest was performedto assess reproducibility of the items.

Results. Overall response rate was 77% (n= 401). A structure in 40 items grouped in 10 dimensions was proposed. TheLISREL approach showed acceptable data fit of the proposed structure. The PLS approach indicated that three dimensions hadthe most impact on the safety culture: ‘Supervisor/manager expectations & actions promoting safety’ ‘Organizational learning—continuous improvement’ and ‘Overall perceptions of safety’. Internal consistency was above 0.70 for six dimensions.Reproducibility was considered good for four items.

Conclusions. The French HSOPSC questionnaire showed acceptable psychometric properties. Classification of the dimensionsshould guide future development of safety culture improving action plans.

Keywords: safety culture, safety climate, safety management, validation studies, psychometrics

Introduction

In France, as in other Western countries, adverse events arefrequent, serious and preventable in more than one-third ofcases [1]. These events have their roots in system failures, careprocesses and working conditions that do not promote safety[2]. Developing a ‘safety culture’ based on a systemic view of

the determinants of safety could be a leverage to improvepatient safety in health care [3].There are many definitions of the safety culture [4]. One

definition comes from the nuclear power industry and hasbeen adapted to healthcare: ‘the safety culture of an organiza-tion is the product of individual and group values, attitudes,perceptions, competencies, and patterns of behavior that

International Journal for Quality in Health Care vol. 25 no. 4

© The Author 2013. Published by Oxford University Press in association with the International Society for Quality in Health Care;

all rights reserved 459

International Journal for Quality in Health Care 2013; Volume 25, Number 4: pp. 459–468 10.1093/intqhc/mzt047Advance Access Publication: 5 July 2013

at Aston U

niversity on January 17, 2014http://intqhc.oxfordjournals.org/

Dow

nloaded from

Page 2: Validation of the French version of the Hospital Survey on Patient Safety Culture questionnaire

determine the commitment to, and the style and proficiencyof ’ a healthcare organization to manage safety [5].Safety culture measurement relies on a combination of

quantitative (individual and self-administered questionnaires) andqualitative (interviews, on-site observations, focus groups)methods. For feasibility reasons, individual and self-administeredquestionnaires are mostly used in health care [6–9]. Thesequestionnaires are distributed to a group of healthcare provi-ders in order to measure their shared perceptions about safety,also called ‘safety climate’. Data collected from individuals areaggregated to represent safety climate at the unit level. Fewstudies combined the use of a quantitative questionnaire withanother source of information such as qualitative measures[10, 11].The Hospital Survey on Patient Safety Culture (HSOPSC)

is a self-administered questionnaire funded by the Agencyfor Healthcare Research and Quality [12]. It measures 12dimensions of the safety culture through 42 items and can beused across disciplines. It is widely used in the USA as wellas in European countries [13–17]. The study of the psycho-metric properties of the different HSOPSC versions showedthat the structure of the original one was partially replicatedbut that some adaptations were needed (dimensions weremerged together, items were deleted or replaced in another di-mension). However, these studies had some limitations. Thereproducibility of the measured items has not been studieddespite the fact that such surveys are used to monitor safetyculture over time. Other dimensions that could be relevant inhealthcare have only been tested in one study [16]. The firstobjective of the described study was to assess the psychometricproperties of the French version in order to propose a vali-dated tool to French hospitals. The second objective was tostudy the hierarchical structure of the measured dimensions inorder to identify relevant area for future research on safetyculture.

Methods

French version of the hospital survey on patient

safety culture

The HSOPSC was translated into French by a group ofFrench researchers in epidemiology, sociology, ergonomics,anesthesiology and management who work in patient safety. Itwas pretested among healthcare providers [18]. No back trans-lation was done but the questionnaire translation was com-pared with another translation independently performed byanother group of French researchers.The French HSOPSC was identical to the American one

except for three items that were added. These items sought toaccount for the influence of peers in the acquisition of a safetyculture, an aspect that was not measured in the original ques-tionnaire. They were a priori grouped into a new dimensionlabeled ‘Training and organizational learning’: (1) patient safetyissues were addressed during my education, (2) in contact withour co-workers, we improve our practices in term of safety,(3) when someone does not respect patient safety because of

a difficult or complex situation, the ward staff does not react.Additionally, respondents’ background information was col-lected.Item responses were measured with 5-point agreement

(from 1 = strongly disagree to 5 = strongly agree) or frequency(from 1 = never to 5 = always) scales. For each item, the per-centage of positive responses (pointing to a developed safetyculture) was calculated. For positively worded items, responses4 and 5 corresponded to positive answers. Inversely, for nega-tively worded items, responses 1 and 2 corresponded to posi-tive answers. Each dimension was measured through three tofour items. For each dimension, a score was calculated. It wasthe mean of the percentages of positive answers to the dimen-sion’s respective items.

Data collection

The survey was conducted in January 2009 in seven hospitalsin South-western France. Hospitals were randomly selectedaccording to their status and number of beds. Eighteen acutecare units of the selected hospitals voluntarily participated in thesurvey: 10 medical (dermatology, endocrinology, pneumology,rheumatology, neurology, internal medicine, gastroenterologyand three cardiology units) and 8 surgical (cardiac and generalsurgery, urology, pediatrics, gynecology, neurosurgery, and twoorthopedic units) units. Full- and part-time healthcare providerswho worked in the unit for at least one month prior the admin-istration of the questionnaire were included in the survey.Quality and risk management assistants coordinated the dis-

tribution and collection of the questionnaires in each hospital.Head nurses were in charge to identify and to distribute aquestionnaire to each eligible healthcare provider. The recom-mended survey completion period was one week. Completionwas voluntary and anonymous. A deposit box was used forcollection.

Statistical analysis

Exploratory analyses. Items were described calculatingresponse rates and percentage of responses in each category.Intercorrelations among items and dimensions were calculatedusing the non-parametric Spearman test as it is adapted toqualitative ordinal variables. High correlation above 0.80indicated that two items were redundant, and some itemscould be deleted. Low correlation under 0.20 indicated thattwo items were weakly related, and some items were misplacedor could be deleted.Internal consistency of the questionnaire and its dimensions

was measured by Cronbach’s alpha coefficient. The consist-ency of each dimension was also measured after droppingitems. Homogeneity was considered good if alpha was greaterthan 0.70 [19, 20].A principal component analysis (PCA) was used to reduce

the number of variables and to identify the emerging compo-nents, with no a priori on the number of components andtheir underlying causal structure [20]. First, the items’loadings on the first component were studied to confirm the

Occelli et al.

460

at Aston U

niversity on January 17, 2014http://intqhc.oxfordjournals.org/

Dow

nloaded from

Page 3: Validation of the French version of the Hospital Survey on Patient Safety Culture questionnaire

unidimensionality of the questionnaire. Then, as dimensionswere correlated (correlation above 0.30), the oblimin rotationmethod was used. The Kaiser’s rule, the scree test and theHorn’s parallel analysis, along with the subjective interpretationof the retained components, were used in order to determinethe number of components. For an item to be considered asbeing adequately represented by a particular component, thesquare of its cosine should be 0.30 or greater [21]. For an itemto be considered as having an adequate contribution to a par-ticular component, its loading should be 0.40 or greater [20].A PCA with an oblimin rotation was finally performed on thefinal French version of the questionnaire, and internal consist-ency of its dimensions was measured.

Confirmatory analyses. A structural equation modeling method(SEM) was used to verify the model hypothesized from theexploratory analyses and to determine how well it fits the data[22, 23]. The SEM consisted in an external model representingthe relationships between the latent (the dimensions) andmanifest variables (their items); as well as an internal modelrepresenting the relationships between the latent variables.Two complementary approaches were used in parallel: (1)

the LInear Structural RELations (LISREL) approach in orderto study the extent to which the data fits the hypothesizedmodel [24] and (2) the Partial Least Square (PLS) approach inorder to estimate the individual scores of the latent variables[25]. Using the LISREL approach, the best fit model wasassessed with the Comparative Fit Index (CFI), and the RootMean Square Error Approximation (RMSEA). CFI valueabove 0.90 and RMSEA value under 0.08 indicate a good fitmodel, CFI value above 0.95 and RMSEAvalue under 0.05 in-dicate an excellent fit model [26, 27]. The jöreskog chi-squaretest was used to compare the original structure in 12 dimen-sions with the hypothesized model. The best model had thelowest chi-square [28].Using the PLS approach, the internal and external model’s

parameters were estimated. For the external model, normal-ized external weights were calculated: they quantify the impactof each item on its dimension (with percentage of relative im-portance of an item’s effect on its dimension). For the internalmodel, structural coefficients were calculated: they representthe impact of each dimension on the safety culture, withoutquantifying it. Homogeneity of the dimensions was measuredusing Dillon–Goldstein’s rho coefficient.

Test–retest

To assess the reproducibility, a test–retest was conducted inanother hospital. Forty-five randomly selected healthcare pro-viders were asked to answer the questionnaire twice with a2-week interval between the test and the retest. The retest wassent out once the test questionnaire had been collected.Internal hospitals’ mailing facilities were used for distributionand collection.Test–retest reliability of the 42 items of the original version

of the questionnaire was assessed by the one-way intra-classcorrelation coefficient (ICC type (1, 1) [29]. Reliability wasconsidered good if ICC was greater than 0.70 [30].

Results

Population

Overall response rate to the survey was 76.5% (n= 401). Itvaried across units from 42.3 to 100%. Most respondents werenurses (45.8%, n= 181). Among respondents, 190 (48.1%) hadworked for 11 years or more in their specialty, 208 (52.8%) hadworked for six years or more in their hospital, and 141 (35.5%)had worked for six years or more in their unit (Table 1).

Validation of a French version of the hospital survey

on patient safety culture

Exploratory analyses. Item response rate ranged from 94.8 to99.8% (Table 2). Four items (A7, A11, F6 and F11) had lowcorrelations with two or more items of their dimension(Table 2); no items had high correlations, above 0.80, with otheritems of their dimension. Cronbach’s alpha was of 0.88 for thequestionnaire and ranged from 0.46 to 0.84 for dimensions.Three dimensions had an alpha above 0.70. The dimensions‘Staffing’ and ‘Training and organizational learning’ had thelowest coefficients.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Table 1 Characteristics of the responders (n= 401)

n %

Staff position 395Physician 55 13.9Nurse 181 45.8Auxiliary nurse 129 32.7Other 30 7.6

Number of years in specialty 395<1 year 38 9.61 to 5 years 95 24.16 to 10 years 72 18.211 to 15 years 36 9.116 to 20 years 50 12.7≥21 years 104 26.3

Number of years in hospital 394<1 year 63 16.01 to 2 years 35 8.92 to 5 years 88 22.3≥6 years 208 52.8

Number of years in clinical unit 397<1 year 84 21.21 to 2 years 42 10.62 to 5 years 130 32.7≥6 years 141 35.5

Working time in clinical unit 397<50% 26 6.5Between 50 and 100% 371 93.5

Participation in risk managementcommittees or structures

397

Yes 105 26.4No 292 73.6

Validation of the French HSOPSC • Safety and management

461

at Aston U

niversity on January 17, 2014http://intqhc.oxfordjournals.org/

Dow

nloaded from

Page 4: Validation of the French version of the Hospital Survey on Patient Safety Culture questionnaire

. . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Table 2 Exploratory factor analysis: response rates, intercorrelations, PCA components, internal consistency and reproducibilityof the 45 items and 13 dimensions measured with the French version of the HSOPSC questionnaire

Dimensions and items Responserate

Intercorrelationbetween items ofthe samedimension

PCA Cronbach’salpha

ICC

<0.20 >0.60

1. Overall perceptions of safety 95.5 0.67A15 Patient safety is never sacrificed to get more work done. 98.8 A10 F7 0.53A18 Our procedures and systems are good at preventing errorsfrom happening.

99.0 F7 0.50

A10 It is just by chance that more serious mistakes do nothappen around here.

98.8 A15 F7 0.50

A17 We have patient safety problems in this unit. 98.5 F7 0.742. Frequency of event reporting 94.8 0.84D1 When a mistake is made, but is caught and corrected beforeaffecting the patient, how often is this reported?

96.0 D2 F2 0.37

D2 When a mistake is made, but has no potential to harm thepatient, how often is this reported?

95.5 D1,D3

F2 0.24

D3 When a mistake is made that could harm the patient, butdoes not, how often is this reported?

94.8 D2 F2 0.60

3. Supervisor/manager expectations & actions promoting safety 97.8 0.83B1 My supervisor/manager says a good word when he/shesees a job done according to established patient safetyprocedures.

99.5 F1 ↓ 0.76

B2 My supervisor/manager seriously considers staffsuggestions for improving patient safety.

98.5 F1 0.70

B3 Whenever pressure builds up, my supervisor/managerwants us to work faster, even if it means taking shortcuts.

98.8 B4 F1 0.66

B4 My supervisor/manager overlooks patient safety problemsthat happen over and over.

98.8 B3 F1 0.51

4. Organizational learning—continuous improvement 97.3 0.59A13 After we make changes to improve patient safety, weevaluate their effectiveness.

99.0 F6 0.60

A6 We are actively doing things to improve patient safety. 99.3 F6 0.57A9 Mistakes have led to positive changes here. 98.5 F6 0.55

5. Teamwork within hospital units 98.3 0.63A1 People support one another in this unit. 99.5 A11 A4 F3 0.62A11 When one area in this unit gets really busy, others help out. 99.3 A1, A4 F11 ↓ 0.59A3 When a lot of work needs to be done quickly, we worktogether as a team to get the work done.

99.5 F3 0.67

A4 In this unit, people treat each other with respect. 99.5 A11 A1 F3 0.706. Communication openness 96.8 0.62C2 Staff will freely speak up if they see something that maynegatively affect patient care.

99.5 0.32

C4 Staff feel free to question the decisions or actions of thosewith more authority.

97.5 0.50

C6 Staff are afraid to ask questions when something does notseem right.

98.8 F5 0.53

7. Feedback and communication about error 95.5 0.64C1 We are given feedback about changes put into place basedon event reports.

98.3 F8 0.64

C3 We are informed about errors that happen in this unit. 97.8 F10 0.11C5 In this unit, we discuss ways to prevent errors fromhappening again.

98.0 F6 0.57

(continued )

Occelli et al.

462

at Aston U

niversity on January 17, 2014http://intqhc.oxfordjournals.org/

Dow

nloaded from

Page 5: Validation of the French version of the Hospital Survey on Patient Safety Culture questionnaire

. . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Table 2 Continued

Dimensions and items Responserate

Intercorrelationbetween items ofthe samedimension

PCA Cronbach’salpha

ICC

<0.20 >0.60

8. Non-punitive response to error 97.5 0.57A12 When an event is reported, it feels like the person is beingwritten up, not the problem.

98.8 0.63

A16 Staff worry that mistakes they make are kept in theirpersonnel file.

99.3 A8 F12 ↓ 0.49

A8 Staff feel like their mistakes are held against them. 99.0 A16 0.509. Staffing 96.3 0.46A2 We have enough staff to handle the workload. 99.0 A7 F9 0.77A14 We work in ‘crisis mode’, trying to do too much, tooquickly.

99.8 A7 F9 0.63

A5 Staff in this unit work longer hours than is best for patientcare.

99.3 A7 F9 0.69

A7 We use more agency/temporary staff than is best forpatient care.

98.3 A2,A14, A5

F10 ↓ 0.64

10. Hospital management support for patient safety 95.5 0.73F1 Hospital management provides a work climate thatpromotes patient safety.

97.0 F8 0.53

F8 The actions of hospital management show that patientsafety is a top priority.

96.8 F8 0.63

F9 Hospital management seems interested in patient safetyonly after an adverse event happens.

97.5 F8 ↓ 0.53

11. Teamwork across hospital units 96.8 0.59F10 Hospital units work well together to provide the best carefor patients.

98.0 F6 F8 0.33

F4 There is good cooperation among hospital units that needto work together.

97.8 F4 0.38

F6 It is often unpleasant to work with staff from other hospitalunits.

98.3 F10, F2 ↓ 0.62

F2 Hospital units do not coordinate well with each other. 97.3 F6 F4 0.5512. Hospital handoffs and transitions 95.5 0.66F11 Shift changes are problematic for patients in this hospital. 97.8 F3, F7 F5 ↓ 0.71F3 Things ‘fall between the cracks’ when transferring patientsfrom one unit to another.

97.5 F11 F4 0.49

F5 Important patient care information is often lost during shiftchanges.

97.3 F4.F5

0.65

F7 Problems often occur in the exchange of information acrosshospital units.

97.3 F11 F4 0.26

13. Training and organizational learninga 0.46H1 Patient safety issues were addressed during my education 95.5 H3 F11H2 In contact with our co-workers, we improve our practices interm of safety

97.0 F3

H3 When someone does not respect patient safety because of adifficult or complex situation, the ward staff does not react.

96.8 H1 F11

PCA, principal component analysis; ICC, intra-class correlation coefficient.In the column entitled PCA, F7 is the factor to which the item had an adequate contribution (factor loading ≥ 0.40). A bold entry correspondto a factor that adequately represents the item (square of the cosine ≥ 0.30). In the column entitled Cronbach’s alpha, ‘↓’ indicates that thealpha of the dimension was higher after dropping the corresponding item.aAdded dimension to the original questionnaire.

Validation of the French HSOPSC • Safety and management

463

at Aston U

niversity on January 17, 2014http://intqhc.oxfordjournals.org/

Dow

nloaded from

Page 6: Validation of the French version of the Hospital Survey on Patient Safety Culture questionnaire

The results of the exploratory analyses and a first PCA (com-ponents loadings of the items are shown in Supplementary mater-ial, Appendix) lead to hypothesize a structure in 10 dimensionsand 40 items: three items from the original version were dropped(A7, A11 and F11) and one added item was kept (Table 3). Twoitems were reworded (F6 and A5).A second PCA was performed on the hypothesized structure.

Before rotation, the study of the first component confirmed theunidimensionality of the questionnaire. After rotation, the Kaiser’srule indicated a solution in 10 components accounting for 58%of the total variance. The Horn’s parallel analysis indicated a so-lution in nine components. No solution could be retained fromthe scree test as there was no obvious break in the scree plot.The internal consistency measured on the structure in 10

dimensions showed that Cronbach’s alpha was above 0.70 forsix dimensions (dimensions number 2 to 5, 10 and 11 + 12)and ranged from 0.55 to 0.67 for the other four.

Confirmatory analyses. Using the LISREL approach, theoriginal structure in 12 dimensions and 42 items was tested(chi-square = 1308.4, df = 741) and showed an acceptable fitto the data (CFI = 0.848; RMSEA = 0.050). The hypothesizedmodel in 10 dimensions and 40 items was tested (chi-square =1199.4, df = 685) and showed an acceptable fit (CFI = 0.855;RMSEA = 0.049). Results of the chi-square test indicated thatthe hypothesized model fitted the data better.

The PLS model was conducted on the 10-dimension struc-ture. The homogeneity of the 10 dimensions was good withrho coefficients from 0.77 to 0.91. External model parametersshowed that the effect of items on their dimension differed(Fig. 1), e.g. one item of dimension 8 and two items of dimen-sion 9 had more effect on their dimension than other items.Internal model parameters showed that the impact on the safetyculture of the dimensions differed. The five dimensions with themost impact were ‘3: Supervisor/manager expectations & actionspromoting safety’ (structural coefficient = 0.226) had the mostimpact, followed by ‘4 + 7: Organizational learning—continuousimprovement’ (0.218), ‘1: Overall perceptions of safety’ (0.205),‘10: Hospital management support for patient safety’ (0.179), ‘11+ 12: Teamwork across hospital units’ (0.170).

Test–retest

Forty-five providers answered twice to the questionnaire. Forthe 42 items of the original version, ICC ranged from 0.11 to0.77 (Table 2). Four items had an ICC above 0.70, 25 itemshad an ICC between 0.50 and 0.70 and 13 items had an ICCof 0.50 or under.

Perceptions of safety across the 18 care units

Health care workers’ perceptions on the 10 dimensions ofsafety culture were studied. It showed that five dimensions had

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Table 3 Dimensions and items of the French version of the HSOPSC questionnaire, before and after exploratory analyses

French structure before exploratory analyses French structure after exploratory analyses

Dimensions Items Dimensions Items

1. Overall perceptions of safety A10, A15,A17, A18

1. Overall perceptions of safety A10, A15, A17,A18

2. Frequency of event reporting D1, D2, D3 2. Frequency of event reporting D1, D2, D33. Supervisor/manager expectations &actions promoting safety

B1, B2, B3,B4

3. Supervisor/manager expectations &actions promoting safety

B1, B2, B3, B4

4. Organizational learning—continuousimprovement

A6, A9, A13 4 + 7. Organizational learning—continuousimprovement

A6, A9, A13, C1,C3, C5

5. Teamwork within hospital units A1, A3, A4,A11

5. Teamwork within hospital units A1, A3, A4,H2,(A11)

6. Communication openness C2, C4, C6 6. Communication openness C2, C4, C67. Feedback and communication abouterror

C1, C3, C5

8. Non-punitive response to error A8, A12, A16 8. Non-punitive response to error A8, A12, A169. Staffing A2, A5, A7,

A149. Staffing A2, A5, A14, (A7)

10. Hospital management support forpatient safety

F1, F8, F9 10. Hospital management support forpatient safety

F1, F8, F9, F10

11. Teamwork across hospital units F2, F4, F6,F10

11 + 12. Teamwork across hospital units F2, F4, F6, F3, F5,F7, (F11)

12. Hospital handoffs and transitions F3, F5, F7,F11

13. Training and organizational learning H1, H2, H3

4 + 7: two dimensions grouped into a single dimension. A bold and underlined entry correspond to an added item. An italic entry correspondto a removed item.

Occelli et al.

464

at Aston U

niversity on January 17, 2014http://intqhc.oxfordjournals.org/

Dow

nloaded from

Page 7: Validation of the French version of the Hospital Survey on Patient Safety Culture questionnaire

a score of 50% or under in more than half of the 18 care units,pointing to poorly developed dimensions (Table 4). They wereas follows: ‘1: Overall perceptions of safety’ (score range, 25.0–71.8%), ‘8: Non-punitive response to error’ (3.5–47.1%), ‘9:Staffing’ (15.0–58.3%), ‘10: Hospital management support forpatient safety’ (15.4–58.8%) and ‘11 + 12: Teamwork acrosshospital units’ (24.6–66.7%).

Discussion

The French version of the HSOPSC questionnaire exploresthe same constructs as the original version does; however,some adjustments were required to fit the French context: twodimensions were merged into one, three items were removedand one was added in the revised version. The final structurein 10 dimensions and 40 items performed better than the ori-ginal one in the sample of the seven French hospitals.The French version of the questionnaire has shown ac-

ceptable psychometric properties. It has shown a good feasi-bility and acceptability of a single survey with high responserates. The internal consistency of the final structure was above0.70 for six dimensions. After exploratory analyses, onlyone out of the three new items was kept and added to thedimension ‘Teamwork within units’. This indicates that thecontent of the dimensions could be improved. The HSOPSCquestionnaire has been translated in several European (Belgium,Netherlands, UK and German-speaking Switzerland) andnon-European countries (Taiwan, Japan) [13–16, 31, 32]. Eventhough other translation validation results converge with ours,the final structure of the tools does differ. This corroborates theneed to adapt the tool to each country according to local waysof being, thinking, behaving and communicating. For

international comparison purposes, a core set of dimensionsconsistently assessed as valid should be defined and measuredin all countries.The study has some limitations. First, selection bias might

have existed as units were selected on a voluntary basis and ashead nurses were responsible for distributing the question-naires. One dimension measures the perceptions of workerson the supervisor actions for safety in the unit (including thehead nurse’s actions). Hence, it is possible that head nurseschose not to include some healthcare providers eligible. Forfuture surveys, list of eligible providers should be collected in-dependently from the management of the surveyed unit.Besides, as most of the respondents are nurses, the final struc-ture probably reflects their perceptions of safety. The survey isbest fitted for examining patient safety climate from a hospitalstaff unit perspective. Second, the drafting up and validationof the French version were based on the PCA and MESstudies carried out on the same data set. The small size of thesample did not allow us to split in half the sample as in atypical cross-validation study. The MES studies conductedon the same data confirmed the hypothesized model, butthis need to be confirmed on different data. Finally, theitems could require some fine-tuning. The results of our ex-ploratory analyses showed that the items of the dimension‘Communication openness’ would not measure this constructprecisely enough in the French version. The reproducibility of26 of the final 40 items was found to be average (ICC > 0.50).As back translation has not been performed, it cannot beruled out that the measurement structure has been changedthrough translation. But, the low reliability also points out theinstability of the aspects measured by the questionnaire, whichare based on professionals’ perceptions of safety (themselveslinked to safety circumstances at a given time, and inherently

Figure 1 Confirmatory factor analysis of the hypothesized structure of the French version of the HSOPSC questionnaire:impact of the manifest variables on the latent variables and of the latent variables on the safety culture. Rectangles representitems or manifest variables; circles represent the dimensions (dim) and the safety culture, or latent variables. ‘W(Nor)’ is thenormalized external weight of each item on its dimension. ‘Reg’ is the structural coefficient of each dimension on the safetyculture. The dark variation shows which dimensions have the most impact on the safety culture.

Validation of the French HSOPSC • Safety and management

465

at Aston U

niversity on January 17, 2014http://intqhc.oxfordjournals.org/

Dow

nloaded from

Page 8: Validation of the French version of the Hospital Survey on Patient Safety Culture questionnaire

instable and subject to change). If culture does not change sorapidly, perceptions do. These results show that test–retest reli-ability should be assessed in studies about psychometric prop-erties of safety climate questionnaires. Future research shouldalso evaluate the impact of response rates on safety climatemeasurements, define a threshold for which safety climate canbe considered developed and compare safety climate measure-ments with results of ethnographic studies and with the evolu-tion of patient safety process or outcomes.The LISREL and the PLS techniques were both used.

These methods are complementary rather than being competi-tive. The LISREL approach allowed us to validate the structureemerging from the exploratory analysis, but no scores could becalculated. The PLS approach made it possible to rank thedimensions according to their impact on safety culture and tocalculate the weight of each item on its dimension. The calcu-lation of scores showed that the dimension ‘Supervisor/manager expectations & actions promoting safety’ turned outto have the most impact. A finding, which was previouslyreported by a Norwegian study aimed at validating a shortversion of the HSOPSC questionnaire [33]. Actions aiming atimproving safety culture are currently being defined and testedin health care [34, 35]. Culture being a multidimensionalconcept, it seems important to target the dimensions that havethe most impact. In the literature, developing a non-punitiveresponse to errors is described as an important area to focuson [4]. However, according to our ranking, it does not appearto be one of the most important dimensions, in spite of itsbeing poorly developed in the surveyed units. In our study,three of the five considered dimensions that are perceived asbeing poorly developed have an important impact on safetyculture. Dimensions to focus on in priority should be ‘Overallperceptions of safety’, ‘Hospital management support forpatient safety’ and ‘Teamwork across hospital units’. Finally, theeffect of items on their respective dimensions was quantified. Ashort version of the French HSOPSC questionnaire could bedrawn up, by including the most important items only.As a conclusion, a French language safety culture measuring

tool has been tested. It has similar psychometrics properties ascompared with those of other translation of the HSOPSCquestionnaire. Since 2010, safety culture evaluation is one ofthe requirements of the French hospitals’ accreditationprogram. In addition, the Ministry of health and the HauteAutorité de Santé (High Authority for Health) launched a re-search program to draw indicators from this questionnaire foraccountability and public diffusion.Confirmatory analyses showed that the dimensions mea-

sured by the French HSOPSC questionnaire do not all havethe same impact on the safety culture of health care units. Thisfinding is important for future research: it should guide thestudy of relationships between measured dimensions, safetyculture and patient safety as well as help drawing up relevantinterventions aiming at improving safety culture.

Supplementary material

Supplementary material is available at INTQHC online.

.................................

.................................

...............

...............

...............

...............

...............

............................................................................................................................................................................................................................................

Table4

Scores

aof

the10

dimension

smeasuredwith

thevalidated

French

versionof

theHSO

PSC

questionn

aire

Dimension

sof

safetyculture

AB

CD

EF

G

12

34

12

34

12

12

12

34

34

1.Overallperceptions

ofsafety

53.2

51.0

37.5

71.8

44.7

60.7

50.0

63.9

25.0

38.2

34.4

55.0

40.2

40.8

50.0

26.1

67.6

36.8

2.Frequencyof

eventreporting

68.8

73.1

70.8

58.8

78.9

65.2

60.0

55.6

73.3

50.0

72.7

73.8

50.7

53.7

61.4

49.3

47.1

75.9

3.Supervisor/m

anagerexpectations

&actions

prom

otingsafety

73.4

57.1

48.1

67.6

81.3

77.3

67.1

63.6

55.6

64.7

58.3

58.3

62.0

72.5

71.1

20.5

63.2

7.4

4+7.Organizationallearning—

continuous

improvem

ent

45.2

58.0

48.0

79.8

50.0

58.0

59.6

43.8

50.7

62.5

47.8

48.6

54.5

54.2

64.8

43.5

43.8

44.4

5.Team

workwithinho

spitalunits

87.5

66.3

72.8

78.0

79.2

61.4

55.9

52.3

62.0

92.6

61.5

60.0

72.7

73.8

86.8

76.1

89.7

73.6

6.Com

municationopenness

61.5

56.0

63.0

55.2

63.3

52.4

52.6

66.7

48.1

74.5

61.1

69.0

60.9

58.7

57.9

53.0

64.7

37.0

8.Non

-punitive

respon

seto

error

42.7

31.0

24.4

20.6

36.8

36.2

26.3

3.7

17.9

19.6

24.6

42.2

43.5

29.8

29.8

24.6

47.1

3.5

9.Staffing

29.2

29.8

21.0

42.2

15.0

42.0

33.3

43.3

25.6

23.5

20.3

20.0

43.5

33.3

35.2

25.8

58.3

25.9

10.H

ospitalm

anagem

entsuppo

rtforpatient

safety

19.5

15.4

27.0

23.4

19.7

32.1

27.9

19.4

39.8

33.8

37.5

33.9

29.5

40.0

56.6

29.3

58.8

22.4

11+12.Teamworkacrossho

spitalunits

36.7

30.8

34.0

33.8

36.1

34.1

28.7

35.2

39.5

31.4

31.3

38.9

44.9

47.5

48.0

24.6

66.7

36.0

Aisauniversityhospital;Bisapublichospitalw

ith300beds

ormore;Cisahospitalw

ithlessthan

300beds;D

toG

areprivateho

spitalswith

200beds

ormore.Unitnum

ber1and2are

medicalunits

and3and4aresurgicalunits.

a The

scoreof

adimension

isthemeanof

thepercentagesof

positiveanswers(indicatingadevelopedsafetyculture)o

nits

items.

Occelli et al.

466

at Aston U

niversity on January 17, 2014http://intqhc.oxfordjournals.org/

Dow

nloaded from

Page 9: Validation of the French version of the Hospital Survey on Patient Safety Culture questionnaire

Acknowledgements

Sandrine BERTHELOT, Hospital Nursing Administrator,Polyclinique de Bordeaux Tondu; Sophie ZAMARON,Director, Quality and Patient Safety Department, CHU deBordeaux; Chantal PETIT, Quality and Patient SafetyManager, CHU de Bordeaux; Maryse LABEYRIE, Qualityand Patient Safety Manager, Clinique Esquirol St Hilaired’Agen; Catherine DELHAIE, Quality and Patient SafetyManager, Polyclinique de Navarre à Pau; StéphanieCAZENAVE, Quality and Patient Safety Assistant Manager,Polyclinique de Navarre à Pau; Marie-Odile CAULIER,Hospital Nursing Administrator, CH d’Orthez; VéroniqueNOIRET, Quality and Patient Safety Manager, CH d’Orthez;Elisabeth ETCHEBERRY, Head Nurse, PolycliniqueAguiléra; Violaine LAPORTE, Quality and Patient SafetyAssistant Manager, Polyclinique Aguiléra; Christine CADOT,Quality and Patient Safety Manager, CH d’Agen; AudreyLARRIVE, Quality and Patient Safety Secretary, CH d’Agen;Maryse PISCAREL, Secretary, CCECQA; Teachers of theInstitute of Public Health, Epidemiology and Development(ISPED), Université Bordeaux 2.

Funding

This work was supported by the Haute Autorité de Santé(National Authority for Health) [grant number CSR – PRn07–34]

References

1. de Vries EN, Ramrattan MA, Smorenburg SM et al. The inci-dence and nature of in-hospital adverse events: a systematicreview. Qual Saf Health Care 2008;17:216–23.

2. Michel P, Quenon JL, Djihoud A et al. A French national surveyof inpatient’s adverse events prospectively assessed with wardstaff. Qual Saf Health Care 2007;16:369–77.

3. The Research Priority Setting Working Group of the WHOWorld Alliance for Patient Safety. Summary of the Evidence on PatientSafety: Implications for Research. Geneva: WHO, 2008.

4. Guldenmund FW. The nature of safety culture: a review oftheory and research. Saf Sci 2000;34:215–57.

5. Health and Safety Commission, Advisory Committee on theSafety of Nuclear Installations (ACNSI). Organising for Safety: ThirdReport of the ACSNI Study Group on Human Factors. Sudbury,England: HSE Books, 1993.

6. Flin R, Burns C, Mearns K et al. Measuring safety climate inhealth care. Qual Saf Health Care 2006;15:109–15.

7. Singla AK, Barrett TK, Weissman JS et al. Assessing patientsafety culture: a review and synthesis of the measurement tools.J Patient Saf 2006;2:105–15.

8. Colla JB, Bracken AC, Kinney LM et al. Measuring patientsafety climate: a review of surveys. Qual Saf Health Care 2005;14:364–6.

9. Nieva VF, Sorra J. Safety culture assessment: a tool for improvingpatient safety in healthcare organizations. Qual Saf Health Care2003;12:17–23.

10. Elder NC, Brungs SM, Nagy M et al. Intensive care unit nurses’perceptions of safety after a highly specific safety intervention.Qual Saf Health Care 2008;17:25–30.

11. Ginsburg L, Norton PG, Casebeer A et al. An educational inter-vention to enhance nurse leaders’ perceptions of patient safetyculture.Health Serv Res 2005;40:997–1020.

12. Sorra JS, Dyer N. Multilevel psychometric properties of theAHRQ hospital survey on patient safety culture. BMC Health ServRes 2010;10:199–212.

13. Hellings J, Schrooten W, Klazinga N et al. Challenging patientsafety culture: survey results. Int J Health Care Qual Assur 2007;20:620–32.

14. Smits M, Christiaans-Dingelhoff I, Wagner C et al. The psy-chometric properties of the ‘Hospital Survey on Patient SafetyCulture’ in Dutch hospitals. BMC Health Serv Res 2008;8:230–9.

15. Waterson P, Griffiths P, Stride C et al. Psychometric properties ofthe Hospital Survey on Patient Safety Culture: findings from theUK. Qual Saf Health Care 2010;19:1–5.

16. Pfeiffer Y, Manser T. Development of the German version of thehospital survey on patient safety culture: dimensionality and psy-chometric properties. Saf sci 2010;48:1452–62.

17. Sarac C, Flin R, Mearns K et al. Hospital survey on patient safetyculture: psychometric analysis on a Scottish sample. BMJ Qual Saf2011;20:842–8.

18. Occelli P, Quenon JL, Hubert B et al. Development of the safetyculture: initial measurements at six hospitals in France. J HealthRisk Manag 2011;30:42–7.

19. Streiner DL, Geoffrey RN.Health Measurement Scales: a Practical Guideto Their Development and Use, 3rd ed. Oxford: Oxford UniversityPress, 2003.

20. Igalens J, Roussel P. Research Methods in Human ResourceManagement (in French). Paris: Economica, 1998.

21. Escoffier B, Pagès J. Simple and Multiple Factor Analyses. Objectives,Methods and Interpretation (in French). Paris: Dunod, 1990.

22. Tenenhaus M, Gauchi JP, Ménardo C. Régression PLS et applica-tions. Rev Stat Appl 1995;43:7–63.

23. Fahmy T, Jaeger J, Jakobowicz E. XLSTAT Support. Paris:Addinsoft. http://www.xlstat.com/fr/home (1 August 2010, datelast accessed).

24. Joreskog KG. A general method for analysis of covariance struc-tures. Biometrika 1970;57:239–51.

25. Jakobowicz E. PLS Path Modeling and PLS regression modulesin the XLSTAT software. 3rd Workshop on PLS Developments,ESSEC Business School of Paris, 2009.

26. Bentler PM. Comparative Fit Indexes in Structural Models. PsycholBull 1990;107:238–46.

Validation of the French HSOPSC • Safety and management

467

at Aston U

niversity on January 17, 2014http://intqhc.oxfordjournals.org/

Dow

nloaded from

Page 10: Validation of the French version of the Hospital Survey on Patient Safety Culture questionnaire

27. Steiger JH, Lind J. Statistically-based tests for the number ofcommon factors. In Paper presented at the Annual Spring Meetingof the Psychometric Society in Iowa City, 1980.

28. Jöreskog KG. Some contributions to maximum likelihood factoranalysis. Psychometrika 1967;32:443–82.

29. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessingrater reliability. Psychol Bull 1979;86:420–8.

30. Fermanian J. Measuring agreement between 2 observers: aquantitative case (in French). Rev Epidemiol Sante Publique1984;32:408–13.

31. Chen I, Li H. Measuring patient safety culture in Taiwan usingthe Hospital Survey on Patient Safety Culture (HSOPSC). BMCHealth Serv Res 2010;10:152–62.

32. Ito S, Seto K, Kigawa M et al. Development and applicability ofHospital Survey on Patient Safety Culture (HSOPS) in Japan.BMC Health Serv Res 2011;11:28–35.

33. Olsen E. Exploring the possibility of a common structuralmodel measuring associations between safety climate factorsand safety behaviour in health care and the petroleum sectors.Accid Anal Prev 2010;42:1507–16.

34. Thomas EJ, Sexton JB, Neilands TB et al. The effect ofexecutive walk rounds on nurse safety climate attitudes: arandomized trial of clinical units. BMC Health Serv Res2005;5:28–37.

35. Sexton JB, Berenholtz SM, Goeschel CA et al. Assessing and im-proving safety climate in a large cohort of intensive care units.Crit Care Med 2011;39:934–9.

Occelli et al.

468

at Aston U

niversity on January 17, 2014http://intqhc.oxfordjournals.org/

Dow

nloaded from