Chapter 4 – Analysis of the relationship between the Focus 2002 survey data and significant- injury frequency rates

SIMCA P+ was unable to model the data in models PLS-UK-SE-US-1 and PLS-UK-SE-US-2 (as indicated by the inability to produce principal components).

 Table 4.2 – Note 2.

The AstraZeneca SIFR distributions were logarithmically transformed within SIMCA P+ by application of Equation 6.  The transformation was performed to reduce the skewness of the SIFR Y block data.  Reducing the skewness made the data more normally distributed, hence increasing the ability of SIMCA P+ to model the data.

                                      Log transformed SIFR =  log₁₀ (SIFR+1)                                           (6)

4.4 – Conclusions

Section 4.4 records the conclusions of the work performed in this chapter.   

4.4.1 – Validity of the PLS models  

Section 2.9.3.2 stated that for a PLS model to be valid the Q² ordinate intercept of the cross permutation validation plot should be no greater than 0.05.  All of the cross permutation validation plots for all models produced were found to have Q² ordinate intercepts less than 0.05.  Apart from PLS-UK1, all Q² intercepts were found to be negative.  All models produced were therefore proven to have some validity i.e. they predict SIFR performance significantly better than chance.  Examination of the response permutation validation plots shows that the R² and Q² values of some of the permuted models are greater than the original model.  This increase is expected.  The expected increase in R² and Q² results from the permuted models being only slightly perturbed from the original model. 

4.4.2 - Predictive ability- national level   

The R² value of the best-fit line of actual versus predicted SIFR was greater than 0.88 for all models produced.  Based upon the R² values and having regard to the number of degrees of freedom, the ability of the PLS models to predict SIFR performance may be described as good in all cases.  This answers research question 8 listed in Table 1.1.  Inspection of the actual versus predicted SIFR prediction plots in Appendix 13 indicates that all of the models produced are able to discriminate between those sites with good SIFR performance from those that have poorer SIFR performance.  The ability of the PLS models to discriminate between sites with similar SIFR performance may be described as moderate.  The ability of PLS-SE2 to model SIFR performance appears to be poorer than PLS-UK2 and PLS-US2.  There may be several possible explanations for this difference that include:   

·        There may be inconsistencies in the threshold of SIFR reporting across the SE sites.

·        Factors other than organisational culture dominate or significantly affect SIFR performance at the SE sites. 

·        The SIFR performance at the SE sites is driven by a population whose organisational culture is not represented by the mean Focus 2002 question responses.

·        The way in which the Focus 2002 questions were translated from English to Swedish introduced degrees of interpretive freedom that in turn gave rise to a greater response variance.

The experimental work done in this project is unable to identify which, if any, of the above factors gave rise to the poorer predictive ability of PLS-SE2 compared to PLS-UK2 and PLS-US2.   Appendix 11 details Pearson correlation coefficients between the Focus 2002 question responses and SIFR performance for the UK, SE and US sites.  The R² values of the best-fit line through the actual versus predicted SIFR for the UK and SE models are greater than the correlation between any single Focus 2002 question response and SIFR performance.  This finding answers research question 10 listed in Table 1.1.

4.4.3 - Predictive ability – international level

It was shown that it was not possible to produce a combined UK, SE, US PLS model (indicated by SIMCA P+ being unable to produce principal components for models PLS-UK-SE-US1 and PLS-UK-SE-US2).  Research question 11 asks:  Can a single model of organisational culture be used to predict SHE performance in more than one nation?  The inability to produce a combined UK, SE, US PLS model based upon the Focus 2002 responses does not necessarily preclude the possibility of creating a multi-nation model.  Subject to the identification of a suitable set of questions, it may be possible to create a PLS model that is able to predict SIFR performance in more than one nation.  It is recommended that further research should be undertaken to identify a modified suite of questions that correlates with SIFR performance in more than one nation.   The inability to produce a single model for all of the three nations is unsurprising.  Appendix 11 details the (Pearson) correlation matrix of the arithmetic mean Focus 2002 question responses and SIFR data for each of the UK, SE and US sites.  The matrix indicated that only one Focus 2002 question response is correlated with SIFR performance above the threshold of significance in all three nations.  It also indicated that a particular Focus 2002 question response may be positively correlated with SIFR in one nation and negatively correlated in another.  A possible explanation for this observation may be that the accident causation models may be different in different nations.  A possible explanation of the inability to identify a set of questions that correlate within all three nations may be the dominance of national culture over organisational culture.  If national culture dominates over organisational culture, the likelihood of finding a set of questions that correlate with safety performance in several nations is decreased.  This decreased likelihood will be a result of the question responses being influenced by national cultural differences rather than how sites in any one nation perceived a particular issue.  The modelling work within this chapter is unable to identify the relative dominance of national versus organisational climate.

4.4.4 – The use of the Focus 2002 question response standard deviation 

The literature review showed that previous research attempts to correlate organisational culture with SHE outcomes had exclusively used the arithmetic mean responses.  Models PLS-UK3, PLS-SE3 and PLS-US3 all used the standard deviation of the Focus 2002 question responses.  All three of the models were shown to be able to model SIFR performance.  The predictive ability of these models is comparable with those models based upon the arithmetic mean responses.   The ability to model SIFR performance based upon the standard deviation of Focus 2002 responses suggests that the distribution of the responses is related to SIFR performance.  Information regarding the relationship of question response arithmetic means and standard deviations can be obtained by inspection of the following model pairs:

·        PLS-UK2 and PLS-UK3.

·        PLS-SE2 and PLS-SE3.

·        PLS-US2 and PLS-US3. 

All of the question responses retained in PLS-US3 are present within PLS-US2.  5 of the questions in PLS-UK3 are also found in PLS-UK2.  5 of the questions retained in PLS-S3 are found in PLS-SE2.  The above comparisons suggest that, for those question responses retained in both the arithmetic mean and standard deviation models, the mean as well as the distribution around the mean is an important metric that is an indicator of SIFR performance.    

4.4.5 - PLS model structure

In Section 3.2 it was noted that the Focus 2002 survey contained questions that addressed all five of Flin et al’s [60] organisational safety culture factors.   PCA models PCA-UK2, PCA-SE2 and PCA-US2 are represented by 4, 4 and 2 principal components respectively.   Table 4.2 shows that all of the resultant PLS models were found to have a maximum number of 2 principal components, rather than an anticipated maximum of 4.  Table 4.2 also indicates that all of the models starting with all of the Focus 2002 question responses are able to account for at least 71% of the SIFR variation (as indicated by Q² cum).  The amount of SIFR variation accounted for by the 1^st component in the 2- component models varies from 0.499 (PLS-SE2) to 0.912 (PLS-UK2).  The average Q² variation accounted for by the first component in all of the PLS models is about:   

 One can therefore conclude that:

The above conclusions answer research question 12 listed in Table 1.1.   The following explanations may account for the low Q² value for the first component in model PLS-SE2 compared to the other PLS models.   

·        The poorer SE models may be due to there being biases or inconsistencies in the way in which the SE sites report SIFR incidents.  Another possible explanation is that there may be greater cultural diversity in SE compared to the UK and US sites.

·        More than one organisational cultural construct is related to SIFR performance in SE.

·        Biases may have been introduced into the Swedish version of the Focus 2002 questionnaire when it was translated from English.

The Focus 2002 questions responses, the number of responses and the relative loadings of those responses retained in the final PLS models are dissimilar for the UK, SE and US nations.  Several possible explanations for the dissimilarities are possible and include:   

4.4.6 – Focus 2002 question responses and their association with desirable SHE performance

The majority of the questions retained in the final PLS models (detailed in Table 4.2) are not directly related to safety.  This observation answers research question 9 listed in Table 1.1.  During the literature review it was found that it is usual to group those questions loading on a particular cultural construct.  Once grouped, it is common for researchers to examine the questions with a view to labelling the construct that best describes the grouped questions.  The usefulness of this practice in industry is questionable.  Grouping several questions together under a single construct label potentially causes useful information to be lost.  Rather than grouping of the questions together under a single construct it is suggested that industry would benefit more from inspection of those questions remaining in the PLS models with a view to identification of specific targeted remedial action to address the undesired state.  An example of this practical approach follows.   Appendix 13 reproduces the graphical outputs of the PLS modelling.  The actual versus predicted SIFR plot for model PLS-UK2 (Appendix 13.7) shows that sites UK2 and UK6 have significantly higher SIFR than the other UK sites.  Simultaneous inspection of the corresponding score weightings scatter plots (Appendix 13.9 and 13.10) illustrates that sites UK2 and UK6 tend to give a higher than average response to the following questions: 

All the other UK sites tended to give lower than average response to these questions.  Section 2.8.2 explained that the above questions were responded to on a five point scale ranging from 1= Agree to 5= Disagree.  Higher than average responses therefore tend toward the ‘disagree’ end of the scale.  Lower than average responses tend toward the ‘agree’ end of the scale.  The observation of the relationship between the above question and desirable safety performance therefore falls in line with expectations i.e. sites that have personnel who are well informed and have clear targets achieve better SIFR performance than those sites whose personnel do not have clear performance targets and are less well informed.  Knowing that sites with undesirable SIFR performance are characterised by responses that are ‘tending to disagree’ with the above questions, managers of the sites can go about instigating remedial action to shift attitudes and satisfactions toward the more desirable end of the scale, and in doing so, perhaps improve site SIFR performance. 

4.4.7 – The use of organisational culture as a proactive safety indicator 

Research question 13 asks: Does the measurement of organisational climate metrics offer a robust tool for industry to predict SHE performance and perform relative risk ranking? The results detailed in this chapter have shown that PLS modelling of the Focus 2002 question response is able to model SIFR performance at the national level within AstraZeneca.  PLS modelling at the multi-national level may also become possible subject to the identification of a set of organisational culture question responses that correlate with safety outcomes in all nations being studied.   The practical application and usefulness of PLS modelling by industry is dependent upon the dynamics of organisational culture.  Chapter 2 established that some researchers have found organisational culture remains stable over a period of several years [35] , whereas others have found that it is more dynamic [31] .  Subject to confirmation that i) organisational culture is stable with time and ii) the correlation between organisational culture and its relationship with SIFR is stable with time, the usefulness of PLS modelling as a proactive safety performance metric looks promising.  This project is unable to determine the stability of organisational culture and its relationship with SHE outcomes with time.  It is recommended that further research be carried out to examine the stability of organisational culture with time.  As PLS is able to identify those questions that are related to desirable and undesirable SIFR outcomes, the use of organisational culture surveys will remain of significant value to industry even if further research into organisational culture discovers that its relationship to SHE outcomes changes over a period of 12 months or more.   

4.4.8 - Summary and conclusions 

By the application of PLS to the Focus 2002 survey responses and GSHE SIFR data, a statistically significant correlation between organisational culture and SIFR performance at AstraZeneca UK, SE and US sites has been established.  Section 2.6 established that previous research attempts have concentrated upon correlation of single dimensions of organisational culture with SHE outcomes.  The results summarised in this chapter represent the first time that several organisational culture factors have been simultaneously correlated with an organisational SHE outcome above the level of statistical significance.   In Section 3.3.2 it was noted that the UK, SE and US AstraZeneca sites support a range of AstraZeneca activities.  The sites therefore represent a range of risk profiles.  The ability to model SIFR performance without consideration of the site relative risk profile suggests that, for AstraZeneca UK, SE and US sites, SIFR performance is not significantly affected by the levels of inherent risk.  This observation puts into question Heinrich’s [73] classic model that specifies that accidents happen as a probabilistic function of the joint occurrence of unsafe acts and unsafe conditions.  It is unlikely that Heinrich’s model is incorrect.  More likely is that industrial risks are already well controlled by engineering and procedural means.  The observed relationship between organisational culture and SIFR performance is most likely to be related to the residual risk remaining after establishment of efficient engineering and procedural controls.   Examination of the structure of the resultant PLS models indicates that one organisational culture construct in the UK and US and two constructs in SE dominate SIFR performance.  Dissimilar accident causation models in each nation may explain the difference in the number of constructs.  Further work in this area is recommended to investigate why the number of constructs differs at the  UK, US and SE sites.   The inability of PLS to resolve sites of similar SIFR performance is of little concern.  It is an established fact amongst SHE professionals that, despite all measures taken to control them, accident frequency rates tend to vary year on year.  Because of this variation, it is proposed that there is little value in attempting to predict the precise SIFR of similar SIFR performing sites.  Of greater practical value to industry is the ability to identify those organisational attributes that discriminate sites with good safety performance from those with significantly poorer safety performance.    Based upon the Focus 2002 survey, it was shown that it was not possible to produce a single PLS model that was capable of predicting SIFR performance across the UK, SE and US sites.  It was suggested that the production of a single PLS UK, SE and US model may be possible after identification of a set of questions that correlate with SIFR performance in each of the nations.