Original Research

A validation study of the Miville-Guzman Universality-Diversity Scale in South Africa

Christina Viljoen, Crizelle Els

Received: 24 Sept. 2022; Accepted: 03 Nov. 2022; Published: 31 Jan. 2023

Copyright: © 2023. The Author(s). Licensee: AOSIS.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background: In order for organisations to best implement diversity management programmes that will create an environment in which diversity is appreciated and utilised optimally to achieve organisational outcomes, they need to first identify the attitudes and orientations that individuals in the organisation have towards diversity. Universal-diverse orientation (UDO) is a largely under-researched topic in South Africa, and currently no instrument measuring UDO has been validated in a South African context.

Aim: The purpose of this study was to determine the psychometric properties of the Miville-Guzman Universality-Diversity Scale (M-GUDS) in South Africa.

Setting: The study was conducted among the general working population in the Gauteng Province in South Africa.

Methods: A quantitative cross-sectional research design was followed. A heterogeneous sample of 255 South African employees from the general working population in Gauteng was selected to complete a voluntary web-based questionnaire.

Results: The results confirm that the M-GUDS is reliable; the factorial validity, and convergent and divergent validity of the M-GUDS are also confirmed. The predictive validity of the instrument is partly confirmed.

Conclusion: This study contributes to the limited research that has been done on UDO in South Africa by providing a valid instrument that can give additional insight into organisations on the orientations and attitudes of employees toward diversity. This provides organisations with a platform to assess their diversity orientation that can be used to inform diversity management programmes.

Contribution: This study contributes to the limited research that has been done on UDO in South Africa, as well as addresses the gap of a lacking instrument to measure UDO in South Africa by validating the M-GUDS.

Keywords: Miville-Guzman Universality-Diversity Scale; universal-diverse orientation; diversity of contact; relativistic appreciation; sense of connection.

Introduction

Over the last 30 years, there has been a drastic change in the dynamics of the South African workforce. The new constitution opened the workforce to an influx of previously disadvantaged cultural groups, as well as an increase in gender diversity and individuals with various disabilities (Viviers, Mans-Kemp & Fawcett 2017). This diverse and unique reality has increased the need for organisations to work towards establishing harmony among their employees and to promote respect and collaboration among its members. Diversity management is a framework that guides organisations in transforming their culture into a more diverse and inclusive environment, by distributing work in a fair and organised manner that counters inequality (eds. Bendl et al., 2015). Organisations that lack diversity management systems, or systems that are poorly executed, run the risk of creating a culture where employees could experience exclusion, conflict, lack of teamwork, gossiping and bullying, or a loss of personal identity (Carrim 2016; Holck, Muhr & Villesèche 2016; Joubert 2017). Organisations need to consider that there are various psychological undertones that drive people’s responses and attitudes towards diversity (April, Ephraim & Peters 2012).

Universal-diverse orientation (UDO) is a construct that specifically explores individual attitudes and resistance towards diversity. Universal-diverse orientation is measured by the Miville-Guzman Universality-Diversity Scale (M-GUDS) and was defined by Miville et al. (1999) as:

[A]n attitude toward all other persons that is inclusive yet differentiating in that similarities and differences are both recognised and accepted; the shared experience of being human, results in a sense of connectedness with people and is associated with a plurality or diversity of interactions with others. (p. 292)

Therefore, UDO refers to the ability of a person to connect with others, based on their similarities, while also fostering an appreciation for others, based on their differences. Universal-diverse orientation also specifically incorporates the ability of an individual to push past the discomfort of diverse interactions, by continually engaging in such situations. It was suggested that the M-GUDS would be useful in creating and assessing interventions that focus on well-being within various settings, and that the development of the individual sub-constructs of UDO may be used to achieve overall well-being within these settings (Miville et al. 2004). Nauly, Purba and Fauzia (2018) found positive results by using the M-GUDS to assess the effectiveness of their diversity training modules, having found that positive diversity attitudes are best formed over time and with continual interactions and engagement with diversity, which is also in alignment with various other studies that followed a similar route (Fu et al. 2018; Kilgo 2015; Kohli et al. 2016). When considering the notion of addressing the undertones that direct the attitudes and responses of each stakeholder toward diversity, it is evident that there is more ground to be won by paying attention to individual attitudes towards diversity, and intentionally developing and shaping these personal attitudes through a well-structured diversity management programme over an extended period of time (April et al. 2012). Furthermore, the M-GUDS appears to be a useful tool to shape diversity management programmes and to assist in evaluating the effectiveness of interventions, as well as guiding organisations in developing the individual perceptions and attitudes of their employees (Celinska & Swazo 2016; Fu et al. 2018).

To assist in the development of South African employees, diversity management programmes could be more effectively measured, developed and implemented. To this end, the M-GUDS may be used as a pre- and post-intervention litmus test to track the development of an organisation’s diversity management processes (Celinska & Swazo 2016; Kohli et al. 2016; Vasquez et al. 2017). The M-GUDS may also be used to determine specific constructs to be developed and assist in identifying the best methods for developing a workforce that is aware of the differences in their team and able to utilise these differences and similarities to achieve better results. However, when considering the benefits that the M-GUDS could hold for our unique workforce, it should also be noted that the assessment measure has not yet been validated in the South African context. It is essential for assessment measures to be developed and tested in specific contexts. Moreover, it cannot simply be assumed that the same results would be achieved in all other contexts, especially not in our diverse nation (eds. Foxcroft & Roodt 2018). The validation of measuring instruments is an integral part of collecting quality data that may be seen as yielding credible results (Kazi & Khalid 2012). Researchers need to ensure that measuring instruments are viable in different contexts, with terms and wording that are understandable, relatable and adequate for the intended purpose, as well as based on a good theoretical foundation (De Klerk, Boshoff & Van Wyk 2009; Kazi & Khalid 2012). Therefore, the main purpose of this study will be to validate the full or long form of the M-GUDS in a general population of South Africa to provide organisations with a measuring instrument that is reliable and valid, catering for all the diverse and unique individuals.

Literature review

Conceptualising universal-diverse orientation

It is important for organisations to recruit and develop members that are aware of the multidimensionality of others, understanding that each individual represents a collection of various traits at a single moment in time, and cannot just be defined as having a single trait (Tatli & Özbilgin 2011). Once members are able to see and understand how their individual traits fit into the organisational context, they might feel more included in the organisation as a whole, while also being able to further build on their own self-concept (Warner 2008). The trait of UDO specifically encompasses an individual’s ability to relate and understand diversity among individuals and can be a useful additive to promote connection among organisational members. Universal-diverse orientation may briefly be defined as the inclination of an individual to be aware of the differences and similarities that exist between people, and the ability to also accept and appreciate these similarities and differences (Miville et al. 1999).

Individuals with a high level of UDO, typically display attitudes towards diversity that are formed from three main components springing from a cognitive, behavioural, and emotional reservoir. These components are referred to as relativistic appreciation (RA), diversity of contact (DC), and sense of connection (SC) (Miville et al. 1999). As the cognitive component of UDO, RA describes the ability of an individual to recognise and appreciate the characteristics that make others different or similar to themselves (Fuertes et al. 2000). Diversity of contact is the behavioural component and portrays a person’s desire and tendency to place themselves in situations in which they are exposed to differences (Kottke 2011); whereas SC is the affective component that refers to a person’s capacity to form an emotional connection to those that are different and similar, while overcoming the discomfort or anxiety that diverse interactions initiate (Fuertes et al. 2000). These three dimensions of UDO are a dynamic and complex social structure that represent either an interest in, or a repulsion to interaction with others, as well as the emotional aspect that could create a rejection of or hesitance towards these interactions (Miville et al. 2004). From these dimensions, it can be seen that individuals could have the intention to interact with diverse others, while simultaneously feeling resistance towards an unknown or different environment. This resistance could then be counteracted by the actual cognitive understanding and appreciation of the aspects that make people different or similar. Sadly, this resistance or anxiety has in the past encouraged society to refrain from voluntary interaction, causing division among various groups instead of encouraging society to grow in an appreciation and utilisation of these differences (Miville et al. 2004). However, it seems that people with a high score in UDO are able to recognise the growth potential and personal development that could come from interactions with others, and these individuals would actively seek to grow their understanding by intentionally engaging in situations that further develop their appreciation (Vasquez et al. 2017).

Up until 2010, there had not been any studies based on UDO within the organisational context. However, Kottke (2011) suggested that there should theoretically be a connection between UDO and diverse workplaces, and among teams in organisational settings. This was supported by later studies that showed how social integration in Japanese companies increased creativity, with this relationship being further moderated and strengthened by organisational leaders with higher levels of UDO (Suzuki & Takemura 2016). This implies that universally diverse leaders can counter the feeling of low cohesion among members by recognising and acknowledging differences and similarities, and then effectively motivating members in the management of this diversity (Fuertes et al. 2000; Suzuki & Takemura 2016). These studies show that there is value in developing universally diverse management in organisations, since the management teams will most likely be the drivers of diversity management in organisations (Abramovic & Traavik 2017). Furthermore, Abramovic and Traavik (2017) determined in their study that individuals who are more oriented towards the goals of others, and who have been exposed to more positive diverse situations, tend to be more engaged and active in diversity management activities. Therefore, there seems to be a connection between the individual’s personal experience in the diverse environment, as well as the manner in which the diversity is managed in the environment.

Measuring and validating universal-diverse orientation

As noted previously, UDO is measured by the M-GUDS, which was originally developed by Miville et al. (1999) as a means to determine the attitudes and orientation of an individual toward the similarities and differences of others. The original extended form of M-GUDS consisted of 45 items in total, with 15 items allocated per construct for RA, diversity of contact, and SC (Miville et al. 1999). Although the M-GUDS was originally developed in the United States of America, a short form (M-GUDS-S) has been adapted and validated in various other countries, including Australia, Asia, and the United Kingdom (Jesiek, Shen & Haller 2012; Kegel & DeBlaere 2014). This shorter version of the M-GUDS was proposed by Fuertes et al. (2000) to make it easier and less time-consuming to administer, while still yielding the desired results. The M-GUDS-S consisted of 15 items in total, with five items allocated to each construct of UDO (Kottke 2011). Despite the short form yielding similar results in factor structure, reliability, and validity in other countries, we need to consider the unique South African context before distributing the questionnaires in local organisational diversity programmes. Considering the extent of diversity and the political history of South Africa, it is reasonable to expect that a measure to assess individuals’ attitudes and orientation toward the similarities and differences in others, may perform differently than the authors of the instrument intended. This necessitates that the M-GUDS be validated for fair usage in the South African context. In the current study, we also endeavoured to validate the original long form of the M-GUDS since the factor scores and relationships among scales are more clearly defined and allow for analysis of the subscales in the instrument. Therefore, the primary objective of this study was to examine the reliability, as well as the construct-, convergent-, divergent- and criterion-related validity of the M-GUDS within the general population of South Africa.

In the current study it was expected that the M-GUDS would be reliable, as also found in several previous studies with Cronbach’s alphas ranging from 0.72 to 0.92 (Çivitci 2020; Han & Pistole 2017; House, Razak & Ashraf 2017; Miville et al. 1999):

H₁: The M-GUDS is a reliable measuring instrument.

In the past contrasting results were found for the construct validity of the M-GUDS. Initially, it was postulated that the components of UDO can be seen as multidimensional, consisting of the three dimensions relativistic appreciation, diversity of contact, and sense of connection (Miville et al. 1999). Nevertheless, after analysis, Miville et al. (1999) concluded that UDO can rather be seen as a unidimensional construct, as the factors are so closely related that they are best observed together, despite being cognitive (relativistic appreciation), behavioural (diversity of contact), and affective (sense of connection) in nature:

H₂: The M-GUDS measures one construct that consists of three interrelated components.

To assess the convergent validity of the M-GUDS, we compared the M-GUDS with the Cultural Intelligence Scale (CQS; Earley & Ang 2003). Cultural intelligence (CQ) can be defined as the ability of an individual to understand, function, detect and embrace diverse social interactions (Earley & Ang 2003). Cultural intelligence comprises four components, which are mental, motivational, and behavioural in nature. The mental component of CQ is derived from the metacognitive CQ and cognitive CQ factors, the motivational component from motivational CQ, and the behavioural component from behavioural CQ (Ang et al. 2007). On the other hand, UDO consists of cognitive (relativistic appreciation), behavioural (diversity of contact), and affective (sense of connection) factors that determine an individual’s orientation, behaviour and attitude towards diverse individuals (Miville et al. 1999). This study aimed to determine the similarities that exist between the cognitive and mental aspects of UDO and CQ, which are represented by RA, metacognitive CQ and cognitive CQ factors:

H₃: The M-GUDS is positively correlated with the CQS (convergent validity).

For the purpose assessing the divergent validity, the M-GUDS will be compared to the short form of the Marlowe-Crowne Social Desirability Scale (SDS; Reynolds 1982). Miville et al. (1999) also compared their model of the M-GUDS with the SDS in their validation study, and as desired, they found an insignificant correlation (0.17) between the two models:

H₄: The M-GUDS has a low correlation with the short form of the SDS (divergent validity).

Finally, criterion-related validity was established by examining the relationships between UDO and two outcome variables – colleague relationships and organisational membership. Colleague relationships can be seen as relations among employees who do not directly report to one another (Sias 2005). It is expected that UDO relates to colleague relationships since it was found that a SC tends to focus more on personal discomfort, and the anxiety or ease with which an individual approaches diverse people or situations, turning the focus more to the individual’s self-efficacy and their reliance on their personal ability to interact with others (Miville et al. 2004). Furthermore, it was seen that increasing an individual’s ability to empathise with others, assists in combating anxiety and fear related to diverse interactions, and trust is strongly interlinked with both the cognitive (relativistic appreciation) and emotional (sense of connection) aspects of UDO (Han & Pistole, 2017; Miville et al. 2006):

H₅: UDO predicts employee relationships (criterion-related validity).

Furthermore, Miville et al. (2004) found that the RA construct of UDO correlates positively with collective self-esteem and creates a positive outlook towards a person’s own group and their place in that group. They concluded that it is important for an individual to have a proper understanding and a positive view of their own social group to show an appreciation for and acceptance of other social groups (Miville et al. 2004). Interestingly, they found that although positive collective self-esteem is highly correlated with UDO, one’s personal feelings of self (self-esteem) are also not connected to one’s ability to appreciate diversity on a group level. Therefore, it is assumed that organisational membership would be predicted by UDO, as it is the sense of belonging that various organisational stakeholders experience by means of support, acceptance, respect and inclusion (Cockshaw & Shochet 2013):

H₆: UDO predicts organisational membership (criterion-related validity).

Research design

A quantitative, cross-sectional research design was employed to determine the relationships and problems that exist within a specific environment at a specific point in time (De Vos et al. 2011; Struwig & Stead 2013).

Research participants

The study sample was selected by means of heterogeneous sampling and consists of participants from the general working population within various sectors and organisations of the Gauteng Province in South Africa, as this is the most densely populated province in the country with more than 16 million inhabitants (Statistics South Africa 2022). More specifically, the sample of participants included South African citizens who are employed within the Gauteng Province and are literate in the English language (N = 255). The distribution of the sample is presented below in Table 1.

An advertisement was placed on the platforms encouraging participants from the Gauteng Province (including areas such as Benoni, Carletonville, Heidelberg, Johannesburg, Kempton Park, Pretoria, Vanderbijlpark and Vereeniging) to participate in the study. The purpose and duration of the questionnaire, and the inclusion criteria were outlined in the advertisement to ensure that the appropriate target population was reached. Interested participants were requested to follow the link to complete the online survey. On the online platform, participants were prompted to read the letter of informed consent that stipulated the purpose of the questionnaire, and notified participants that participation is voluntary, that they have the right to withdraw from the study at any time, and that their responses will be treated as confidential.

Measuring instrument(s)

Biographical information for the participants was collected. This included age, gender, cultural group, language, ethnicity, occupation, organisational tenure, and job position tenure.

Universal-diverse orientation was measured by the full form of the M-GUDS. This measuring scale consist of 44 items divided into three subscales, including diversity of contact, relativistic appreciation, and sense of connection (Miville et al. 1999). These three subscales are measured by a six-point scale ranging from 1 (strongly disagree) to 6 (strongly agree), with 15 items (four reversed scored) measuring diversity of contact, 15 items (four reversed scored) measuring RA, and 14 items (five reversed scored) measuring SC.

Cultural intelligence was measured with the CQS. This scale was developed by Ang et al. (2007) and it consists of four subscales, which include metacognitive CQ, cognitive CQ, motivational CQ, and behavioural CQ. These subscales were each measured by a range from 1 (strongly disagree) to 7 (strongly agree) on a seven-point scale with four items for metacognitive CQ, six items for cognitive CQ, and five items for both motivational CQ and behavioural CQ

For the purpose of testing social desirability, the short form of the Marlowe-Crowne SDS was used (Crowne & Marlowe 1960; Reynolds 1982). The scale was adapted from 18 true items and 15 false items to a reduced five true items and eight false items (Crowne & Marlowe 1660; Reynolds 1982).

Colleague relationships were measured by means of the nine items derived from the English version of the questionnaire on the Experience and Assessment of Work (VBBA scale), as developed by Van Veldhoven et al. (2002). These items were measured by a four-point scale ranging from 1 (always) to 4 (never).

Finally, organisational membership was assessed with the Psychological Sense of Organisational Membership Scale (PSOM) to determine the respondents’ workplace belongingness (Cockshaw & Schochet 2010). This added an additional 18 items to the questionnaire rated on a five-point Likert scale ranging from 1 (not at all true) to 5 (completely true).

Statistical analysis

The statistical analyses were conducted with SPSS – version 23 (IBM SPSS Inc. 2016) and Mplus 7.2 (Muthén & Muthén 2014). Confirmatory factor analysis (CFA) was implemented to test the hypotheses as outlined by this study (Hurley et al. 1997). For construct validity, the goodness-of-fit for the CFA was determined by the χ2 statistic, comparative fit index (CFI ≥ 0.90; Byrne 2010), the Tucker-Lewis index (TLI ≥ 0.90; Byrne 2010), the root mean square error of approximation (RMSEA ≤ 0.08; Browne & Cudeck 1993), and the standardised root mean square residual (SRMR ≤ 0.05; Hu & Bentler 1999). Exploratory structural equation modelling (ESEM) was used to account for the multiple related dimensions in the convenience of one model (Dicke et al. 2018). This method allowed for each item to load onto all the other factors in the model, implying an acceptance that there is a likelihood of a small loading to exist between each item and all the other factors represented in a model (Marsh et al. 2009). It was proposed that this method would be more suited since previous research for the M-GUDS also showed that the construct for UDO was identified as an unidimensional construct with high correlations between the factors, instead of a multidimensional construct with more defined sub-constructs that are only related to one another (Fuertes et al. 2000; Miville et al. 1999). The fit of the ESEM model was also determined by the same fit indices used for the CFA models.

Pearson correlations were applied to determine the relationships between the latent variables, with the statistically significance considered at p ≤ 0.05, and practical significance with a medium effect (r ≥ 0.30) or a large effect (r ≥ 0.50; Cohen 1988). Reliability was determined by the Cronbach’s alpha coefficient (α ≥ 0.70; Morera & Stokes 2016).

Results

The construct (factorial) validity for the long form of the M-GUDS was firstly tested as three CFA models, yielding unsatisfactory results. Initially, the analysis for the first CFA model (Model 1) was run with all the items for each factor included. Since unsatisfactory results were attained, all the items with low loadings and non-significance were removed for the analysis of Model 2; however, this model also presented with a poor fit. Consequently, a third CFA model (Model 3) was analysed with the remaining items being equally parcelled. Although the model fit for Model 3 was an improvement from the previously tested models, the results were still not on par with the desired specifications. Therefore, the parcelled items were further analysed as an ESEM model (Model 4). According to this method, target rotation was applied, and cross-loadings were allowed on all the other factors. Lastly, the ESEM model was translated through a set-ESEM approach into a CFA model for the structural model (Dicke et al. 2018). Refer to Table 2 for the results attained from the analysis of the measurement models.

From Table 2, it can be noted that the CFI and TLI for Model 1 were both below the cut-off point of 0.90. Although the RMSEA yielded significant results of above 0.05, the SRMR was substantially higher than the proposed cut-off point of ≤ 0.05. In an attempt to improve the poor fit of Model 1, all the items with factor loadings less than 0.30 were removed from the model. More specifically, DCT9: ‘I do not know too many people from other countries’; DCT11: ‘I have not seen many foreign films’; and DCT12: ‘I am not very interested in reading books translated from another language’ were removed for diversity of contact. For relativistic appreciation, RAP11: ‘It is very important that a friend agrees with me on most issues’; RAP12: ‘Knowing how a person is similar to me is the most important part of being good friends’; RAP13: ‘It is often hard to find things in common with people from another generation’; RAP14: ‘Placing myself in the shoes of a person from another race is usually too tough to do’; and RAP15: ‘It is hard to understand the problems that people face in other countries’ were removed. Furthermore, SCN9: ‘Getting to know someone of another race is generally an uncomfortable experience for me’; and SCN14: ‘I am often embarrassed when I see a physically disabled person’ were removed for SC. Resultantly, the analysis for Model 2 was completed with 10 items remaining for RA, 12 items for diversity of contact, and 12 items for SC.

Despite all the items with low loadings and non-significant items being removed from the analysis, Model 2 still yielded unsatisfactory results with CFI and TLI values below 0.90, and an SRMR value greater than 0.05. Therefore, a load-balancing technique was used by pairing the items with the highest loading into parcels with the items with the lowest loading for a single factor (Little et al. 2013). This process was repeated until all the items had been equally parcelled. Consequently, Model 3 consisted of a three-factor model with five-item parcels for RA, six parcels for diversity of contact, and six parcels for SC. Although this method resulted in a significant value for the SRMR, the RMSEA only indicated a moderate fit (RMSEA = 0.08) and the CFI and TLI were still slightly below the cut-off point of 0.90. It is also noted that the correlations for all the factors of UDO were statistically significant in Model 3. The correlations between diversity of contact and RA (r = 0.77), RA and SC (r = 0.70), as well as diversity of contact and SC (r = 0.85) were all statistically significant to a large effect.

Next, the researcher attempted a relatively novel approach to structural equation modelling known as ESEM. As noted in Table 2, the ESEM model (Model 4) yielded a better fit for the data compared to the three CFA models, with both the CFI and TLI above 0.90, an RMSEA above 0.05, and an SRMR below 0.05 (CFI = 0.95; TLI = 0.91; RMSEA = 0.06; SRMR = 0.03).

Table 3 provides a tabulated representation of the factor loadings of the item parcels of each UDO factor.

Table 3 indicates that all the parcelled items for relativistic appreciation showed strong factor loadings with the lowest factor loadings being RAP1 (λ = 0.64, SE = 0.10) and RAP5 (λ = 0.62, SE = 0.07), and the highest being RAP3 (λ = 0.87, SE = 0.07). For a sense of connection, SCN1 displayed the lowest loading (λ = 0.37, SE = 0.09), while SCN3 was not significant (λ = 0.28, SE = 0.09; p = 0.002); although the rest of the parcels yielded acceptable loadings with the highest being SCN6 (λ = |0.89|, SE = 0.05). The item parcels for diversity of contact, mostly yielded good factor loadings with the highest loading being DCT4 (λ = 0.89, SE = 0.07). It is, however, noted that the item parcel for DCT1 yielded only a slight loading (λ = 0.17, SE = 0.07; p = 0.024), while also having a higher loading to RA (λ = 0.25, SE = 0.08; p = 0.003) and a significant loading to a SC (λ = 0.42, SE = 0.08; p = 0.001).

Next the reliability and correlations between the factors for the M-GUDS were calculated. All the reliability coefficients obtained for the M-GUDS dimensions were sufficient, with Cronbach’s alpha coefficients of 0.79 for diversity of contact, 0.79 for RA and 0.74 for SC. Therefore, H₁ is supported. The results also suggest that all three factors are positively correlated (p ≤ 0.001).

Reference is given to correlations in Table 4 to determine the convergent and criterion-related validity between the M-GUDS and the remaining variable in the model.

From the results obtained in Table 4, it can be noted that the factors for the M-GUDS are positively related to the different dimensions of the CQS. Diversity of contact was found to have a statistically significant correlation with all the CQS factors. However, all the correlations were only practically significant to a small effect, with the exception of motivational CQ (r = 0.46; medium effect). Despite being insignificantly correlated with cognitive CQ, RA has a statistically significant correlation with metacognitive CQ (r = 0.37; medium effect), motivational CQ (r = 0.53; large effect), and behavioural CQ (r = 0.30; small effect). Finally, a SC showed a statistical significance with all the factors of the CQS. The correlation between a SC and CQS was practically significant to a large effect with the motivational CQ (r = 0.66), and to a small effect with metacognitive CQ (r = 0.26), cognitive CQ (r = 0.23), and a very small effect with behavioural aspects CQ (r = 0.15).

Additionally, it is noted that H₅ and H₆ are not supported by the above-mentioned data. Interestingly, colleague relationships were only significantly correlated with diversity of contact (r = 0.17; small effect), while there were no significant correlations with any of the other factors. Organisational membership yielded significant correlation with the diversity of contact factor (r = 0.16; small effect) and the RA one (r = 0.20; small effect).

Divergent validity

Divergent validity was established by comparing the M-GUDS with the SDS. This was achieved by running an additional CFA model that was used to determine the relationships between the UDO factors and social desirability, and this model showed a good fit (CFI = 0.94; TLI = 0.92; RMSEA = 0.06; SRMR = 0.04). The results for the divergent validity are displayed in Table 5.

In support of H₄, the results in Table 5 show that social desirability did not yield statistically significant results when compared to the UDO factors, and all the correlations were below the suggested threshold of 0.85 for discriminant validity (r’s ≤ 0.85; Brown 2015).

Criterion-related validity

Next, the set-ESEM structural model (Model 5) was tested to examine the criterion-related validity of the M-GUDS. For criterion-related validity to be determined, consideration was taken of the size and direction of the standardised beta coefficient values (β) and the statistical significance of the paths, which was set at p < 0.05. Since the ESEM method allows for all the items to load onto all factors represented in the model, Model 4 (ESEM) was set into a CFA model by using a set-ESEM approach. This was achieved by only allowing loading onto the factors that would hold theoretical value and are within the same set of constructs (Dicke et al. 2018). Model 5 (set-ESEM) only presented in minor changes in the fit indices, with the only changes viewed in CFI and SRMR. Nevertheless, the fit indices for the structural model also showed acceptable model fit (CFI = 0.94; TLI = 0.92; RMSEA = 0.06; SRMR = 0.04). The regression results for the structural paths, as set out by the hypotheses, are presented in Table 6.

From the above-mentioned results, it can be noted that diversity of contact significantly predicted colleague relationships (β = 0.20, SE = 0.10, p = 0.046), while RA and SC did not significantly predict colleague relationships (relativistic appreciation: β = −0.03, SE = 0.10, p = 0.780; SC: β = −0.03, SE = 0.09, p = 0.784). Therefore, H₅ is only partially supported by the structural model. Furthermore, organisational membership was not significantly predicted by diversity of contact (β = 0.10, SE = 0.10, p = 0.354) or SC (β = −0.13, SE = 0.10, p = 0.180). However, organisational membership was significantly predicted by RA (β = 0.21, SE = 0.11, p = 0.044). Therefore, the information provided in Table 6 also only partially supports H_6.

Discussion

This study aimed to validate the M-GUDS within the general working population in the Gauteng Province of South Africa. Specifically, the objective was to determine the reliability, construct validity, convergent and divergent validity, as well as the criterion-related validity of the M-GUDS within the given population.

For the first objective, the study aimed to determine the reliability of the M-GUDS measuring instrument in the South African context. The results from the study confirmed that all three factors of the M-GUDS displayed favourable Cronbach’s alpha coefficients (Morera & Stokes 2016). When considering that reliability indicates the internal consistency of a measuring instrument, the results for the study show that diversity of contact, RA, and SC yield consistent results that are in line with previous research (House et al. 2017; Mallinckrodt et al. 2014; Nunnally & Bernstein 1994). Therefore, sufficient proof for H₁ was accepted.

The second objective attempted to determine the construct-, convergent-, divergent- and criterion-related validity of the M-GUDS in the South African context. For construct validity, the hypothesis was to determine whether the M-GUDS measures one construct that consist of three interrelated dimensions (H₂). In the current study, the results indicated that M-GUDS consists of three dimensions (i.e. diversity of contact, relativist appreciation and sense of connection) that are related to one another. More specifically, this would mean that there is some conceptual overlap between the dimensions. From previous studies, it was expected that the M-GUDS would be portrayed as one construct with three dimensions that are closely related to one another (Fuertes et al. 2000; Miville et al. 1999). Therefore, the results for construct validity accepted H₂. Furthermore, studies also showed that the relationships between the dimensions of the M-GUDS were stronger and more closely related to the original long form of the M-GUDS, as was administered in this study. However, we found results that were more in line with the research on the short form of the M-GUDS. In the short form of the M-GUDS, there was slightly less overlap between the dimensions, although they still had strong relationships. Similar results were found in our study, and it is suggested that future research should explore the usefulness of administering the dimensions separately.

Convergent validity was determined by considering the relationships that exist among the factors for the M-GUDS and the CQS. These relationships are considered to determine whether the M-GUDS is able to gain similar results as another measuring instrument that measures a theoretically similar idea or phenomenon (Taherdoost 2016). For convergent validity, H₃ aims to determine whether the M-GUDS is positively related to the CQS. In this study, it was found that the majority of the dimensions of UDO yielded positive relationships with the factors for CQ, with the exception of the relationship between RA and cognitive CQ. Therefore, H₃ was accepted. The non-significance between RA and cognitive CQ would imply that a person can have an appreciation for the similarities and differences in others, even though they have not gained any specific experience or knowledge of the different cultures, morals or norms that differentiates one person or group from another. In other words, they might appreciate a diverse characteristic or phenomenon, without taking the time and effort to research or gain more information on the characteristic or phenomenon. Even though both RA and cognitive CQ are the cognitive aspects for UDO and CQ, these two cognitive aspects do not theoretically measure the same dimension of cognitive function. From literature, it is seen that there is also another cognitive component for CQ, named metacognitive CQ. Metacognitive CQ specifically measures the mental processes involved in understanding other cultures and increases one’s knowledge about other cultures (Van Dyne et al. 2012). In the present study, it was revealed that there is a stronger relationship between metacognitive CQ and RA. Practically, this would imply that a person who is able to notice and appreciate the similarities and differences of others, should also be able to conceptualise these differences in their mind, as well as have control over their thought processes towards diversity (Ang et al. 2007).

Furthermore, it was noted that all of the relationships between diversity of contact and the dimensions of CQ were significant. The strongest relationship was found between diversity of contact and motivational CQ. This implies that individuals who are more committed and interested in diverse interactions would likely have the capacity to focus their attention and energy on learning about and engaging in diverse situations. This would theoretically make sense, since diversity of contact refers to the interest of an individual to interact with diversity (Fuertes et al. 2000), while motivational CQ is an individual’s capacity to interact and turn their attention to diverse interactions (Ang et al. 2007). Therefore, these results show that there should be a capability and motivation to interact with diverse others who drive an individual to engage in the actual interaction with diversity. Sense of connection also yielded significant relationships with all the dimensions of CQ. More specifically, a strong relationship was noted with motivational CQ, and a weaker relationship was noted compared to behavioural CQ. The results from this study, therefore, imply that people who can push beyond the initial discomfort of diverse interactions and form an emotional connection with diverse other (sense of connection), will likely display a higher likelihood of focusing their effort and energy on diverse interactions (motivational CQ). Additionally, these results also show that people who are more comfortable with diverse interactions (sense of connection), will to some extent also have the proficiency to adapt and alter their behaviour and verbal cues to best suit the situation (behavioural CQ; Ang et al. 2007). This is in line with previous research that suggests that an individual needs to be able to overcome the initial discomfort with differences to develop a connection with those they interact with (Kottke 2011), despite not feeling fully comfortable with their own ability to interact and adapt their behaviour in such situations (Ang et al. 2007). This is further substantiated in literature since Miville et al. (2004) found that SC is strongly related to an individual’s self-efficacy and their personal view of their ability to successfully interact with others. Therefore, the weaker relationship between SC and behavioural CQ could be that the participants have a self-perception that they are able to engage well with diverse others, although in practice they might not necessarily display this ability.

Next, divergent validity was determined by assessing the relationship between the M-GUDS and the SDS. The results in this study indicate that there is no relationship between the two constructs. Theoretically, it would make sense that they are not related, and the results show therefore that the M-GUDS does not statistically relate to another construct with which it should theoretically not relate (Taherdoost 2016). Therefore, H₄ stating that the M-GUDS displays divergent validity from theoretically unrelated instruments, was accepted. The study results corroborate the results of previous research by Miville et al. (1999), who identified no significant relationships between the M-GUDS and the SDS. Additionally, these results in our study also give further proof of divergent validity, as well as showing that the participants who completed the questionnaire were not attempting to present themselves in a more or less favourable manner (Reynolds 1982).

Criterion-related validity was determined in further support of the second objective. Specifically, this study was interested to determine whether the dimensions of the M-GUDS predict colleague relationships (H₅) and organisational membership (H₆). The results from the study found that only one dimension of UDO (i.e. diversity of contact) predicts colleague relationships, and the other two dimensions, RA and SC, did not predict colleague relationships. Therefore, H₅ was rejected. The results further imply that colleague relationships would be stronger in organisations in which individuals show a higher interest and intention in connecting with others who are diverse. On the other hand, simply having an appreciation for others might not encourage employees to build stronger colleague relationships. Previous studies have found that RA is a predictor of collective self-esteem (Miville et al. 2004), and therefore the results that show no prediction between RA and colleague relationships were unexpected. However, according to Miville et al. (2004), one needs to first have an appreciation of one’s own group and position in that group before being able to appreciate the role and place of those in other groups. Therefore, these results call for additional research on other factors that might affect this relationship.

Similarly, the non-significant relationship between SC and colleague relationships would suggest that the level of comfort that others feel with diversity will not necessarily encourage employees to improve their relationships with colleagues. Since SC specifically considers an individual’s perception of their self-efficacy in interacting with others, it could mean that they see themselves as competent in diverse interactions, but they do not necessarily have the desire to interact with others (Han & Pistole 2017). Previous research showed that trust highly predicts an individual’s SC, and this would allow them to be more open and understanding with others (Han & Pistole 2017; Miville et al. 2006; Thompson et al. 2002). Therefore, it could be noted that participants might not trust their colleagues, resulting in a lack of agreeableness, anxiety and discomfort (Miville et al. 2006). These results indicate that future studies could determine whether trust could be a moderator in the relationship between a SC and colleague relationships.

The results for the predictive relationship between UDO and organisational membership (H₆₎, suggest that organisational membership is only significantly predicted by RA, while diversity of contact and a SC did not predict organisational membership. Therefore, H₆ is only partially supported. The results from this study consequently suggest that employees who are able to appreciate the contribution that diverse others make, are likely to experience a greater sense of belonging in the organisation, as they would feel as though they also play a significant part in the organisation. On the other hand, individuals who are interested in other cultures and diverse settings (diversity of contact), and individuals who are comfortable with interactions in diverse settings (sense of connection), might not necessarily feel as though they are more connected to their organisation. Although it might be surprising that the affective component of UDO (sense of connection) is unrelated to both colleague relationships and organisational membership, it is noted that previous research has found that this component refers more to the individual’s self-efficacy and their reliance on their personal ability to interact with others (Miville et al. 2004). Consequently, it seems that a SC is focused more on the individual’s perceived personal ability to interact with others than on the actual interaction with others. This differentiation was noticed in later studies that renamed the construct in short to comfort with differences to account for the internal experience of diverse situations, instead of an outward connection with diverse others (Fuertes et al. 2000; Kottke 2011). The criterion-related validity of the M-GUDS is therefore still questionable in this study sample and it is recommended that future research should explore the outcomes of UDO within an organisational context.

Practical implications

In this study, it was found that the M-GUDS is a reliable and valid tool to measure UDO in the South African context. With these results, organisations in the Gauteng Province could incorporate the M-GUDS in their diversity management programmes to gain a better understanding of their employees’ attitudes and orientation toward diversity. Additionally, they may use the results from the M-GUDS as a guide to determine whether the behavioural, cognitive, or affective aspects of UDO should be addressed during the diversity management processes. For instance, colleague relationships can be improved within organisations by implementing interventions that would develop diversity of contact. Diversity of contact is specifically developed by improving an individual’s self-efficacy (Miville et al. 2004). Furthermore, organisational membership could be improved by implementing interventions that allow employees to explore more diverse situations and creative works, while also practising reserving their judgement and appreciating the art, moral viewpoints or artists (Thompson et al. 2002). Consequently, organisations could use these results to improve the diversity attitudes of their employees by providing an environment where they can develop their self-efficacy on a personal level to build on teamwork, colleague relationships, and organisational membership. This in turn will improve creativity, cohesion, and collaboration while benefitting the organisation as a whole. Similarly, the M-GUDS could be used as a means of assessing the effectiveness of a new diversity management policy or practice by conducting a pre- and post-implementation test after an extended period of time after it has been implemented.

Limitations and recommendations

Despite the favourable results achieved in this study, it should be noted that the research study was not without limitations. Since the study consisted of an online survey, distributed via Facebook, to the general Gauteng working population, it should be considered that there is a large section of the employed population that could not be reached, and the possible bias of recruiting participants via social media platforms (i.e. excluding potential participants that do not use these platforms). Therefore, it is recommended that future studies make use of other data-gathering methods (i.e. other social media platforms, pencil-and-paper format, etc.), expanding the study to include the nine other provinces in South Africa, or to conduct the study within specific industries.

This study followed a cross-sectional design of the study and therefore account cannot be taken of specific events or circumstances that might have influenced the responses of participants at the time that the study was conducted. Therefore, it is suggested that a longitudinal design should be followed in future research to corroborate the results found in this study.

As the long-form of the M-GUDS was presented in addition to the other measuring instruments utilised for validation, the questionnaire was time-consuming and this might have resulted in employees being reluctant to complete the survey. This might also account for the low response rate that was achieved (N = 255); so it is suggested that future studies validate and implement the short form of the M-GUDS (M-GUDS-S) to overcome these limitations.

Lastly, it is suggested that bias and equivalence should be tested in future studies, especially considering the vast and diverse nature of the South African workforce and the different perspectives and backgrounds on diversity that exist among the different age, ethnic, culture, gender and language groups.

Acknowledgements

This article was submitted as part of mini-dissertation for a Master’s of Commerce degree in Industrial Psychology at the School of Industrial Psychology and Human Resource Management of the North-West University in December 2020 under the supervision of Prof. C.E. The title of the submitted mini-dissertation is ‘A validation study of the Miville-Guzman Universality-Diversity Scale in South Africa’.

The author(s) declare that they have no financial or personal relationship(s) that may have inappropriately influenced them in writing this article.

C.V. fulfilled the role of the primary researcher, and this study formed part of her Master’s research. She was responsible for the conceptualisation of the article, collecting of the data, the interpretation of the research results, and the writing of the article. C.E. acted as supervisor, and thus played an advisory role in this study and assisted in the conceptualisation of the study and the writing of the research article.

Ethical approval was obtained from the Economic and Management Sciences Research Ethics Committee of the North-West University (NWU-00774-18-S4. 26 October 2018).

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

The data that support the findings of this study are available on request from the corresponding author.

The views and opinions expressed in this article are those of the author and do not necessarily reflect the official policy or position of any affiliated agency of the authors.

References