Assessment of Multicultural Competence Scales for Social Work Practitioners

Assessment of Multicultural Competence Scales for Social Work Practitioners

Karina L. Walters, Ph.D.

Darrell P. Wheeler, Ph.D., M.P.H. 

Assessment of Multicultural Competence Scales for Social Work Practitioners is the first step in a comprehensive assessment of multicultural counseling competencies among social work students.  It is a multi-site study, with data drawn from a cross-sectional mail survey of masters level social work students from 5 schools of social work, and is designed to assess the appropriateness and validity of two self-report inventories used in counseling psychology.

This article presents preliminary results.  (For further background information on the study, see Practice and Research, Spring, 1997.) 

      For nearly four decades, social work educators have emphasized the necessity for multiculturally competent practice (Van Soest, 1995). Social work researchers, however, have not adequately documented practitioner levels of multicultural competence and how these levels of cultural competence ultimately affect practice interactions, treatment efficacy, and treatment outcomes (McMahon & Allen-Meares, 1992). The discrepancy between the call for multiculturally competent practice and the lack of documentation is due in part to insufficient development of multicultural competence practice measures.

      This study addresses such inadequacies with the development and administration of a multicultural social work practice inventory that assesses behavioral and attitudinal levels of multicultural knowledge, awareness, practitioner-client relationship behaviors, and skills (i.e., KABS). KABS consists of modified items from two multicultural self-report inventories utilized in counseling psychology (i.e., the MCI by Sodowsky, Taffe, Gutkin, & Wise, 1994; and the MCAS-B by Ponterotto, Sanchez, & Magids, 1991).

      Data in this preliminary report were drawn from a cross-sectional mail survey of 442 masters level social work students from 5 schools of social work (1 west coast, 1 Midwest, 1 southeast, 1 northeast, and 1 Mid-Atlantic region). A multimethod approach was employed to examine reliabilities, observed interscale correlations, and disattenuated correlations across instruments as indicators of the comparability of the constructs being measured by each scale for social work practice students. There were four main objectives of the study: (1) to review the psychometric properties of the MCI and the MCAS-B for their compatibility and appropriateness for social work practice; (2) to examine the factorial (i.e., construct) validity of each instrument; (3) to examine the dimensional nature of the competency construct across both scales by combining them in a factor analysis; and (4) to examine the criterion-related and convergent validity of the instruments. 

Method

Participants

      Of the 442 participants, 87% were women and 13% were men. Yearly average household income ranged from $28,000 to $37,999 for the entire sample with 20% of the sample reporting a yearly household income in excess of $80,000. Over three-quarters of the sample were first year MSW students. Thirty-six percent reported previous multicultural training experience. In terms of racial self-identification, 9% of the sample self-identified as Asian/ Pacific Islander, 12% as Black/African American/Carribean, 10% as Latino/a, 66% as White/European American, and 2% as multiracial. The majority of the sample self-identified as mainly heterosexual (91%). Approximately 9% self-identifyed as gay, lesbian, or bisexual. 

Measures

In addition to basic background characteristics, questionnaire items focused on multicultural competencies (i.e., KABS) and levels of sensitivity to discriminatory attitudes. 

Multicultural Counseling Competencies. The two measures used to assess multicultural counseling competencies were the MCI and the MCAS-B.  The MCI (Sodowsky et al., 1994) is a multifactor 40 item self-report measure of multicultural counseling competencies. Participants were asked to indicate “how accurately each statement describes you when working in a multicultural counseling [replaced with social work practice] situation.” Each item was scored using a 4-point scale ranging from very inaccurate (1) to very accurate (4). The MCI consists of four subscales: knowledge (11 items), awareness (10 items), relationship behaviors (8 items), and skills (11 items). The subscales have demonstrated moderate to high reliability in the present sample, ranging from .65 for the awareness subscale to .85 for the skills subscale. The MCI items tend to be more behavioral than attitudinal.

            The MCAS-B (Ponterotto et al., 1994) is a multifactor 45 item self-report measure of multicultural counseling competencies. Participants were asked to rate the “truth of each item as it applies” to them on a 7-point scale ranging from not at all true (1) to totally true (7). The scale consists of three subscales: knowledge/skills scale (28 items), awareness scale (14 items), and a social desirability cluster (3 items). The awareness and knowledge/skills scales demonstrated adequate internal consistency in the present sample (alpha = .72 and alpha = .87, respectively). Previous research raised face validity concerns regarding the combination of knowledge and skills into a single scale (Pope-Davis & Dings, 1995). The high correlations with the awareness scale suggest that there may be only one dimension of multicultural competence being measured, despite previous factor analytic studies that suggest there might be two dimensions. Ponterotto et al. (1996) stated that the scales need to be interpreted with caution until further rigorous testing is employed.  In contrast to the MCI, the MCAS-B items tend to be more attitudinal than behavioral.Discriminatory Attitudes. The Quick Discrimination Index (QDI; Ponterotto, Burkard, et al., 1995) is a 30-item Likert-type self-report inventory of gender and racial attitudes and general sensitivity to discriminatory attitudes. The total score for the QDI measures “overall sensitivity, awareness, and receptivity to cultural diversity and gender equality.” Participants were asked to rate their level of agreement from strongly disagree (1) to strongly agree (5) in response to items such as “I feel ok about my son or daughter dating someone from a different race,” or “I think that it is (or would be) important for my children to attend schools that are racially mixed.” Internal consistency in the present sample was high (alpha = .87). 

      Procedure

       This study constituted the first step in a comprehensive assessment of multicultural counseling competencies among social work students. The five schools of social work were contacted and asked to distribute the self-administered surveys to their entire first year MSW class within the first 2 weeks of their first semester (the second year class was also given the survey at one site). The study’s title “ Multicultural Training Project Survey” and eligibility criteria (enrolled in MSW program) were clearly printed on the top page of the questionnaire. The introduction invited voluntary participation and respondents were not remunerated for participation in the study. Completion of the survey took approximately 45 minutes. The response rates for the schools ranged from 18% to 63%.  

Results

The MCI and MCAS-B:  Social Workers Compared with Psychology Interns

In examining the mean levels of multicultural competence across the MCI measure for all subscales (see Figure 1), social work student scores in the present sample were comparable with psychology graduate student intern scores in another study (data extracted from the Pope-Davis & Dings, 1994). Psychology students had slightly higher knowledge and skill scale scores, which is not surprising since they were already in field internships when they participated in the Pope-Davis & Dings (1994) study, and, as a result, most likely had a few years of clinical classroom preparation already. Interestingly, social work students’ mean scores were slightly higher than

psychology interns with respect to self-awareness and relationship behaviors. In contrast, however, in a comparison of the mean scores between social work students and psychology students on the MCAS-B, the psychology interns tend to score considerably higher for both knowledge/skills (mean = 131.8 vs. 102.9) and awareness subscales (mean = 86.4 vs. 75.1).

Construct Validity of the MCI and MCAS-B

      Within each instrument and across scales, we examined interscale correlations and  inter-correlations. The observed interscale correlations within each instrument were fairly comparable to those obtained in previous studies, suggesting that each instrument is not performing in an unusual manner for social work students. Overall, the intercorrelations between the instruments (see Table 1) also suggest that there are only a few problems with the scales. Specifically, the correlation for MCI knowledge with the MCAS-B knowledge/skills subscale was fairly strong     (r = .52), as was the correlation between the MCAS-B knowledge/skills with MCI skill (r = .51). The correlation, however, between the MCAS-B awareness and MCI awareness was quite low

(r = .04), suggesting that these two constructs might not be capturing the same phenomenon. 
 
 

Table 1. Intercorrelations and Reliabilities for the MCAS-B and MCI.

      Factorial validity.  Factor analysis was used in this study for two purposes: (1) to establish the factor structure of the MCI, MCAS-B, and the combination of MCI-MCAS-B among social work students; and (2) to establish empirically the construct validity for the competency constructs (i.e., KABS) among social workers. A principal components analysis using varimax rotation on all factors was performed and resulted in a 4-factor solution for the MCI, a 5-factor solution for MCAS-B, and a 4-factor solution for the combined scale. Examination of the scree tests confirmed the factor solutions. The 4-factor extraction for the MCI (n = 279) accounted for 23% of the common variance and resulted in factor loadings and eigenvalues as follows: Skills (.46-.73; eigen = 8.99); Knowledge (.43-.67; eigen = 2.73); Awareness (.41-.70; eigen = 2.32); and Relationship Behavior (.46-.63); eigen = 1.99). Interestingly, the MCAS-B (n = 351) revealed five factors in contrast to the anticipated two factors. The 5-factor extraction for the MCAS-B accounted for 19% of the common variance with the following factor loadings: Skills/Knowledge items (.38-.74; eigen = 8.3); Multicultural Readings/Concepts (.65-.80; eigen =  4.6); Social/History (.41-.70; eigen = 3.02); Discrimination (.27-.70; eigen = 1.80); and Awareness (.42-.66; eigen = 1.5). The 4-factor extraction for the combined MCAS-MCI scale (n = 241) produced a 4-factor solution that accounted for 18% of the common variance. The factor loadings were as follows: Knowledge (.35-.72; eigen = 14.47); Skills (.31-.68; eigen = 5.42); Awareness (.34-.71; eigen =  4.09); Relationship (.32-.67; eigen = 3.51). Discussion of the 5-factor structure is beyond the scope of this report as is the combined scale analysis.  We will, however, be analyzing these findings in future analyses.

      In selecting the best factor structure for social work students, our primary criteria were the interpretability and clarity of each factor in the final factor loadings. Using this as a guideline, it is clear that the 4-factor structure and constructs found in the MCI (i.e., KABS) is the model best represented in our data, a conclusion confirmed by combined scale factor analysis. The factor analytic solution reflected the anticipated item groupings; thus, the MCI provides preliminary evidence of the multicultural counseling construct’s (i.e., KABS) applicability to social work students. Using the Group-Difference approach to assessing criterion-related validity (Walsh & Betz, 1990), we examined the MCI score differences between those who had previous multicultural training and those who had none. For clarity in interpretation, we included only first year MSW students (n = 339). Analyses of the previous training variable were mixed: the “trained” sample scored significantly higher than the comparison group on two of the MCI subscales. Specifically, students who had previous multicultural training scored significantly higher than students who had no previous training in terms of multicultural competency skills (mean = 3.19 vs. mean = 3.04), t(239) = 2.33, p. < .05, and awareness (mean = 3.04 vs. mean = 2.73),  t(239) = 2.65, p < .05 . There were no significant differences in training level by multicultural knowledge or relationship competencies.

      We also wanted to examine if the subscales of the MCI were appropriately correlated with similar but distinct constructs. For example, if the participants’ scores on competency were strongly correlated with their scores on a measure of sensitivity to discrimination (e.g., QDI), this would provide more evidence that the multicultural competency subscales had construct validity. We chose the QDI and again used only first year students. Results indicated that competency knowledge (r = .32, p < .000); awareness (r = .34, p < .000); and skill (r = .24, p <.00) were all moderately positively correlated with sensitivity to discrimination, providing further evidence of construct validity for the knowledge, awareness, and skill competency constructs among social work students. 

Conclusion

The psychometric properties of the MCI and the MCAS-B appear fairly strong for both instruments. Internal consistency was moderate to high across all subscales. Preliminary factor analyses suggest that the MCI as a competency measure might be applicable to social work practice. Moreover, analyses provided evidence for the factorial, criterion, and convergent validity of the dimensional nature of the competency constructs (i.e., KABS). The results are preliminary and should be interpreted with caution until further analyses support these findings. Despite the preliminary nature of the findings, they appear to support using the MCI as a measure of multicultural practice competencies in social work training as well as practice. Future development of social work-specific multicultural practice instruments should incorporate the multicultural competency KABS constructs. The discrepancy between the call for multiculturally competent practice and the lack of documentation is due, in part, to insufficient development of multicultural competence practice measures. 

The study’s principal investigators are Karina L. Walters, Ph.D. and Darrell P. Wheeler, Ph.D., M.P.H. 
 

                                    References 

McMahon, A., & Allen-Meares, P. (1992). Is social work racist? A content analysis of recent literature. Social Work, 2(2), 43-52. 

Ponterotto, J. G., Burkard, A., Rieger, B. P.,  Grieger, I., D’Onofrio, A., Dubuisson, A., Heenehan, M., Millstein, B., Parisi, M., Rath, J. F., & Sax, G. (1995). Development and initial validation of the Quick Discrimination Index (QDI). Educational and Psychological Measurement, 55(6), 1016-1031. 

Ponterotto, J. G., Rieger, B. P., Barrett, A., Harris, G., Sparks, R., Sanchez, C. M., & Magids, D. (1996). Development and initial validation of the Multicultural Counseling Awareness Scale. In G. R. Sodowsky & J. C. Impara (Eds.), Multicultural Assessment in Counseling and Clinical Psychology Lincoln, NB: University of Nebraska Press. 

Ponterotto, J. G., Sanchez, C. M., & Magids, D. M. (1991). Initial development and validation of the Multicultural Counseling Awareness Scale (MCAS). Poster presented at the annual meeting of the American Psychological Association, San Francisco.  

Pope-Davis, D. B., & Dings, J. G. (1994). An empirical comparison of two self-report multicultural counseling competency inventories. Measurement and Evaluation in Counseling and Development, 27(2), 93-102. 


Pope-Davis, D. B., & Dings, J. G. (1995). The assessment of multicultural counseling

       competencies. In J. G. Ponterotto, J. M. Casas, L. A. Suzuki, & C. M.Alexander

       (Eds.), Handbook of multicultural counseling (pp. 287-311). Thousand Oaks, CA: Sage. 


Sodowsky, G. R., Taffe, R. C., Gutkin, T. B., & Wise, S. L. (1994). Development of the Multicultural Counseling Inventory: A self-report measure of multicultural competencies. Journal of Counseling Psychology, 41(2), 137-148. 


Van Soest, D. (1995). Multiculturalism and social work education: The non-debate about competing perspectives. Journal of Social Work Education, 31(1), 55-66. 


Walsh W. B., & Betz, N. E. (1990). Tests and assessment (2^nd ed.). Englewood Cliffs, NJ: Prentice-Hall.