Patient-clinician language concordance is a critical component of the language access equation and is considered the gold standard for communication. As a result of lack of validated testing standards, measures, and tools, Kaiser Permanente developed the Clinician Cultural and Linguistic Assessment Initiative to ensure verifiable linguistic proficiency in clinical encounters and has established a standard level whereby the clinician is deemed to have a qualifying level of proficiency in communicating directly with patients independent of an interpreter. Our benchmarking efforts in language concordance have been rooted with the key aim to identify talented bilingual and bicultural clinician workforce and to establish the systems foundation to coordinate appropriate language services. We share accomplishments, lessons learned, and promising practices to inform future efforts in language concordance. (Am J Public Health. 2011;101:205-208.)
LANGUAGE DIFFERENCES between patients and clinicians jeopardize communication, leading to compromised care, increased health disparities and inequities, dissatisfaction with care, and inefficiency in the health care system.[
National Linguistic and Cultural Programs, a division of Kaiser Permanente National Diversity, developed the Clinician Cultural and Linguistic Assessment (CCLA) Initiative as an overarching strategy in language concordance. The initiative had the following goals: (
Despite federal and state legislation, as well as accreditation efforts to ensure equitable health care for all populations, systematic solutions to reduce clinical care disparities in health outcomes among linguistic minorities are still in developmental stages. The most recent area of focus centers on standards and measures for the assessment of professional health care interpreters and bilingual staff. Clinician linguistic proficiency is often the forgotten variable of the language access and health equity equation. Although several language proficiency tests exist, none fill the specific requirements for evaluating clinicians.[
As a call to action, National Linguistic & Cultural Programs led the collaboration with Alameda Alliance for Health in the development of The Provider Linguistic Proficiency Project, funded by The California Endowment in 2002. This initial phase focused on addressing the dearth of research, data, and standards in language concordance.
Our collaboration developed the Kaiser Permanente -- Clinician Cultural and Linguistic Assessment (KP-CCLA) tool, which assesses clinicians' ability to communicate in the language of service in a primary care setting. We focused initial pilot efforts in the most commonly interpreted language in the San Francisco Bay Area: Spanish.
The KP-CCLA is designed to reflect language-based knowledge, skills, and abilities required of bilingual clinicians to successfully complete language-based tasks during the course of their practice. The assessment is considered both a proficiency-based and a criterion-referenced examination. It measures knowledge, skills, and abilities using guided interviews, rather than a set of open-ended questions, so that language samples of an appropriate range and depth can be elicited from the clinician candidates. The KP-CCLA assesses candidates' target language proficiency along 5 dimensions (see the box on this page).
The examination consists of both objective and subjective assessment components. The objective assessment comprises scoring units that determine the candidate's level of proficiency, and the subjective assessment supplements the data on other dimensions, such as fluency, pronunciation, customer service, and cultural proficiency. The subjective portion plays an additional role in determining recommended interventions to further enhance the candidate's linguistic skills.
Examination scoring is conducted by teams of 2 qualified and trained raters to ensure inter-rater reliability. Each rater scores the candidate's performance independently, and upon completion, raters compare their scores, identify any differences in their assessments of each scoring unit, and work to come to a consensus for any differences. Consensus scoring provides an additional check against the objective scoring system, ensuring that any novel, questionable, or otherwise interesting responses have every opportunity to be considered acceptable.
Pilot findings indicate that the Spanish version of the KP-CCLA is able to correctly identify physicians' ratings (based on process of self-report and confirmation by peer evaluation) to their actual candidate ability, providing substantial evidence of the examination's criterion validity (Table 1).
The KP-CCLA assessment of overall medical Spanish is able to discern proficiency levels of low-, medium-, and high-proficiency candidates, because the mean number of objective errors for each group improves with each level of ability (Table 1). Furthermore, the standard deviation becomes smaller as proficiency increases; the range also narrows for each group. These results are expected because higher proficiency candidates make fewer errors, meaning that their scores will be closer to each other.
In addition to the quantitative evidence of criterion validity, considerable evidence of content validity was established during the examination's development. Diverse constituents, including physicians, linguists, health care interpreters, health administrators, and legal representatives, were convened to examine the KP-CCLA's criterion and content validity for a clinical setting in the target language (see the box on this page). This Spanish Expert Panel was convened in 2006 and established the valid cut score to set the standard level in which providers can communicate independently of an interpreter. The passing score was set at 80% (average of all competencies) to ensure safe and effective communication in clinical encounters.
Although the Spanish KP-CCLA was the first and only validated and reliable language assessment specifically designed to assess provider linguistic proficiency in clinical encounters, the assessment has now been developed in 15 other languages on the basis of threshold and priority languages: Amharic, Arabic, Armenian, Burmese, Cantonese, Farsi, French, Japanese, Khmer, Korean, Mandarin, Portuguese, Russian, Tagalog, and Vietnamese. To enhance testing administration, we leveraged the use of Interactive Voice Response technology to administer the KP-CCLA over the telephone. Use of technological innovation increases testing administration reliability, ensures cost-effectiveness, and enables wide-reaching access. The KP-CCLA has also had external applicability in other organizations. As of January 2010, the total number of internal and external assessments administered was 1017.
Kaiser Permanente's decade of learning has opened new possibilities and opportunities to address one of the most critical problems facing health care organizations today, ensuring quality and meaningful communication. The KP-CCLA research and development process, lessons learned, and promising practices have continued to inform the developing field of language concordance. The pilot paved way for new strategies and interventions, such as (
The Kaiser Permanente-Clinician Cultural and Linguistic Assessment tool has been disseminated or is in the process of implementation in the following external sectors at the time of publication:
- Nonprofit sector
- Health care network
- Children's hospital
- Public sector
- State primary care medical society
- County public health department
- City and county public hospital
- Academic sector
- Regional academic health care system
- University academic medical center
- School of medicine
The objective assessment involves measuring communicative competence. This principle assessment area is defined as the ability to meaningfully and accurately understand and produce the target language in a medical setting in a culturally appropriate way. The components of communicative competence are:
- Grammatical competence: if a doctor can use grammar to construct meaning, then the individual has a degree of grammatical competence.
- Discourse competence: if a doctor can use language effectively in a logical and connected way to persuade or explain, he or she is demonstrating discourse competence.
- Sociolinguistic competence: if a doctor can adjust his or her style of speaking according to the age, gender, or educational status of a patient, and use culturally appropriate language, then he or she can be said to possess sociolinguistic competence.
- Strategic competence: if a doctor describes a procedure but does not have the terminology in Spanish, he or she can be said to be using strategic competence.
The areas of subjective assessment are:
- Fluency: the ease with which a candidate can produce native-like target language, including the degree of hesitation, the clarity of speech sounds, and the appropriate use of rhythm, stress, and intonation.
- Pronunciation: the degree of target language phonology, accent, and related comprehensibility.
- Customer service: the ability to make medical issues and concepts accessible to the patient.
- Cultural proficiency: the ability to recognize and respect the patients' expressed beliefs. It also includes comprehension of idiomatic and colloquial speech.
Reprints can be ordered at http://
This article was accepted May 3, 2010.
This study was funded and supported in part by The California Endowment (grant 20012268). The initial phase of the Kaiser Permanente-Clinician Cultural and Linguistic Assessment (KP-CCLA) Spanish research study was conducted in collaboration with Al-ameda Alliance for Health. We would like to acknowledge Kaiser Permanente Northern and Southern California Regions as well as external hospitals, health plans, health systems, medical schools, and other health care organizations for participating and adopting the use of the KP-CCLA tool. The National Council on Interpreting in Health Care was contracted to conduct a national environmental scan of industry practices and tools related to language assessment for various health care and non-health care professions, with the specific goal of understanding what may be relevant for assessing clinicians. University of Arizona was contracted to review and validate the initial Kaiser Permanente Spanish assessment tools. ALTA Language Services, Inc. was contracted to provide KP-CCLA testing administration.
The KP-CCLA Spanish Pilot completed and submitted its final institutional review board report to the Kaiser Foundation Research Institute on April 20, 2010.
PHOTO (COLOR): The Kaiser Permanente-Clinician Cultural and Linguistic Assessment was awarded the National Committee for Quality Assurance's "Recognizing Innovations in Multicultural Health Care Awards" at a Washington, DC ceremony in October 2009. From left to right: Ignatius Bau, former Director of Health Systems, The California Endowment; Oscar Lanza, Manager, National Linguistic & Cultural Programs, Kaiser Permanente National Diversity; Dr. Garth Graham, Deputy Assistant Secretary for Minority Health, Office of Minority Health.
By Gayle Tang, MSN, RN, Gayle Tang was with Kaiser Permanente National Diversity, Oakland, CA.; Oscar Lanza, IMG, Oscar Lanza was with Kaiser Permanente National Diversity, Oakland, CA.; Fátima Marinely Rodriguez, MPH, Fátima Marinely Rodriguez was with Kaiser Permanente National Diversity, Oakland, CA. and Annie Chang, BA, Annie Chang was with Kaiser Permanente National Diversity, Oakland, CA.