A Comprehensive Psychometric Examination of the Lesbian, Gay, and Bisexual Knowledge and Attitudes Scale for Heterosexuals (LGB-KASH).

Heterosexual attitudes toward individuals with other sexual orientations and identities, such as lesbian, gay, bisexual, transgender, intersex, asexual, etc. (LGBTQIA+), have been conceptualized as multidimensional and wide-ranging. With the objective of measuring these attitudes, Worthington and colleagues developed the Lesbian, Gay, and Bisexual Knowledge and Attitudes Scale for Heterosexuals (LGB-KASH). In this article, 540 college students were administered the LGB-KASH; these data were examined for their psychometric properties of the instrument using a sequential approach. The first step included descriptive analysis, including internal consistency measures and association rules, followed by the evaluation of dimension structure with factor analysis (confirmatory/exploratory), and, finally, item analysis using non-parametric (Mokken scale analysis) and parametric approaches (item response theory). Specific to the LGB-KASH factor structure, the current structure was not supported by these data; however, alternative subsets of items that could be used to revise the original instrument to obtain a more psychometrically sound measure are suggested.

Keywords: LGBTQIA+; psychometric analysis; association rule analysis; item response theory (IRT); Mokken scale analysis

In recent decades, numerous social, political, economic, and other gains have been achieved within and for the LGBTQIA+ community (lesbian, gay, bisexual, trans*, queer/questioning, and all other non-heteronormative and non-gender-normative identities). Increased awareness of and improved social attitudes toward these individuals have motivated these changes. Yet there is still plenty of work to be done to increase cultural competency toward LGBTQIA+ identified individuals.

It is useful to acknowledge the ever changing and evolving language of LGBTQIA+. LGBTQIA+ is a common acronym for the collective of individuals who identify as lesbian, gay, bisexual, trans*, queer/questioning, intersex, asexual, and others who do not claim normative gender and heterosexual identities. For the purpose of this study, the term queer will be used as the umbrella term to refer to individuals who may in any way identify as non-normative concerning their gender or sexuality. Even the LGBTQIA+ acronym may seemingly grant priority to certain identities within this broad population. It would be irresponsible to ignore that this term has long held a derogatory meaning and could potentially trigger unpleasant feelings; however, the term queer has been reclaimed as a term of empowerment in recent decades, especially since the development of academic "queer theory." This theory uses the term to challenge the categorization of gender and sexuality and expands the discussion beyond the commonly referenced "gay" and "lesbian" identities to be inclusive of all identities that deviate from the normative or dominant. Terms involving the "homo-" prefix (e.g., homosexual, homophobia, homonegativity) will be avoided unless it is used to mirror language used in previous research, as it is indicative of the restrictive notion that the individuals referenced have a romantic or sexual attraction toward or participate in sexual activity with someone of the same sex or gender. Also, this prefix maintains a negative connotation from previous decades in which homosexuality was still classified as a mental disorder by the American Psychiatric Association (APA).

Although queer is often used as a broad and encompassing term when discussing both sexual orientation and gender identity, the terms and language used within and about the LGBTQIA+ community is immense, and there are many other more specific terms that could be discussed. Due to the nature of this study attempting to create a measure that is more inclusive of non-normative gender identities, two terms that warrant further clarification are trans*/transgender and cisgender. Trans* or transgender is another umbrella term that refers to people whose self-identification, anatomy, appearance, manner, expression, behavior, and/or other perceptions differ from conventional or cultural expectations of congruent gender expression and designated birth sex. It should be noted that not everyone whose appearance or behavior is gender-atypical will self-identify as transgender. The asterisk (*) on trans* is alluding to the common technique used when searching through databases where an asterisk on the end of a prefix will return all the terms that include that prefix. For example, trans* could refer to transgender, transsexual, transman, or transwoman. but in general it is considered inclusive of all non-cisgender identities. Cisgender is a term that refers to someone who identifies either by nature or by choice with the sex and gender they were assigned at birth and who conforms to the mainstream gender-based societal expectations. This is also often referred to as "gender normative." This highlights the difficulty in trying to stay current in defining and measuring attitudes with this quickly evolving minority population.

A recent meta-analysis shows that lesbian, gay, and bisexual individuals experience a higher rate of mental disorders among other negative outcomes such as increased distress, higher rates of suicide attempts, and a greater likelihood of substance abuse (Meyer, [18]). This research is careful and deliberate when explaining that an LGBTQIA+ identity itself is not a disorder or considered negative or causal of such outcomes, but the minority stress model proposes that these results can be attributed to the excess social stressors related to the stigma and prejudice of the dominant-minority conflict. Dentato ([6]) generally defined minority stress as "the relationship between minority and dominant values and resultant conflict with the social environment experienced by minority group members." In other words, the minority stress model asserts that attitudes and behaviors from the dominant group create explicit and implicit biases, prejudices, and discrimination against minority group members. For almost two decades, an adapted version of the minority stress model has been proposed to help explain several of these negative outcomes experienced by members of the queer community (Meyer, [16], [17]).

Researchers have been measuring attitudes for decades and have consistently found a relationship between one's attitude and their behaviors (see Cialdini, Petty, & Cacioppo, [5], for a review). The cyclical relationship between attitudes, the creation of stereotypes and stigmas, and the consequent behaviors are all contributing factors to the aforementioned negative outcomes as explained by the minority stress model. It is important to have valid and reliable measures of such attitudes and behavior from the dominant culture. Furthermore, understanding the personal treatment of LGBTQIA+ individuals has broader and more systematic implications such as the shifting of social norms and their influence on policy as well as meeting physical and mental health care needs of LGBTQIA+ individuals by culturally sensitive providers. Professional organizations such as the American Psychological Association's (American Psychological Association, [3]) guidelines for practice with LGBTQIA+ individuals and the U.S. Department of Health and Human Services have set goals to advance LGBT health in Healthy People 2020. The ability to evaluate the care administered is an integral part of determining if practice guidelines and goals are being met.

To evaluate the competency of professionals, improve mental and physical health outcomes for LGBTQIA+ individuals, and assess individual and social attitudes, it is imperative to have a reliable, culturally sensitive, and relevant assessment tool for both public and research use. In a previous study, Beck and colleagues (2014) examined the potential utility of a campus Safe Zone program as an intervention to increase the cultural competency of training clinicians in providing mental health services to individuals from the queer community. These researchers noted difficulty formulating a pre-/post-test design due to limitations in measurement options. Specifically, they noted most of the current measures assess attitudes toward specific identities that may fall under the queer label such as the Riddle Scale (Riddle, [24]), the Nungesser Homosexual Attitudes Inventory–Revised (Shidlo, [28]), the Homosexuality Attitude Scale (Kite & Deaux, [11]), the Attitudes Toward Lesbians and Gay Men Scale (Herek, [9]), and the Lesbian, Gay, and Bisexual Knowledge and Attitudes Scale for Heterosexuals (LGB-KASH; Worthington, Dillon, & Becker-Schutte, [38]). Unfortunately, these fail to capture the full spectrum of the community within one tool. The LGB-KASH is to date the most evolved and nuanced measure of attitudes toward the LGB population and, subsequently, was used in the study. However, due to its psychometric properties and lack of a more contemporary conceptualization of sexuality, Beck and colleagues (2014) cautioned that it could still present significant problems.

The aim of the current study was to assess the psychometric properties and, if warranted, provide recommendations for improved constructs for the LGB-KASH (Worthington et al., [38]), as well as, if warranted, provide recommendations for improving this instrument. The measure contains 28 items and includes five scales: Hate, LGB Knowledge, Religious Conflict, LGB Civil Rights, and Internalized Affirmativeness. Item scores consist of ratings on a 1 to 6 ordinal Likert scale (1 = very uncharacteristic of me or my views; 6 = very characteristic of me or my views). Table 1 shows the contents of the 28 items composing the five subscales/dimensions of the LGB-KASH. Higher scores in the items composing the Hate and Religious Conflict scales are associated with negative perceptions (more hate toward and conflict with non-heterosexual individuals), whereas higher scores in the items composing the LGB Knowledge, LGB Civil Rights, and Internalized Affirmativeness scales are associated with more openness toward and acceptance of non-heterosexual individuals.

Table 1. Descriptive statistics for items and scales and measures of internal consistency (N = 540).

• 12 Note. *Correlations between each item and the subscale scores computed without the item,
• 13 LGB = lesbian/gay/bisexual.

The instrument was developed in a series of four studies reported in the same article that consisted of: item generation, preliminary item screening, and exploratory examination of structure (study 1, n = 422); estimation of internal consistency and testing of measurement model structure (study 2, n = 574); estimation of test-retest reliability and convergent validity (study 3, n = 45); and examination of sensitivity to differences across sexual orientation identities (study 4, n = 190). It is noteworthy that study 2 was conducted using confirmatory factor analysis and tested three hypothetical structure models for the 28 items, but none of these models resulted in satisfactory fit to the data by a number of fit indices. Among these three models, the model that provided the best fit was a 5-factor model with intercorrelated factors. Worthington and colleagues ([38]) concluded that this 5-factor model provided a fair fit to the data, which was selected as the preliminary structure for the instrument. Further, these researchers also concluded that there was the possibility of improvement in confirmatory goodness-of-fit via model re-specification, and that with respect to the stability and validity of the instrument.

In order to meet the study aim (see Figure 1), this comprehensive psychometric evaluation included: (1) descriptive analysis including internal consistency measures and association rules (a data mining technique); (2) evaluation of dimension structure with factor analysis (confirmatory and exploratory factor analysis); and (3) item analysis with non-parametric (Mokken scale analysis) and parametric ordinal item response theory (IRT) methodologies. This final step ensured the robustness of the recommendations provided.

Graph: Figure 1. Data analysis steps used in the study.

This psychometric study had institutional review board approval from a large university from the southeastern United States. Six hundred college students in an Introduction to Psychology course were enrolled and administered questionnaires. Items were in random order, and a quality control indicator was included to ascertain if participants were responding randomly. The students participated in research as part of the course curriculum. The study was posted on the course Web page. Overall, the sample was 19 years old on average (SD = 1.2 years) and predominately identified as White/Caucasian (79.5%; Black or African American: 13.6%, Hispanic or Latina/Latino: 1.2%, Mixed or Other: 4.9%). Approximately two thirds of the participants identified as female (66.8%), one third as male (32.5%), but just under 1% identified as trans* or other (0.7%). As expected, a large majority of the participants identified as heterosexual or straight (87.1%), while the remaining 13% endorsed asexual (8.2%), gay or lesbian (1.0%), bisexual (1.7%), or queer, other, or a combination of sexual orientation identities (1.7%). Although 600 students responded to the questionnaire, a final sample of 540 was obtained after excluding questionnaires with incorrect answers to the quality control item, excluding LGBTQ respondents, and those with seven (at least 75%) or more missing item responses.

A combination of classical psychometric methods, a data mining approach, and parametric and non-parametric IRT methods was used for testing, exploring the instrument structure, and examining the usefulness of each item. Combining two analytical paradigms in performing item analysis (parametric and non-parametric) offers advantages in quality of the resulting instrument because classical psychometric analysis focuses on estimating and testing the dimensional structure of the items, whereas the IRT methods focus on the performance of the items in classifying subjects along an underlying latent continuum (Kelly, Kallen, & Suarez-Almazor, [10]; Kyngdon, [12]; Waugh & Chapman, [37]). The analysis strategy used in this study is summarized in Figure 1. The analyses were conducted using SAS software (version 9.4) and the R environment for statistical computing (version 3.1.2).

Descriptive statistics such as mean, standard deviation, and median were computed for each item and scale. Cronbach's alpha (internal consistency measure), change in alpha when an item is dropped, and corrected item-dimension correlations (correlations between each item and the dimension/subscale scores computed without the item) were estimated. Association rule analysis was conducted on the data. This is an exploratory data mining technique for any type of categorical data that provides sets of rules for co-occurring situations. The a priori association rule algorithm searches for the most frequently occurring item set in the dataset (Agrawal, Imieliński, & Swami, [2]). All 28 items were used to generate the association rules. The simplest association rules consist of two variables with their respective category that are related, an antecedent variable and a consequent variable (but no causality is implied). For example, gender (female) ≤LGTBQ+ acceptance (yes) implies that when gender is female (antecedent variable), then "yes" they accept LGTBQ+ individuals (consequent variable). The strength of these associations is measured by three quantities that are referred to in the data mining literature as support, confidence, and lift. A support = 60%, confidence = 90%, and lift = 2.1 in the above example can be interpreted as follows: the set (gender (female) = >LGTBQ+ acceptance (yes)) occurs 60% (support) in the complete dataset. Among the respondents who were female, 90% (confidence) accepted LGTBQ+ individuals, and respondents who are females are twice or 2.1 (lift) times more likely than males to accept LGTBQ+ individuals. The generated rules are descriptive/exploratory in nature and can be used after psychometric analysis in editing or removal of the item (Swiger, Raju, Breckenridge-Sproat, & Patrician, [31]). The association analysis was conducted with the R package arules (Hahsler, Grun, & Hornik, [8]).

A confirmatory factor analysis (CFA) was conducted to test the factor structure of the LGB-KASH using a structural equation model fitted with maximum likelihood, and the Satorra-Bentler scaled test statistic and robust standard errors (Rosseel, [26]) were used to correct for non-normality if present. The model fit was assessed using the ratio of the χ2 statistic by its degrees of freedom (χ2/df) as well as other commonly used indices such as comparative fit index (CFI), root mean square error of approximation (RMSEA), and standardized root mean square residual (SRMR). The typically used values of χ2/df smaller than two, CFI values greater than 0.95, RMSEA values smaller than 0.06, and SRMR values smaller than 0.08 are all indicative of good fit (Tabachnick & Fidell, [32]). The CFA was fitted in R using the lavaan package (Rosseel, [26]). Because the original factor structure was not supported by the CFA, the underlying structure was explored using an oblique-rotated exploratory factor analysis (EFA) with principal axes extraction and squared multiple correlations as estimates of the prior communalities, and restricted to factors with eigenvalues ≥1. The package psych (Revelle, [23]) was used to perform EFA analysis.

Non-parametric methods and parametric models were used to examine item performance.

This involved the employment of several sub-steps.

Mokken Scale Analysis (MSA). MSA (Mokken, [19]; Sijtsma & Molenaar, [29]) is a non-parametric approach to analysis of survey instruments with the objective of obtaining scales that measure a single trait each, allow person-ordering at the trait level by means of the total scale score, and may provide a hierarchical ordering of items according to their increasing level of agreeability with or endorsement of the latent trait. Given a set of items, the approach consists of four steps: (1) assessment of dimensionality, (2) assessment of monotonicity, (3) assessment of scale properties, and (4) assessment of whether a hierarchical ordering of items is feasible. As opposed to other IRT methods, MSA does not require fitting of models.

Dimensionality assessment is akin to cluster analysis of the items, and it is conducted by a computer algorithm that searches for optimal groupings of items with a minimal level of scalability, as measured by Loevinger's item scalability coefficient (Molenaar, [20]), a non-parametric measure computed from inter-item covariances, ranging from 0 to 1, with values closer to 1 indicating items that better separate on a unidimensional trait (low trait total scores from high trait total scores). For a set of items comprising one scale, the overall scalability coefficient H (after homogeneity) for the scale uses the information from the individual item scalability coefficients to summarize the strength of the relationship between the total score and the trait. A higher overall H coefficient reflects a more accurate person ordering on the trait levels by means of the total scale score. For scales to be used in practice, the following rules of thumb are suggested: 0.3 ≤ H < .4 means a weak scale; 0.4 ≤ H < 0.5 a scale of medium strength; H ≥ 0.5 a strong scale; and H < 0.3 means the items are not scalable (Mokken, [19]).

Monotonicity assessment examines whether higher overall trait levels (estimated by the total scale scores) correspond, probabilistically, to higher item scores (i.e., as one moves from the lower levels of the latent trait to the higher levels, the probability of obtaining higher item scores should not decrease) for each item. It can be conducted in two sub-steps. First, response function plots are examined for each item to visually assess whether increases in the item score correspond, on average, to increases in the total scale score (Figure 2a). If sections of the curves are not monotonically increasing (Figure 2b), then the second sub-step consists of testing statistically, using non-parametric methods, whether the local decreases are significantly different from a non-increasing trajectory. Statistically significant difference is indicative of a violation of the monotonicity assumption, but also can be affected by small sample size. The sample size of 540 was considered sufficiently powered to detect relevant deviations from the monotonicity assumption with significance tests.

Graph: Figure 2. Hypothetical item response functions for an ordinal item with responses ranging from 1 to 6 in a scale composed of four items (total scale score ranging from 4 to 24). Panel A: For this particular item, the monotonicity assumption holds. Higher overall trait levels (estimated by the total scale scores) correspond, probabilistically, to higher item scores, i.e., as one moves from the lower levels of the latent trait to the higher levels, the probability of giving higher item scores is monotonically increasing. Panel B: Violation of the monotonicity assumption. For this particular item, higher overall trait levels do not necessarily correspond, probabilistically, to higher item scores.

Reliability assessment was conducted by computing measures of internal consistency such as Cronbach's alpha, Guttman's lambda-2, or the Molenaar Sijtsma statistic (Molenaar & Sijtsma, [21]) and ensuring that these measures reached acceptable levels.

Unidimensionality, monotonicity, and internal consistency allow ordering persons by their level of a trait, based on their total scale scores; however, they do not assess whether the items constitute a hierarchical scale. The assessment of hierarchical order of items was conducted in two sub-steps. First, for each scale, a plot with the items' summary response functions was examined to visually determine if the individual item functions intersect (they may overlap, but not cross), as in Figure 3a. For scales with curves that cross (Figure 3b), non-parametric methods were used to determine whether the crossing sections were significantly different from overlapping trajectories. Statistically significant differences indicate items might not form a hierarchical scale. For this study, the implementation of MSA was conducted using the mokken package in R (van der Ark, [35], [36]). For step 1, the computer algorithm searched for groups of items with a lower bound scalability with coefficients of 0.5.

Graph: Figure 3. Hypothetical summary response functions for four ordinal items with individual item responses ranging from 1 to 6 (total scale score ranging from 4 to 24). Panel A: Hierarchical ordering of items with respect to endorsement of the latent trait. On average, item 1 provides the highest endorsement of the latent trait, followed by item 2, then item 3, and item 4. Panel B: The set of four items do not constitute a hierarchical scale.

These IRT methods model agreeability with or endorsement of a latent trait based on the responses to a set of items (Bond & Fox, [4]). Given a set of items assumed to measure the same construct (i.e., unidimensional of Hate), IRT models assist in estimating the values of the latent trait and also examining the properties of the individual items (Lord, [13]). Although there are several different types of IRT models, the two types of polytomous IRT models used in this study were graded response model (GRM) (Samejima, [27]) and partial credit model (PCM; Masters, [15]) because they are used for items with ordered or Likert-type responses. Both divide the responses into different blocks, also known as thresholds. These thresholds are used in modeling conditional probabilities of trait endorsement. It is important to check more than one technique to evaluate the model fit., Raju and Colleagues ([22]) provided a full explanation of these methods. The parametric IRT models generate statistics that indicate whether an item fits the model or not. A chi-squared statistic with infit and outfit statistics indicating "bad" items is generated for each item with higher infit and outfit values indicating bad item fit. GRM and PCM analyses were conducted using the R packages ltm (Rizopoulos, [25]) and eRm (Mair & Hatzinger, [14]), respectively.

A total of 540 questionnaires were included in the present analyses. Approximately 1.34% of data were missing, which were assumed to be missing at random. Low missing percent is considered manageable (Acuna & Rodriguez, [1]) and therefore imputed using the chain equations method (Van Buuren, [33]) with one of the best (Stekhoven & Buehlmann, [30]) ensemble technique random forest (Doove, van Buuren, & Dusseldorp, [7]) as the underlying prediction algorithm. The imputation was conducted using the R package mice (Van Buuren & Groothuis-Oudshoorn, [34]). Due to the small percent of individual item data missing, a single imputation was deemed sufficient prior to conducting the main analyses.

Descriptive statistics for items and scales and measures of internal consistency are shown in Table 1. Cronbach's alpha was adequate for all dimensions/subscales (≥0.7). However, six items had corrected correlations ≤0.4, and removal of individual items on two dimensions (Hate and Religious Conflict) decreased internal consistency of the remaining items to levels below the commonly recommended 0.7 threshold. Table 2 shows the top association rules for the item pairs, ranked by lift, with support of at least 50%. These top rules were all found in the Knowledge scale. Exploring the first rule, 54% (support) of the total respondents indicated that they were not knowledgeable about the significance of the Stonewall riot and that they were not familiar with the work of the National Gay and Lesbian Task Force. Next, among respondents who indicated they were not knowledgeable about the significance of the Stonewall riot, 86% (confidence) indicated they were not familiar with the work of National Gay and Lesbian Task Force. Finally, respondents who indicated they were not knowledgeable about the significance of the Stonewall riot were 1.34 (lift) times more likely to indicate they were not familiar with the work of the National Gay and Lesbian Task Force, compared to those who indicated having knowledge of the Stonewall riot.

Table 2. Top five association rules by descending order of lift with support ≥.50 and lift ≥1.10. (N = 540).

Results from the CFA are presented in Table 3. Although all hypothesized path coefficients were significantly different from 0 at the 0.01 (uncorrected) significance level, and only four items had standardized paths ≤0.4 in magnitude, the CFA model did not provide adequate fit to the item data, as indicated by the χ2/df ratio and the other three fit indices. Based on these results, the hypothesized 5-dimension structure of the 28-item set was not supported by the data. Thus the underlying structure of the item data was explored with an EFA, whose results are shown in Table 4. Only three factors had eigenvalues ≥1, and this 3-factor solution explained 93% of the common variance (43% of the total variance), which suggested three underlying latent variables. However, the EFA results suggested that two hypothesized original dimensions, Hate and LGB Civil Rights, lay on the same latent variable (Factor 1), but their scale scores are in opposite directions (lower endorsement of Hate corresponds to higher endorsement of LGB civil rights). As per the CFA results, the estimated correlation between these two hypothesized dimensions is large at −0.77 (see Table 3). The EFA results also suggest some items of the Internalized Affirmativeness dimension lay in Factor 1 as well. As per the CFA results, the estimated correlation between the Internalized Affirmativeness and LGB Civil Rights dimensions is large at 0.74 (see Table 3). These findings further suggest the hypothesized 5-dimension structure for the 28 items is not stable.

Table 3. Confirmatory factor analysis of the original LGB-KASH structure (N = 540).

• 14 Fit statistics:
• 15 χ2/df = 1117.5/340 = 3.29, p < .0001
• 16 CFI = 0.844
• 17 RMSEA (90%CI) = 0.065 (.061,.069); Ho: RMSEA < .05; p < .0001
• 18 SRMR = 0.087
• 19 Note. p < 0.01 for all path coefficients. χ2/df: Goodness of fit χ2 statistic divided by its degrees of freedom; CFI: Comparative Fit Index; RMSEA: Root Mean Square Error of Approximation; SRMR: Standardized Root Mean Square Residual; see Table 1 for item labels.

Table 4. Exploratory factor analysis of the 28 LGB-KASH items (N = 540).

20 Note. h2: communality, proportion of common variance explained by the factor solution. Loadings <0.3 in absolute value not shown. See Table 1 for item labels.

Prior to analysis, the items composing the original Hate and Religious Conflict scales were reverse-coded so that all 28 items were in the same direction (higher scores associated with positive perceptions). The results of the MSA procedure are shown in Table 5. All the items were used in the first step of MCA. The algorithm selected 19 out of the 28 items forming five scales. The results suggest these 19 items were scalable, and each scale, respectively, allowed reasonable ordering of persons by their level of the latent trait based on their total scale scores. The first Mokken scale (scale 1 in Table 5) was composed of nine items (H1, H2, H3, C1, C2, C5, I3, I4, I5) belonging to three (H1, H2, H3) of the original scales. The second Mokken scale was composed of four (K1, K2, K3, K4) out of five items from the original Knowledge scale. The remaining three Mokken scales were composed of two items each, belonging to the original LGB Civil Rights, Religious Conflict, and Internal Affirmativeness, respectively. All but two of the Religious Conflict items were dropped. With the exception of the fifth Mokken scale, all scales had internal consistency measures >0.7. The final step in MSA with hierarchical scales indicates that two additional items (C1 and I4) can also be removed. A more conservative MSA with hierarchical assessment suggests 17 items.

Table 5. Results of Mokken scale analysis.

21 Note. H = overall scalability coefficient; MS = Molenaar Sijtsma; OK = no deviation found; non sig = deviation found but nonsignificant; sig = significant deviation found; removed = item removed to obtain a hierarchical scale; Ranking = order of items in the hierarchy of endorsement to the latent trait; Mean item scores are shown for reference. Items in the original Hate (H) and Religious Conflict (R) scales were reverse-coded prior to the Mokken analysis. See Table 1 for item labels.

The EFA shown in Table 4 was the starting point used to assess dimensionality of the item set. The eight items with negative loadings on Factor 1 were reverse-coded prior to analysis, so all items in this factor were in the same direction (positive perceptions). Next, the two IRT models (PCM and GRM) were fitted individually to the items composing each of the three factors. Table 6 shows the results of the PCM analysis. High values of infit or outfit statistics with goodness-of-fit tests p < 0.001 indicated seven ill-fitting items. The result suggested dropping three items from the original Hate scale, three items from the original Religious Conflict scale, and one item from the original Internalized Affirmative scale. The results of the GRM approach were analogous to those of the PCM approach. The item information curves produced by the GRM analysis for the first factor (16 items) are shown in Figure 4. The items with the lowest contributing information were the same items with worst fit statistics in the PCM analysis. Similar results were obtained for the remaining two factors. Again, these findings further suggest the original hypothesized 5-dimension structure for the 28 items are not stable.

Table 6. Results of the partial credit model analysis.

22 *Ill-fitting items; see Table 1 for item labels.

Graph: Figure 4. Item information curves for the first factor (16 items) generated by the graded response model. Items with higher peak contribute more to the overall aggregate information than items with lower peak. The highest peaked item was: "Hospitals should acknowledge same-sex partners equally to any other next of kin" (C2). Some of the lowest peaked items that did not contribute to the overall information are: "I have difficulty reconciling my religious views with my interest in being accepting of LGB people" (R4), "Hearing about a hate crime against a LGB person would not bother me" (H5), "I sometimes think about being violent toward LGB people" (H4), and "I have conflicting attitudes or beliefs about LGB people" (R7).

The psychometric examination of the 28-item five-factor LGB-KASH instrument presented here resulted in a lack of support of the original instrument structure: the CFA did not result in at least fair fit; the EFA suggested three underlying dimensions instead of five; and the non-parametric and parametric ordinal IRT procedures suggested dropping at least seven items (25%). All these results suggest instability in the originally proposed structure by Worthington et al. ([38]). The selected item sets from the IRT procedures presented in this article could be used to revise the original instrument in subsequent studies to obtain a more psychometrically sound measure.

Comparing the parametric and non-parametric IRT approaches is not straightforward in this case since the parametric analysis began with an assessment of dimensionality based on an EFA, while the non-parametric approach began with a clustering of items. In terms of the original items retained by each procedure, the MSA approach retained 19 items, while the parametric procedures retained 21. The main result difference between the two approaches is that the MSA procedure removed all but two of the seven Religious Conflict items, while the parametric IRT models removed only three of these items. One approach in improving the scale could be to remove or modify items common in both the MSA and IRT approaches.

The data confirm the need for an updated and comprehensive measure of attitudes toward the LGBTQIA+. In addition to paying close attention to overall measure and structure reliability of the instrument, attention should be paid to item content as well. As mentioned above, the quality of the items in the LGB-KASH have been called into question in the scientific literature as well. In a rapidly changing social climate, it is important to ensure any instrument assessing attitudes is general enough to not be outdated or obsolete in a relatively short time period. Items also need to have some degree of generalizability to ensure that respondents from diverse backgrounds and beliefs glean the same understanding from what is being asked. Finally, ensuring that items are simplified, capturing targeted information, as opposed to "loaded" items that may assess attitudes, multiple constructs also affects instrument validity and subsequent utility.

The quality of an instrument is integral to the integrity of findings being reported. The need to assess attitudes as it relates to the LGBTQIA+ population is rapidly expanding. In addition to assessing individual attitudes and beliefs from the U.S. population at large, recent developments in mental health treatment and health care as a whole are in need of metrics for evaluating competency of practitioners. For those interested in studying behavior change with targeted interventions, or making policies, there also needs to be measures that are consistent and reliable across time points.

The present study is not without limitations. A limitation of this instrument is that it does not distinguish between attitudes toward gay men, bisexuals, and lesbians separately; in that sense, all these groups are lumped together. Yet it may be that some people hold differing opinions and attitude toward such individuals. Further, data for the present study were provided by a convenience sample belonging to a particular population (college students) in a particular geographic location (the Deep South), so their answers to the items may differ from those obtained from the original normative adult population. Furthermore, the alternative instrument structures resulting from the IRT analyses presented here are exploratory and in need of validation in separate datasets. That being said, the psychometric analysis framework provided in this study (see Figure 1) can be used as a psychometric analysis template in analyzing and improving the instrument.

This study adds to the literature by providing a comprehensive psychometric evaluation of the LGB-KASH, arguably the current "gold standard" instrument for measuring attitudes toward LGBTQIA+ individuals. Current societal and policy efforts endorse provision of person-centered and culturally competent health care, workplace, and educational environments. Attitudes toward LGBTQIA+ individuals, nonetheless, remain complex and multidimensional in the United States, as evidenced in the current political debate and ongoing reactions to the historic ruling of the Supreme Court of the United States legalizing same-sex marriage. In order to examine the degree of acceptance of diversity versus stigma and prejudice in health care, workplace, and educational environments, a generalized and psychometrically sound instrument is needed to measure attitudes toward LGBTQIA+ individuals.

The results of this article strongly suggest the original LGB-KASH instrument structure is unstable and unreliable; however, the selected item sets from the psychometric procedures presented here could be used to revise the original instrument in subsequent studies and thus obtain a more psychometrically sound measure. Moreover, mixed-method approaches including cognitive interviewing and focus groups might identify additional dimensions for the development of new items to improve assessment of LGBTQIA+ attitudes. Finally, this article provides a step-by-step approach in performing a comprehensive psychometric analysis that may be useful to other research groups in the development or revision of measures.

No potential conflict of interest was reported by the authors.