Causes of group differences studied with the method of correlated vectors: A psychometric meta-analysis of Spearman’s hypothesis
Joep Dragt (master thesis, 2010).
Study 1: Effect of Language Bias in Subtests
When comparing test scores of people who lack a desirable level of proficiency in the target language and bilinguals (i.e., most immigrants) against the test scores of native speakers, a distinction is usually made between verbal and nonverbal tests. Subtests with a substantial verbal component measure to an undesirable extent proficiency in the language of the test taken and underestimate the level of g of the tested nonnative speakers (see te Nijenhuis & van der Flier [1999] for a review of Dutch studies). The more limited the language skills, the larger the underestimate. Language bias plays a clear role in the testing of immigrants in Europe, but also in the testing of Blacks in South Africa, where the English used in the test, it is sometimes the second or even third language of the Black test taker.
In a study of Dutch immigrants, using a mixture of culture-loaded and culture-reduced tests te Nijenhuis and van der Flier (2003) found that the highly verbal subtest Vocabulary of the GATB is so strongly biased that it depresses the score on Vocabulary by 0.92 SD, leading to an underestimate of g based on GATB IQ, with as much as 1.8 IQ points due to this single biased subtest alone, whereas the other 7 subtests combined show only very little bias. However, one should not forget that subtests with a strong verbal component usually constitute only a small part of a test battery; due to the use of sum scores the strong bias in tests with a verbal component becomes diluted.
Looking at the effect of length of residence in the Netherlands on the scores on various intelligence tests also shows the influence of language. Tests without a verbal component show small to negligible correlations with length of residence, tests with a verbal component show moderate correlations, while language proficiency tests show large correlations (see te Nijenhuis & van der Flier, 1999; see van den Berg, 2001, p. 37). All these findings regarding the clear but modest role of language bias are in line with the findings of language bias when testing Hispanics who do not have a desirable level of proficiency in the target language or who are bilingual (Lopez, 1997; Pennock-Román, 1992).
In the US studies on Spearman’s hypothesis it is usually native-born Blacks and Whites who are compared. Therefore language bias is not the problem that it is in the study of immigrants, and Blacks and Whites in South Africa. However, studies of Hispanic immigrants may show language bias. In order to combine the diverse studies for a meta-analysis the effects of language bias had to be taken into account. We did this by leaving out subtests with a substantial language component for immigrants in Europe; Blacks in South Africa; and Mexican immigrants in the US for some studies. When there were still at least seven subtests left we recomputed the correlation between d and g and included that data point in the meta-analyses. Therefore Table 5 of Study 3 in some cases shows two correlations between d and g: one for all subtests and another after excluding one or more subtests with a substantial language component.
Continue reading →