IQ-Motivation : réfutation de Angela Lee Duckworth

L’étude “Role of test motivation in intelligence testing” (2011) menée par Duckworth et ses collègues est régulièrement citée par les partisans “anti-QI” comme une preuve à l’encontre de la “thèse du QI”. Il y a premièrement un grand malentendu. Comme Duckworth et al. ont noté dans leur conclusion :

It is important not to overstate our conclusions. For all measured outcomes in Study 2, the predictive validity of intelligence remained statistically significant when controlling for the nonintellective traits underlying test motivation. Moreover, the predictive validity of intelligence was significantly stronger than was the predictive validity of test motivation for academic achievement. In addition, both Studies 1 and 2 indicate that test motivation is higher and less variable among participants who are above-average in measured IQ. These findings imply that earning a high IQ score requires high intelligence in addition to high motivation. Lower IQ scores, however, might result from either lower intelligence or lack of motivation. Thus, given closer-to-maximal performance, test motivation poses a less serious threat to the internal validity of studies using higher-IQ samples, such as college undergraduates, a popular convenience sample for social science research (43). Test motivation as a third-variable confound is also less likely when experimenters provide substantial performance-contingent incentives or when test results directly affect test takers (e.g., intelligence tests used for employment or admissions decisions).

Mais où est la relation avec “g” (le facteur d’intelligence générale) ? Il n’y a aucune raison de supposer que les gains de QI vont perdurer après que les participants aux tests aient reçu leur paiement. Dans ce cas, l’étude ne vaut rien. Elle ne nous dit rien sur la malléabilité du QI par le biais des programmes sociaux, qui ont tous échoué à produire des effets durables sur le QI. Nous en arrivons maintenant aux problèmes techniques de l’étude de Duckworth. Le blogueur Statsquatch donne le premier coup :

A bad apple in Duckworth’s IQ-Motivation meta-analysis?

The paper’s meta-analysis of the 46 studies was very good but they neglected to make a “forest plot” that graphs the effect sizes and confidence intervals from each study.  So I did it.  The size of the effect circle is proportional to the inverse of the effect’s standard error, i.e., bigger studies that get more weight have bigger circles.

Two things are apparent: the study outcomes are highly variable, i.e., they are heterogeneous, and there are only three large experiment (2, 3, and 4 in the graph) that showed motivation leading to a large improvement in IQ score and hence are very influential.

The three experiments were run on special ed kids and written up in one paper by Bruening and Zella (1978) while the first author was at the Oakdale Center for Developmental Disabilities.  A few years later Bruening admitted to fabricating data for other experiments on retarded kids at the same center and the case became a text book example of scientific fraud.  Although there were no allegations of fraud on the 1978 paper, I re-ran the meta-analysis with out Bruening’s data and found the the estimate (using the random effects model) was now 0.48 SD and was no longer significant ( p = .07).  This makes me uneasy about accepting Duckworth’s results without further replication.

Sackett et al. “High-Stakes Testing in Higher Education and Employment” (2008) donnent le second coup. Voici ce qu’ils avaient à dire à propos de la précédente étude de Duckworth “Self-Discipline Outdoes IQ in Predicting Academic Performance of Adolescents” (2005) sur l’effet de la motivation et l’auto-discipline sur le rendement scolaire :

“They applied a range restriction correction to one of the six outcome measures (GPA) and reported that whereas the corrected IQ–GPA correlation (.49) was larger than the uncorrected value (.32), it remained lower than the self-discipline–GPA correlation (.67). Although this is true, note that their conclusion (self-discipline accounts for more than twice as much variance as IQ) no longer holds after one takes range restriction into account. In addition, we applied range restriction corrections to other outcomes; in the case of predicting procrastination (as measured by the time homework was begun), for example, IQ had a higher correlation (-.28 corrected, -.18 observed) with procrastination than did self-discipline (-.26) after we corrected for range restriction, which is clearly at odds with the authors’ conclusion.”

Il y a cependant quelque chose que personne ne semble avoir noté. L’un des plus gros problèmes avec la théorie de Duckworth serait de supposer que la motivation est un trait malléable (i.e., modifiable par le biais de l’éducation parentale). L’étude sur le rôle de la motivation/auto-discipline n’a de sens qu’à la seule condition où la motivation serait 1) largement malléable et 2) largement indépendante du QI. Le cas contraire signifierait que les programmes sociaux subventionnés peuvent difficilement stimuler la motivation des enfants.

Et c’est le troisième coup contre l’étude de Duckworth. Si la motivation était fortement malléable et plus important que le QI pour prédire la rendement scolaire, nous devrions nous attendre à ce que le statut socio-économique (SES) soit plus important que le QI pour prédire, par exemple, la probabilité d’obtenir un diplôme universitaire dans la mesure où les parents à fort SES seraient plus aptes à stimuler la motivation de leurs enfants par le moyen d’une meilleure éducation parentale. Pourtant, lorsque nous comparons le rôle indépendant du QI et le rôle indépendant du SES sur le rendement scolaire, nous remarquons que le QI (à SES parental constant) est beaucoup plus important que le SES parental (à QI constant) pour prédire la probabilité d’obtenir un diplôme universitaire et la probabilité de décrochage scolaire.

Of the whites who dropped out never to return, only three-tenths of 1 percent met a realistic definition of the gifted-but-disadvantaged dropout (top quartile of IQ, bottom quartile of socioeconomic background.)
(The Bell Curve, p. 148)

In terms of this figure, a student with very well-placed parents, in the top 2 percent of the socioeconomic scale, had only a 40 percent chance of getting a college degree if he had only average intelligence. A student with parents of only average SES – lower middle class, probably without college degrees themselves – who is himself in the top 2 percent of IQ had more than a 75 percent chance of getting a degree.
(The Bell Curve, p. 153)

L’effet du SES parental est insuffisant pour offrir le moindre support à la théorie de la motivation. Il est utile de rappeler que le QI est un prédicteur important de longévité, d’accident de voiture, d’accident de travail. Le QI (autres facteurs mis à part) prédit aussi un faible taux de criminalité, taux de naissance illégitime, de divorce, etc. La motivation ne peut pas tenir compte de tous ces résultats sociaux (voir Gottfredson sur le facteur g).