Are cognitive assessment methods equal? A comparison of think aloud and thought listing

The comparability of data generated by think aloud and thought listing was investigated. High math-anxious students completed two sets of mathematics problems. During one set they thought aloud while problem solving and during the other they listed their cognitions after completing each problem. Assessment method order and problem set order were counterbalanced in a split-plot design. The dependent variables included 11 cognitive content variables, subjective anxiety, number of problems correct, and amount of time for each problem set-assessment phase. Students were significantly more anxious and took longer to list thoughts than to think aloud. Differences on number of problems correct approached significance. On the average, think aloud produced about twice as many thoughts as thought listing and significantly more thoughts in four categories: review of information, strategic calculations, conclusions, and attention control. Thought listing produced significantly more positive problem-solving evaluations and more positive self-evaluations. An analysis on proportions revealed the same differences plus a significantly greater proportion of thoughts in five other categories for thought listing. The implications for cognitive assessment were discussed.