Friday, September 30, 2016

Test Taking Speed: Predictors and Implications

Summary:

This journal article uses a quantitative test taking approach to understanding the key connections between giving students extended time on reading comprehension based time tests. The researchers had 253 students complete measures for processing speed, reading fluency, and self-reporting of their abilities in test taking and reading of a standardized pencil based reading comprehension test.

1. The researchers initial quest was to identify whether students needed extra time on standardized reading comprehensive tests, and whether extended time students were those with learning disabilities like ADHD. The extended time students are often times prescribed by diagnosticians, education professionals, and counselors due to an inability to complete a comprehensive reading test in 20 minutes like the Nelson Denny Reading Test (NDRT). 

2. Beginning the experiment, the researchers collected their observables of 253 undergraduates at one medium-sized public college, and one large private university. The observables included 124 freshmen, 109 sophomores, and 20 other upper classmen, which had a racial distribution of 75% white, 9% Asian, 8% Hispanic, and 8% African American. Of which 24% of the total observables (i.e. students) self reported having a disability diagnosis of ADHD or other psychological disorder at one point in their lives.

3. For the standardized test, the researchers used the NDRT, Form G, as the proxy timed academic test. Though they never timed the observables, but solely told them to complete the test as quickly and accurately possible. The usual testing time for this test is 20 minutes. At the outset of the test they took a “Reading Rate” score, which involved students reading the first minute worth of the first passage, and noting it on the test so it could be listed for the comprehension score. 

4. For conclusive results of the study, it found that the average student took 17.6 minutes to complete the NDRT comprehension measure test. While 22% of the observables failed to complete the NDRT in less than 20 minutes. Of that 22% of observables, 78% had not reported any disability condition before the test. The researchers also found that processing speed was related to reading fluency, but processing speed only had a slight positive relationship to comprehension performance. In reference to the “Reading Rate” score, the researchers found that it did not correlate to the reading comprehension score. Yet, it did not correlate to the time it took to complete the test, with those that read faster completing usually under the 20 minute mark.

In conclusion, the researchers noted that reading comprehension is a need in all tests, classroom environments, and high stake settings. Of note from the study was that more than 20% of the observables failed to complete the NDRT in 20 minutes, but most of them had no disability. Thus proving an over reliance of diagnosticians to base a disability on timed tests solely for student performance. This could be due to other factors such as the answer sheet being separate from a test booklet or an individual’s test taking style influencing the failure in performance. While in reference to accuracy with comprehension it was mostly due to an individual’s knowledge base and intelligence in identifying words and lexicons properly. While for speed it was mostly due to individual personality characteristics. One of the biggest things found by the researchers was that regardless of completion times those that performed under the 20 minutes, and those that went over 20 minutes, both had even scores across the board. Thus allowing the researchers to note that more time on a test doesn’t necessarily produce better results.

Critique:

This study does not really influence forecasting, it more so puts pressure on speed-reading in a time-constrained environment (the test in this case was not timed). In its application to forecasting, it proves that speed-reading is merely a modifier. A modifier that can be useful if a given person has the right speed and accuracy to execute the actions of his duties as noted in the “So Much to Read, So Little Time; How Do We Read, and Can Speed Reading Help” post on the Advanced Analytic Techniques blog by Roland Blatnik. Also the study shows that diagnosis of disabilities cannot be solely based on a test, otherwise it sets a false forecasting for one’s own individual future due to the stigma it can bring in society. The authors also rightfully state the limitations of the study of the test not being officially timed; not accounting for variables that may influence a student’s performance (i.e. personality and test-taking traits); or involving any true variable manipulation. All of which can be accounted for in a later test to increase the robustness of the already established study. Personally, I think if they took this timed test and changed it into a time based scenario it would prove interesting, particularly in a business model. A model where large passages of information are provided and a decision must be reached within the timed limit that weighs both profits and consequences to test reading comprehension with decision-making. This could then ultimately be used to assess capabilities of forecasting in this modern information age we live in.

Sources

Test-Taking Speed: Predictors and Implications

So Much to Read, So Little Time; How Do We Read, and Can Speed Reading Help

Lovett, B. J., Lewandowski, L. J., & Potts, H. E. (2016). Test-Taking Speed Predictors      and Implications. Journal of Psychoeducational Assessment,             0734282916639462.             <http://jpa.sagepub.com/content/early/2016/03/18/0734282916639462.full.pd     f>.

Rayner, K., Schotter, E. R., Masson, M. E., Potter, M. C., & Treiman, R. (2016). So Much           to Read, So Little Time How Do We Read, and Can Speed Reading Help?. Psychological Science in the Public Interest, 17(1), 4-34.                                                    <http://psi.sagepub.com/content/17/1/4.full.pdf+html>.