Abstract |
For the most part, the experiment could not discern an impact as a result of providing the equipment and training for TI-Navigator. As shown in the figure below, we found a modest effect for Geometry achievement using the NWEA End of Course Geometry test. This figure shows the outcome measure in standardized units. However, this impact was not reflected in CST Geometry scores. In Algebra, while we found no overall difference, there was some evidence of a small negative impact for students scoring “below basic” on the CST and, holding pretest score constant, for English proficient students. The results of the NWEA End of Course Algebra I test did not reflect those same results. Implementation must be considered in interpreting these findings. Our surveys and observations make clear that this implementation was not a fair test of the difference TI-Navigator might make if used more extensively. Of the 19 teachers originally assigned to the treatment group, about half did not use the system at all for instruction. Of the remaining nine teachers, only three could be considered “Comprehensive-Implementers.” Of those three, only one used TI-Navigator daily. Technical glitches deterred many from using the system after previous failed attempts. Overall use may have been constrained by the fact that California prohibits calculator use on state tests. Impact on NWEA Geometry Achievement Our results also must be qualified by the fact that, while finding differences on one test, we did not find differences on the other test. The significant amount of attrition, both at the teacher and student levels, although not believed to be associated with the program being tested, raises issues about generalizability. For example, it is clear that in both experimental conditions, lower scoring students were significantly more likely to not have posttests, indicating that our findings are not applicable to the lowest scoring students in these districts. Overall, we found that the TI-Navigator affected the average number of minutes the technology was used. The teachers with TI-Navigator reported using the technology about 15 minutes more per week per class period than teachers without. Future exploratory analyses may prove useful in suggesting whether extent of usage can account for student outcomes. In particular, since TINavigator resulted in greater technology use, examining the correlation between technology use and achievement may suggest a mechanism by which TI- Navigator could be effective. Future studies of TI-Navigator will benefit from greater support for implementation. We also recommend continuing to include Geometry in the topics to which TI-Navigator is applied, since the positive result found in this experiment should be replicated.
|