2008 NCTI Technology in the Works Award Winner — Fraction Sense

Schools across the country are paying more attention to students’ achievement in mathematics, not only for AYP scores, but as part of a renewed focus on 21st century skills and STEM education. Fractions have proven a persistent stumbling block for children and teachers alike. This study of Fraction Sense shows that thoughtfully designed technology can create an effective learning approach for children and their teachers. This 2008 NCTI Technology in the Works award shows how collaborative research can overcome the realities of implementation challenges. 

Participants:

Lauri Susi Chief Executive Officer Spotlight for Learning, LLC	John Laskarzewski Chief Financial Officer Spotlight on Learning, LLC	Dr. Dave Edyburn Professor, Department of Exceptional Education University of Wisconsin-Milwaukee

IntelliTools Classroom Suite 4 Serves as a Platform for Math Instruction

While assistive technology (AT) has created amazing new opportunities to elevate the talents and capabilities of youth with disabilities more fully to the surface, Lauri Susi, John Laskarzewski, and Dave Edyburn argue that there has been little effort to build conceptual development in mathematics into AT design. Despite general public awareness of the importance of math and science, they also find that mathematics instruction has received little emphasis in the public schools in the face of greater attention applied to reading instruction, an emphasis they also find reflected in many of the AT systems on the market today.

Lauri states the team’s case for design simply, saying, “We know that the area of fractions, decimals, and percents are very difficult for students, especially students with special needs, and there’s not really anything out there to help them from a software perspective.” Dave adds that these conceptual underpinnings are essential to handling algebraic concepts and operations in student futures.

Responding to this need, John and Lauri of Spotlight on Learning created Fraction Sense, a planned sequence of fraction activities that use on-screen, manipulatives to build concrete understanding of the concepts involved in fraction operations. It was created within IntelliTools Classroom Suite 4 (ICS4) because of the pre-existing templates that make lesson design simple. John adds that ICS4 also has the benefit of “a scripting language allowing for more sophisticated interactions.” More importantly, Lauri continues,

Our background is not necessarily programming, we were really designing from an educational and instructional perspective.  A major reason we use this platform is it already has data collection built in that gives teachers feedback (on student growth). Hopefully teachers will be interested in picking up the information and trying to create from it.

This refers to the ability of end-user teachers to modify presentations and activities, or add to them with new content, in ICS4 for their own individual uses, also without programming experience. This feature offers great flexibility and adaptability for various school district goals and curricula.

Continuing on the design of both technology and their collaborative study to help gauge the efficacy of Fraction Sense, Lauri continues,

We wanted a clear sequence that would model best practice for teachers and give kids really good practice with problem solving with immediate feedback. But we didn’t want to target Special Ed classes because we felt that one of the problems is that students can be identified as having math disabilities when, in fact, they haven’t had good instruction. We wanted to get Fraction Sense in an inclusive classroom environment for all students.

Proving Effectiveness

With Fraction Sense, Spotlight on Learning was building on successful teaching applications they had built in ICS4, and Lauri and John created the activities based on best-practices literature. It is well documented that the use of concrete manipulatives, physical objects that demonstrate mathematical ideas and relations, is far more effective than teaching algorithmic processes alone. It can lead to the ability to apply mathematical processes to novel problem solving based in real understanding, not just the isolated ability to follow an algorithm ‘recipe’ are cued to do so. The inclusion of on-screen manipulatives that could be moved about with input devices made complete sense, especially designed to be tied to direct instruction, and took the process to a higher level than some of the pre-existing tools, as Lauri explains,

There’s a National Library of Virtual Manipulatives, but it’s really more exploratory than instructional, and there’s no feedback for students — unless teachers are actually facilitating any of that development, they probably aren’t really understanding what they’re doing with them.

Nonetheless, school districts that are sorely pressed to meet complex Annual Yearly Progress (AYP) goals need proof that technologies are effective, not just well-designed. The NCTI Technology in the Works Award allowed John and Lauri, with Dave’s professional research design support, to augment their own anecdotal data with a structured study.

The team implemented the program, then, to determine if onscreen manipulative-based instruction would lead to measurable increases in mastery for students with disabilities and other special categories of students, and to allow them to maintain those gains over time.  Ambitiously, they also held the goal of determining whether it would actually allow them to outperform ‘regular’ student peers in math attainment, owing to design tailored to meet the processing and conceptual needs of special students.

Implementation in Public Schools — The Reality

A school district in the northeast served as an important fourth partner. The project involved a true experimental design with control groups using traditional instruction using just the textbook and pencil and paper algorithms. The experimental groups involved training for the teachers, then use of Fraction Sense. Teacher interviews and growth and maintenance assessments were built in, and data would be collected using ICS4 and regular curricular tests. Despite this clean design, and John’s indication that things worked out well in the end and the district was responsive and cooperative, there was tremendous stress as the careful design began to unravel. This was due to a substantial change to the model for math instruction at the experimental schools only a couple weeks before the study began, as John continues,

Instead of having the regular home room teacher rotate through the process of teaching their kids, regrouping for math, what happened was they assigned one teacher to teach all sections of math. So that presented the dilemma of what do we use for the control group because now you had just two teachers, both of which were teaching math, instead of having one school be a control, and the other school having four teachers teaching math. Now we only had one teacher teaching math.

Dave relates that the changes in the school schedule and teaching assignments meant that the research became a ’quasi-experimental’ design. A third school location had to be brought in, and characteristics in the experimental and control groups were substantially changed. The two groups were not equal in size, and the experimental group could not be balanced — it actually had higher initial math attainment than those in the control group, which limited the team’s ability to interpret results.

Lauri counsels that even though schools’ goals may appear to be well aligned with those of a technology development team doing research, there can be subtle differences in emphasis that come into play, or other goals that actually end up demanding more attention. She emphasizes the critical need to work with building principals to smooth the research process as much as possible, but also to ensure that even higher level administration is aware of research projects. Dave adds that

In schools there can be administrative change, upheaval, challenges with boards of education… those are just the realities of doing research in public schools. This school was extremely cooperative but various things affect research quality as people respond to needs just in running their buildings.

Flexible Research Design and Methods

The team’s equanimity in understanding that instructional changes were not intended to cause problems for the project, but were based in exigencies within the district, helped them to recover. Despite substantial challenges, John explains,

It went pretty smooth by comparison to what it could have been. Reflecting back, although we were thrown, with Dave’s guidance we were able to get back on track. With his help, we adjusted and adapted our strategies and moved forward. And we did get it done.

Indeed, Dave’s analysis showed that most students in the experimental group improved. Among the top strata of improving students, 33% had learning disabilities or ADHD. Further, 58% of the students with disabilities had “gain scores in the top 20% of all students,” a very encouraging result given historical challenges, and “three of the top five scores belonged to students with SLD or ADHD.”

Similarly, 44% of the ’students from poverty’ scored within the same top 20% of all scores, illustrating a profound impact for this population, including students of ethnic minority.

Changes in the experiment design did not permit a valid comparison of instructional modes directly, but the overall gain was higher for the treatment group. Dave sums up, indicating, “Certainly we need to replicate the study, but the findings are that this type of software intervention can help all kids, and that kids who are the most troubled can gain the most — that’s huge. There aren’t a lot of things out there that can do that.” John relays that teacher comments showed important results as well, including that:

After looking at our reports one teacher said, ‘You know, I just noticed that one of the girls in the 5th grade was even with her peers, this is the first time that she’s ever accomplished that because she has ADHD. If there were a speck of dust floating in the sunlight that would distract her to the point where she never ever finished any of her work.’  Another commented, ‘The kids have never seen such progress with math facts from September to December.’

Engagement as a Design Principle

Video-taping and observation also demonstrated new levels of quiet, on-task behavior in the labs that did not go unnoticed by other teachers, as Lauri relates,

One of the things that ended up happening is the district is purchasing more Classroom Suite with Fraction Sense, and they’re putting it in all the buildings because so many of the teachers walked by and saw the kids working and wanted it. They actually found money, which was not easy…

Dave puts a fine point on what was occurring, arguing, “Engagement is the issue here — that’s the design principle.” Lauri explains that this was, in fact, the guide for the project in total:

We believe that it’s not the software alone, it’s how it’s implemented. For this project it was pairing it with strong training and implementation. We used a model called SOS in which you structure the environment, optimize student time, and provide students with reinforcement. We trained them to use minimal teacher talk and to have specific routines so the kids knew what to do without getting directions each time.

Math Challenges Among Educators

The team was also courageous in confronting math as a generational challenge. Research has demonstrated many teachers themselves suffer with gaps in their own understanding based on how they were taught and other ongoing societal factors. Sensitively, Laurie shares, correspondingly, that, “Working with teachers did show some limitation in expertise that affected the ability to understand and teach fractions.” Further, as Lauri notes, “Teachers still tend to focus on the procedural stuff, it’s all functional math or memorizing for fluency with facts.” This is often the case, Lauri has seen, even though to their credit most teachers do “want more problem-solving for students.” Encouragingly, one provocative indication of the ability of an AT to directly ‘assist’ teachers as well as students came as an anecdotal finding uncovered in teacher interviews, as John relates,

They came to understand some of the things they were teaching better because they saw the models and were able to explain the models to the students, and therefore, one of the teachers especially, a relatively young one, said that ability to see that on the screen and to explain it to kids was a big jump in her education.

Dave offers the optimistic view that “With the STEM Initiatives† (PDF) (Science, Technology, Engineering, and Mathematics), math is now getting pushed, too,” and the team believes this may lead to more development in math related AT’s.

Competent Researchers in Schools + University Analysis =
Successful Collaboration

John and Lauri’s role encompassed training, logistics of work with the school district, and observation and data collection, while Dave focused on research methodology and analysis. The team’s work model primarily involved E-mail work augmented by a small handful of phone calls and personal meetings at conferences. Dave explains that the relative ease of adapting the project to garner useful information despite substantial changes in the teaching model at the schools was the result of careful planning when developing the initial proposal. Commenting on the value of teamwork, he glowingly describes the interaction with John and Lauri,

From my vantage point, we don’t have people who are in the field enough to do the kind of data collection and are as knowledgeable about instruction as these guys, so it was a great partnership. They were the on-site researchers, and it added more value than if we had sent a graduate student because they understood how schools work. In terms of NCTI, I think this is kind of the model of collaboration they’ve always been promoting. I think it’s still pretty rare, but for people looking for research partners, I would offer these folks only to the highest bidder!

Lauri responds to this humility from the researcher by saying,

John and I had not done a ‘real’ research project, and we were able to ask a lot of questions and work back and forth with different ideas as the design was not going the way we wanted it to. A major part of his role became educating us. The collaboration was just great.

Buying Technology Isn’t Enough to Promote Learning

Noting that much technology ends up on shelves and expanding on Lauri’s earlier comments, the team strongly emphasizes that careful implementation of technology and research results is critical, as reinforced by their own substantial results. They believe that teachers need assistance to accomplish real integration of new tools. Dave elaborates, explaining,

In the literature they’re advocating a position called ‘translators’ who would help translate research and put it in classrooms in a practical way. Then we need staff development people who understand research as well as realities of schools. We need to get more data-based evidence as to whether there’s a need for having people like John and Lauri  in  a district who are not bound to a classroom, and would have resources for staff development, technology tools, and data collection. I’m not convinced that just buying [software authoring tools] is going to make a difference for schools.  It’s about also valuing people who aren’t in front of kids all day too, who can help build these  systems in education.

Based on their success, John and Lauri have added revised units of Fraction Sense in ICS4. Still, the team encourages school districts to spend economic stimulus and other funding on staffing support and training, not just technology — in short, to extend their own collaborative model into the classroom — to create the kind of real change they believe is possible for kids.

†  STEM initiatives are projects that seek to reform and unify education in science, technology, engineering, and mathematics into a rational and naturalistic process. Many states have STEM projects, often connected with state governor’s offices.