Smartxt

Using Universal Design to Enhance Student Learning August 8, 2009

Gianna Durso-Finley, PhD Mercer County Community College

Abstract
Universal Design for Learning (UDL) is a technique for designing course materials and pedagogy that emphasizes accessibility and student engagement. This approach has heretofore been applied primarily to special needs and developmental studies populations. The Smartxt project, based in the California Community College System Chancellor’s Office, has now begun to apply UDL principles to college-ready students. The project involves nearly 200 students in Introductory Sociology classes at three community colleges – Laney College and American River College in California and Mercer County Community College in New Jersey. Participating faculty developed UDL-based learning strategies and used Kurzweil 3000, a text-to-speech software, to embed reading and learning strategies directly into electronic versions of the course texts, creating a “teacher within the text.” This enabled students to hear their text as they read it and to receive learning strategies and guidance directly from their professors wherever and whenever they worked on their course materials. The impact of the Smartxt approach on students was evaluated using a locally-enhanced survey designed to align with national benchmarks from the Community College Survey of Student Engagement (CCSSE). Statistical findings from Fall, 2008 indicate that students using the Smartxt approach perceived themselves to be working significantly more on higher-order thinking skills than the national comparison group. Very preliminary findings from Mercer County Community College alone also indicate that Smartxt project students performed higher on a standard final exam, although the N was insufficient to allow for credible statistical manipulation. The project and the Smartxt Outcomes research continues.
Introduction
Imagine a standard door. Although functional, it is not accessible to everyone. In deference to the Americans with Disabilities Act, let us add a push button that opens the door. This change does certainly enable additional people to use the door, perhaps even some of those in wheelchairs. However the door is still an obstacle to many others, such as someone without the use of her hands. Now imagine that we have added sensors to the doorway, so that the door opens automatically. Many more people can use the door, and can do so without drawing special attention to any one group in particular. This example from architecture illustrates some of the key principles of Universal Design (Burgstahler, 2001, p. 3). The goal is not simply to retro-fit existing facilities with minor modifications that slightly enhance access. Instead, the goal of Universal Design is to create a seamless environment that enhances usage for all groups, while minimizing distinctions (Scott, McGuire, & Shaw, 2003, p. 369).
Following the dictates of section 504 of the Rehabilitation Act of 1973 and the Americans with Disabilities Act of 1990, colleges have worked to address the needs of students with special issues and concerns, including physical differences and alternative learning styles. But the efforts thus far have largely been on the model of adding push buttons to doors, as noted in the example above (Pace & Schwartz, 2008, p. 21). Colleges created departments of special services, and they set up learning laboratories equipped with screen readers and staffed by tutors knowledgeable in the field of special education. These segregated areas send a clear message to students and faculty that these spaces, and these students, are different and apart. Building again on the door analogy, the Universal Design for Learning approach instead encourages faculty to create materials and pedagogy that in effect add sensors to the door so that all can learn with a minimum of distraction and separation.
Universal Design for physical space focuses on nine principle issues: equity, flexibility, simplicity, perception, tolerance, ease, and approachability (Connell, Jones, Mace, Mueller, Mullick, Ostroff, Sanford, Steinfeld, Story & Vanderheiden, 1997). When these principles are extended to the realm of education, they need to be modified only slightly: inclusiveness, physical access, delivery to all, information access, interaction, feedback, and a capacity for the demonstration of knowledge gained (Burgstahler, 2001, p. 5). These can be further distilled into core principles of UDL that create a framework for the development of course materials and pedagogy (Rose & Meyer, 2002, p. 75): to support cognitive, strategic, and affective learning through flexible methods of presentation, expression, and engagement. Although the name and the approach of Universal Design implies impact and implementation throughout the curriculum, the UDL strategy has typically been applied only in special services areas in colleges, continuing to reinforce distinctions rather than creating truly universal learning strategies.
The time to break the silos separating special needs students from other students is at hand, especially in the community college. Several trends are emerging that highlight the need for a truly Universal Design for Learning, The first trend is centered on student diversity. At the League for Innovations in the Community College: Innovations 2009 conference, keynote speakers reminded attendees that there is no such thing as a “typical” community college student. Students in community college sector are diverse along nearly all axes of comparison – age, race, ethnicity, gender, sexuality, religion, and certainly level of preparedness for college work. Sessions at the Innovations 2009 conference that centered on the First Year Experience addressed a method of combining developmental classes with a college-level class in a learning community that enhanced student outcomes.
The second trend is a growing awareness that reaching and teaching the millennial student requires a very different approach from standard pedagogy (McGlynn, 2007). Digital natives require a different type of learning experience, and their digital immigrant faculty are scrambling to find ways to meet students on common ground. Millennials also strive for a more communal, connected learning experience, and group work in classes has only truly scratched the surface in addressing that need (McGlynn, 2007). The problem is that, “Students are different, but a lot of educational material is not (Johnson, Levine, & Smith, 2009, p. 6).”
The third trend reflects technological improvements that enable faculty to create student centered course materials. Community college faculty have long been working to shift from teacher-centered learning to student-centered classrooms that reflect the best practices in undergraduate teaching (Chickering & Gamson, 1987). However, this effort has largely been limited to pedagogy, because the transformation of the actual course materials has taken a very different route. Textbook publishers have been working diligently to create snazzy materials enhanced with video, web links, crossword puzzles and self-tests. Whole course shells now exist for online instructors. The problem with all this is that they exist separately from and sometimes in opposition to the ways in which specific, individual faculty want the students to use the course materials. Students can do all the publisher’s self-tests, but if that is not what is on their own professors’ exams, then the utility is minimal. And students are savvy enough to know what is and is not worth their time (Nathan, 2005, p. 110-113). Technology now exists, however, for faculty themselves to relatively easily customize course materials in ways that are authentic to the student experience in their classes. Adobe, for example, allows faculty to put voice notes in students’ papers, so that students can hear the faculty member’s focus and emphasis on suggestions for improvement. The Kurzweil 3000 text-to-speech software used in the Smartxt project takes this capacity to another level, enabling faculty to put voice-based and text-based suggestions and strategies for students directly into the course materials.
The increasing diversity of the community college student population, combined with the need to reach and teach millennial students effectively, create pressure to transform the curriculum in ways that truly engage the learner. This pressure, combined with technological improvements that allow faculty to tailor course materials, create the capacity for bringing genuine Universal Design for Learning to life. After all, if Universal Design for Learning is truly universal, then it should be beneficial for all students, not just those with special needs. If developmental students benefit from hearing text out loud as they read it, might not other students learn better that way as well? Community colleges are not in the business of weeding out students (Scott, McGuire, & Shaw, 2003, p. 374). Rather, the community college mission is one that undertakes to educate all students.
Developing an authentic Universal Design for Learning, one that enables all students, at all levels of preparedness–with or without diagnosed learning disabilities – is an intrinsic good. It aligns with the community college mission of student access and success. UDL also supports the community college mission of creating a safe space for learning for all types of students, respecting diversity along all axes. Universal Design is, by its very nature, a better approach to teaching. Alongside the ontological reasons for shifting to a UDL pedagogy, the Smartxt project team has argued that UDL can and should be tested across the population. If UDL is indeed effective at reaching all students in an integrated environment (and not just special needs students), then colleges should eventually move toward a seamless approach to teaching all students.
The Smartxt Project
The coordinator of the Smartxt project is Stacey Kayden, a Learning Specialist at Laney College in Oakland, California. Under the auspices of the California Community College Chancellor’s Office, Kayden has been recruiting faculty who teach developmental courses and college-level courses to begin testing the hypothesis that UDL can benefit all students. If UDL is a successful teaching strategy, it could enable the re-integration of developmental and special needs students with college-level students in new and effective ways. Universal Design for Learning could also create an environment in which students with learning disabilities, both diagnosed and undiagnosed, can meet on a level playing field with other students, without calling special attention to themselves.
In 2004, the Smartxt team began working with instructors in developmental courses and in learning laboratories at community colleges in California. Using Kurzweil 3000 software and electronic texts, faculty and instructors created interactive materials to enhance the student experience working with course content. In 2004-05, the implementation was relatively small-scale, but informal student feedback emphasized that the instructor-enhanced audible texts reduced their anxiety about reading course materials. In 2005-06, an expanded implementation, research findings, based on student self-reporting, showed improvement in the following student outcomes: retention, comprehension, and writing skills (Jaw, 2006, p. 3). Pre- and post-surveys with students for the 2006-07 project showed that students using the Smartxt pedagogy had an easier time staying focused and understood better what they needed to learn from the material (Christie, 2007, p. 13).
While these results are extremely promising, they were limited in several key aspects. First, they reflect only the input of students in the special needs categories, which Universal Design for Learning purports to obviate and eventually eliminate. Second, they make use primarily of student self-reported data of within-semester change. For example, students responded to questions about their attitude toward reading textbooks before using the Smartxt approach and afterwards. There is no doubt that student self-reported data is a powerful indicator of affective change, but these data lack a comparison group outside of the Smartxt experience. We would have no way of knowing, for example, if the improvement noted by Smartxt students in understanding what it was that they were supposed to learn from the materials is better than that of students working with a course materials in a traditional way, but benefitting from hindsight at the end of the semester.
In 2008, the Smartxt program coordinators began recruiting faculty for a project that would reach college-level students across multiple institutions. For personnel reasons, the team chose to work initially with Introduction to Sociology. The Sociology Smartxt team through Fall, 2008 consists of three faculty members: Pam Chao at American River College, Inger Stark at Laney College, and this author, thus far the only team member outside California. The team met at Diablo Valley College during the Summer of 2008 in order to develop effective learning strategies, to hammer out some logistical details, and to develop an authentic assessment strategy for the effectiveness of UDL for college-ready students in Introductory Sociology classes.
Several key decisions were made. First, the faculty agreed that it would be acceptable, and perhaps even beneficial for the study, if different professors used different text books. Second, the team agreed that, despite lab access on campus, the digital divide was still an important feature of many community college students’ experience. Therefore the computer-based e-text enhancements could be required activities during class time or in a laboratory, but that overall, student participation in the Smartxt components of the class would be optional, allowing the students to earn extra credit. The team felt that, if the Smartxt enhancements to the course materials were of sufficient utility, then the students would gravitate toward them without being forced to do so. With respect to assessment, the team agreed that measures of student engagement could be evaluated across the colleges involved in the project. But the team concluded that it did not make sense to try to apply a standard final exam for all the students involved, because the student learning outcomes and course goals, as expressed in the course documentation at the three colleges, were not in full alignment.
The Sociology team also addressed several key logistical issues. It was decided that the students would need home access to the e-texts. Therefore, the Kurzweil 3000 software was provided to students on USB drives, rather than the standard learning lab computer-based installation. The team chose Box.net to be the upload/download site for enhanced digital text chapters, rather than a server-based model, so that students could easily retrieve chapters from any computer with internet access. The larger Smartxt team, which had been working with publishers to digitize material for several years, provided logistical and technical support. The two colleges in California worked with Cengage to digitize the text they were using. At Mercer, the Smartxt students used the same text as the rest of the college. Allyn & Bacon already had a digital version of the text, which the sociology editor agreed to open up for insertion points in order to allow the professor to incorporate active learning enhancements for the students.
The Sociology Smartxt Research Design
As noted above, the strength of the research supporting the Smartxt project was the emphasis on affective change for students during their time using the software to enhance their access to text material. However, there were several gaps created by this research approach. One aspect that was lacking in the Smartxt research was the ability to compare students’ affective relation to course material with that of students who had not participated in the Smartxt program. Another aspect lacking the previous Smartxt research was a cognitive dimension, i.e. an indicator of actual student learning experiences and outcomes. The research effort reflected in this paper aims to correct those deficits by drawing comparisons between Smartxt students’ engagement with course material and the levels of engagement experienced by a national comparison group of students who were not using of the Smartxt approach. In order to facilitate the comparison, the author drew on the data provided by The Community College Survey of Student Engagement (CCSSE).
The CCSSE is a nationally-normed and benchmarked survey of student engagement with their institution, coursework, faculty, and other students (www.ccsse.org ). The survey is administered annually to over 500 colleges. Several state systems, including most recently New York, have made the CCSSE their student survey of choice. The CCSSE group (based at UT Austin) have themselves done substantial research linking CCSSE indicators of engagement with other student success indicators, such as retention, persistence, and graduation rates. For the purposes of this study, the most important CCSSE components are engagement with coursework, including time spent with the text, as well as the level of cognitive connection to the course content as measured by questions related to Bloom’s taxonomy. The CCSSE has a companion survey that operates at the level of the individual class section called the Course Feedback Form (CFF). The CFF is a student evaluation of instruction form that contains numerous questions that can be matched directly back to the CCSSE. In this way, individual faculty can compare the level of student engagement in their sections with that of their home institution as well as the national benchmarks that the CCSSE provides. While the CCSSE carries a substantial fee for adopting institutions, the CFF is freeware, as long as credit is given to the CCSSE organization. In addition to the questions that align with the CCSSE, the author added questions about which aspects of the Kurweil 3000 software students used most frequently (see Appendix for sample questions). The three sociology faculty implemented the CFF in their sections in the last two weeks of the Fall semester.
Given the different texts being used at the Smartxt Sociology project team colleges, cognitive student learning outcomes were evaluated only at Mercer County. Using questions drawn directly from the text’s test bank, sociology faculty at Mercer jointly developed a standard assessment measure consisting of 40 questions. In Spring, 2008, students in all sections of Sociology 101 at Mercer took an assessment test of course content. Results from that test showed that the author’s students scored higher on average than students from sections across the college. Therefore, no attempt in the Smartxt evaluation was made to compare the Smartxt students with the general Mercer population of Sociology 101 students. Rather, in Fall 2008, a 30-question version of the same instrument was used to evaluate student learning outcomes in the UDL section compared with sections that did not incorporate the UDL e-text enhancements. All sections were those taught by the author, so as to control for the effect of the professor.
Findings
The Course Feedback Form (CFF) was completed by 180 of the 192 students registered in Smartxt courses across the three colleges. The weighted average responses are presented in the column entitled UDL. The column entitled CCSSE presents the weighted average responses for the over 300,000 students who took the CCSSE nationally in Spring 2008. The left-most column provides the exact text of the questions that appeared on the CCSSE and on the CFF. While some of the questions and the comparisons did not seem to yield much difference between the student experience nationally and the student experience in the Smartxt project classes, there was one area of questions that did present a clear distinction. The findings on the Bloom’s Taxonomy-based questions on higher-order critical thinking skills immediately stood out from the other results. Those marked with an * are statistically significant at the .05 level or better.

CCSSE/CFF Question UDL
N=180 CCSSE
N=342,835
Memorizing facts, ideas, or methods from your courses and reading so that you can repeat them in pretty much the same form 2.81 2.82
Analyzing the basic elements of an idea, experience, or theory 3.25* 2.84
Synthesizing and organizing ideas, information, or experiences in new ways 3.08* 2.71
Making judgments about the value or soundness of information, arguments, or methods 3.01* 2.55
Applying theories or concepts to practical problems or in new situations 3.11* 2.64

The findings in the chart above indicate that students working with the Smartxt approach report that they are focusing significantly more on higher order thinking skills than the comparison group of all students who completed the CCSSE. One possible explanation is that students using the Smartxt approach “process” their textbooks and materials more independently, so that in-class time can be devoted to building on a pre-existing base level of comprehension. It is then important to understand what aspects of the software may be enhancing student work with course materials.
For students across the three colleges, the most often used feature of the software was to access the faculty’s embedded notes, with 30.4% of students reporting that they used this feature Often or Very Often. The next most common choice for students (22.6%) was to use the highlighting feature of the software which allows students to build automatic outlines. The third feature in terms of usage was the ability to hear the text read aloud, with (16.6%) of students reporting high levels of usage. Students also rated access to the dictionary as a valuable feature of the software.
Looking only at the Mercer students on the measures of cognitive student learning outcomes, the N was too low to allow for statistical credibility. However, it is still worth looking at, given the exploratory nature of this first round of research. In the Smartxt section, 14 of the 22 students completed one or more of the optional activities that incorporated UDL principles. The average score for these students was 78.8% on the standard final, with a standard deviation of 3.8. The average score on the final exam for the 48 students in the author’s non-Smartxt sections was 71.3% with a standard deviation of 4.0. The standard deviations are provided for informational purposes only. The number of students involved does not lend itself to deep interpretation of the data, but it seems unlikely that the Smartxt users are a dramatically different, self-selected population.
The in-class pedagogy for the students was identical across sections, other than the fact that the Smartxt section had 4 days in the lab during which they learned and practiced the software. Days that were lab days for the Smartxt class were days that the other sections spent in group-based active learning. It seems likely that, if anything, the Smartxt students had less in-class instruction time than the other sections. As a teaching note here, the author uses personal response systems (aka clickers) in class pedagogy. After each exam, the students in the Smartxt section were asked in an in-class clicker question about how much (if at all) they felt that working with the Smartxt enhancements improved their score on the exam. The 14 users overwhelmingly reported that the Kurzweil 3000 based e-enhancements had helped them with the exams.
Directions for Future Activity
In February, 2009, the Smartxt team presented the research findings to the community colleges in California that have been working on the developmental side of the Universal Design for Learning project. Representatives from the California Community Colleges Chancellor’s Office were greatly impressed by the findings on higher-order thinking skills. The project is expanding in California, despite severe budgetary restrictions facing the state as a whole.
The Sociology Smartxt team continues with the experimental UDL learning design in Spring 2009, welcoming new faculty to the mix. The CFF Survey that aligns with the Community College Survey of Student Engagement will be administered again. It will be very instructive to determine if the Smartxt pedagogy is robust, in the sense that it can be transferred to new team members. Recent research does indicate that even a minimal amount of training in UDL pedagogy can make important differences in helping faculty design activities that benefit all students while avoiding singling out any particular groups (Spooner et al, 2007, p. 108).
In addition, the author will administer the standard final for Spring 2009. With a second group of students working through the UDL activities, there should be sufficient N to begin to make some claims about cognitive student learning outcomes. These findings both in the area of student engagement and in the area of cognition are certainly encouraging. This Spring 2009 round of implementation will provide more clarity in terms of the robustness of the findings from Fall 2008.
There are still some barriers to overcome with the Smartxt approach to UDL. Once the project is “over” and Cambium Learning stops donating the Kurzweil 3000 software in portable USB form, it is likely to be too expensive for students to purchase individually. The way it has been used in the past is that institutions purchase site licenses and distribute the software throughout learning laboratories. The point of UDL, however, is to open the gateways between students not to segregate them in labs. Therefore a more open model will be needed. The Smartxt UDL concept of providing students access to a “teacher-within-the-text” is most certainly not limited to the Kurzweil 3000 software. Other platforms, designs, and text-to-speech tools exist. Working with publishers, faculty and software developers, the goal would be to have a wide range of tools and options available to all students that reflect the pedagogy of the faculty the student actually encounters.
The Sociology Smartxt project, and the assessment of its impact, can move toward demonstrating that UDL is not only intrinsic good but also a crucial enhancement to the student learning experience. And if so, all faculty, but perhaps community college faculty in particular, should be encouraged to incorporate UDL principles into their course pedagogy, through whatever means are available. If the Universal Design for Learning Smartxt project is ever to be truly scalable, it must be easy for faculty to access, adopt, and adapt to their own teaching styles and a reflection of the needs of all of their students.

References:

Burgstahler, S. (2001). Universal Design of Instruction. University of Washington: DO-IT. Retrieved from ERIC March 23, 2009.

Chickering, A. & Gamson, Z. (1987). Seven principles for good practice in undergraduate education. American Association for Higher Education, Washington, DC.

Christie, D. (2007). Evaluation Report: Laney Enhanced Digitized Text Project. Sacramento, CA: California Community Colleges Chancellor’s Office..

Connell, B.R., Jones, M., Mace, R., Mueller, J., Mullick, A., Ostroff, E., Sanford, J., Steinfeld, E., Story, M., & Vanderheiden, G. (1997). The Principles of Universal Design. Raleigh, NC, North Carolina State University, Center for Universal Design.

Jaw, E. (2006). Program Evaluation: Universal Design for Learning (UDL Project) Universal Learning Using Assistive Technology. Sacramento, CA: California Community Colleges Chancellor’s Office

Johnson, L., Levine, A. & Smith, R. (2009) The 2009 Horizon Report. Austin, TX: The New Media Consortium.

McGlynn, A. (2007). Teaching Today’s College Students: Widening the Circle of Success. Madison, WI: Atwood Publishing.

Nathan, R. (2005). My Freshman Year: What a Professor Learned by Becoming a Student. Ithaca, NY: Cornell University Press.

Pace, D. & Schwartz, D. (2008). Accessibility in post secondary education: Application of UDL to college curriculum. US-China Education Review. 5 (12), 20-26.

Rose, D.H. & Meyer, A. (2002). Teaching Every Student in the Digital Age: Universal Design for Learning. Alexandria, VA: Association for Supervision and Curriculum Development.

Scott, S., McGuire, J. & Shaw, S. (2003). Universal Design for Instruction: A New Paradigm for Adult Instruction in Postsecondary Education. Remedial and Special Education. 24 (6), 369-379.

Spooner, F., Baker, J., Harris, A., Ahlgrim-Delzell, L. & Browder, D. (2007). Effects of Training in Universal Design for Learning on Lesson Plan Development. Remedial and Special Education. 28 (2), 108-116.

Appendix

Relevant Questions from the SmarTxt – Student Evaluation – Fall 2008
(adapted from the “Course Feedback Form” c.2006 CCSSE)

Q4 During this current semester, how much has this course emphasized the following?
Memorizing facts, ideas, or methods from your courses and reading so that you can repeat them in pretty much the same form
Analyzing the basic elements of an idea, experience, or theory
Synthesizing and organizing ideas, information, or experiences in new ways
Making judgments about the value or soundness of information, arguments, or methods
Applying theories or concepts to practical problems or in new situations

Q7 Please rate how often you used the following aspects of the Electronic Textbook and the Kurzweil Learning software
Listened to the text read out loud
Used the dictionary for definitions
Read the Instructor’s embedded notes to focus my reading
Used the Highlighting / Outlining
Copied and pasted notes into a Word document
Listened to my own writing read out loud back to me