Lisa Bennett and Cathy Yun were writing a grant proposal when it hit them: they couldn't answer their most important questions with the data they had. Bennett, the coordinator of the multiple subject teacher credential program at Fresno State's Kremen School of Education, and Yun, the coordinator of the school's early childhood education teacher credential program, wanted to understand how their candidates were progressing towards developing skills and knowledge for teaching during their time in the Fresno teacher preparation programs.
With a goal of understanding the candidate experience holistically, Bennett and Yun wanted to bring together data including coaching feedback, student reflections, dispositions surveys, course based assessments, performance assessment scores and advising sessions. However, like many schools of education across the CSU, the data at Fresno State were stored in a hodgepodge of disconnected systems including paper files, homegrown databases and spreadsheets, and vendor software. Locating and gaining access to the data was challenging, and data from the same teacher candidates couldn't be linked across the different systems. Some important data points weren't tracked or stored systematically at all.
Faced with the need to get a comprehensive perspective to support program improvement, Bennett and Yun knew the existing mishmash wouldn't work. Bennett recalls wondering, "How on earth are we going to even access let alone analyze all of the pieces of paper that are all over the place?" They also saw that colleagues with different needs—such as grant reporting and accreditation—were running into similar data challenges. Seeing an opportunity to overhaul the data system in a way that could provide broad benefits across the school of education, Bennett and Yun began investigating possible software solutions, with a goal of having a system that would allow supervisors, coordinators, and faculty to be able to track candidate progress through the program and in their development of key teaching skills.
Choosing a path forward together
For a major shift in data systems to be successful, Bennett and Yun felt it was important that their colleagues in the Kremen School of Education have a voice in the process. They identified three software vendors who offered a candidate data management product, and invited each vendor to provide an online demonstration of their respective products and their capabilities. After meeting with each of the three potential vendors and examining the different strengths of each product, Bennett and Yun invited Watermark's Tk20 to Fresno State to provide an in-person demonstration and question/answer session with all the faculty and staff.
They also used these gatherings as an opportunity to air out and articulate the shortcomings of the current system that faculty hoped a new system would resolve. Getting a better sense of their colleagues' concerns and needs helped Bennett and Yun build enthusiasm for how a new system could support their shared work, such as common course-based assessments.
"We really got our faculty to understand what we were trying to achieve and how this would replace a system that … was too complicated and no one knew how to use," Bennett says.
Bennett and Yun credit Dean Paul Beare and Associate Dean Laura Alamillo for trusting and supporting them to do the research on options for candidate data management systems. In the end, the decision to adopt Tk20, one of the three platforms considered, was unanimous.
From decision to implementation
Getting a comprehensive new data system into place requires strategic planning, coordination, and technical expertise. Bennett and Yun asked Kremen School of Education administrator Andrea Perez to oversee the process. Perez was enthusiastic about joining the effort. "When I was approached with the idea of implementing Tk20 as a data management system to better drive our program as a whole, it was right on track with the current needs of our coordinators, teacher candidates, and university coaches … I couldn't wait to get started," she says.
With the help of Tk20 implementation project manager Bianca Salinas and critical cooperation from Fresno State's information technology team, Perez designed a phased implementation plan based on the needs and priorities of each program. Different cohorts and programs were assigned to be integrated into the system in different semesters, beginning with the single subject credential program in the fall of 2017.
"Andrea's deep knowledge of the program's needs and existing assessment practices … made for a highly successful implementation," says Salinas. "[She] routinely looks for ways to make existing processes more efficient and thinks about how to eliminate any unnecessary work from students or evaluators."
The plan also addressed how each participant would be trained and guided through the process. Bennett and Yun acknowledge that the implementation would not have been possible without Perez's dedication. "She has worked tirelessly with such great energy to support all of our candidates, all of our faculty, our coaches," Bennett says. "She puts her contact information on the platform so people can contact her in real-time for support."
Salinas emphasizes the importance of the support as a key component of a successful implementation. "Fresno created customized guides to provide users so they could quickly and easily adopt the tool and a have a central place to answer frequently asked questions. This helped onboard users quickly and minimize issues that could stall a user's engagement in the platform," says Salinas.
She adds, "Another creative technique Fresno used was training students on how to assist their mentor teachers to navigate Tk20. It's common for students to be technically savvy so Fresno leveraged this and [called on students] to spread knowledge and help mentor teachers troubleshoot issues. This technique was not only efficient but also a great way of fostering collaboration between the student and their mentor teacher."
Early volunteers among the Kremen School of Education faculty also helped demystify the process for their colleagues. Perez, Bennett, and Yun all give credit to Emy Lopez Phillips, the coordinator of single subject credentials, for being willing to pilot Tk20 in the first semester of implementation and serve as a showcase of the process for other programs. They also credit Jeanie Behrend, the coordinator of Fresno State's teaching performance assessment, for willingly jumping into the deep end, moving to the new system while simultaneously revising the assessment itself.
Early evidence of positive impacts
While the implementation at Fresno State is still underway, faculty and staff have begun to experience some of the benefits of the work, particularly in tracking the development of candidates through their fieldwork experiences. "It's already been a critical tool for helping us support our candidates," Yun says. "Our coaches are able to provide more immediate feedback to our students," Perez agrees.
There's more transparency for teacher candidates as well. "Students have access to their information, observations, and scores over the span of the program," says Perez. Bennett and Yun say they've seen an increase in students using the data to advocate for themselves, to say 'hey, I need a reflection with my coach,' or to bring data from their assessments to the table when talking about their personal development with their coaches and mentors.
These early successes are also increasing the appetite for reliable, relevant data in the Kremen School of Education. At a recent faculty inquiry meeting, faculty members were eager to discuss other data and features that could be added to the system, and energized to contribute to the work.
One expansion of the original implementation plan is already in progress. In spring 2018 the team began to pilot of an offline, app-based version of Tk20 using support from the S. D. Bechtel, Jr. Foundation. The hope is that the offline app will make it easier for university coaches to record observation data for their candidates in the field, so they can spend less time re-entering notes and more time giving high-quality feedback.
Advancing data-led program improvement across the CSU system
The data system work at Fresno State is well-timed with other initiatives taking place in the CSU system. Inspired in part by Fresno's successes, the CSU Educator Quality Center (EdQ Center) recently awarded grants to two additional CSU campuses to pilot Tk20 with funding from the S. D. Bechtel, Jr. Foundation. The Fresno State team will work with the implementation teams from these campuses in an advisory capacity to share the lessons learned from their experience.
In addition, the Fresno State team is helping to refine Tk20 itself. "Through the implementation process Fresno has validated how our tools meet their needs but we have also received valuable feedback on ways to improve our product so we can continue to meet their needs and similar needs of other institutions as well," Salinas says.
Fresno State's vision for a data system to inform improvement in their programs is also aligned with the EdQ Center's development of the EdQ DataView. Both initiatives are part of a CSU-wide shift in how our community uses data to understand and inform efforts to strengthen educator preparation.
"[We] look at this tool as a way to triangulate data and help us understand our program … and understand where we can continue to grow," says Bennett.
Austen Adair, Vice President of Sales at Watermark, looks at Fresno's data work as an exemplar for other educator preparation institutions. "A huge factor in the successful advancement of data usage is centered around clear and concise vision and goal setting. Cathy and Lisa have done a fabulous job of creating a central mission of collecting reliable and valid data to generate useful insights for individual practitioners and for the unit as a whole," says Adair.
"Cathy and Lisa are truly an inspiration," Adair continues. "I began working in this field to improve data literacy in higher education to ultimately improve an institution's ability to produce quality candidates. For me, Cathy and Lisa embody this spirit … this core value gets them out of bed each day. It pushes them to facilitate difficult conversations to create a culture of improvement. Their sense of purpose is crystal clear, and for me, that is the quintessential mark of a true educator."
Acknowledgements
It takes a community to make large-scale change possible. In addition the individuals mentioned above, the Fresno State Tk20 implementation team wants to thank all of their colleagues who contributed to this work. Special thanks go to Colleen Torgerson, Brenna Barks, Orlando Leon, Robert Guinn, Pawn Sayphengsy, Dolores Charest, Renee Flores, Heather Horsley, Suzanne Brandl, Kara Allen, Christina Macias, Brooke Berrios, Selena Van Horn, all of the Kremen School of Education university coaches, all graduates of Fresno State's Master's Degree in Early Childhood Education, Jon Dolle, Macy Parker, Jana Luft, and the staff of the CSU Educator Quality Center.