Good morning, CharterFolk.
Today we are pleased to share a Contributor Column from Ben Sayeski, the Managing Partner at ESC Matrix.
We provide a bio for Ben below.
Ben is the co-founder of ESC where he manages client engagements and strategic partnerships. He served as a high school economics teacher and elementary school principal before becoming the executive director of the Darden/Curry Partnership at the University of Virginia and Chief Education Officer of the Andre Agassi Foundation for Education. Dr. Sayeski earned his Ph.D. in Administration and Supervision from the University of Virginia, his M.A. in Secondary Education from the Ohio State University, and his B.A. in Political Science from Furman University.
Time for An Education Check-Up
Mark Twain said, “The man who does not read has no advantage over the man who cannot read.” The same is true for data. For any stakeholder in education, the refusal to make use of available data betrays, at best, a naïve misunderstanding of student performance metrics or, at worst, a politically motivated intellectual dishonesty. Just as doctors monitor basic health indicators such as cholesterol, glucose, and iron levels of their patients and ensure their patients have access to and understand those data, education leaders and advocates have an obligation to make available and use key health indices of their educational systems.
Simply put, everyone touched by the education enterprise should be able to answer the fundamental question that surrounds the purpose of schooling: “Are students learning?” These data should be public, accurate, and unobscured. What do we need to know?
- What were the academic outcomes of X school or Y grade?
- How does that performance compare to prior years? That is, what is the trend?
- How do these results compare to district and state outcomes?
- How does X school compare to the peers of my choosing (e.g., by demographics, location, or size)?
Strong administrators and tenacious advocates know the answers to these questions. Yet, their capacity to know that information was hard-won, secured by scanning PDFs or sorting through multiple Excel files. Yes, the structural impediments to knowing the answers to these questions are many. Here are just of few of the largest: (a) incomplete or corrupted datasets from the state that make longitudinal datasets nearly impossible, (b) the capacity and availability of data analysts to aggregate and disaggregate data, (c) visualization tools to make the data comprehensible, and (d) most importantly, the political will to make the answers to these questions publicly available. While challenging, these impediments are not insurmountable; the answers to all of these questions should be at anyone’s fingertips at all times.
The first three impediments have technical and structural fixes. Our recent work in Oklahoma with the Statewide Charter School Board, the Oklahoma Public School Resource Center, and the Missouri Charter Public School Association provides a model demonstration. In partnership with these organizations, we built elegant and intuitive data visualization tools that addressed the technical challenges plagued by many states. The work involved cleaning and reconstructing datasets to ensure an accurate, longitudinal, and integrated platform. However, what makes the work in Oklahoma and Missouri noteworthy was the political will necessary to bring these data to the public.
I have spent most of my career working with large data sets, both publicly available (i.e., aggregated) and controlled (i.e., student-level). There is always a moment of truth when I present school administrators or invested stakeholders, such as foundations and advocacy organizations, with their data. How people respond usually falls into two categories: those who ignore or excuse unfavorable outcomes and those who seize the opportunity for reflection. The former dismisses the value of basic learning outcomes. The latter lean into the data with the mixed emotions akin to those of someone staring at enlightening health metrics. In short, they suspend their ego to get an honest assessment of their performance and figure out what needs to change.
I do not discount the political pressures a board member, superintendent, principal, parent, or association feels to support, defend, and cheerlead for their organization. Affinity for and loyalty to an organization are the backbone of any organization. However, embracing a clear-eyed picture of the current status is not incompatible with championing the home team.
Beyond the power of what data can provide, there lies the question of what is missed when we either do not have access to sound data or fail to make use of the data we have. First, we lose solutions to our biggest challenges. One of the most common points of resistance I hear to making school data publicly available is the problem with identifying underperforming schools. The worry is that calling out underperforming schools will cause a culture problem. Certainly, there are hard-working individuals at these schools who are committed to the students they serve. Yet, when we mask those schools, we also mask schools and solutions that are excelling in some of the most historically challenging areas of education. Like all good big data analyses, comparison data allows us to show where those solutions are hiding in plain sight. Those solutions can provide the tools necessary for hard-working underperformers to experience meaningful change.
Here is an example from Oklahoma. It is not uncommon for a governor, legislator, or state department official to ask questions like: Are all schools with high concentrations of poverty performing similarly? What about high-poverty schools that have high Hispanic populations? If there are differences across schools, what can be learned from the data?
In Oklahoma, the question of high poverty and high Hispanic populations can be answered in a few clicks of the OPSRC visualization. The image (below) shows all elementary schools in the state with student populations of at least 90% economically disadvantaged and at least 75% Hispanic. On the visualization, four schools stand apart from the rest. The yellow lines that trail each point reflect prior years’ performance. These trend lines provide a richer picture of overall performance. The availability and access to timely data can be transformative. Here, the picture is clear: Something good and potentially scalable is happening in these schools.
A second thing we lose when we fail to embrace data is the inability of education stakeholders to do their jobs. Without access to basic instructional health metrics, these individuals are deprived of the information needed to make informed decisions, legislate, govern, manage, instruct, and advocate. In the absence of data, anecdotes and emotion play a supersized role, as seen in statements like “this well-known person loves this program” and “this school has been the heart of this community.” Anecdotes and emotions come home to roost in the charter sector when authorizers must choose which schools to close, open, and scale.
Consider Oklahoma, again. Below is an image of all the charter middle schools in the state. How might this data visualization elevate discussions of closures, expansions, and support?
Finally, a third thing we lose when we fail to use metrics as our guide is the ability to set expectations. Like most things in life, everyone wants to know: “What are my numbers and how do they compare to others?” Knowing if we are above, below, or at the mean helps us set expectations. Whenever I share a dataset with principals, they invariably find their school and compare it to schools with similar student demographics and size, without prompting. In fact, hands down, it is the very first thing that every user of our visualizations does when they first get access to our tools. This is normal, as it should be, and the quickest way to gain perspective. This relative view of performance also sets a standard and target for everyone in the system.
A great example of this comes from New York City. The two images below show the relative performance of African American students served at charter and non-charter public schools in New York City and the state. The first image shows that charter schools outperformed the district by 16% in the 2017–2018 school year. In the second image, that performance increased to 17% in the 2022–2023 school year. Those impressive and sustained results should change the expectations of all educators in New York City. These data should also inform and lead discussions in NYC relative to the reduction, capping, or elimination of charter schools. In light of such differences, imagine taking this schooling option away from Black families in NYC.
The education sector writ large is slowly emerging from a self-imposed COVID-19 hibernation from accountability. It turns out that absolving ourselves from student performance outcomes does not make our obligation to educate students go away. But even as political will begins to shift back to a focus on student achievement, traditional systems are slow to move. The good news is, as a leader in education nimbleness, the charter sector does not need to wait on anyone to accelerate this return. With a little bit of political will, we can all get the results we are looking for on our next education health check-in.