Education leaders have a critical role in developing confident, curious, and consequential data cultures in the age of artificial intelligence (AI). That role depends as much on the culture part of the equation as the data.
Before any bot employed pre-trained transformers to generate human-ish responses to prompts, before we could send avatars instead of ourselves to boring meetings, or have an AI assistant take notes in those boring meetings, there was data. Data fuels AI.
Transparency is an important principle for shaping AI policies. Yet pursuing transparency is only half helpful without knowing how to interrogate the underlying data and safety implications revealed by that transparency.
Gone are the days of siloed data analytics teams as AI mandates a new level of hyper-interdisciplinarity for education leaders. Leapfrogging foundational needs, like leadership data fluency, invites a post-AI-rush repeat of the post-pandemic K-shaped recoveries.[i]
AI and machine learning are ushering in a new data-fueled paradigm in computational thinking (CT).[ii] Without foundational data cultures, educational leaders will struggle to keep pace with the speed of AI development ill-equipped to inform and protect students from AI harms.
While we focus on student data literacy[iii], let’s also reconsider data fluency for leaders.
3-C Data Cultures: Confident, Curious, and Consequential
In a “3-C Data Culture”, confidence and curiosity show up in leadership data conversations, and consequential outcomes result from those conversations. One way 3-C Data Cultures grow is when top leaders let people see and hear them asking questions about data.
There are three types of data that matter in nearly every context, and each has an associated practical query strategy that leaders can apply to help grow 3-C Data Cultures.
Three Leadership Data Types and Questions: Readiness, Confidence, and Action
We will consider snapshots of three data types, and a core query strategy for each, within an AI context, though these three types of data show up across all contexts and the associated question types should also be asked across all types of data.
Readiness Data: “Is it real?”
Readiness data helps leaders decide if something is ready for the next phase in its lifecycle. Is the new AI platform ready to go live and scale? Is a student population’s AI knowledge ready for a new AI application? Is our curriculum ready to meet our students’ AI learning needs? The core query strategy here is to ask data questions concerned with basic understanding of qualitative and quantitative data, to discover if the starting point of the readiness data is understandable in real terms.
This can start with basic questions, like getting comfortable asking about the “n.”[iv] How can you assess or understand data if you do not understand the sample size implications of what you are looking at?
Confidence Data: “Is it righteous?”
Confidence data helps leaders decide if a recommendation is trustworthy in their context. Beyond basic sample size questions, we need to understand the source and proximity of our data to the issue we are trying to address. Is the consultant’s recommendation about a new AI tool based on an appropriate, inclusive, and reliable population? Were there any restrictions on who and how data was collected? The core query strategy here is to ask data questions concerned with belief in the data’s level of objectivity, and consideration of any subjectivity, or bias, in the data.
This can start with questions about what (if any) data was excluded from the recommendation, or if the data has been normalized[v]. If you have ever wondered why you are shown data that is “all good,” yet your lived experience as a leader of the issue at hand is that it is not going well, your data may not be righteous.
Action Data: “Is it responsive?”
Action data helps leaders connect a recommendation to an issue they need to solve. Understanding and believing the data is good, but being able to act on the data is better.
How can we act on this data to address an AI equity issue? Does the data illuminate specific steps we can take? The core query strategy here is to ask data questions concerned with taking focused, data-informed action on issues that are important to your students and your setting.
This can start with questions about alignment with current practical needs or longer-term strategic goals. The age of AI compounds the challenge of focus that is ever-present for education leaders. Building a 3-C Data Culture through questioning offers the focus-sharpening option to insist that any data you are presented with meets leadership alignment goals.
Don’t Always Believe Your Own Eyes
A final note on questioning your way to a 3-C Data Culture: As time-to-decision decreases, the lure of dashboard simplicity increases. It is important that leaders take the time to understand and personally engage with the red, yellow, and green on any data dashboards they encounter. Not all green indicators on dashboards are created equal. This can get tricky because, frankly, data visualizations are seductive. Leaders must insist upon clear, concise, metrics that they understand, believe in, and can act on.
The real, righteous, and responsive framework is simple, but not always easy. Asking straightforward knowledge building questions can be an act of courage and requires vulnerability, acknowledgement that leaders don’t have all the answers, and a willingness to take a learning curve in stride. In the age of AI, asking questions across interdisciplinary teams can be a powerful way to create the conditions in which leadership data fluency can emerge to make data understanding, belief, and action the norm in your organization.
Author: Mary Lang, CoSN Driving K-12 Innovation Advisory Board Member
Published on: November 19, 2024
CoSN is vendor neutral and does not endorse products or services. Any mention of a specific solution is for contextual purposes.
[i] In a K-shaped recovery, some parts of society experience strong growth while others continue to decline. What we saw in the pandemic was that longstanding inequitable access issues went largely unaddressed. Schools with strong pre-pandemic technology infrastructures and resources, recovered at a faster and steadier pace, while schools already struggling when the pandemic struck, recovered at a slower or declining rate. This type of recovery is called “K-shaped” because when charted together, different segments may diverge, resembling the two arms of the letter “K.” Schools with strong infrastructures recovered on the “up” arm trajectory of the K, while those with weak infrastructures followed the K’s “down” arm trajectory.
[ii] Dr. Matti Tedre, of Finland’s Generation AI Project calls this a shift from “CT 1.0” (rule-driven), to “CT 2.0” (data-driven).
[iii] Data Science 4 Everyone is one example of the focus on advancing data science education for K-12 students.
[iv] In statistics, ‘n’ refers to the sample size, which is the number of observations or data points collected in a study. Sample size affects the reliability and validity of results, and the generalizability of the findings to a larger population. It is good to keep in mind that in general, when you are asking about “the n” it is understood to mean the sample size of what is being studied , while “N” (capitalized) indicates the total population size being considered. So, n = sample size and N = population size, and if you have a subgroup sample size, it is indexed as n_i for subgroup i.
[v] Splunk explains: “In simple terms, data normalization is the practice of organizing data entries to ensure they appear similar across all fields and records, making information easier to find, group and analyze. There are many data normalization techniques and rules.” Read their article here: Data & Database Normalization Explained: How To Normalize Data.
About the author
Mary Lang serves as the Chief Education Justice Officer at the Center for Leadership, Equity and Research (CLEAR) is a Researcher at the Center for Artificial Intelligence and Digital Policy a data equity Co-facilitator with DataReframed and a member of CoSN’s Driving K-12 Innovation Advisory Board. A member of the inaugural 2024 cohort of the EdSAFE AI Alliance Women in AI Fellows, Mary holds a Master’s degree from USC’s Annenberg School, is a graduate of the UCLA School of Education and Information Studies Women’s School Leadership Academy, and has certifications from the Institute of the Future as a Strategic Foresight Practitioner, Stanford’s d.school in Designing for Social Systems, and from the Dataassist We All Count Project in the Data Equity Framework. Mary has taught at Stanford University’s graduate school of business, in the organizational behavior discipline.
