Is Your Data Equitable?

By Betsy Gardner • NOVEMBER 9, 2020

Screenshot of the data equity tool showing overrepresentation in NYC 311 data

“If equity isn’t intentional from the beginning, we’re not going to end up with equitable outcomes. It won’t happen by accident,” said Alena Stern, senior data scientist at the Urban Institute. “We need to be really intentional about making equity part of the process.” For many cities, this means using data: creating policy based on collected information or utilizing algorithms or scouring datasets to drive decision making. However, as Stern and her colleague Ajjit Narayanan know, most data is far from representative.

Narayanan, a data science analyst, and Stern conducted 39 interviews with leaders in six different cities across the U.S. to learn the challenges innovative cities face in their efforts to embed equity in their use of data and technology; Stern and Narayanan discovered that city officials needed tools to audit how representative their data is, as well as to measure the outcomes of their equity work. They wanted to help cities utilize data in a way that drives equity, instead of deepening inequality, which led them to create the Spatial Equity Data Tool.

“The tool uses data from the American Community Survey (ACS), which is a Census data product that tells us about the demographics and populations of different neighborhoods and cities. So once a dataset is uploaded into the tool, it measures the levels of geographic and demographic representativeness of the data,” explained Narayanan. “And it does that by essentially comparing the user-uploaded data to the ACS data and measuring disparities.” For example, if a city government wanted to install public WiFi outposts, they could upload data on their free WiFi locations, and compare it to the ACS data on households without internet. The tool would produce a map that shows if certain geographies are over- or under-served with the new WiFi spots, and it would show a chart of demographic groups in those areas. Thanks to this comparison, the city could determine if the WiFi hotspots are in fact in the areas that are struggling with low internet connectivity.

Screenshot of the data equity tool showing overrepresentation in NYC 311 dataBoth Narayanan and Stern point out that the tool can tell users about the size and scale of disparities in the data, but that it can’t explain why they exist. If one group is overrepresented in the data, it could be due to data collection issues, historic inequities, or even the objectives of the city’s program. For instance, the city putting in free WiFi hotspots might be targeting tourists, and so is purposefully prioritizing connectivity in the downtown and not in residential areas. In the Spatial Equity Data Tool, users can decide which population they want to use as a baseline to compare their data to. So in this instance, instead of choosing the ACS data that shows households without the internet, the city might choose to compare with a population that is more relevant to their project goals.

This tool was made for anyone, not just government officials, to compare datasets. Stern and Narayanan hope that city planners, GIS experts, community members, nonprofits, and advocates will all use existing open data portals to pull and compare data. Users don’t need a highly technical background to upload and analyze the tool’s outputs, because they didn’t want it to become a tool that is only used by data specialists. They also don’t want it to replace traditional community outreach or engagement. “One thing we say is that this tool is only one part of a larger decision making process that involves lots of local input and community input,” said Narayanan. “This should be used in concert with a whole host of other local expertise.”

Narayanan and Stern hope that this tool will guide decision makers toward more equitable inputs overall; if policymakers notice a gap in their data, they have the opportunity to correct that by directly engaging the underrepresented communities and rethinking their engagement strategies. It also helps reframe current and future data collection practices. “This is the start of standardizing definitions of equity and bias in datasets, to make it easy and quick to compare representativeness across different datasets or across different cities or even across different policy domains,” said Narayanan. “Our hope is that before cities use any kind of data to make a decision, they understand the limitations of the data.” The Institute’s companion guide to creating equitable technology programs also provides recommendations and takeaways for cities that want to understand and address equity and bias in their use of data and technology, all part of their larger Accelerating Innovation for Inclusion project.

“A benefit of using this tool is educating folks to be more critical consumers of data, and to really think critically about ways in which datasets might not be representative,” said Stern. As creators, Narayanan and Stern welcome user stories, questions, and feedback on the tool, either as emails or on their GitHub. Above all else, they truly hope to see “that cities really are creating these formal processes to do this equity analysis and ask questions every time that they’re using data and technology. And that the tool is part of the process.”

To access the Spatial Equity Data Tool, and provide user feedback and questions, visit https://apps.urban.org/features/equity-data-tool/index.html

About the Author

Betsy Gardner

Betsy Gardner is the editor of Data-Smart City Solutions and the producer of the Data-Smart City Pod. Prior to joining the Ash Center, Betsy worked in a variety of roles in higher education, focusing on deconstructing racial and gender inequality through research, writing, and facilitation. She also researched government spending and transparency at the Lincoln Institute of Land Policy. Betsy holds a master’s degree in Urban and Regional Policy from Northeastern University, a bachelor’s degree in Art History from Boston University, and a graduate certificate in Digital Storytelling from the Harvard Extension School.