This report was supported in part by a grant from the John D. and Catherine T. MacArthur Foundation.

Executive Summary

• Though data is essential to understanding public safety, police data is rarely reliable.

• Surveillance technology distorts crime statistics, giving the illusion that crime is concentrated in predominantly BIPOC and low-income neighborhoods that are already over-policed.

• Independent audits and data verification can help produce a more accurate picture of what crime looks like and where it happens.

I.              Introduction

Nearly every day in America, journalists report on crime rates and trends, treating the numbers we are given by police as an objective reflection of the world around us. But the alarming reality is that police don’t merely report crime data; they shape it through countless policies, decisions, and (increasingly) mass surveillance. The skewed data that regulators, politicians, and the public turn to shape policing policy often reflect departments’ decisions, not the reality of crime in our communities. And newer surveillance technologies are making a bad situation worse, gathering ever-more skewed data and giving officers ever more power to shape the public policy narrative. Left unaddressed, these practices risk creating a skewed data feedback loop, where artificially inflated crime rates in low-income BIPOC communities justify ever more surveillance, further widening the crime rate and policing gap compared to less policed communities. This report focuses on data skewing abuses in New York City to help elucidate trends that impact countless of the 18,000+ law enforcement agencies across the country. From big city police departments to rural sheriffs, law enforcement’s unchecked control of public safety statistics is giving police the power to police themselves. We detail which police data practices are most unreliable, while highlighting safeguards that every locality can implement to regain control over public safety data.

II.            The Problem with Crime Statistics

How safe are our cities? We all want to know: residents navigating our neighborhoods; policymakers crafting public safety programs; voters holding them accountable. We need the facts to understand our neighborhoods’ wellbeing now and over time, and to achieve consensus on what works to make them safe.

But the “facts” we typically get—crime statistics—don’t capture reality. Part of the problem is unavoidable. Year after year, the National Crime Victimization Survey shows that Americans experience crime at a higher rates than they report it to police.[1] Unfortunately, we can’t extrapolate from reported crime rates to true crime rates: underreporting varies from year to year, from crime to crime, according to cities’ features (such as segregation), and with victims’ and perpetrators’ personal characteristics (such as race and socioeconomic status, which affect confidence in law enforcement).[2]

Beyond this unavoidable slippage between true and reported crime rates, police data also fails to track reality due to selective policing of particular offenses and particular communities. Consider, for example, cannabis-related policing in NYC. Through 2011, marijuana-related criminal court summonses hovered below 10,000 per year before increasing to over 13,000/year in 2013, 17,000/year in 2015, and 21,000/year in 2017.[3] Summonses plummeted in the ensuing years before dropping to zero when NYS decriminalized marijuana in 2021.[4] Throughout, the NYPD issued a hugely disproportionate rate of summonses to residents of BIPOC communities—so much so that the state now gives residents of those communities priority to run licensed cannabis dispensaries.[5]

Did marijuana use boom in the 2010’s and drop dramatically after 2017? Does marijuana use vary along race lines? The evidence doesn’t support these differences in marijuana use. It shows that police enforcement of petty offenses increased, peaked in 2017, and decreased.[6] It also shows that Black and white New Yorkers don’t have significantly different rates of cannabis use—a fact confirmed by the wide geographic distribution of NYC’s 2,900 cannabis shops.[7]

NYC’s pattern of marijuana-related summonses neatly demonstrates how selective policing drives crime statistics up and down, and concentrates offenses in particular communities, independent of underlying rates of activity. So do criminal summonses for public drinking, public urination, and other ticketable offenses. Between January 2022 to June 2023, summonses for these offenses went up twenty times in NYC, with nine of ten summonses issued to BIPOC New Yorkers.[8] Were BIPOC communities inundated with free beer and coffee, increasing offense rates? Of course not: under Mayor Eric Adams, the NYPD prioritized issuing criminal summonses for activity that officers previously treated as civil offenses.[9]

Selective policing also skews data for serious crimes. Between 2004 and 2012, the NYPD stopped 4.4 million mostly BIPOC New Yorkers as they went about their daily lives.[10] The vast majority were completely innocent (and in 2013, a federal judge ruled that the NYPD’s stop-and-frisk practice violated their rights against unreasonable searches).[11] On the rare occasion that officers did turn up weapons, they were 50% more likely to recover them from white New Yorkers than from Black or Latinx New Yorkers.[12] But that’s obscured in the crime statistics, because officers stopped eight BIPOC New Yorkers for every white person, greatly inflating offenses attributed to BIPOC people.[13]

III.          More of the Same: Predictive Policing

The court that rejected the NYPD’s stop-and-frisk practice stated the obvious: officers who target Black and Latinx youth based on their race and age perpetuate racist policing. But what about a patrol-planning computer algorithm that directs officers to police BIPOC neighborhoods over and over again? “Predictive policing” tools claim to use historic crime data to predict where crimes will occur so that police departments can allocate their resources effectively.[14] These algorithms ingest historic police data—in NYC, as in many other cities, a record of racist and corrupt police practices—and dispatch officers to neighborhoods where over-policing has inflated recorded offenses.[15] That provides an objective-seeming, supposedly data-driven cover for the same old, same old patrol patterns. In NYC, the evidence is in the NYPD’s stop-and-frisk data, a reasonable proxy for officers’ activities while on patrol. According to the NYPD’s court-appointed monitor, the NYPD has reduced racial disparities in reported stops since 2013, but unreported stops are increasing—as is the monitor’s concern that the NYPD is disproportionately focused on BIPOC New Yorkers.[16]

Some predictive policing tools claim to use historic crime data to predict who will commit crimes, rather than where they will occur. Patternizr, the NYPD’s in-house predictive policing software, uses historic crime data to identify patterns of crime committed by the same person or people.[17] The NYPD bills Patternizr as an algorithm “without bias” because it doesn’t explicitly include race when it profiles suspects.[18] What the department fails to account for is that race isn’t a checkbox that can be unchecked—it’s engrained and embedded into the NYPD’s data, which is a faithful record of historically racist policing tactics like stop-and-frisk. When Patternizr generates profiles and searches for individuals who fit them, the algorithm perpetuates whatever bias is baked into the data—even if that means identifying the same individuals who fit profiles built on racially biased data over and over.[19] Individuals who may or may not have any connection to a set of incidents have the potential to be labelled as repeat offenders just because Patternizr thinks they fit a profile—and due to automation bias, officers may be inclined to believe it.

IV.          ShotSpotter Skew

A Geographically and Racially Skewed Map of Gun Violence

When ShotSpotter compared its gunshot alerts to 911 calls, it found that 80% of the events associated with ShotSpotter alerts aren’t called in to police.[20] All other things equal, a neighborhood without ShotSpotter might call in 20 gunshots per month, whereas a neighborhood with ShotSpotter would get five times as many ShotSpotter alerts. Add the fact that ShotSpotter is mostly installed in BIPOC neighborhoods, and it’s a formula for racially skewed data on gun violence. ShotSpotter detectors inflate and concentrate a city’s awareness of gunshots in BIPOC communities while missing gunshots entirely in neighborhoods where they’re absent.[21]

Cities that report both shootings and ShotSpotter alerts document the ShotSpotter effect. Louisville, KY’s ShotSpotter installation targets its majority-Black West End neighborhood and Newburg, its largest Latinx community (map A below).[22] As expected, the city’s history of ShotSpotter alerts (map B) corresponds exactly to its placement of ShotSpotter sensors. But actual, confirmed gun violence in Louisville is far more equally distributed (map C). ShotSpotter doesn’t see the whole picture.

Feedback Effect

Any city that looks to ShotSpotter alerts to measure gun violence will exaggerate the concentration of gun crimes in BIPOC communities. That’s bad for understanding gun violence. It’s doubly bad when cities use ShotSpotter data to justify installing more surveillance technology in supposedly high-crime communities. In April 2024, NYC Mayor Eric Adams announced that the city would place Evolv weapon detectors in the subway system where ShotSpotter data shows the biggest problems with gun crime.[26] ShotSpotter is disproportionately installed in NYC’s BIPOC communities, so it follows that Evolv detectors will be, too—irrespective of true rates of gun violence across the city.[27] In turn, weapon detectors will turn up increased numbers of guns (and objects incorrectly identified as guns), concentrating weapon detection in these stations and reinforcing exaggerated perceptions of where gun crimes happen.

V.             The Detector Effect

Like ShotSpotter, all detectors increase a community’s awareness of events, real and perceived, independent of any change in underlying offense rates. When cities install speed cameras, documented traffic violations go up. It’s a matter of capacity: in New York City, officers issued 107,970 speeding tickets in 2021; its speed cameras issued almost four and half million.[28] On the face of it, that looks like a reason to keep using speed cameras—when detectors find what they are looking for, they seem to justify their continued use.

But poor-quality detectors contribute to the degradation of police data and public safety. Evolv and other metal-detecting weapon detectors routinely, predictably fail to detect certain weapons: they miss some guns on purpose to reduce false alarms and miss other weapons because they don’t contain iron.[29] These missed weapons are absent from police data. And more importantly, they suggest that weapon detectors provide a false sense of security, not a real safety measure.

The situation is even worse for home surveillance cameras, video doorbells, and associated neighborhood surveillance apps—Nextdoor, Neighbors, Citizen—where users are encouraged to post footage of supposed crimes and suspicious persons. Viewing home videos of supposedly suspicious activity increases users’ awareness of perceived criminality. That’s the detector effect. But there’s no evidence that cameras and apps contribute meaningfully to reporting or solving actual crimes. Police officers surveyed about home surveillance apps report wasting time watching countless videos of noncriminal activity.[30] Meanwhile, app users have persistently reported innocent BIPOC neighbors to the police on platforms like Nextdoor—so much so that in 2020, Nextdoor shut down its “Forward to Police” feature.[31]

 VI.          Police Databases

Gang Databases Exaggerate Gang Activity

According to the NYPD, a person commits a gang-motivated crime when they act on behalf of a gang.[32] On top of that, a person commits a gang-related crime if they commit any criminal offense—intimate partner violence, personal drug possession—and are an identified or suspected gang member.[33] Because any crime counts and suspected gang membership suffices, gang-related offenses inflate the number of crimes attributable to gangs: the NYPD tallied 264 gang-motivated crimes and over one thousand gang-related crimes in 2013, the most recent year for which GangStat figures are available.[34]

That was before the NYPD vastly expanded its database of suspected gang members. Since 2013, the NYPD has profiled tens of thousands of Black and Latinx youth as gang members based on their race, clothes, friends and neighborhoods.[35] This expanded NYPD gang database is a direct extension of the department’s stop-and-frisk practices: it continues to mark Black and Latinx youth for police monitoring in the absence of any evidence that they’ve broken the law.[36] The NYPD hasn’t allowed the public a peek at GangStat reports in years, but its vastly expanded pool of suspected gang members supports an inflated count of gang-related crimes.

Gang databases also inflate the severity of crimes attributable to supposed gang members. When the NYPD stopped Keith Shenery with a small bag of marijuana and a folding knife in 2018, prosecutors could have charged him with a misdemeanor.[37] Because he was wrongfully included in the NYPD gang database, they charged Shenery with felony possession of a weapon and unlawful possession of marijuana instead.[38] This isn’t an isolated event. The Racketeer Influenced and Corrupt Organizations Act (RICO) and related state laws—laws passed to combat powerful organized crime groups—allow prosecutors to use relatively small offenses to hold people responsible for crimes that others committed.[39] After the NYPD arrested 120 suspected gang members in a raid on poor Black and Latinx communities in the Bronx in 2016, one in four was convicted of serious gang crimes despite personally doing nothing more than selling marijuana.[40] These convictions inflate gang crime counts, obscure the true toll of gang activity.

The NYPD’s DNA Database

In violation of New York State genetic privacy laws, the NYPD has compiled a database of almost 34,000 predominantly BIPOC New Yorkers’ DNA.[41] Many have never been arrested, much less convicted of any crime, and some are just children.[42] But each person is checked and rechecked for criminal involvement every day, ad infinitum, as the NYPD scans its database for DNA that matches crime scene evidence. The detector effect applies here: repeatedly checking the database increases law enforcement’s awareness of enrolled individuals who commit offenses—much as stop-and-frisk, ShotSpotter, and gang databases put BIPOC communities under the microscope. When the NYPD finds offenders using DNA, that seems to justify continued and intensified DNA-based policework.

But contrary to popular TV narratives, DNA can be a terrible detector. Common forensic lab techniques can amplify the DNA signal of a tiny number of cells—the kind of sample left behind when someone sheds a hair or touches a doorknob.[43] As a consequence, anyone who visits a crime scene can become a suspect, like Terrell Gills, who touched a checkout screen at his local Dunkin Donuts and was wrongly arrested for robbing the coffee shop.[44] DNA testing has its place—the Innocence Project, for example, has used scientifically rigorous techniques to exonerate hundreds of wrongfully convicted individuals.[45] But as used by the NYPD, DNA testing also contributes to baseless arrests, all visited upon a captive population of mostly BIPOC New Yorkers who were unfortunate enough to be enrolled in its DNA database.

VII.        Independent Checks on Crime Data

Crime data is incomplete and badly skewed, but it’s not irredeemable. Homicide data is comparatively reliable, in part because it’s hard to hide or concoct murders in the U.S., and in part because police reports can be independently verified using vital statistics records.[46] Though homicides are a poor proxy for other crime rates, cities can and should apply the same auditing standard to other crime statistics.[47] Much of the data needed to audit other crime statistics already exists. Car thefts and car-crash related offenses can be checked against insurance claims, Department of Motor Vehicles and auto repair records—even hospital records. The distribution of petty vehicular offenses like speeding can be checked against the telemetric data collected by most late-model cars.[48] Burglary counts can be checked against insurance claims. The distribution of gun-related crimes and assaults may be verifiable using aggregated and anonymized data for patients treated at hospitals who are victims of violent crime. The relationship between these datasets is rarely a one-to-one match. Insurance data, hospital utilization, and other external data will be skewed by countless variables: race, ethnicity, insurance status, immigration status, and much more. Still, this external data provides a starting place, an external anchor against which to model official police data, gaining insight to where official statistics fall short and where they overstate crime. External data can overturn misleading police narratives about where crime does and doesn’t happen.

Audits aren’t an exception, they’re the norm in every other industry. Outside of policing, companies and government agencies routinely face audits for both their finances and performance. Hospitals must continually demonstrate to external auditors that they provide safe and effective care to remain accredited and receive insurance reimbursements.[49] Public schools answer to local and state authorities, who are in turn bound by federal performance standards that address student performance, including equity for disadvantaged and high-need students.[50] Publicly traded companies undergo annual financial audits by independent certified public accountants. Trust but verify—or don’t trust and verify—is the norm for evaluating performance in industries where important interests are at stake. The same approach to crime data can check and correct police narratives about crime so that we can answer the question we all want to know: how safe are our cities?

Social Media Graphics