This report was supported in part by a grant from the John D. and Catherine T. MacArthur Foundation.

We extend a special thanks to our external reviewers: Dave Maass, Director of Investigations, Electronic Frontier Foundation; and Conor Healy, Director of Government Research, IPVM

Executive Summary

• Billions in surveillance technology is sold annually with completely unsubstantiated, outlandish marketing claims.

• These endemic practices frequently appear to constitute deceptive advertising, violating federal and state consumer protections.

• Regulators are beginning to take action against some of the worst offenders, but many surveillance firms appear to make marketing claims with impunity.

• A growing body of independent analysis documents surveillance systems’ ineffectiveness, errors, and bias.

I.              Introduction

Seemingly no matter what public safety threats communities want to address, surveillance companies are there to sell their technology as the solution. But while firms are quick to offer a slick sales pitch, they are rarely able to offer evidence that their products work. Police departments now routinely spend tens of millions of dollars on a single surveillance tool, often without any independent verification of marketing claims.[1] Unproven surveillance reaches beyond large cities to an ever-increasing number of America’s 18,000 state and local police departments.[2] Growing evidence suggests many of these tools are not accurate, effective, or equitable enough to be purchased at any price.[3] Even when manufacturers’ claims do not meet the legal standard of deceptive advertising or fraud, all of this sales hype contributes to a climate where cities waste staggering sums on surveillance that often doesn’t work as expected.

II.            Blurry Computer Vision

Few surveillance systems are as widely used, and widely overhyped, as the automated license plate readers that increasingly track vehicles all across the country. One leading vendor, Flock, makes some truly astounding claims about its product. For instance, the firm claimed it caused “a 70% reduction in burglaries in San Marino, CA,” a truly remarkable impact.[4] These sorts of claims raise numerous methodological questions about how one measures burglaries and how one would know that Flock is responsible for the improvement. But in this case, the point is moot since burglaries actually rose in San Marino in the three years following Flock installation.[5] Similarly, Flock’s marketing materials claim it was so effective in Dayton, Ohio that it caused a “43% decrease in crime,” not just in burglaries, but in crime overall.[6] This is even more remarkable since the city hadn’t even installed Flock when the improvements happened.[7]

But Flock’s marketing team isn’t content just claiming to have solved crimes in individual communities, boasting nothing short of the claim that “10% of reported crime in the U.S. solved by Flock!”[8] Think of the magnitude of the assertion that one company singlehandedly remade American policing on an unprecedented scale, all while remaining rather anonymous. The reaction from many experts is this is “problematic,” “border[ing] on ludicrous….”[9] Even Flock’s research consultant distanced himself from the study, saying “I personally would have done things much differently.”[10] The reality of automated license plate readers is that they routinely find expired registrations and inspection stickers, but it’s unclear if they are typically helpful for much else.[11]

Flock’s marketing claims may be stunning, but they’re far from unique amongst video surveillance firms. Take the video surveillance as a service (“V-SaaS”) company Rhombus, whose software integrates camera feeds, sensors and access controls. V-SaaS products can be hard for researchers to objectively evaluate, but that didn’t stop Rhombus from boldly proclaiming its “#1 Cloud-Based Security Cameras” are “100% better than [Rhombus competitor] Verkada” and “outperform other providers across all metrics.”[12] However, they never say what those metrics are.[13] Worse yet, when researchers did the time-consuming and expensive work of methodically comparing V-SaaS products, Rhombus lagged many competitors.[14] Rhombus ultimately took down its most brazen ads, which may have violated Federal Trade Commission regulations.[15]

III.          Full Metal Racket

For decades, companies sold walk-through metal detectors and wand devices that, although inconvenient and frequently obtrusive, would consistently detect metal. That changed when Evolv started marketing something purportedly different: what it claimed was not a “dumb” metal detector, but a convenient, AI-based weapon detector (“Walk through at the pace of life without always stopping, removing coats or backpacks, or emptying pockets.”)[16] That sales pitch was enough to lease their detectors for roughly ten times more each year than it would cost to buy a traditional metal detector outright.[17] The only problem: it’s actually a metal detector. Evolv’s software is smart enough to look for specific shapes, scanning for metal cylinders to find gun barrels, but that means it falsely flags the metal tubes in umbrellas, Chromebook hinges, three-ring binders and other metal items as weapons, too.[18] Beyond that, Evolv “routinely miss[es] non-ferrous weapons” including those made with plastics and non-ferrous metals.[19] It even misses some ferrous weapons on purpose, including handguns, to reduce its substantial false alarm rate.[20] The detectors’ performance is so troubling that the company is being sued in New York after an Evolv detector allegedly failed to detect a large knife that a student brought to school and used to stab a peer.[21] The company even appears willing to move forward with installations it knows ahead of time are a bad idea. In March 2024, Evolv’s CEO told investors that “subways in particular are not a place that we think is a good use case for us, both for the CONOP [operational concerns] and being below ground and the interference with the railways.”[22]) Two weeks later, New York City mayor Eric Adams announced the city’s partnership with Evolv to pilot its detectors in the NYC subway system.[23]

Evolv also faces scrutiny for claiming that it works faster than other metal detectors, promising that patrons can pass through with “no stopping” and a “line-free experience….”[24] Its ads show crowds racing through Evolv detectors, but we never see what happens to the 5 to 10% (in one school district, 60%) of patrons who are falsely flagged as having weapons.[25] The reality for those wrongly stopped for an umbrella or Chromebook, and everyone waiting behind them, is stopping, waiting, and broken promises from Evolv.[26] The FTC began investigating Evolv’s marketing in late 2023.[27] The Securities and Exchange Commission began its own inquiry in March 2024.[28]

Evolv’s competitors also favor fanciful speed claims. Xtract One claims speeds 7x that of metal detectors.[29] Athena and SoundThinking advertise speeds up to 10x faster than metal detectors.[30] Unsurprisingly, none of these companies provide supporting evidence, though Athena responded to queries by removing speed claims from its marketing materials.[31]

The same pattern plays out with computer vision weapons detection systems, which scan video feeds for any objection that resembles a gun. One leading firm, ZeroEyes, pledges “zero false positives,” promising their software will never falsely identify ordinary objects as weapons.[32] If that’s the case, why did ZeroEyes mistakenly put Brazoswood High School on lockdown in October 2023, when it mistook a student’s arm for a gun? [33] Unarmed individuals have been shot and even killed by police who claimed they were brandishing guns.[34] Surveillance technology that misidentifies innocent objects as weapons can upend—and end—lives.

IV.          Facial Inquisition

As facial recognition has drawn increasing scrutiny for errors and bias, developers have rushed to build up their credibility with government examinations of their technology, particularly analysis by the National Institute of Standards and Technology (“NIST”). SAFR, a facial recognition and access control company, was early to this trend, boasting in its 2020 marketing materials 99%+ accuracy (attributed to NIST), extraordinary speed, and the ability to identify faces “even in challenging real-world conditions where faces are in motion [or] at difficult angles.”[35] In fact, NIST found SAFR was only 14% accurate at identifying profile angle photos, as well as being much slower than advertised.[36]

Facial recognition firm Clearview AI gained global notoriety for scraping the web to compile a database of billions of images users could run facial recognition searches against.[37] Its only advertised product is this type of one-to-many facial recognition search, where a single photo is compared to a large database.[38] Clearview AI sells its product to police agencies, which by 2023 had conducted over one million searches in the U.S.—effectively putting us all in a “perpetual police line-up.”[39] These indisputable facts about Clearview AI’s product don’t stop the company from invoking NIST data to claim its facial recognition is “99+% accurate across all demographics,” a statistic that is only true for a much simpler one-to-one match, where only two photos are compared head-to-head.[40]

Facial recognition isn’t just used in policing but is increasingly part of how we prove who we are everywhere from the airport to the internet. During the Covid-19 pandemic, the identity verification firm ID.me tried to combine the frustrations of facial recognition with the Internal Revenue Service (“IRS”), and the results were disastrous. Despite pledging to reduce customers’ wait times and unemployment fraud, customers often waited more than twice as long as the two hours advertised by ID.me; in at least one case, verification took days.[41] On top of this, ID.me claimed to verify identities using only one-to-one matching, by comparing a customer’s webcam photo to their photo ID.[42] What it didn’t disclose is that it also compared customers’ webcam photos to a large database of face images using a more biased, more error-prone, and typically slower facial recognition technique as an anti-fraud measure.[43] In its pitch to the IRS, ID.me even exaggerated the extent of fraudulent unemployment claims by hundreds of billions of dollars.[44] A U.S. House of Representatives Oversight Committee concluded that ID.me misled the IRS and several senators have urged the U.S. Federal Trade Commission (“FTC”) to take action.[45]

Facial recognition firms haven’t just gotten in trouble for what they claim they do, but what they claim they don’t do. Until 2023, the V-SaaS firm Verkada insisted it didn’t “do facial recognition,” rebutting concerns it might be running afoul of local facial recognition bans in a growing number of cities across America.[46] But this was flatly false. Verkada cameras scan faces they see in real time, looking for a match against a watch list.[47] The company stopped doing business in Texas, Illinois, Baltimore, and Portland, Oregon, where it operated in violation of facial recognition laws, and months later, conceded that it uses facial recognition.[48]

And Verkada isn’t the only surveillance vendor attempting to skirt laws by misconstruing the nature of its technology. In January 2024, the CEO of the video surveillance company Eagle Eye claimed that there are three different types of “video surveillance face technology”: face extraction, face matching, and face recognition.[49] Make no mistake: all three “types” of technology described are facial recognition. Companies like Eagle Eye use additional, misleading distinctions in attempts to circumvent the law.

V.             Lethal Weapon

Every policing technology can alter people’s lives, but none carry graver consequences than weapons systems. For years we’ve been sold the promise that a device straight out of science fiction could replace guns, stopping suspects without killing them.[50] Axon advertises its “non-lethal,” now-ubiquitous Taser with the promise that users can stop “a threat without taking a life.”[51] Axon examined 1,201 “field uses” of Taser and claimed that 99.75% did not result in serious injury.[52] But a major news outlet’s independent inspection of autopsy reports for 1,081 victims killed following Taser use indicated that the stun gun was a cause or contributing factor in 15% of deaths. Alarmingly, 26% of deaths were attributed to “excited delirium,” a now-discredited cause of death that has been outlawed as a classification in California and a growing list of jurisdictions.[53] This suggests the true percentage of cases where Taser was a contributing factor is much higher. Another distortion in the data is that Axon actively attempts to steer Taser-related death investigations: as one lawyer put it, “[f]rom the minute they find out someone dies, they’re doing everything they can behind the scenes… so the case goes away.”[54] The situation is so grim that some departments and victims’ families have sued Axon over its fraudulent safety claims.[55]

VI.          Unpredictive Policing

When so many policing technologies struggle to get things right in real time, predictive policing tools manage to go even a step further, claiming to use historic crime data to predict events before they take place. But predictably, they don’t. The leading firm PredPol, for example, claimed that it “looks forward and projects where and when crime will most likely occur.”[56] In fact, PredPol simply averaged recent policing activity, providing technological cover for sending police back to the places they already frequented.[57] Rather than giving law enforcement a crystal ball, this approach traps officers in a cycle of repeatedly targeting the same communities over and over again.

This approach also meant that PredPol likely both missed lots of crimes that actually happened while chasing threats that never materialized. One study of 23,631 PredPol predictions in Plainfield, NJ showed that its predictions matched reported crimes less than half of one percent (0.5%) of the time.[58] SoundThinking, PredPol’s parent company, no longer claims that the product predicts anything and simply says that it “automates the planning of directed patrols.”[59]

 VII.        Man’s Worst Friend?

A Boston Dynamics’ promotional video shows its robot dog Spot climbing a ramp as a caption reads “NAVIGATES CHALLENGING TERRAIN.”[60] The digital pup marches through pouring rain as the screen says “RAIN AND DUST PROTECTED.”[61] And in a 2022 Superbowl ad, it hands out beers and dances with human pals.[62] Someone seeing the commercials might be impressed until they look in Spot’s operating manual. Read the fine print and it turns out Spot doesn’t do well in rain—“Spot has difficulty walking, and may fall, on slippery surfaces.”[63] And Spot isn’t quite ready to hit the dance floor, or the bar. If you see a robot dog offering you a beer or a dance, manufacturers warn you to stay at least 2 meters away. Why? This adorable dance partner can occasionally step on your feet or even amputate a finger.[64] These sorts of claims may not meet the legal criteria for deceptive advertising practices, but the Spot you see on your screen is far removed from the capabilities of the robots we now see on our streets.

Boston Dynamics isn’t the only robotics firm making questionable marketing claims. Robot maker Knightscope manufactures a large, slow-moving surveillance robot that drew snickers when it sauntered into the Times Square subway station as part of a NYPD pilot program. The firm claims that “[t]echnologies like Knightscope… are known to be effective in reducing crime.”[65] The evidence? Two tiny, questionable data points: the Huntington Park, CA police department credits Knightscope’s K5 with a year-over-year drop in crime reports (48 to 26) and corresponding increase in arrests (11 to 14).[66] Knightscope also takes credit for decreased crime in a Las Vegas housing complex, though local authorities don’t agree (“I cannot say it was due to the robot,” a police spokesperson said).[67] In truth, there isn’t enough data to back Knightscope’s claims, and every reason to think that police robots amount to “security theater.”[68] Two dubious data points—one disavowed by the robot user—don’t amount to a reasonable basis for advertising claims.[69]

VIII.      Conflicts of Interest

All too often, police aren’t just the end users of surveillance technologies, but boosters for it as well, working with tech companies to drive public adoption and acceptance of invasive new products. Amazon drew widespread condemnation for its partnerships with thousands of police departments across the country, agreements that not only gave departments access to Ring surveillance data, but which pressed officers to promote Amazon’s products, often in violation of police departments’ codes of ethics.[70] While officers publicly praised the tech giant’s video doorbells, the reality was quite different. A survey of 40 law enforcement agencies in eight states showed that Ring wasted officers’ time with racoon videos and other noncriminal footage, and that most agencies attributed either zero arrests to Ring or couldn’t speak to its efficacy.[71] That didn’t stop the public praise. Instead, officers continued to tout the product, with Amazon giving free Ring doorbells to police departments in exchange while working to keep the deal secret.[72]

Amazon’s scheme may be egregious, but it’s hardly the only conflict of interest in this space. According to SoundThinking, its gunshot detectors work: “ShotSpotter has a 97% accuracy rate, including a 0.5% false positive rate” when identifying and locating gunfire.[73] But ShotSpotter counts gunshot alerts as accurate unless it hears otherwise from officers.[74] That is, it counts on departments with a vested interest in ShotSpotter’s success, and in their own success, to report mistakes. That’s a conflict of interest and a recipe for overstating the product’s accuracy. In fact, ShotSpotter sensors have confused fireworks, backfiring cars, and other loud bangs for gunshots, and by the company’s own admission, street noise and ordinary features of cityscapes cause ShotSpotter to miscalculate sounds’ locations.[75]

IX.          Reining in Surveillance Advertising

The stakes couldn’t be higher for surveillance advertising. Ineffective tools consume police budgets and can create a false sense of safety, with millions pouring into unproven technology while social services with a more proven track record of reducing crime go underfunded. Predictive policing and ShotSpotter compound the racist over-policing of BIPOC communities. Tasers can be unsafe, even lethal. In some cases, surveillance advertising claims are so exaggerated that they actually are illegal.

The FTC prohibits “unfair and deceptive acts or practices in or affecting commerce.”[76] Unlawful deception “involves information that is important to consumers”[77] and is “likely to mislead customers acting reasonably under the circumstances.”[78] Product representations are misleading if “the advertiser lacked a reasonable basis—or adequate substantiation—for asserting that the message was true.”[79] This captures not only flat out lies, but conflicted, disingenuous and badly substantiated claims. The FTC additionally insists that comparative advertising be truthful.[80]

Companies don’t have to articulate falsehoods to be held accountable for bogus advertisements, either. By FTC standards, an advertisement is deceiving if it includes content “that is likely to mislead a consumer about the “nature, characteristics [or] qualities” of a product.[81] Lastly, companies are liable for the statements made indirectly, through endorsers. Endorsements must “reflect [endorsers’] honest opinions, findings, beliefs, or experience.”[82] Companies can’t use endorsers to misrepresent their products: statements “that would be deceptive if made directly by the advertiser” are off limits for endorsements as well.[83]

Beyond the FTC, nearly every state has parallel consumer protections that restrict similar deceptive practices, at times even going farther than federal law.[84] While many of the claims collected above would appear to meet this legal threshold, few of these surveillance firms have faced any scrutiny from regulators.

While the FTC recently took a step to address surveillance advertising by questioning Evolv, the sector generally hasn’t faced the same scrutiny as other sectors that pose a serious threat to health and safety. In other industries, advertisements that market risky products must meet even more rigorous substantiation standards.[85] The FTC applies a high standard to health and safety claims because they lead consumers to believe that a product is safe and healthy to use.[86] In these cases, the FTC requires “competent and reliable scientific evidence” for substantiation, including objective evidence gathered by relevant professionals using accepted techniques “to yield accurate and reliable results.”[87]

The same standards should apply to surveillance vendors, who also hold Americans’ lives in their hands. Even applying the FTC’s broader standards for advertising can eliminate surveillance companies’ most outrageous deceptions. The FTC has the authority and the infrastructure to crack down on false claims. What they need to do now is take action against surveillance technology vendors to make sure that companies follow the letter of the law.

Social Media Graphics