This report was supported in part by grants from Borealis Philanthropy Disability Inclusion Fund and the John D. and Catherine T. MacArthur Foundation.
We extend a special thanks to Public Interest Privacy Center President Amelia Vance and an anonymous reviewer for their feedback.
EXECUTIVE SUMMARY
Schools increasingly turn to spyware, noise detectors, and other invasive mental health prediction tools, with predictably poor results. These error-prone systems flag non-existent crises and miss real dangers.
Mental health surveillance alienates students, making it more likely that they will self-censor and isolating them from teachers and online mental health resources.
Student spyware routinely outs LGBTQ+ students and puts BIPOC youth at risk of police encounters.
New tech appears to be displacing evidence-based mental health screening.
I. Introduction
School surveillance is nothing new. Cameras have long watched over hallways and classrooms.[1] Students routinely run a gambit of metal detectors. But now, tech vendors are selling unproven and federally funded tools that claim to assess students’ states of mind and predict threats that students pose to themselves or others.[2] But psychological surveillance impedes effective mental health interventions; outs LGBTQ+ students; and gives police pretext to enter our schools and homes. Laptop-scanning spyware can invent mental health crises,[3] transforming a coach’s email about basketball shooting drills into a threat of firearms attack.[4] Noise detectors wiretap school buildings to find threats, but often only call police and school officials to the site of laughter and slamming lockers.[5] U.S. students’ mental health crisis is real,[6] but psychological surveillance solutions are fake. Sadly, funding is flowing to these invasive, error-prone systems and displacing evidence-based mental health screenings, treatment, and support.[7]
II. What Students Need vs. What They Get
One in five U.S. students struggles with a significant mental health problem by high school.[8] Most don’t receive mental health services,[9] but those who do most often receive services at school.[10] To identify students who need support, best practices suggest universal screening—evaluating the needs of every student—using a proven tool suited to a school’s student body.[11] Thereafter, screening results should be interpreted in light of students’ personal histories and unique circumstances.[12] Effective screenings require care, transparency, and practices that foster honest answers to deeply personal questions.[13] Practitioners must take pains to respect students’ agency, privacy, and overall sense of safety. These thoughtful, bespoke forms of care stand in stark contrast to “one-size-fits-none” methodology of automated psychological surveillance.
Student Spyware’s Gotcha Approach
A student googles “Why am I depressed?” while an administrator secretly watches their search activity.[14] This is an advertisement for Securly, software that surveils “all student activity across email, documents, social media, and web browsing.” Securly’s secretive algorithm scores students on the number and severity of supposed oversteps they make by using words or viewing materials that the company flags as supposedly problematic.[15] It notifies administrators and police if students’ scores exceed a seemingly arbitrary level.[16] While the company offers limited parental education materials, many schools opt out of giving any notice.[17] Even worse, the company completely hides what it considers to be the most problematic terms and materials, making meaningful bias evaluation impossible.[18]
“More broadly, this surveillance chills students’ digital lives: not just what they say publicly, but what they allow themselves to Google in the safety of their own rooms.”
Securly is just one of countless student monitoring services, including Gaggle, GoGuardian, Social Sentinel, and Bark, that secretly send students’ computing activity to administrators without notifying students.[19] Gaggle reported thirteen-year-old transgender student for reflecting on a suicide attempt and subsequent therapy in a school writing assignment.[20] The psychological surveillance software retraumatized the student after school officials “freaked out” about his nonexistent mental health crisis.[21] “I was trying to be vulnerable with this teacher and be like, ‘Hey, here’s a thing that’s important to me because you asked,” he later reflected.[22]
This is not an isolated incident. A United States Senate report warns that spyware-induced “[e]scalations and mischaracterizations of crises” can hurt students in the long run “due to stigmatization and differential treatment following even a false report.”[23] When psychological surveillance tools hallucinate crises, they worsen students’ real struggles. More broadly, this surveillance chills students’ digital lives: not just what they say publicly, but what they allow themselves to Google in the safety of their own rooms.[24]
Spying on Students, Badly
Student spyware frequently misunderstand students’ states of mind. This isn’t surprising: spyware companies don’t follow best practices for mental health screening. Instead, they appear to err on the side of caution and flag too many students. Some even have a financial incentive to do so—Gaggle sells online counseling.[25] But tools’ tendency to “overdo it” renders them useless. As one school administrator put it:
“We tried Bark for self-harm notifications. We got tons of alerts, but they were all false positives. Even at its lowest level if a student searched for random things, it would flag them.”[26]
As another teacher put it, “You’re going to get 25,000 emails saying that a student dropped an Fbomb in a chat.”[27]
Student spyware doesn’t understand context, and it doesn’t appear to try. One school attempted to use GoGuardian to stop students from searching for the term “ass” on school computers.[28] Instead of blocking students from using an inappropriate term, GoGuardian flagged words like “class” and “assignment.”[29] In another school, GoGuardian reported any student writing the word “toy.”[30] Social Sentinel flagged a basketball coach’s message to players as portending violence. The coach’s offense? “Workouts are Tuesday and Wednesday after school. Friday morning is shooting.”[31] The list goes on: Gaggle reprimanded a student editor for receiving another student’s email submission.[32] Bark deemed “snow,” the purple heart emoji, “pen” and “school bus” to be drug slang.[33] As one student journalist reported, BTS fans are out of luck (fans of the Korean pop sensation favor purple hearts).[34] More recent editions of Bark’s “Teen Speak Code” for parents note that emojis that can be innocuous can also have inappropriate meanings: a taco emoji, for example, may refer to genitals.[35]
But when companies like Bark err on the side of over-flagging, they leave it to schools to sort out sexual references from Mexican food. In one large survey, 65% of teachers reported being tasked with following up on alerts from student spyware, despite being overburdened with their actual jobs.[36] Students and administrators even report that student spyware interferes with classwork, wrongly blocking research sources and flagging writing assignments.[37] GoGuardian has flagged educational websites about Black History Month, literary classics, Bush v. Gore, and the Bible for containing keywords the company associates with sexual content.[38]
Student spyware also appears to misinterpret marginalized students disproportionately. Securly has wrongly labeled informational LGBTQ+ news and health websites as “pornography.”[39] Until recently, simply saying “gay” or “lesbian” within Gaggle or GoGuardian’s earshot was a no-go: the companies flagged the words as indicators that a health or safety intervention might be necessary.[40] Parents and organizations criticized this policy, and a Gaggle insider even revealed that the company’s policy of flagging LGBTQ+-related terms had routinely outed students.[41] (A third of LGBTQ+ students confirmed that internet monitoring had outed them or someone they know.[42]) Initially, Gaggle doubled down on its homophobic policy.[43] It eventually relented, but it never apologized for outing students.[44] (LGBTQ+ youth’s risk of homelessness skyrockets when they are outed,[45] as does transgender youth’s risk of abuse.[46]) GoGuardian and Gaggle continue to flag normal conversations about sex, putting teens exploring their sexuality at risk of humiliation and worse.[47]
“Absent a crisis, activating the police wastes officers’ time while still putting students at serious risk of police violence.”
Calling the Cops
Student spyware routinely notifies (and effectively dispatches) police officers to students suspected of being in crisis—especially after hours, when teachers are off work and three quarters of alerts come in.[48] Nearly half of teachers in one large scale study reported student spyware alerts being relayed to law enforcement.[49] Baltimore public schools provide a case in point. By day, GoGuardian notifies school counselors and social workers when students are flagged as at risk of self-harm.[50] By night and on weekends, those alerts go straight to the Baltimore police.[51] Instead of encountering a trained mental health professional, students are met by police officers who show up armed and unannounced at their homes.[52] At the University of Connecticut, Social Sentinel updates campus police, not mental health workers, when spyware flags students at risk of self-harm.[53] In Florida, legislators mandated a police database of student spyware records and other school records.[54]
Police aren’t trained mental health professionals. They can and do routinely make mental health crises worse.[55] The New York City Police Department (“NYPD”), for example, routinely uses force in encounters with students: it handcuffs 10% of children during “child in crisis” interventions.[56] Worse yet, almost 95% of the NYPD’s child in crisis interventions involve BIPOC students despite their heightened susceptibility to police violence.[57] Police interactions also increase students’ risk of subsequent involvement with the criminal justice system.[58] And police involvement in mental health crises can even be deadly. In 2020, Salt Lake City police shot a 13-year-old with autism after his mother called 911 for a mental health informed intervention, leaving him with life-threatening injuries and a long-term physical disability.[59]
Absent a crisis, activating the police wastes officers’ time while still putting students at serious risk of police violence. Social Sentinel alerted San Antonio school administrators that a student posed a safety threat after she posted hyperbolic claims about a TV sitcom to social media.[60] School administrators weren’t fooled by “if you dont fall in love with jess again im gonna kill you.”[61] After school hours, police are left to sort through such messages—and possibly arrive at students’ doorsteps. This has occurred before: Baltimore Police admitted multiple visits to student homes for “wellness checks” in response to student spyware flags.[62] The danger of sending police to the homes of Baltimore public school students (90% of whom are Black or Latinx) can’t be overstated.
Spyware Meets ChatGPT: Tutoring Bots and Possible Entrapment
ChatGPT, a generative artificial intelligence tool that can perform internet research and draft papers, worried teachers when students started using the tool to do their homework.[63] Now, some schools are adopting a tutoring program that uses the same technology. Khan Academy’s tutoring chatbot, Khanmigo, uses generative AI to answer students’ questions.[64] It also flags students who discuss self-harm and other forbidden topics,[65] introducing the same range of concerns associated with traditional student spyware, from escalating imagined mental health crises to botching the response to students who need support.[66] As a rule of thumb, Khanmigo suggests not saying anything to it that wouldn’t be appropriate within earshot of an elementary school classroom.[67] It sends the entire log of off-limit chats to every one of a flagged student’s teachers and guardians.[68] One can just imagine the chatbot unthinkingly entrapping high school students. Khanmigo: “What are three main themes of Dante’s Inferno?” Student: “hopelessness, torture, and hell.” Khanmigo: gotcha.
III. More of What Students Get: Bogus Threat Detectors
Noise Detectors
Tech vendors take advantage of school communities’ understandable anxiety about gun violence by selling tools that amount to overhyped noise detectors. Some these noise detectors claim to compare a room’s sounds to the known “sounds patterns” of emotions like aggression and fear; they send alerts to school administrators or police when they make a match.[69] Other noise detectors claim to learn the normal sound level of a room and send an alert when they sense sounds above the norm, potentially indicating aggression or an incident between students.[70]
“Louroe’s so-called ‘aggression- and gunshot-detection’ microphones were confused by the sound of children slamming lockers in a Nevada elementary school.”
But schools are unpredictably noisy, without “normal” sound levels, and noise detectors can’t cope. Louroe’s so-called “aggression- and gunshot-detection” microphones were confused by the sound of children slamming lockers in a Nevada elementary school.[71] A test of Sound Intelligence, the software inside of Louroe devices and other noise detectors, showed that it falsely flagged normal school sounds like singing, laughing, coughing, and energetic speech, but didn’t flag the sound of students screaming.[72] Noise detectors can be tuned and retuned in an attempt to get it right, but as it stands, they simply don’t work.
Despite these flaws, some noise detectors are designed to work with schools’ public announcement (“PA”) systems. One Ohio charter school paired a Sound Intelligence noise detector with an Axis Communications sound system.[73] When a loud sound triggers the noise detector, the PA system plays an automated warning message over the room’s loudspeaker.[74] Cue a school choir, or playful students laughing. “You must calm down or risk having law enforcement contacted. You must calm down.”[75] This is a system that is useless and worse: schools and police may overreact or, alternatively, ignore the system that cries wolf.
Noise-detecting hardware needlessly invites a dangerous police presence into schools. Schools from San Diego to Georgia give “administrators, campus security or police officer[s]” permission to monitor and respond to threat-detector alerts.[76] Other schools around the country have installed hardware that can automatically alert police to perceived threats.[77] But police responses push students into early encounters with the criminal legal system.
Threat Detectors Meet Disproven Psychology
In an online classroom based in Hong Kong, students attend class while software tries to read their faces: who is learning? Who is bored? Who is angry, even dangerous?[78] So-called “emotion recognition” has made inroads in the U.S., too: Zoom nearly added the tool to its platform in 2022,[79] and Intel tested emotion-recognition for schools.[80]
But emotion recognition doesn’t work. These tools, which claim to detect when students are bored, distracted, confused, frustrated, or angry, can’t do anything of the sort: they are based on junk science with documented discriminatory effects.[81] It is common for emotional recognition tools to overattribute negative emotions to Black people.[82] It is common for them to fail more often than they succeed, because people use expected expressions (like smiling when they are happy) only 20% to 30% of the time.[83] People’s emotional indicators—their body language, their tone of voice—vary across cultures and for people with neurodivergences and disabilities.[84] A student’s facial tics, for example, might lead to an emotion recognition system wrongly determining that the student is distracted and disengaged from a lesson, or cheating on an exam.[85] Educators and administrators may choose to respond to this dubious information by punishing students or identifying them, in the words of one emotion detection developer, as “problem pupils.”[86] AI companies abroad have already begun developing emotion recognition systems for the expressed purpose of assessing the mental wellbeing and threat level of students.[87] Fortunately, bogus emotion-detecting tech is still rare in schools and should stay that way.
IV. The Youth Mental Health Crisis and the True Cost of Student Spyware and Threat Detectors
As U.S. Surgeon General Vivek Murthy puts it, the state of American children’s mental health constitutes a public health crisis.[88] The Center for Disease Control surveys a nationally representative sample of high school students every two years. In 2021, 42% of high school students reported persistent feelings of sadness or hopelessness.[89] This astonishing figure cannot be written off as a temporary effect of the COVID-19 pandemic: 37% of students already reported persistent sadness or hopelessness in 2019.[90] 18% of students made a suicide plan and 10%—one in ten students— attempted suicide in 2021.[91] Students’ self-reports are confirmed by pediatric emergency departments, which report an increase in drug-poisoning and self-harm-related visits.[92]
In the face of this public health crisis, students need proven, efficacious, humanely administered behavioral and mental health screenings and services—not expensive tech toys that treat students as safety threats. Bogus school-safety tech is expensive. Contracts reveal costs of tens or hundreds of thousands of dollars per year.[93] This money would be better spent on increasing teacher salaries, keeping class sizes small, and other endeavors that improve student-teacher bonds—a key factor in ensuring students in need of help are not overlooked and receive support earlier rather than later.[94] Cost and opportunity costs aside, spying hurts students. It manufactures crises where they don’t exist, botches the responses to students’ real mental health needs, outs LGBTQ+ students, and dispatches police where they don’t belong. In a world where most of us acquire the information we need online, spying cuts students off from resources: students report they avoid researching mental health online for fear of monitoring.[95] For students’ sake, psychological surveillance has got to go.
-
[1] "Fast Facts: School safety and security measures," National Center for Education Statistics, June 22, 2022, https://nces.ed.gov/fastfacts/display.asp?id=334.
[2] “DOJ Awards Almost $190 Million in Grants to Support School Safety,” Office of Justice Programs, October 14, 2022, https://www.ojp.gov/files/archives/pressreleases/2022/doj-awards-almost-190-million-grants-support-school-safety.
[3] Daniel Villarreal, “Minneapolis Schools Are Spying on Queer Students & Outing Them to Teachers and Parents,” LGBTQ Nation, October 16, 2021, https://www.lgbtqnation.com/2021/10/minneapolis-schools-spying-queer-students-reporting-teachersparents/.
[4] Lam Thuy Vo and Peter Aldhous, “Your Dumb Tweets Are Getting Flagged To People Trying To Stop School Shootings,” BuzzFeed News, October 31, 2019, https://www.buzzfeednews.com/article/lamvo/social-sentinel-school-officials-shootingsflag-social-media.
[5] Jack Gillum and Jeff Kao, “Aggression Detectors: The Unproven, Invasive Surveillance Technology Schools Are Using to Monitor Students,” ProPublica, June 25, 2019, https://features.propublica.org/aggression-detector/the-unproven-invasive-surveillancetechnology-schools-are-using-to-monitor-students/.” Jeff Kao and Jack Gillum, “Reverse-Engineering an Audio Aggression Detection Algorithm,” 2020, https://bpb-usw2.wpmucdn.com/sites.northeastern.edu/dist/0/367/files/2020/02/CJ_2020_paper_52.pdf. See also Gillum and Kao, “Aggression Detectors.”
[6] “Youth Risk Behavior Survey Data Summary & Trends Report 2011-2021” (Centers for Disease Control and Prevention National Center for HIV, Viral Hepatitis, STD, and TB Prevention Division of Adolescent and School Health, 2022), https://www.cdc.gov/healthyyouth/data/yrbs/pdf/YRBS_Data-Summary-Trends_Report2023_508.pdf.
[7] Haskins, Caroline. “Revealed: How One Company Surveils Everything Kids Do and Say In School.” BuzzFeed News, November 1, 2019. https://www.buzzfeednews.com/article/carolinehaskins1/gaggle-school-surveillance-technology-education.
[8] “Ready, Set, Go, Review: Screening for Behavioral Health Risk in Schools” (Rockville, MD: Office of the Chief Medical Officer, Substance Abuse and Mental Health Services Administration, 2019), https://www.samhsa.gov/sites/default/files/ready_set_go_review_mh_screening_in_schools_508.pdf.
[9] “School Mental Health Quality Guide: Screening” (University of Maryland School of Medicine: National Center for School Mental Health, January 27, 2020), https://www.schoolmentalhealth.org/media/som/microsites/ncsmh/documents/qualityguides/Screening-1.27.20.pdf.
[10] “Screening,” National Center for School Mental Health.
[11] “Ready, Set, Go, Review,” Substance Abuse and Mental Health Services Administration. See also Emily Goodman-Scott, Peg Donohue, and Jennifer Betters-Bubon, "The case for universal mental health screening in schools,” Counseling Today, September 5, 2019, https://ct.counseling.org/2019/09/the-case-for-universal-mental-health-screening-in-schools/.
[12] “Ready, Set, Go, Review,” Substance Abuse and Mental Health Services Administration.
[13] “Ready, Set, Go, Review,” Substance Abuse and Mental Health Services Administration.
[14] “Manage Student Wellness Alerts Effectively and Manage Cases,” Securly, accessed June 22, 2023, https://www.securly.com/respond.
[15] “Securly Aware - Student Wellness Monitoring,” Securly, accessed October 23, 2023, https://www.securly.com/aware?utm_term=&utm_source=adwords&utm_medium=ppc&utm_campaign=Securly_US_Brand_search_2021.
[16] “Securly Aware,” Securly. See also Senator Elizabeth Warren and Senator Ed Markey, “Constant Surveillance: Implications of Around-the-Clock Online Student Activity Monitoring,” March 2022, https://www.warren.senate.gov/imo/media/doc/356670%20Student%20Surveillance.pdf.
[17] Warren and Markey, “Constant Surveillance.”
[18] See, for example, Haley Zapal, “2023 Emoji Meanings: An Emoji Dictionary for Parents,” Bark (blog), July 2, 2020, https://www.bark.us/blog/emoji-slang-guide/.
[19] FAQs: Bark for Schools,” Bark, May 19, 2023, https://support.bark.us/hc/en-us/articles/360050483491-FAQs-Bark-forSchools. See also Rachel Franz, “Get Gaggle Out.” See also “Monroe-Woodbury Central School District,” accessed June 22, 2023, https://www.mw.k12.ny.us. See also Ari Sen and Derêka K. Bennett, “Tracked,” Dallas News, September 20, 2022, https://interactives.dallasnews.com/2022/social-sentinel/.
[20] Villarreal, “Spying on Queer Students.”
[21] Villarreal, “Spying on Queer Students.”
[22] Villarreal, “Spying on Queer Students.”
[23] Warren and Markey, “Constant Surveillance.”
[24] Mark Keierleber, “New Research: Most Parents and Teachers Have Accepted Student Surveillance As a Safety Tool — But See the Potential For Serious Harm,” The 74, September 21, 2021, https://www.the74million.org/article/new-research-mostparents-and-teachers-have-accepted-student-surveillance-as-a-safety-tool-but-see-the-potential-for-serious-harm/.
[25] Mark Keierleber, “The Spy Tech That Followed Kids Home for Remote Learning—and Won’t Leave,” Fast Company, September 20, 2021, https://www.fastcompany.com/90677509/the-spy-tech-that-followed-kids-home-for-remote-learningand-wont-leave.
[26] Michael S. Lockett, “School Computer Monitoring Program Makes Botched Debut,” Juneau Empire, October 18, 2019, https://www.juneauempire.com/news/school-computer-monitoring-program-makes-botched-debut/.
[27] Mark Keierleber, “Gaggle Surveils Millions of Kids in the Name of Safety. Targeted Families Argue It’s ‘Not That Smart,’” October 12, 2021, https://www.the74million.org/article/gaggle-surveillance-minnesapolis-families-not-smart-ai-monitoring/.
[28] rubens34, “Need Help with GoGuardian,” Reddit Post, R/K12sysadmin, February 5, 2021, www.reddit.com/r/k12sysadmin/comments/ldao6v/need_help_with_goguardian/.
[29] rubens34, “Need Help with GoGuardian.”
[30] neatlyfoldedlaundry, “Anyone Else HATE GoGuardian?!,” Reddit Post, R/K12sysadmin, June 15, 2022, www.reddit.com/r/k12sysadmin/comments/vd3lmo/anyone_else_hate_goguardian/.
[31] Vo and Aldhous, “Dumb Tweets.”
[32] Keierleber, “Gaggle Surveils Millions.”
[33] Haley Zapal, “2023 Drug Slang Emojis: Here’s What Every Parent Needs to Know,” Bark (blog), May 26, 2021, https://www.bark.us/blog/drug-slang-emojis/.
[34] Genie Tang, “Opinion: Bark Technologies, the Dangers of Third-Party Trackers for Children,” HS Insider, October 12, 2022, https://highschool.latimes.com/troy-high-school/opinion-bark-technologies-the-dangers-of-third-party-trackers-for-children/.
[35] Zapal, “2023 Emoji Meanings.”
[36] “Software That Monitors Students May Hurt Some It’s Meant to Help,” Education Week, August 8, 2022, sec. Technology, Classroom Technology, https://www.edweek.org/technology/software-that-monitors-students-may-hurt-some-its-meant-tohelp/2022/08.
[37] djwired, “Gaggle Creates Sprawling Surveillance Industrial Complex That Targets 5 Million Kids Who Can’t Opt out,” Reddit Post, R/Tech, November 2, 2019, www.reddit.com/r/tech/comments/dqjejl/gaggle_creates_sprawling_surveillance_industrial/. whackathon2002, “Securly Sucks,” Reddit Post, R/Computers, February 19, 2019, www.reddit.com/r/computers/comments/asb94a/securly_sucks/. Userp2020, “Securly vs Bark?,” Reddit Post, R/K12sysadmin, October 25, 2020, www.reddit.com/r/k12sysadmin/comments/jhp88f/securly_vs_bark/.
[38] Electronic Frontier Foundation, “Red Flag Machine,” accessed November 2, 2023, https://redflagmachine.org.
[39] James Factoria, “Surveillance Programs Are Reportedly Targeting, Outing LGBTQ+ Students,” Them, October 19, 2021, https://www.them.us/story/surveillance-programs-reportedly-targeting-outing-lgbtq-students. Todd Feathers, “Schools Use Software That Blocks LGBTQ+ Content, But Not White Supremacists,” Vice, April 28, 2021, https://www.vice.com/en/article/v7em39/schools-use-software-that-blocks-lgbtq-content-but-not-white-supremacists.
[40] Mark Keierleber, “Gaggle Drops LGBTQ Keywords from Student Surveillance Tool Following Bias Concerns,” 74 Million, January 27, 2023, https://www.the74million.org/article/gaggle-drops-lgbtq-keywords-from-student-surveillance-tool-followingbias-concerns/
[41] Rachel Franz, “Get Gaggle out of Schools Today,” Fairplay (blog), May 17, 2022, https://fairplayforkids.org/get-gaggle-out-ofschools-today/.
[42] Keierleber, “Gaggle Drops LGBTQ Keywords.”
[43] Keierleber, “Gaggle Drops LGBTQ Keywords.”
[44] Keierleber, “Gaggle Drops LGBTQ Keywords.”
[45] Susan Tebben, “LGBTQ+ Youth Would Be Put at Greater Risk of Homelessness under Ohio Forced Outing Bill,” Ohio Capital Journal, July 10, 2023, https://ohiocapitaljournal.com/2023/07/10/lgbtq-youth-put-at-greater-risk-of-homelessness-underforced-outing-bill-proposed-in-ohio/.
[46] Brian C. Thoma et al., “Disparities in Childhood Abuse Between Transgender and Cisgender Adolescents,” Pediatrics 148, no. 2 (August 1, 2021): e2020016907, https://doi.org/10.1542/peds.2020-016907.
[47] Lisa Railton, “Nudity & Sexual Content: What Is Gaggle Seeing?,” November 5, 2021, https://www.gaggle.net/blog/nuditysexual-content-what-is-gaggle-seeing. See also Jason Kelley, “How GoGuardian Invades Student Privacy” (Electronic Frontier Foundation, October 30, 2023), https://www.eff.org/deeplinks/2023/10/how-goguardian-invades-student-privacy.
[48] Ryan Dorsey [@ElectRyanDorsey], “@BaltCitySchools Monitors Students’ Chrome Books for Keyword Searches Indicating Interest in Self-Harm, and Then Sends Police—Not Qualified Professionals—to Intervene. This Is Crazy.,” Tweet, Twitter, September 30, 2021, https://twitter.com/ElectRyanDorsey/status/1443369691351760897. See also Keierleber, “Spy Tech Followed Kids.”
[49] Elizabeth Laird et al., “Hidden Harms: The Misleading Promise of Monitoring Students Online” (Center for Democracy and Technology, August 3, 2022), https://cdt.org/insights/report-hidden-harms-the-misleading-promise-of-monitoring-studentsonline/.
[50] Jaisal Noor, “Cops in Baltimore Schools Are Monitoring Students’ Laptops,” The Real News Network, October 4, 2021, http://therealnews.com/cops-in-baltimore-schools-are-monitoring-students-laptops.
[51] Noor, “Cops in Baltimore Schools.”
[52] Noor, “Cops in Baltimore Schools.”
[53] Grace McFadden, “UConn Police Use AI to Track Student Social Media,” The Daily Campus, October 3, 2022, https://dailycampus.com/2022/10/03/uconn-police-use-ai-to-track-student-social-media/.
[54] Benjamin Herold, “To Stop School Shootings, Fla. Will Merge Government Data, Social Media Posts,” Education Week, July 27, 2018, sec. Technology, Data, https://www.edweek.org/technology/to-stop-school-shootings-fla-will-merge-government-datasocial-media-posts/2018/07. See also Benjamin Herold, “Florida Plan for a Huge Database to Stop School Shootings Hits Delays, Legal Questions,” Education Week, May 31, 2019, sec. Leadership, School Climate & Safety, https://www.edweek.org/leadership/florida-plan-for-a-huge-database-to-stop-school-shootings-hits-delays-legalquestions/2019/05.
[55] Nicholas Turner, “We Need to Think Beyond Police in Mental Health Crises,” Vera Institute of Justice, July 7, 2018, https://www.vera.org/news/we-need-to-think-beyond-police-in-mental-health-crises.
[56] “Data Brief,” Advocates for Children of New York, June, 2021, https://www.advocatesforchildren.org/sites/default/files/library/police_response_students_in_crisis.pdf?pt=1.
[57] “Data Brief,” Advocates for Children of New York.
[58] “Improving New York City’s Responses to Individuals in Mental Health Crisis,” (New York City Public Advocate Jumaane D. Williams, September 25, 2019), https://www.pubadvocate.nyc.gov/reports/improving-new-york-citys-responses-to-individualsin-mental-health-crisis/.
[59] Rachel Treisman, “13-Year-Old Boy With Autism Disorder Shot By Salt Lake City Police,” NPR, September 9, 2020, sec. National, https://www.npr.org/2020/09/09/910975499/autistic-13-year-old-boy-shot-by-salt-lake-city-police.
[60] Vo and Aldhous, “Tweets Flagged.”
[61] Vo and Aldhous, “Tweets Flagged.”
[62] Liz Bowie, “Baltimore City Student Laptops Are Monitored for Mentions of Suicide. Sometimes, the Police Are Called,” Baltimore Sun, October 12, 2021, https://www.baltimoresun.com/education/bs-md-laptops-monitoring-20211012-a2j3vsytijhhjj36n57ri5zdhi-story.html.
[63] “What Is Chat GPT? - 10 Ways to USE Chat GPT for Students,” College Vidya (blog), October 14, 2021, https://collegevidya.com/blog/how-to-use-chat-gpt-for-students/.
[64] Chris Sadler, “The Future of AI Tutoring in Higher Ed,” New America (blog), April 4, 2023, https://www.newamerica.org/oti/briefs/the-future-of-ai-tutoring-in-higher-ed/.
[65] Natasha Singer, “New A.I. Chatbot Tutors Could Upend Student Learning,” New York Times, June 8, 2023, https://www.nytimes.com/2023/06/08/business/khan-ai-gpt-tutoring-bot.html?searchResultPosition=3.
[66] “Privacy Policy,” Khan Academy, accessed October 25, 2023, https://www.khanacademy.org/about/privacy-policy#sharing. Natasha Singer, “New A.I. Chatbot Tutors Could Upend Student Learning,” New York Times, June 8, 2023, https://www.nytimes.com/2023/06/08/business/khan-ai-gpt-tutoring-bot.html?searchResultPosition=3.
[67] “Why Was My Khanmigo Conversation Flagged?,” Khan Academy Help Center, March 10, 2023, https://support.khanacademy.org/hc/en-us/articles/13871273505933-Why-was-my-Khanmigo-conversation-flagged-.
[68] Daniel De Angulo, “Khan Academy’s Approach to Prompt Engineering for Khanmigo,” Khan Academy Blog, October 23, 2023, https://blog.khanacademy.org/khan-academys-7-step-approach-to-prompt-engineering-for-khanmigo/. See also “What Happens If My Child or Student’s Khanmigo Conversation Gets Flagged?,” Khan Academy Help Center, September 21, 2023, https://support.khanacademy.org/hc/en-us/articles/14394569357069-What-happens-if-my-child-or-student-s-Khanmigoconversation-gets-flagged-.
[69] Derek van der Vorst, “Audio Analytics: Enhancing IP Cameras with Intelligent Ears,” Secure Insights (blog), September 5, 2019, https://www.axis.com/blog/secure-insights/audio-analytics-ip-cameras/.
[70] “What Are Aggression Events?,” HALO Smart Sensor, accessed October 27, 2023, https://halodetect.com/docs/events/.
[71] Gillum and Kao, “Aggression Detectors.”
[72] Jeff Kao and Jack Gillum, “Reverse-Engineering an Audio Aggression Detection Algorithm,” 2020, https://bpb-usw2.wpmucdn.com/sites.northeastern.edu/dist/0/367/files/2020/02/CJ_2020_paper_52.pdf. See also Gillum and Kao, “Aggression Detectors.”
[73] Steve Levine, “Audio Sensor System Able to Warn Schools of Potential Violence inside Classrooms,” ABC News, April 28, 2022, https://abc6onyourside.com/news/local/audio-sensor-system-able-to-warn-schools-of-potential-violence-inside-classroomsworthington-columbus-ohio-safe-passage-consulting-april-28-2022.
[74] Levine, “Audio Sensor System.”
[75] Levine, “Audio Sensor System.”
[76] Karla Rendon-Alvarez, “Youth Campaign, School’s Smart Sensors Combat Teen Vaping,” NBC San Diego, February 27, 2020, https://www.nbcsandiego.com/news/local/youth-campaign-schools-smart-sensors-combat-teen-vaping/2274114/. See also “The Knowhere Capus Safety System,” (Drift Net Securities, July, 2020), https://campussafetyconference.com/wpcontent/uploads/2020/07/campus-safety-virtual-booth-and-bag.pdf. “Early County School System to Implement AI Security System,” Drift Net, accessed October 25, 2023, https://www.driftnet.com/resources/press/early-county-school-system-toimplement-ai-security-system.
[77] Gillum and Kao, “Aggression Detectors.” See also “Schools,” Sound Intelligence, accessed October 25, 2023, https://www.soundintel.com/markets/overview/education/.
[78] Milly Chan, “This AI Reads Children’s Emotions as They Learn CNN Business,” CNN, February 17, 2021, https://www.cnn.com/2021/02/16/tech/emotion-recognition-ai-education-spc-intl-hnk/index.html.
[79] Kate Kaye, “ACLU and Human Rights Groups Fight Zoom’s Emotion AI Plan - Protocol,” May 11, 2022, https://www.protocol.com/enterprise/zoom-emotion-ai-aclu-rights.
[80] Kate Kaye, “Class Tests Intel AI to Monitor Student Emotions on Zoom,” Protocol, April 17, 2022, https://www.protocol.com/enterprise/emotion-ai-school-intel-edutech.
[81] Lewis, “Microsoft Removing Emotion Recognition.”
[82] Lauren Rhue, “Racial Influence on Automated Perceptions of Emotions,” November 9, 2018, https://doi.org/10.2139/ssrn.328176. See also Lewis, “Microsoft Removing Emotion Recognition.”
[83] Taylor Telford, “‘Emotion Detection’ AI Is a $20 Billion Industry. New Research Says It Can’t Do What It Claims,” Washington Post, August 1, 2019, https://www.washingtonpost.com/business/2019/07/31/emotion-detection-ai-is-billion-industry-newresearch-says-it-cant-do-what-it-claims/.
[84] Taylor Telford, “‘Emotion Detection’ AI.”
[85] Albert Fox Cahn, Caroline Magee, Eleni Manis, and Naz Akyol. “Snooping Where We Sleep: The Invasiveness and Bias of Remote Proctoring Services.” Surveillance Technology Oversight Project, November 11, 2020, https://www.stopspying.org/snooping.
[86] Roth and Enzer, “How Tech Treats Students With Disabilities Like Criminals.” See also Michael Standaert, “Smile for the Camera: The Dark Side of China’s Emotion-Recognition Tech,” The Guardian, March 3, 2021, sec. Global development, https://www.theguardian.com/global-development/2021/mar/03/china-positive-energy-emotion-surveillance-recognitiontech.
[87] Opsis Emotion AI. “Solutions.” Accessed November 3, 2023. https://opsis.ai/solutions/.
[88] Kelly Davis and Danielle Fritze, “Young People’s Mental Health In 2020: Hope, Advocacy, and Action for the Future,” (Mental Health America, December 9, 2020), https://mhanational.org/sites/default/files/Young%20People's%20Mental%20Health%20Report%202020%20with%20Program%20Appendix%2012.8.20.pdf. Caitlynn Peetz, “Kids’ Declining Mental Health Is the ‘Crisis of Our Time,’ Surgeon General Says,” Education Week, April 25, 2023, https://www.edweek.org/leadership/kids-declining-mental-health-is-the-crisis-of-our-time-surgeon-generalsays/2023/04.
[89] “Youth Risk Behavior Survey,” Centers for Disease Control and Prevention.
[90] “Youth Risk Behavior Survey,” Centers for Disease Control and Prevention.
[91] “Youth Risk Behavior Survey,” Centers for Disease Control and Prevention.
[92] Lakshmi Radhakrishnan, Kelly Carey, Kathleen P. Hartnett, et al., “Pediatric Emergency Department Visits Before and During the COVID-19 Pandemic — United States, January 2019–January 2022,” (Centers for Disease Control and Prevention, February 25, 2022), http://dx.doi.org/10.15585/mmwr.mm7108e1.
[93] See, for example, Mark Keierleber, “‘Don’t Get Gaggled’: Minneapolis School District Spends Big on Student Surveillance Tool, Raising Ire After Terminating Its Police Contract,” October 18, 2020, https://www.the74million.org/article/dont-get-gaggledminneapolis-school-district-spends-big-on-student-surveillance-tool-raising-ire-after-terminating-its-police-contract/.
[94] Masakazu Hojo, “Association between Student-Teacher Ratio and Teachers’ Working Hours and Workload Stress: Evidence from a Nationwide Survey in Japan,” BMC Public Health 21, no. 1 (2021): 1635, https://doi.org/10.1186/s12889-021-11677-w. See also Stephanie Burroughs, “To Build Strong Student-Teacher Bonds, Keep Class Size Small,” ASCD (blog), June 11, 2021, https://www.ascd.org/blogs/to-build-strong-student-teacher-bonds-keep-class-size-small. See also “Supporting Child and Student Social, Emotional, Behavioral, and Mental Health Needs” (U.S. Department of Education, Office of Special Education and Rehabilitative Services, 2021), https://www2.ed.gov/documents/students/supporting-child-student-social-emotionalbehavioral-mental-health.pdf.
[95] Mona Wang and Gennie Gebhart, The Electronic Frontier Foundation (blog), February 27, 2020, https://www.eff.org/deeplinks/2020/02/schools-are-pushing-boundaries-surveillance-technologies.
Social Media Graphics
