New York City鈥檚 free online therapy platform for teens may violate state and federal laws protecting student data privacy, lawyers from the New York Civil Liberties Union and advocates charged in a letter Tuesday to the city鈥檚 Education and Health Departments.

, a $26 million partnership between the city Health Department and teletherapy giant Talkspace launched in late 2023, connects city residents between ages 13 and 17 with free therapists by text, phone, or video chat.

In less than a year, roughly 16,000 students have signed up, Health Department officials said. Sign-ups disproportionately came from youth who identified as Black, Latino, Asian American and female and live in some of the city鈥檚 lowest-income neighborhoods, .

Information shared with a therapist is subject to stringent protections under the federal Health Insurance Portability and Accountability Act, or HIPAA. But before connecting with a therapist through Teenspace, teens go through a registration process that asks for personal information like their name, school, mental health history, and gender identity. Advocates are concerned such information is being improperly collected and could be misused.

For one, teens enter the registration information before securing parental consent 鈥� a possible violation of federal student privacy laws, the letter contends.

And families don鈥檛 get a chance to review the privacy policy 鈥� which discloses that registration information can be used to 鈥渢ailor advertising鈥� and for marketing purposes 鈥� before entering the registration information, advocates allege. There鈥檚 an option for teens to request that their data be deleted from the company鈥檚 platform, but it鈥檚 hard to find, according to advocates.

鈥淚t鈥檚 all very invasive,鈥� said Shannon Edwards, a parent and founder of AI For Families, an organization that seeks to help families navigate artificial intelligence, who co-authored the letter along with NYCLU and the Parent Coalition for Student Privacy. 鈥淚t鈥檚 also very unclear that parents understand what they鈥檙e getting themselves into.鈥�

Advocates also pointed to the risk of a potential data breach 鈥� something the city has in recent years.

Advocates say similar about have been circulating for years and questioned whether city officials did sufficient due diligence or built in enough additional privacy safeguards before inking the contract.

鈥淚t鈥檚 the opacity of the relationship here, and the failure to make manifest what the city is doing to ensure there isn鈥檛 this data accumulation and sharing for inappropriate purposes,鈥� said Beth Haroules, a senior attorney at the NYCLU who co-authored the letter.

Health Department spokesperson Rachel Vick said the agency has 鈥渢aken additional steps to protect the data of Teenspace users and ensure information is not collected for personal gain, including stipulations that require all client data to remain confidential during and after the completion of the city鈥檚 contract and barring use of data for any purpose other than providing the services included in the contract.鈥�

Client data is destroyed after 30 days if a teen doesn鈥檛 connect with a therapist, officials said.

A spokesperson for Talkspace referred questions to the Health Department.

The extent to which Teenspace is subject to state and federal laws governing student privacy in educational settings is somewhat murky, given that the contract is with the city鈥檚 Health Department, not its Education Department.

But NYCLU attorneys contend 鈥渢he City cannot absolve itself of its responsibility to provide the protections inherent in federal and state laws鈥imply because the contract sits with DOHMH instead of DOE. The service is promoted on public school websites, and it is DOE鈥檚 responsibility to ensure that student data is protected, regardless of which City agency signs the contract.鈥�

Parents may be more inclined to trust the platform because it has a 鈥渟tamp of approval鈥� from the school system, Edwards added.

A Health Department spokesperson didn鈥檛 specify whether the program is subject to education privacy laws, but said it鈥檚 鈥渘ot a school based service.鈥�

Teenspace has been the city鈥檚 highest-profile effort to address the ongoing youth mental health crisis.

鈥淲e are meeting people where they are with a front door to the mental health system that for too long has been too hard to find,鈥� said Ashwin Vasan, the city鈥檚 health commissioner, in May.

Some teens have praised the program, noting it鈥檚 a way to bring mental health care to young people who may not otherwise have access.

But some mental health providers have argued it can鈥檛 replace the kind of intensive care a clinician provides, especially for kids with severe mental health challenges.

Company officials shared in May that they had helped 36 teens navigate serious incidents including reports of suicide attempts and abuse 鈥� cases they referred to child protective services, in-person therapists, or hospitals.

Talkspace CEO Jon Cohen previously told Chalkbeat the company uses an artificial intelligence algorithm to scan transcripts of therapy sessions to help identify teens at risk of suicide.

Even advocates critical of Teenspace鈥檚 privacy protections acknowledge the severe shortage of mental health providers and say teletherapy can play a role in filling the gap.

鈥淲e know you cannot find providers 鈥� there is such a need,鈥� said Haroules. But advocates said the city can do more to ensure its vendors are meeting strict standards for data privacy, especially with such sensitive information.

鈥淓veryone thinks, well, mental health is important for kids, these kids of services are required 鈥� when on the other side is: 鈥楬ow are they getting to it?鈥欌�� said Edwards. 鈥淚t doesn鈥檛 matter what the app is, there has to be a standard.鈥�

This was originally published by Chalkbeat. Chalkbeat is a nonprofit news site covering educational change in public schools. Sign up for their newsletters at .

Update: The bill was ordered to the inactive file on the last night of the session Aug. 31. It had been amended to keep mandatory police notification requirements if a student assaults or threatens a teacher. The bill would have still let teachers choose to call the police if a student is using or possessing controlled substances, and it would also decriminalize willful disturbance by students.

During Zuleima Baquedano鈥檚 first year as a teacher, she faced an important choice. 

One of her students had difficulty controlling her emotions. One day, she had a meltdown and kicked Baquedano down.

The principal asked Baquedano if she wanted to call the police, because the incident legally counted as assault. But not long before, the student had moved in with her family after being in and out of foster care, was beginning the diagnostic process for her disability and had been working with Baquedano on coping mechanisms. 

鈥淎ny contact with police would have really put all of that in jeopardy,鈥� Baquedano said. 鈥淐alling the police, getting Child Protective Services involved and all that would have completely just ruined any kind of progress she鈥檇 made.鈥� 

Baquedano decided against calling the police. 鈥淚鈥檓 never going to regret advocating for her, despite the fact that several teachers told me I couldn鈥檛 let her get away with it, and that she did this on purpose when they didn鈥檛 even know her,鈥� she said. 

She had a choice because she worked at a charter school in Los Angeles. Staff at traditional public schools don鈥檛 have the same freedom: Under California law, they are required to make a police report if a student assaults them 鈥� and can be prosecuted if they don鈥檛. 

A bill before the Legislature in its final week . 

But what supporters see as a common sense bill, opponents see as going too far, raising partisan tensions in an election year in which crime and education are top of mind for many voters. 

A difficult path to the Senate

, a San Jose Democrat, has been trying to get similar legislation passed for four years.

鈥淭he data very clearly shows that when law enforcement is required to come onto campus, those that they choose to arrest are disproportionately people with disabilities and students of color,鈥� Kalra said in an interview. 

 found that students with disabilities make up 26% of school arrests, despite being 11% of total enrollment. According to a , students of color are handcuffed by police at a disproportionate rate 鈥� 20% of Black students compared to 9% of white students. 

鈥淭his bill is really a turning point in addressing issues around school climate,鈥� said Oscar Lopez, an associate managing attorney at Disability Rights California, a sponsor of the bill. 

This is the first time Kalra鈥檚 bill has made it to the Senate, and it wasn鈥檛 easy. It barely squeaked out of the Assembly by a vote of just 41-22, with seven Democrats voting 鈥渘o.鈥� 

鈥淚t鈥檚 unfortunate that a common sense bill like this has struggled so hard to make it through the Legislature,鈥� Kalra said. 

And opposition is organizing.

Last week, Senate Republicans , listing concerns about school safety, drug possession and the relationship between schools and law enforcement. 

鈥淭he bottom line is this is going to make our school campuses less safe,鈥� Senate GOP Leader  of San Diego told CalMatters. 鈥淚t鈥檚 going to endanger our students, teachers, administrators and even the law enforcement professionals who have to serve on these campuses.鈥�

Law enforcement officials worry that AB 2441 could open the door to eliminating school resource officers. 

鈥淪chool officials and law enforcement should work together, especially when it comes to pupils whose behavior violates the law and puts school safety in jeopardy,鈥� said Cory Salzillo, legislative director of the California State Sheriffs鈥� Association. 鈥淩emoving requirements just runs counter to that notion.鈥�

If AB 2441 were to pass, there would still still be times when staff are required to call the police. Under federal law, local education agencies must call law enforcement if a student has a firearm or is caught selling controlled substances. 

Some opponents have also raised concerns about school administrators鈥� ability to discern between students who are selling controlled substances or just possessing them 鈥� a task they think should be left to law enforcement, particularly .

鈥淪chools are not isolated in the community, so when there are crimes being committed, even if it鈥檚 simple possession of a controlled substance, that鈥檚 something that law enforcement should be aware of,鈥� Salzillo said. 

The California Department of Public Health plans to announce a new fentanyl education campaign on Wednesday. 

鈥淔entanyl is so dangerous that we need to be all hands on deck on dealing with that crisis on our school campuses,鈥� Jones said. 鈥淩emoving this requirement of reporting is just unbelievable to me at this point in time.鈥� 

Because of an amendment to the bill, staff would also need to notify law enforcement if someone needed immediate medical attention. 

After the Senate Republican Caucus released its analysis 鈥� and sent it to its entire press list for the first time 鈥� supporters of the bill accused them of fear mongering and spreading misinformation. 

鈥淭here鈥檚 been a lot of untruths shared and promoted by the opposition to this bill,鈥� said Rachel Bhagwat, legislative advocate at ACLU California Action, a bill sponsor. 

Jones denied that鈥檚 what鈥檚 happening. 

鈥淐alifornia voters and taxpayers are fed up with the criminal justice system in California right now,鈥� he said. 鈥淭hey鈥檙e fed up with the progressive wing that鈥檚 continuing to decriminalize crime.鈥� 

Preventing the school-to-prison pipeline

 that when young people face severe discipline at school 鈥� such as police interaction, suspension or expulsion 鈥� they are less likely to graduate high school and more likely to go to prison. 

鈥淭he interpretation of normal, age-appropriate behaviors as being threatening and criminal and dangerous is leading to a situation where young people are not getting educational opportunities in school, and they鈥檙e being funneled into further criminal contact and the criminal system,鈥� Bhagwat said. 

Under current state law, staff are required to try other methods 鈥� such as meeting with parents, speaking with a psychologist, creating an individualized education plan or restorative justice programs 鈥� before resorting to something more severe. 

鈥淏etween counseling and other programs, there are methods to use that don鈥檛 involve punitive consequences such as a misdemeanor crime,鈥� Naj Alikhan, senior director of marketing and communications for the Association of California School Administrators, wrote in a statement to CalMatters.

The bill would also get rid of a clause that makes it a crime to 鈥渨illfully disturb鈥� public schools and meetings. Under this provision, students could be criminally prosecuted for running in hallways or knocking on doors. 

鈥淚t鈥檚 somewhat of a vague term,鈥� Kalra said, 鈥渁nd it鈥檚 been used against students who might have behavior issues. There鈥檚 a lot of different reasons why a student may be causing a disturbance and we want to give schools the ability to decide how they want to handle those situations.鈥� 

An amendment to the bill would make it an infraction for someone to prevent a school staff member from calling the police. 

Baquedano 鈥� who  before the Senate education committee in July and now teaches in Santa Ana 鈥� said that if the bill passes, there are serious situations, like having a deadly weapon or being in possession of drugs, where she would still call.  

鈥淭here鈥檚 an assumption that we鈥檙e going to stop calling the police, and that鈥檚 not the case,鈥� she said. 鈥淭he idea that we wouldn鈥檛 have that common sense is a little insulting.鈥� 

It鈥檚 a decision Baquedano said teachers deserve to have. 

鈥淧eople should trust us 鈥� the professionals in the situation, who鈥檝e been trained, who鈥檝e gone through education to do this 鈥� they should be trusting our judgment,鈥� she said. 鈥淲e鈥檙e the ones who best know our students. We spend all these hours with them a year, sometimes more than parents do.鈥�

Kalra remains optimistic that AB 2441 will pass the Senate this week and make it to Gov. Gavin Newsom鈥檚 desk. 

鈥淵ou would hope,鈥� he said, 鈥渢hat legislators would understand the need for us to support all students, and I鈥檓 hopeful that at least we can get this bill through to see that it鈥檚 not going to create some doomsday outcome.鈥�

This was originally published on .

As the eight-year-old startup rolled out Los Angeles Unified School District鈥檚 flashy new AI-driven chatbot 鈥� an animated sun named 鈥淓d鈥� that AllHere was hired to build for $6 million 鈥� a former company executive was sending emails to the district and others that Ed鈥檚 workings violated bedrock student data privacy principles. 

Those emails were sent shortly before 蜜桃影视 first reported last week that AllHere, with in investor capital, was in serious straits. A June 14 statement on the company’s website revealed a majority of its employees had been furloughed due to its 鈥渃urrent financial position.” Company founder and CEO Joanna Smith-Griffin, a spokesperson for the Los Angeles district said, was no longer on the job. 

Smith-Griffin and L.A. Superintendent Alberto Carvalho went on the road together this spring to unveil Ed at a series of high-profile ed tech conferences, with the schools chief dubbing it the nation鈥檚 first 鈥減ersonal assistant鈥� for students and leaning hard into LAUSD鈥檚 place in the K-12 AI vanguard. He called Ed鈥檚 ability to know students 鈥渦nprecedented in American public education鈥� at the ASU+GSV conference in April. 

Through an algorithm that analyzes troves of student information from multiple sources, the chatbot was designed to offer tailored responses to questions like 鈥渨hat grade does my child have in math?鈥� The tool relies on vast amounts of students鈥� data, including their academic performance and special education accommodations, to function.

Meanwhile, Chris Whiteley, a former senior director of software engineering at AllHere who was laid off in April, had become a whistleblower. He told district officials, its independent inspector general’s office and state education officials that the tool processed student records in ways that likely ran afoul of L.A. Unified鈥檚 own data privacy rules and put sensitive information at risk of getting hacked. None of the agencies ever responded, Whiteley told 蜜桃影视. 

鈥淲hen AllHere started doing the work for LAUSD, that鈥檚 when, to me, all of the data privacy issues started popping up,鈥� Whiteley said in an interview last week. The problem, he said, came down to a company in over its head and one that 鈥渨as almost always on fire鈥� in terms of its operations and management. LAUSD鈥檚 chatbot was unlike anything it had ever built before and 鈥� given the company鈥檚 precarious state 鈥� could be its last. 

If AllHere was in chaos and its bespoke chatbot beset by porous data practices, Carvalho was portraying the opposite. One day before 蜜桃影视 broke the news of the company turmoil and Smith-Griffin鈥檚 departure, spotlighted the schools chief at a Denver conference talking about how adroitly LAUSD managed its ed tech vendor relationships 鈥� 鈥淲e force them to all play in the same sandbox鈥� 鈥� while ensuring that 鈥減rotecting data privacy is a top priority.鈥�

In a statement on Friday, a district spokesperson said the school system 鈥渢akes these concerns seriously and will continue to take any steps necessary to ensure that appropriate privacy and security protections are in place in the Ed platform.鈥� 

鈥淧ursuant to contract and applicable law, AllHere is not authorized to store student data outside the United States without prior written consent from the District,鈥� the statement continued. 鈥淎ny student data belonging to the District and residing in the Ed platform will continue to be subject to the same privacy and data security protections, regardless of what happens to AllHere as a company.鈥� 

A district spokesperson, in response to earlier questioning from 蜜桃影视 last week, said it was informed that Smith-Griffin was no longer with the company and that several businesses 鈥渁re interested in acquiring AllHere.鈥� Meanwhile Ed, the spokesperson said, 鈥渂elongs to Los Angeles Unified and is for Los Angeles Unified.鈥�

Officials in the inspector general鈥檚 office didn鈥檛 respond to requests for comment. The state education department “does not directly oversee the use of AI programs in schools or have the authority to decide which programs a district can utilize,” a spokesperson said in a statement.

It鈥檚 a radical turn of events for AllHere and the AI tool it markets as a 鈥渓earning acceleration platform,鈥� which were all the buzz just a few months ago. In April, Time Magazine education technology companies. That same month, Inc. Magazine dubbed Smith-Griffin in artificial intelligence in its Female Founders 250 list. 

Joanna Smith-Griffin, Founder and CEO of AllHere, joins on stage as he delivers the keynote at on Unlocking the Potential of AI. With Ed, students receive personalized support to help them excel in school.

鈥� allherek12 (@allherek12)

Ed has been similarly blessed with celebrity treatment. 

鈥淗e鈥檚 going to talk to you in 100 different languages, he鈥檚 going to connect with you, he鈥檚 going to fall in love with you,鈥� Carvalho said at ASU+GSV. 鈥淗opefully you鈥檒l love it, and in the process we are transforming a school system of 540,000 students into 540,000 鈥榮chools of one鈥� through absolute personalization and individualization.鈥�

Smith-Griffin, who graduated from the Miami school district that Carvalho once led before going onto Harvard, couldn鈥檛 be reached for comment. Smith-Griffin鈥檚 LinkedIn page was recently deactivated and parts of the company website have gone dark. Attempts to reach AllHere were also unsuccessful.

鈥楾he product worked, right, but it worked by cheating鈥�

Smith-Griffin, a former Boston charter school teacher and family engagement director, founded AllHere in 2016. Since then, the company has primarily provided schools with a text messaging system that facilitates communication between parents and educators. , the tool relies on attendance data and other information to deliver customized, text-based 鈥渘udges.鈥� 

The work that AllHere provided the Los Angeles school district, Whiteley said, was on a whole different level 鈥� and the company wasn鈥檛 prepared to meet the demand and lacked expertise in data security. In L.A., AllHere operated as a consultant rather than a tech firm that was building its own product, according to its contract with LAUSD obtained by 蜜桃影视. Ultimately, the district retained rights to the chatbot, according to the agreement, but AllHere was contractually obligated to 鈥渃omply with the district information security policies.鈥� 

聽The contract notes that the chatbot would be 鈥渢rained to detect any confidential or sensitive information鈥� and to discourage parents and students from sharing with it any personal details. But the chatbot鈥檚 decision to share and process students鈥� individual information, Whiteley said, was outside of families鈥� control.聽

In order to provide individualized prompts on details like student attendance and demographics, the tool connects to several data sources, according to the contract, including , an online tool used to track students鈥� special education services. The document notes that Ed also interfaces with the stored on , a cloud storage company. , the Whole Child platform serves as a central repository for LAUSD student data to help educators monitor students鈥� progress and personalize instruction. 

Whiteley told officials the app included students鈥� personally identifiable information in all chatbot prompts, even in those where the data weren鈥檛 relevant. Prompts containing students鈥� personal information were also shared with other third-party companies unnecessarily, Whiteley alleges, and were processed on offshore servers. Seven out of eight Ed chatbot requests, he said, are sent to places like Japan, Sweden, the United Kingdom, France, Switzerland, Australia and Canada. 

Taken together, he argued the company鈥檚 practices ran afoul of data minimization principles, a standard cybersecurity practice that maintains that apps should collect and process the least amount of personal information necessary to accomplish a specific task. Playing fast and loose with the data, he said, unnecessarily exposed students鈥� information to potential cyberattacks and data breaches and, in cases where the data were processed overseas, could subject it to foreign governments鈥� data access and surveillance rules. 

Chatbot source code that Whiteley shared with 蜜桃影视 outlines how prompts are processed on foreign servers by a Microsoft AI service that integrates with ChatGPT. The LAUSD chatbot is directed to serve as a 鈥渇riendly, concise customer support agent鈥� that replies 鈥渦sing simple language a third grader could understand.鈥� When querying the simple prompt 鈥淗ello,鈥� the chatbot provided the student鈥檚 grades, progress toward graduation and other personal information. 

AllHere鈥檚 critical flaw, Whiteley said, is that senior executives 鈥渄idn鈥檛 understand how to protect data.鈥� 

鈥淭he issue is we鈥檙e sending data overseas, we鈥檙e sending too much data, and then the data were being logged by third parties,鈥� he said, in violation of the district鈥檚 data use agreement. 鈥淭he product worked, right, but it worked by cheating. It cheated by not doing things right the first time.鈥�

In a 2017 policy bulletin, the district notes that all sensitive information 鈥渘eeds to be handled in a secure way that protects privacy,鈥� and that contractors cannot disclose information to other parties without parental consent. A second policy bulletin, from April, outlines the district鈥檚 authorized use guidelines for artificial intelligence, which notes that officials, 鈥淪hall not share any confidential, sensitive, privileged or private information when using, prompting or communicating with any tools.鈥� It鈥檚 important to refrain from using sensitive information in prompts, the policy notes, because AI tools 鈥渢ake whatever users enter into a prompt and incorporate it into their systems/knowledge base for other users.鈥� 

鈥淲ell, that鈥檚 what AllHere was doing,鈥� Whiteley said. 

鈥楢cid is dangerous鈥�

Whiteley鈥檚 revelations present LAUSD with its third student data security debacle in the last month. In mid-June, a threat actor known as 鈥淪p1d3r鈥� began to sell for $150,000 a trove of data it claimed to have stolen from the Los Angeles district on Breach Forums, a dark web marketplace. LAUSD Bloomberg that the compromised data had been stored by one of its third-party vendors on the cloud storage company Snowflake, the repository for the district鈥檚 Whole Child Integrated Data. The Snowflake data breach may be one of the largest in history. The threat actor claims that the L.A. schools data in its possession include student medical records, disability information, disciplinary details and parent login credentials. 

The chatbot interacted with data stored by Snowflake, according to the district鈥檚 contract with AllHere, though any connection between AllHere and the Snowflake data breach is unknown. 

In its statement Friday, the district spokesperson said an ongoing investigation has 鈥渞evealed no connection between AllHere or the Ed platform and the Snowflake incident.鈥� The spokesperson said there was no 鈥渄irect integration鈥� between Whole Child and AllHere and that Whole Child data was processed internally before being directed to AllHere.

The contract between AllHere and the district, however, notes that the tool should 鈥渟eamlessly integrate鈥� with the Whole Child Integrated Data 鈥渢o receive updated student data regarding attendance, student grades, student testing data, parent contact information and demographics.鈥�

Earlier in the month, a second threat actor known as Satanic Cloud claimed it had access to tens of thousands of L.A. students鈥� sensitive information and had posted it for sale on Breach Forums for $1,000. In 2022, the district was victim to a massive ransomware attack that exposed reams of sensitive data, including thousands of students鈥� psychological evaluations, to the dark web. 

With AllHere鈥檚 fate uncertain, Whiteley blasted the company鈥檚 leadership and protocols.

鈥淧ersonally identifiable information should be considered acid in a company and you should only touch it if you have to because acid is dangerous,鈥� he told 蜜桃影视. 鈥淭he errors that were made were so egregious around PII, you should not be in education if you don鈥檛 think PII is acid.鈥� 

L.A. parents and students, we want to hear from you.  using AllHere鈥檚 Ed:

, the coalition expressed concerns that AI-enabled student monitoring tools could foster discrimination against marginalized groups, including LGBTQ+ youth and students with disabilities. The Education Department鈥檚 Office for Civil Rights should issue guidance on the appropriate uses of emerging classroom technologies, the lawmakers wrote, and crack down on practices that run afoul of existing federal anti-discrimination laws. 

鈥淲hile the expansion of educational technology helped facilitate remote learning that was critical to students, parents and teachers during the pandemic,鈥� the lawmakers wrote, 鈥渢hese technologies have also amplified student harms.鈥� 

Lawmakers asked the Education Department鈥檚 civil rights office whether it has received complaints alleging discrimination facilitated by education technology software and whether it has taken any enforcement action related to potential civil rights violations. 

The letter comes in response to a recent national survey of educators, parents and students, the findings of which suggest that schools鈥� use of digital tools to monitor children online have based on their race, disability, sexual orientation and gender identity. The survey, conducted by the nonprofit Center for Democracy and Technology, found that while activity monitoring has become ubiquitous in schools and is intended to keep students safe, it鈥檚 used regularly as a discipline tool and routinely brings youth into contact with the police.

Findings from the CDT survey, lawmakers wrote, 鈥渞aise serious concerns about the application of civil rights laws to schools鈥� use of these technologies.鈥� Letter signatories include Democratic Reps. Lori Trahan of Massachusetts, Sara Jacobs of California, Hank Johnson of Georgia, Bonnie Watson Coleman of New Jersey and Adam Schiff of California. Trahan, who serves on the House Energy and Commerce Committee鈥檚 Innovation, Data and Commerce Subcommittee, has previously called for tighter student data privacy protections in the ed tech sector. 

The monitoring tools, such as those offered by for-profit companies GoGuardian and Gaggle, rely on artificial intelligence to sift through students鈥� online activities and flag school administrators 鈥� and sometimes the police 鈥� when they discover materials related to sex, drugs, violence or self-harm. 

Two-thirds of teachers reported that a student at their school was disciplined as a result of activity monitoring and a third said they know a student who was contacted by the police because of an alert generated by the software. 

Children with disabilities were more likely than their peers to report being watched, and special education teachers reported heightened rates of discipline as a result of activity monitoring. The findings, researchers argue, that entitle children with disabilities equal access to an education. Even beyond the technologies, students with disabilities are subjected to disproportionate levels of school discipline, including restraint and seclusion, when compared to their general education peers. 

Half of all students said their schools responded fairly to alerts generated by monitoring software, a sentiment shared by just 36% of LGBTQ+ youth. In fact, LGBTQ+ youth were more likely than their straight and cisgender peers to report that they or someone they know was disciplined as a result of monitoring. And nearly a third of LGBTQ+ youth reported that they or someone they know was outed because of the technology. 

More than a third of teachers said their school monitors students鈥� online behaviors outside of school hours 鈥� and sometimes on their personal devices. 

In a similar student survey, released this month by the American Civil Liberties Union, a majority of respondents expressed worries that the monitoring tools 鈥� despite being designed to keep them safe 鈥� could actually cause harm and a third said they 鈥渁lways feel鈥� like they鈥檙e being watched. 

蜜桃影视 has reported extensively on schools鈥� use of digital surveillance tools to monitor students鈥� online behaviors, and the tools鈥� implications for youth civil rights. The company Gaggle previously flagged to administrators student communications that referenced LGBTQ+ keywords like 鈥済ay鈥� and 鈥渓esbian.鈥� The company says it halted the practice last year in the wake of pushback from civil rights activists. 

Given the survey findings, the lawmakers urged the Education Department to clarify 鈥渉ow educators can fulfill their civil rights obligations鈥� as they develop policies related to artificial intelligence, whose rapidly evolving role in education more broadly 鈥� including students鈥� use of tools like ChatGPT 鈥� has become a topic of debate. 

鈥淭his research is particularly concerning due to linkages between school disciplinary policies and incarceration rates of our nation鈥檚 youth,鈥� the coalition wrote, adding concerns that the tools can create hostile learning environments. 

Half of teachers say they know a student at their school who was disciplined or faced negative consequences for using 鈥� or being accused of using 鈥� generative artificial intelligence like ChatGPT to complete a classroom assignment, , a nonprofit think tank focused on digital rights and expression. The proportion was even higher, at 58%, for those who teach special education. 

Cheating concerns were clear, with survey results showing that teachers have grown suspicious of their students. Nearly two-thirds of teachers said that generative AI has made them 鈥渕ore distrustful鈥� of students and 90% said they suspect kids are using the tools to complete assignments. Yet students themselves who completed the anonymous survey said they rarely use ChatGPT to cheat, but are turning to it for help with personal problems.

鈥淭he difference between the hype cycle of what people are talking about with generative AI and what students are actually doing, there seems to be a pretty big difference,鈥� said Elizabeth Laird, the group鈥檚 director of equity in civic technology. 鈥淎nd one that, I think, can create an unnecessarily adversarial relationship between teachers and students.鈥�   

Indeed, 58% of students, and 72% of those in special education, said they鈥檝e used generative AI during the 2022-23 academic year, just not primarily for the reasons that teachers fear most. Among youth who completed the nationally representative survey, just 23% said they used it for academic purposes and 19% said they鈥檝e used the tools to help them write and submit a paper. Instead, 29% reported having used it to deal with anxiety or mental health issues, 22% for issues with friends and 16% for family conflicts.

Part of the disconnect dividing teachers and students, researchers found, may come down to gray areas. Just 40% of parents said they or their child were given guidance on ways they can use generative AI without running afoul of school rules. Only 24% of teachers say they鈥檝e been trained on how to respond if they suspect a student used generative AI to cheat. 

The results on ChatGPT鈥檚 educational impacts were included in the Center for Democracy and Technology鈥檚 broader annual survey analyzing the privacy and civil rights concerns of teachers, students and parents as tech, including artificial intelligence, becomes increasingly engrained in classroom instruction. Beyond generative AI, researchers observed a sharp uptick in digital privacy concerns among students and parents over last year. 

Among parents, 73% said they鈥檙e concerned about the privacy and security of student data collected and stored by schools, a considerable increase from the 61% who expressed those reservations last year. A similar if less dramatic trend was apparent among students: 62% had data privacy concerns tied to their schools, compared with 57% just a year earlier. 

Those rising levels of anxiety, researchers theorized, are likely the result of the growing frequency of cyberattacks on schools, which have become a primary target for ransomware gangs. High-profile breaches, including in Los Angeles and Minneapolis, have compromised a massive trove of highly sensitive student records. Exposed records, investigative reporting by 蜜桃影视 has found, include student psychological evaluations, reports detailing campus rape cases, student disciplinary records, closely guarded files on campus security, employees鈥� financial records and copies of government-issued identification cards. 

Survey results found that students in special education, whose records are among the most sensitive that districts maintain, and their parents were significantly more likely than the general education population to report school data privacy and security concerns. As attacks ratchet up, 1 in 5 parents say they鈥檝e been notified that their child鈥檚 school experienced a data breach. Such breach notices, Laird said, led to heightened apprehension. 

鈥淭here鈥檚 not a lot of transparency鈥� about school cybersecurity incidents 鈥渂ecause there鈥檚 not an affirmative reporting requirement for schools,鈥� Laird said. But in instances where parents are notified of breaches, 鈥渢hey are more concerned than other parents about student privacy.鈥� 

Parents and students have also grown increasingly wary of another set of education tools that rely on artificial intelligence: digital surveillance technology. Among them are student activity monitoring tools, such as those offered by the for-profit companies Gaggle and GoGuardian, which rely on algorithms in an effort to keep students safe. The surveillance software employs artificial intelligence to sift through students鈥� online activities and flag school administrators 鈥� and sometimes the police 鈥� when they discover materials related to sex, drugs, violence or self-harm. 

Among parents surveyed this year, 55% said they believe the benefits of activity monitoring outweigh the potential harms, down from 63% last year. Among students, 52% said they鈥檙e comfortable with academic activity monitoring, a decline from 63% last year. 

Such digital surveillance, researchers found, frequently has disparate impacts on students based on their race, disability, sexual orientation and gender identity, potentially violating longstanding federal civil rights laws. 

The tools also extend far beyond the school realm, with 40% of teachers reporting their schools monitor students鈥� personal devices. More than a third of teachers say they know a student who was contacted by the police because of online monitoring, the survey found, and Black parents were significantly more likely than their white counterparts to fear that information gleaned from online monitoring tools and AI-equipped campus surveillance cameras could fall into the hands of law enforcement. 

Meanwhile, as states nationwide pull literature from school library shelves amid a conservative crusade against LGBTQ+ rights, the nonprofit argues that digital tools that filter and block certain online content 鈥渃an amount to a digital book ban.鈥� Nearly three-quarters of students 鈥� and disproportionately LGBTQ+ youth 鈥� said that web filtering tools have prevented them from completing school assignments. 

The nonprofit highlights how disproportionalities identified in the survey could run counter to federal laws that prohibit discrimination based on race and sex, and those designed to ensure equal access to education for children with disabilities. In a letter sent Wednesday to the White House and Education Secretary Miguel Cardona, the Center for Democracy and Technology was joined by a coalition of civil rights groups urging federal officials to take a harder tack on ed tech practices that could threaten students鈥� civil rights. 

鈥淓xisting civil rights laws already make schools legally responsible for their own conduct, and that of the companies acting at their direction in preventing discriminatory outcomes on the basis of race, sex and disability,鈥� the coalition wrote. 鈥淭he department has long been responsible for holding schools accountable to these standards.鈥�

Since the pandemic began, Baltimore City Public Schools officials have with GoGuardian, a digital surveillance tool that promises to identify youth at risk of harming themselves or others. When GoGuardian flags students, their online activities are shared automatically with school police, giving cops a conduit into kids鈥� private lives 鈥� including on nights and weekends.

Such partnerships between schools and police appear startlingly widespread across the country with significant implications for youth, according to . Nearly all teachers 鈥� 89% 鈥� reported that digital student monitoring tools like GoGuardian are used in their schools. And nearly half 鈥� 44% 鈥� said students have been contacted by the police as a result of student monitoring. 

The pandemic has led to major growth in the number of schools that rely on activity monitoring software to uncover student references to depression and violent impulses. The tools, offered by a handful of tech companies, can sift through students鈥� social media posts, follow their digital movements in real-time and scan files on school-issued laptops 鈥� from classroom assignments to journal entries 鈥� in search of warning signs. 

Educators say the tools help them identify youth who are struggling and get them the mental health care they need at a time when youth depression and anxiety are spiraling. But the survey suggests an alternate reality: Instead of getting help, many students are being punished for breaking school rules. And in some cases, survey results suggest, students are being subjected to discrimination. 

The report raises serious questions about whether digital surveillance tools are the best way to identify youth in need of mental health care and whether police officers should be on the front lines in responding to such emergencies. 

鈥淚f we鈥檙e saying this is to keep students safe, but instead we鈥檙e using it punitively and we鈥檙e using it to invite law enforcement literally into kids鈥� homes, is this actually achieving its intended goal?鈥� asked Elizabeth Laird, a survey author and the center鈥檚 director of equity in civic technology. 鈥淥r are we, in the name of keeping students safe, actually endangering them?鈥�

Among teachers who use monitoring tools at their schools, 78% said the software has been used to flag students for discipline and 59% said kids wound up getting punished as a result. Yet just 45% of teachers said the software is used to identify violent threats and 47% said it is used to identify students at risk of harming themselves. 

The findings are a direct contradiction of the stated goal of student activity monitoring, Laird said. School leaders and company executives have long maintained that the tools are not a disciplinary measure but are designed to identify at-risk students before someone gets hurt.

The Supreme Court鈥檚 recent repeal of Roe v. Wade, she said, further muddles police officers鈥� role in student activity monitoring. As states implement anti-abortion laws, that data from student activity monitoring tools could help the police identify youth seeking reproductive health care. 

鈥淲e know that law enforcement gets these alerts,鈥� she said. 鈥淚f you are in a state where they are looking to investigate these kinds of incidents, you鈥檝e invited them into a student鈥檚 house to be able to do that.鈥�

A tale of discrimination

In Baltimore, counselors, principals and school-based police officers receive all alerts generated by GoGuardian during school hours, according to by The Real News Network, a nonprofit media outlet. Outside of school hours, including on weekends and holidays, the responsibility to monitor alerts falls on the police, the outlet reported, and on numerous occasions officers have shown up at students鈥� homes to conduct wellness checks. On , students have been transported to the hospital for emergency mental health care. 

In a statement to 蜜桃影视, district spokesperson Andre Riley said that GoGuardian helps officials 鈥渋dentify potential risks to the safety of individual students, groups or schools,鈥� and that 鈥減roper accountability measures are taken鈥� if students violate the code of conduct or break laws.

鈥淭he use of GoGuardian is not simply a prompt for a law enforcement response,鈥� Riley added.

Leading student surveillance companies, including GoGuardian, have maintained that their interactions with police are limited. In April, Democratic Sens. Elizabeth Warren and Ed Markey warned in a report that schools鈥� reliance on the tools could violate students鈥� civil rights and exacerbate 鈥渢he school-to-prison pipeline by increasing law enforcement interactions with students.鈥� Warren and Markey focused their report on four companies: GoGuardian, Gaggle, Securly and Bark. 

In , Gaggle executives said the company contacts law enforcement for wellness checks if they are unable to reach school-based emergency contacts and a child appears to be 鈥渋n immediate danger.鈥� In on the company鈥檚 website, school officials in Wichita Falls, Texas, Cincinnati, Ohio, and Miami, Florida, acknowledged contacting police in response to Gaggle alerts.

In some cases, school leaders ask Securly to contact the police directly and request they conduct welfare checks on students, the to lawmakers. Executives at Bark said 鈥渢here are limited options鈥� beyond police intervention if they identify a student in crisis but they cannot reach a school administrator. 

鈥淲hile we have witnessed many lives saved by police in these situations, unfortunately many officers have not received training in how to handle such crises,鈥� in its letter. 鈥淚rrespective of training there is always a risk that a visit from law enforcement can create other negative outcomes for a student and their family.鈥� 

In its , GoGuardian states the company may disclose student information 鈥渋f we believe in good faith that doing so is necessary or appropriate to comply with any law enforcement, legal or regulatory process.鈥� 

Meanwhile, survey results suggest that student surveillance tools have a negative disparate impact on Black and Hispanic students, LGBTQ youth and those from low-income households. In a letter on Wednesday to coincide with the survey鈥檚 release, a coalition of education and civil rights groups called on the U.S. Department of Education to issue guidance warning schools that their digital surveillance practices could violate federal civil rights laws. Signatories include the American Library Association, the Data Quality Campaign and the American Civil Liberties Union.

鈥淭his is becoming a conversation not just about privacy, but about discrimination,鈥� Laird said. 鈥淲ithout a doubt, we see certain groups of students having outsized experiences in being directly targeted.鈥�

In a youth survey, researchers found that student discipline as a result of activity monitoring fell disproportionately along racial lines, with 48% of Black students and 55% of Hispanic students reporting that they or someone they knew got into trouble for something that was flagged by an activity monitoring tool. Just 41% of white students reported having similar experiences. 

Nearly a third of LGBTQ students said they or someone they know experienced nonconsensual disclosure of their sexual orientation or gender identity 鈥� often called outing 鈥� as a result of activity monitoring. LGBTQ youth were also more likely than straight and cisgender students to report getting into trouble at school and being contacted by the police about having committed a crime. 

Some student surveillance companies, like Gaggle, monitor references to words including 鈥済ay鈥� and 鈥渓esbian,鈥� a reality company founder and CEO Jeff Patterson has said was created to protect LGBTQ youth, who face a greater risk of dying by suicide. But survey results suggest the heightened surveillance comes with significant harm to youth, and Laird said if monitoring tools are designed with certain students in mind, such as LGBTQ youth, that in itself is a form of discrimination.聽

In its letter to the Education Department鈥檚 Office for Civil Rights Wednesday, advocates said the disparities outlined in the survey run counter to federal laws prohibiting race-, sex- and disability-based discrimination. 

鈥淪tudent activity monitoring is subjecting protected classes of students to increased discipline and interactions with law enforcement, invading their privacy, and creating hostile environments for students to express their true thoughts and authentic identities,鈥� the letter states. 

The Education Department鈥檚 civil rights division, they said, should condemn surveillance practices that violate students鈥� civil rights and launch 鈥渆nforcement action against violations that result in discrimination.鈥�

Lawmakers consider youth privacy

The report comes at a moment of increasing alarm about student privacy online. In May, the Federal Trade Commission announced plans to crack down on tech companies that sell student data for targeted advertising and that 鈥渋llegally surveil children when they go online to learn.鈥� 

It also comes at a time of intense concern over students鈥� emotional and physical well-being. While the pandemic has led to a greater focus on youth mental health, the May mass school shooting in Uvalde, Texas, has sparked renewed school safety efforts. In June, President Joe Biden signed a law with modest new gun-control provisions and an influx of federal funding for student mental health care and campus security. The funds could lead to more digital student surveillance.

The results of the online survey, which was conducted in May and June, were likely colored by the Uvalde tragedy, researchers acknowledged. A majority of parents and students have a favorable view of student activity monitoring during school hours to protect kids from harming themselves or others, researchers found. But just 48% of parents and 30% of students support around-the-clock surveillance. 

鈥淪chools are under a lot of pressure to find ways to keep students safe and, like in many aspects of our lives, they are considering the role of technology,鈥� Laird said. 

Last week, the Senate designed to improve children鈥檚 safety online, including new restrictions on youth-focused targeted advertising. The effort comes a year after a showing that the social media app Instagram had a harmful effect on youth mental well-being, especially teenage girls. One bill, the Kids Online Safety Act, would require tech companies to identify and mitigate any potential harms their products may pose to children, including exposure to content that promotes self-harm, eating disorders and substance abuse.

Yet the legislation has faced criticism from privacy advocates, who argue it would mandate digital monitoring similar to that offered by student surveillance companies. Among critics is the Electronic Frontier Foundation, a nonprofit focused on digital privacy and free speech. 

鈥淭he answer to our lack of privacy isn鈥檛 more tracking,鈥� the . The legislation 鈥渋s a heavy-handed plan to force technology companies to spy on young people and stop them from accessing content that is 鈥榥ot in their best interest,鈥� as defined by the government, and interpreted by tech platforms.鈥� 

Attorney Amelia Vance, the founder and president of Public Interest Privacy Consulting, said she worries the provisions will have a negative impact on at-risk kids, including LGBTQ students. Students from marginalized groups, she said, 鈥渨ill now be more heavily surveilled by basically every site on the internet, and that information will be available to parents鈥� who could discipline teens for researching LGBTQ content. She said the legislation could force tech companies to censor content to avoid potential liability, essentially making them arbiters of community standards. 

鈥淲hen you have conflicting values in the different jurisdictions that the companies operate in, oftentimes you end up with the most conservative interpretations, which right now is anti-LGBT,鈥� she said.