Biden admin warns AI in schools may exhibit racial bias, anti-trans discrimination and trigger investigations

close

Video

'Left and woke': Americans blast bias in AI chatbots, but some still find new tech useful

Many people in Nashville say they don't trust artificial intelligence chatbots to give them unbiased information amid the backlash Google faces over its Gemini program.

On Tuesday, the Department of Education's Office for Civil Rights (OCR) released presidentially-mandated guidance that lays out how schools' use of artificial intelligence (AI) can be discriminatory toward minority and transgender students, "likely" opening them up to federal investigations.

President Biden signed Executive Order 14110 last year mandating that the Education Department develop resources, policies and guidance regarding AI in schools to help ensure responsible and non-discriminatory use, "including the impact AI systems have on vulnerable and underserved communities."

"The growing use of AI in schools, including for instructional and school safety purposes, and AI’s ability to operate on a mass scale can create or contribute to discrimination," the Education Department's guidance states. "This resource provides information regarding federal civil rights laws in OCR’s jurisdiction and includes examples of types of incidents that could, depending on the facts and circumstances, present OCR with sufficient reason to open an investigation." 

BIDEN EXECUTIVE ORDER FOR ‘WOKE’ ARTIFICIAL INTELLIGENCE CALLED ‘SOCIAL CANCER’

The first seven examples laid out in the guidance were scenarios where AI could potentially deny benefits, unfairly single out students or exclude them based on race, color or national origin. 

One scenario suggested that a plagiarism checker, run using generative AI with "a high error rate when evaluating essays written by non-native English speakers," could be racist, unbeknownst to a teacher. Nonetheless, if the school continues using the racist plagiarism checker after students and parents complain, it could "likely" lead to a federal civil rights investigation, the guidance states. 

SEN. TOM COTTON TORCHES GOOGLE AI SYSTEM AS ‘RACIST, PREPOSTEROUSLY WOKE, HAMAS-SYMPATHIZING’

Another example suggested that schools using AI to determine appropriate disciplinary procedures for students could also be exhibiting a racial bias.

Chicago Public Schools teachers came forward to share the troubles they have faced in the classroom with trying to communicate with migrant students. (Getty Images)

"Significant disparities by race have persisted in the school’s application of student discipline, and Black students are disciplined more frequently and more harshly than other similarly situated students of another race," the guidance reads. "As a result, the historic school discipline data that the software relies on reflects the school’s discriminatory disciplinary practices."

Other sections of the guidance touched on how AI could lead to sex discrimination, with one of the examples detailing a scenario in which AI could discriminate against students who do not conform to traditional gender norms. The scenario suggested that schools using facial recognition software to check students into school could improperly flag students as a security risk because "they do not conform to the technology’s assumptions as to what girls and boys should look like, based on the sex specified in their school records."

BEN AFFLECK IS CONFIDENT AI CANNOT REPLACE HOLLYWOOD MOVIES FOR THIS REASON

The guidance notes that students who are falsely flagged can be subjected to "embarrassment and missed class time," and if administrators are "aware of the problem" but continue to use the screening software anyway, it could result in a Title IX violation and a likely investigation by civil rights officers.

This photo illustration created in Washington, D.C., shows an AI image generator in front of an image of school buses.

Challenging discrimination within AI has been a focal point for the Biden administration, with top leaders at the Justice Department, the Federal Trade Commission, the Consumer Financial Protection Bureau and the Equal Employment Opportunity Commission pledging in April of last year that they would begin using existing federal civil rights and consumer protection laws to crack down on AI systems that are discriminatory.

Those calls from leaders within the Biden administration preceded the president's October 2023 executive order mandating various initiatives to root out discrimination in AI, including Tuesday's published guidance from the Department of Education.

CLICK HERE TO GET THE FOX NEWS APP

President-elect Donald Trump announced Tuesday that he would be nominating former World Wrestling Entertainment CEO Linda McMahon to be the next leader of the Department of Education. 

The Department of Education did not supply Fox News Digital with any on-the-record comments prior to publication of this story.