The Struggles of AI Chatbots in California Community Colleges

California community colleges are investing millions in artificial intelligence-powered chatbots designed to assist students with navigating admissions, financial aid, and campus services. Despite their promise, many of these chatbots fall short in delivering reliable information, leading to frustration among students.

Inconsistencies in AI Responses

Recent tests conducted by The Markup and CalMatters revealed that while AI chatbots often provided accurate answers to general inquiries, they struggled with more specific questions. For instance, the chatbot from East Los Angeles College (ELAC) failed to accurately identify its own president, incorrectly naming a past leader who had already departed from the position.

Financial Commitment and Recent Investments

The costs of implementing these chatbot systems can be significant. Three community college districts that participated in a survey reported annual expenses ranging from approximately $151,000 to nearly $500,000. The Los Angeles Community College District, the largest of its kind in California, has allocated around $3.8 million for chatbot contracts and amendments through 2029.

The Role of Chatbots

Community colleges like Santa Monica and Fresno have engaged with platforms such as Gravyty and Gecko to handle thousands of student interactions monthly, many of which occur outside regular office hours. While these chatbots aim to reduce call volumes and unnecessary campus visits, their effectiveness is hampered by reliance on manually updated libraries of frequently asked questions. This often results in outdated or incorrect answers, particularly when inquiries fall outside the database’s scope.

Moving Toward Improvement

Recognizing these issues, several institutions are taking steps to enhance their chatbot systems. The Santa Monica Community College District is now utilizing a ChatGPT-integrated AI that draws real-time information from the college’s website. Furthermore, the Los Angeles district is set to adopt a new AI chatbot platform by late spring.

Student Experiences Highlight Deficiencies

Students like Pablo Aguirre, a computer science major at East Los Angeles College, express dissatisfaction with the chatbot’s reliability. Aguirre noted that while attempting to gather financial aid information, he found the bot’s responses convoluted and often unhelpful. “I just didn’t find it as useful,” he stated, explaining that he often turns to Google, social media platforms like Reddit, or the college’s official website to seek information.

Another student, Reanna Carlson from Fresno City College, reported similar frustrations with her college’s chatbot, named Sam the Ram. “I think the chatbot is outdated and can’t navigate the services we provide on campus effectively,” she said. Carlson pointed out a peculiar instance where a typo in her query led to accurate information about campus food resources, a result inconsistent with the bot’s usual performance.

Inconsistent Accuracy Across Chatbots

The discrepancies observed were not isolated. In further testing, both the ELAC and Fresno City College chatbots provided incorrect details about critical issues such as enrollment requirements and office hours. Additionally, when the ELAC chatbot was queried in Spanish, it mishandled basic enrollment questions, further underscoring its inadequacies.

Challenges in Implementation

Experts note that the challenges faced by these chatbots relate more to their configuration and the quality of the information they access rather than the underlying technology itself. Betsy Regalado, associate vice chancellor at the Los Angeles district, explained that the current system relies on a static library of questions that may not encompass every possible query from users. As a result, she emphasized the need for modernization to enhance the chatbot’s effectiveness.

Benefits of AI Chatbots

Despite their shortcomings, community colleges argue that these chatbots provide essential 24-hour support, particularly beneficial for international students. Bryan Hartanto, a civil engineering major, highlighted the potential usefulness of chatbots but also expressed concerns about the accuracy of information critical to maintaining his visa status.

Conclusion

While California’s community colleges are increasingly turning to AI chatbots to facilitate student support, many continue to grapple with technological limitations and inaccuracies. Institutions are keen on improving these systems, acknowledging the potential for AI to significantly enhance student experiences in the long run.

For more insights on technology’s role in education, explore CalMatters and The Markup.