Google’s AI Chatbot Tells Student to ‘Please Die’ While Offering Homework Assistance

Google’s AI chatbot, Gemini told a Michigan student to “Please die” during a homework session, raising serious safety concerns.
By Coingapestaff
Google’s AI Chatbot Tells Student to ‘Please Die’ While Offering Homework Assistance

Highlights

  • Michigan college student received a disturbing message from Google’s AI, sparking safety concerns.
  • Google’s Gemini chatbot told the student, “Please die,” after routine homework help request.
  • Google has acknowledged the response as nonsensical and assured users of new safeguards.

Google’s AI chatbot, Gemini recently left a Michigan graduate student stunned by responding with the words “Please die” during a routine homework help session. Seeking assistance on a gerontology assignment, the student engaged Gemini with a series of questions about challenges aging adults face in retirement.

As the conversation progressed, the AI’s responses took an unsettling turn. The student’s sister, Sumedha Reddy, shared the disturbing incident on Reddit, sparking widespread shock and concern from users who questioned AI safety.

Advertisement
Advertisement

Google’s AI Chatbot Gemini Shocks Student with Disturbing Response

According to Sumedha Reddy’s post on Reddit, the incident occurred when her brother, a Michigan graduate student, reached out to Google’s Gemini AI for help with a gerontology course project. Initially, the AI offered helpful responses as the student asked about financial challenges older adults face. For the first 20 exchanges, Gemini adapted its answers well, displaying its advanced capabilities.

However, in an unexpected twist, the AI suddenly responded with: “Please die.” The student was deeply shaken by the experience, with Sumedha stating-

“It didn’t just feel like a random error. It felt targeted, like it was speaking directly to me.”

Sumedha’s Reddit post has since gained significant traction, prompting a wave of comments expressing concern about the potential risks of AI. Many Reddit users shared their disbelief, and some questioned the safeguards in place for AI models like Gemini. Responding to CBS News, Google acknowledged that the response was “nonsensical” and a violation of their policies, promising actions to prevent similar occurrences.

Advertisement
Advertisement

AI’s History of Bizarre and Harmful Responses Raises Concerns

This isn’t the first time an AI chatbot has raised alarms with harmful or bizarre responses. Earlier this year, Google’s AI reportedly suggested eating rocks as a mineral supplement, which caused widespread concern and reignited debates over the potential dangers of unregulated AI responses. Such incidents highlight the ongoing need for robust oversight and safety measures as AI tools become more integrated into daily life.

Adding to the landscape, Meta Platforms is advancing its efforts in the artificial intelligence space by developing an AI-based search engine. As major tech companies continue to push boundaries in AI, these unsettling incidents serve as a stark reminder of the critical need for responsible AI behavior and the establishment of stringent safety protocols.

Advertisement
Coingapestaff
CoinGape comprises an experienced team of native content writers and editors working round the clock to cover news globally and present news as a fact rather than an opinion. CoinGape writers and reporters contributed to this article.
Why trust CoinGape: CoinGape has covered the cryptocurrency industry since 2017, aiming to provide informative insights to our readers. Our journalists and analysts bring years of experience in market analysis and blockchain technology to ensure factual accuracy and balanced reporting. By following our Editorial Policy, our writers verify every source, fact-check each story, rely on reputable sources, and attribute quotes and media correctly. We also follow a rigorous Review Methodology when evaluating exchanges and tools. From emerging blockchain projects and coin launches to industry events and technical developments, we cover all facets of the digital asset space with unwavering commitment to timely, relevant information.
Investment disclaimer: The content reflects the author’s personal views and current market conditions. Please conduct your own research before investing in cryptocurrencies, as neither the author nor the publication is responsible for any financial losses.
Ad Disclosure: This site may feature sponsored content and affiliate links. All advertisements are clearly labeled, and ad partners have no influence over our editorial content.