Chinese Researchers Introduces Tool to Correct AI Hallucination

A groundbreaking solution to the problem of AI hallucination in Multimodal Large Language Models (MLLMs) has been designed by a group of scientists from the University of Science and Technology of China and Tencent’s YouTu Lab.

Solving AI Hallucination: Introducing Woodpecker

The solution was introduced through a published research paper titled “Woodpecker: Hallucination Correction for Multimodal Large Language Models.” This research was published on the pre-print server arXiv.

Woodpecker utilizes three different AI models. This differs from the MLLM that is being corrected for hallucinations. The models are GPT-3.5 turbo, Grounding DINO and BLIP-2-FlanT5. Their fusion facilitates a system where an evaluation is carried out to first identify the hallucinations and then command the model that is under correction for the hallucinations to regenerate its result according to its data.

This is not the first time an attempt is being made to correct the challenge of hallucination in AI models. Prior to this time, existing solutions involved an instruction-tuning approach which  required the model to be retrained with a particular data. However, these methods were data and computation intensive which equally means that they were expensive. 

In line with the inspiration behind its name, the Woodpecker framework works in five different stages including key concept extraction, question formulation, visual knowledge validation, visual claim generation, and hallucination correction.

Hallucination in AI Models

For context, AI hallucination is typically said to have happened when an AI model generates outputs with a high level of confidence but are not in alignment with the information embedded in its training data. 

These scenarios have largely been experienced with Large Language model (LLM) research. An example of AI applications that use LLM and are at risk of these hallucinations include OpenAI’s ChatGPT and Anthropic’s Claude.

According to a note in the research paper, “Hallucination is a big shadow hanging over the rapidly evolving Multimodal Large Language Models (MLLMs), referring to the phenomenon that the generated text is inconsistent with the image content.” 

With the release of new chatbot models like GPT-4, especially its visual variant GPT-4V as well as other visual systems that process picture and text into generative AI modality, such incidents of hallucination are imminent and Woodpecker is deemed a workable solution.