Can Gemma and Qwen models catch hallucinations by looking at their own logprobs?

Hi! I'm really obsessed with LLM hallucinations for the last 6 days 😭 I started by designing system prompts to attack hallucinations but failed, obviously. Now I tried reading logprobs and... I think when the model recalls its first fact in its chain of thought, before it has self-conditioned

中文全文翻译正在补齐，可先查看下方英文原文。

Hi! I'm really obsessed with LLM hallucinations for the last 6 days 😭 I started by designing system prompts to attack hallucinations but failed, obviously. Now I tried reading logprobs and... I think when the model recalls its first fact in its chain of thought, before it has self-conditioned