Weekend Challenge: AI Fact-Check -- Systematically Verify Claims and Spot Misinformation

Your uncle sends a WhatsApp message: 'Study proves: AI will make 40% of all jobs obsolete by 2027!' A LinkedIn post claims: 'Companies using AI increase revenue by an average of 300%.' A news site writes: 'New EU regulation bans all AI-generated images on social media starting September.' It all sounds plausible. But is it true?

The core problem: Our brain confuses plausibility with truth.

When a claim sounds logical, when it contains a number, when it names a source ('according to a Harvard study'), we believe it -- without checking. Psychologists call this plausibility bias: the more convincingly something is worded, the less we question it. And in the age of AI-generated text, the problem grows because AI can produce perfectly plausible-sounding misinformation.

Why you need AI for fact-checking -- and why you must distrust it at the same time:

AI is a powerful research tool. It can put claims in context, find logical contradictions, point out missing sources, and suggest alternative explanations. But: AI itself can hallucinate -- inventing convincing-sounding facts. That is why this challenge teaches you not only HOW to use AI for fact-checking, but also HOW to verify AI answers themselves.

The task (25 minutes, 3 phases):

Phase 1 -- Select a claim and break it down (7 min)

Choose a claim you recently heard or read and are not sure about. If nothing comes to mind, use one of these examples:

- 'AI-generated texts are now indistinguishable from human texts -- no detector achieves more than 50% accuracy.'
- 'German companies spend on average less than 2% of their IT budget on AI.'
- 'According to an MIT study, using ChatGPT saves an average of 37% of working time on writing tasks.'
- 'The EU has decided that all AI-generated content must be labeled starting 2025.'
- 'OpenAI uses as much electricity per ChatGPT query as a light bulb in one hour.'

Copy this prompt:

'You are an experienced fact-checker and investigative journalist. Your task is to systematically break down claims and check their accuracy. You are neutral, thorough, and honest -- even when the result is uncomfortable.

The claim I want to check:
[Paste the claim you want to verify here]

Where I found the claim:
[e.g., LinkedIn post from an influencer / WhatsApp forward / News article on site X / Podcast Y / Comment under a YouTube video]

Analyze the claim in the following steps:

1. Break down into sub-claims
Break the claim into its individual verifiable components. Often one sentence contains multiple statements that need to be checked separately. For each sub-claim:
- What exactly is being claimed?
- Is it a factual claim (verifiable) or an opinion (not verifiable)?
- What implicit assumptions are embedded that are not stated?

2. Source analysis
If the claim names a source (study, institution, expert):
- Does this source actually exist?
- What does the original source actually say -- and was it accurately represented?
- What context does the claim omit?
- Who funded or conducted the study? Are there potential conflicts of interest?

If NO source is named:
- What type of source would be needed to support this claim?
- Is it suspicious that no source is cited?

3. Logic check
Examine the internal logic:
- Does the claim contain logical fallacies? (e.g., correlation presented as causation, generalization from a single case, false dichotomy)
- Are the numbers realistic? Do they fit the order of magnitude?
- Does the claim contradict itself or known facts?

4. Context check
What is missing?
- What important context does the claim conceal?
- Is there a simpler explanation for what is being claimed?
- Who benefits if this claim is believed? (Cui bono?)

5. Overall assessment
Rate on this scale:
- TRUE: Supported by reliable sources
- MOSTLY TRUE: Correct in essence, but details are imprecise or context is missing
- HALF TRUE: Contains true elements but also significant errors or misleading omissions
- MOSTLY FALSE: Some elements are correct, but the core claim is wrong or heavily distorted
- FALSE: Not supportable by sources or demonstrably wrong
- UNVERIFIABLE: Cannot be confirmed or refuted with available information

For the rating: explain in 2-3 sentences why you reached this verdict.

6. What I should verify independently
What steps can I take myself to verify the result? Name concrete, actionable research steps -- not vague suggestions like look into it further.'

Read the analysis carefully. Pay special attention to point 6: the AI gives you hints there on how to verify its own answer. This is crucial because the AI itself could be wrong.

Phase 2 -- Verify the AI answer itself (10 min)

Now comes the most important step: you check whether the AI analysis itself is reliable. Because AI can hallucinate about factual statements -- especially with specific numbers, study results, and dates.

Copy this prompt:

'Now I want to critically question your own analysis. Be ruthlessly honest with yourself.

1. Uncertainty audit:
Go through every fact, every number, and every source you mentioned in your analysis. For each individual point:
- How confident are you that this information is correct? (Scale: very confident / fairly confident / uncertain / could be hallucinated)
- If you are not very confident: why not? What exactly is the uncertainty?

2. Hallucination warning:
Which parts of your answer have the highest risk of being fabricated or misremembered? Mark these spots explicitly.

3. Missing perspectives:
- Is there a counter-position to your assessment that you did not consider?
- Could someone with the opposite opinion have good arguments?
- Do you perhaps have a confirmation bias -- did you unconsciously look for confirmation of a preconceived opinion?

4. Verifiable sources:
Give me 3-5 concrete URLs or search terms I can use to independently verify your analysis. For each:
- What exactly should I find there?
- Which result would confirm your analysis, and which would refute it?

5. Overall confidence:
How reliable is your analysis overall, on a scale from 1 (very uncertain, much speculation) to 10 (fully supported by verifiable sources)? Justify the number.'

This step is the heart of the challenge. Most people accept AI answers as given. But an AI that admits where it is uncertain is more valuable than one that confidently asserts everything. Read the uncertainty sections closely and verify at least one of them with a real source (Google, Wikipedia, original source).

Phase 3 -- Build your personal fact-checking toolkit (8 min)

Now you build a reusable system you can deploy for every future claim.

Copy this prompt:

'Based on the fact-checking exercise we just did: help me create a personal fact-checking toolkit that I can use for any dubious claim from now on.

1. The 5-question quick check:
Create a compact questionnaire with exactly 5 questions I should ask myself for EVERY claim -- whether LinkedIn post, news article, or family group chat. The questions must:
- Take under 30 seconds to run through
- Be answerable without research (common sense only)
- Immediately flag a warning signal when something is off

Phrase them as yes/no questions where every No is a warning signal.

2. Red flags -- the most common patterns:
List the 10 most common patterns that indicate misinformation or distortion. For each pattern:
- How do I recognize it? (Concrete example)
- Why does it still work so often? (Psychological reason)
- How should I respond?

3. Trustworthy source list:
Create a list of sources and methods I can use for quick verification:
- For statistical claims about my country/region
- For AI and tech claims
- For health claims
- For political claims
- For viral social media claims
For each category: 2-3 specific starting points (websites, databases, methods).

4. The compact AI fact-check prompt:
Condense the detailed prompt from Phase 1 into a compact version I can quickly copy for any claim. Maximum 8 lines, but with the essential checking steps. This prompt should become my standard tool.

5. How to verify AI answers:
Give me 5 concrete rules for when I can trust an AI answer and when I cannot. Make the rules specific enough that I can apply them immediately.'

Why this challenge matters:

We live in an era where information and misinformation can look identical. AI-generated texts sound perfect. Deepfakes keep improving. Social media algorithms reward emotional, polarizing content -- regardless of whether it is true. The ability to systematically check claims is no longer an academic exercise. It is an everyday skill like reading and math.

Three examples of how fact-checking works in practice:

Example 1 -- The LinkedIn statistic:
Claim: 'Companies with an AI strategy grow 5x faster than the market average.'
Breakdown: What counts as an 'AI strategy'? Which market average? What time period? Which companies were compared?
Result: HALF TRUE. McKinsey studies show a correlation, but the '5x' factor is heavily exaggerated and ignores survivorship bias: only successful companies report about their AI strategy.
Lesson: Numbers without context are almost always misleading. Ask: 'Compared to what? Over what time period? Who was measured?'

Example 2 -- The WhatsApp warning:
Claim: 'Starting next month, WhatsApp will permanently store all messages and share them with authorities.'
Breakdown: Which authorities? In which country? On what legal basis?
Result: FALSE. A regularly recurring hoax that has circulated in variations for years. WhatsApp uses end-to-end encryption and technically cannot read message content.
Lesson: Chain messages with urgent tone ('Starting next month!') and no source citation are almost always false.

Example 3 -- The podcast fact:
Claim: 'GPT-5 scored an IQ of 155 on an intelligence test -- higher than 99% of all humans.'
Breakdown: Which IQ test? Who conducted the test? Is an IQ test even meaningful for AI?
Result: UNVERIFIABLE to MISLEADING. Individual benchmarks exist, but IQ tests for AI are methodologically problematic: AI solves pattern recognition tasks differently than humans, and test results vary greatly depending on the test version.
Lesson: When a complex topic is reduced to a single impressive number, important context is almost always missing.

Get even more out of it:
- Counter-perspective: 'Find the three strongest arguments that the claim is actually true -- even if you rate it as false overall. What might I be overlooking?'
- Pattern recognition: 'I see this type of claim often on [LinkedIn / Twitter / in my family]. What systematic bias is behind it? And how can I respond faster next time?'
- Check your own claims: 'Here is a text I wrote [paste text]. Check whether I am making unsupported claims or using misleading statistics. Be critical.'
- Evaluate a media source: 'I regularly read [media name]. How would you rate this source? What are its strengths and weaknesses? Where should I consult a second source?'

Privacy note:

When fact-checking with AI, you reveal information about your media consumption and your doubts. This is generally unproblematic, but: do not enter personal data about third parties (e.g., 'My colleague John Smith claims...'). Phrase things neutrally and without names.

Your learning outcome: You have learned a systematic approach to checking claims -- instead of either blindly believing or blanket-rejecting them. You now know how to break a statement into sub-claims, analyze sources, spot logical errors, and identify missing context. You have experienced that AI is a powerful fact-checking tool -- but only if you question its answers. And you have a compact toolkit with 5 quick questions, 10 red flags, and a reusable prompt that you can deploy for any dubious claim from now on. The key insight: the question is never 'Do I believe this?' -- it is 'What evidence supports it?'