{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "a1b2c3d4-0001",
   "metadata": {},
   "source": [
    "# Propaganda Pipeline — Usage Tutorial\n",
    "\n",
    "This notebook shows how to use `propaganda_pipeline.py` to detect propaganda techniques in text using LLMs (via DSPy + OpenAI).\n",
    "\n",
    "## Pipeline overview\n",
    "\n",
    "```\n",
    "Text\n",
    " │\n",
    " ▼\n",
    "Analisis.run()          ← runs all 23 technique detectors in sequence\n",
    " │  (or run_consistency() for robustness via multiple trials)\n",
    " ▼\n",
    "candidates list         ← one dict per technique: answer, confidence, span, rationale\n",
    " │\n",
    " ▼\n",
    "RunJuez()               ← LLM judge scores candidates, synthesizes final answer\n",
    " │\n",
    " ▼\n",
    "result dict             ← judged, topk, synthesis, selected_models_posthoc\n",
    " │\n",
    " ├── show_rubric_table_from_result()   ← per-technique rubric scores table\n",
    " ├── synthesis_report()               ← narrative summary card\n",
    " └── visualize_spans()                ← highlighted text with color-coded spans\n",
    "```\n",
    "\n",
    "## Detected techniques (23 total)\n",
    "\n",
    "| # | Name | Description |\n",
    "|---|------|-------------|\n",
    "| 1 | REPETITION | Persuasion through repetition of words/phrases |\n",
    "| 2 | EXAGERATION | Hyperbole or minimization without evidence |\n",
    "| 3 | OBFUSCATION | Intentional vagueness or confusion |\n",
    "| 4 | LOADED_LANGUAGE | Emotionally charged terms without reasoning |\n",
    "| 5 | WHATABOUTISM | Deflecting criticism by pointing to others |\n",
    "| 6 | KAIROS | Exploiting timing/urgency to manipulate |\n",
    "| 7 | KILLER | Conversation killers that suppress discussion |\n",
    "| 8 | SLIPPERY | Slippery slope fallacy |\n",
    "| 9 | SLOGAN | Brief striking phrases used as sole argument |\n",
    "| 10 | VALUES | Appeal to abstract values instead of evidence |\n",
    "| 11 | RED_HERRING | Introducing irrelevant information |\n",
    "| 12 | STRAWMAN | Misrepresenting someone's position |\n",
    "| 13 | FEAR | Appeal to fear or prejudice |\n",
    "| 14 | AUTHORITY | Fallacious appeal to authority |\n",
    "| 15 | BANDWAGON | Appeal to popularity |\n",
    "| 16 | CASTING_DOUBT | Attacking credibility instead of the argument |\n",
    "| 17 | FLAG_WAVING | Appeal to group pride/identity |\n",
    "| 18 | SMEAR_POISONING | Poisoning the well / reputation attacks |\n",
    "| 19 | TU_QUOQUE | \"You do it too\" / appeal to hypocrisy |\n",
    "| 20 | GUILT_BY_ASSOCIATION | Discrediting by association |\n",
    "| 21 | NAME_CALLING | Derogatory labeling |\n",
    "| 22 | CAUSAL_OVERSIMPLIFICATION | Attributing complex outcomes to a single cause |\n",
    "| 23 | FALSE_DILEMMA | Presenting only two options when more exist |"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1b2c3d4-0002",
   "metadata": {},
   "source": [
    "---\n",
    "## 1. Setup\n",
    "\n",
    "Import the pipeline components and configure the LLM. Replace `YOUR_OPENAI_API_KEY` with your key or load it from an environment variable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a1b2c3d4-0003",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from propaganda_pipeline import (\n",
    "    Configuracion,\n",
    "    Analisis,\n",
    "    run_consistency,\n",
    "    RunJuez,\n",
    "    visualize_spans,\n",
    "    synthesis_report,\n",
    "    show_rubric_table_from_result,\n",
    ")\n",
    "\n",
    "# Configure the LLM — gpt-4o-2024-08-06 is recommended for best accuracy\n",
    "cfg = Configuracion(\n",
    "    model_name=\"gpt-4o-2024-08-06\",\n",
    "    api_key=os.environ.get(\"OPENAI_API_KEY\", \"YOUR_OPENAI_API_KEY\"),\n",
    ")\n",
    "cfg.setup()\n",
    "print(\"LLM configured successfully.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1b2c3d4-0004",
   "metadata": {},
   "source": [
    "---\n",
    "## 2. Define the input text\n",
    "\n",
    "The pipeline works on a single string. It can be in English or Spanish."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a1b2c3d4-0005",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Example: False Dilemma + Appeal to Fear\n",
    "TEXT = (\n",
    "    \"Referring to your claim that providing medicare for all citizens would be costly \"\n",
    "    \"and a danger to the free market, I infer that you don't care if people die from \"\n",
    "    \"not having healthcare, so we are not going to support your endeavour.\"\n",
    ")\n",
    "\n",
    "print(f\"Text ({len(TEXT)} chars):\")\n",
    "print(TEXT)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1b2c3d4-0006",
   "metadata": {},
   "source": [
    "---\n",
    "## 3. Run the detectors\n",
    "\n",
    "### Option A — Single run (`Analisis.run`)\n",
    "\n",
    "Fast, but may produce inconsistent results across runs due to LLM stochasticity.\n",
    "Use this during development or for quick tests."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a1b2c3d4-0007",
   "metadata": {},
   "outputs": [],
   "source": [
    "analisis = Analisis()\n",
    "\n",
    "# Single run — returns a list of 23 candidate dicts\n",
    "candidates_single = analisis.run(TEXT)\n",
    "\n",
    "# Quick peek at the structure of one candidate\n",
    "import json\n",
    "example = candidates_single[0]\n",
    "print(json.dumps({k: v for k, v in example.items() if k != \"raw\"}, indent=2, ensure_ascii=False))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1b2c3d4-0008",
   "metadata": {},
   "source": [
    "### Option B — Consistency run (`run_consistency`)\n",
    "\n",
    "Runs the full analysis `trials` times and consolidates results by detection ratio.\n",
    "A technique is reported as detected only if it appears in at least `threshold` fraction of trials.\n",
    "\n",
    "- `trials=5, threshold=1.0` → technique must be detected in **all 5 runs** (strictest)\n",
    "- `trials=5, threshold=0.6` → technique must be detected in at least **3 of 5 runs**\n",
    "\n",
    "Each consolidated candidate has an extra `ratio` field."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a1b2c3d4-0009",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Recommended for production — runs 5 trials and requires unanimous agreement\n",
    "candidates = run_consistency(analisis, TEXT, trials=5, threshold=1.0)\n",
    "\n",
    "# Show techniques detected (answer = \"Sí\") with their ratio\n",
    "detected = [(c[\"model\"], c[\"ratio\"], c[\"confidence\"]) for c in candidates if c[\"answer\"] == \"Sí\"]\n",
    "print(f\"Detected {len(detected)} technique(s):\")\n",
    "for model, ratio, conf in detected:\n",
    "    print(f\"  {model:35s}  ratio={ratio:.2f}  confidence={conf:.2f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1b2c3d4-0010",
   "metadata": {},
   "source": [
    "---\n",
    "## 4. Inspect individual candidates\n",
    "\n",
    "Each candidate dict has these standard keys:\n",
    "\n",
    "| Key | Type | Description |\n",
    "|-----|------|-------------|\n",
    "| `model` | str | Technique name (e.g. `\"FALSE_DILEMMA\"`) |\n",
    "| `answer` | str | `\"Sí\"` (detected) or `\"No\"` |\n",
    "| `confidence` | float | Model certainty in [0, 1] |\n",
    "| `ratio` | float | Detection rate across trials (consistency mode only) |\n",
    "| `rationale_summary` | str | Brief explanation of the decision |\n",
    "| `span` | list[str] | Literal text excerpts supporting the detection |\n",
    "| `labels` | dict | Technique-specific metadata |\n",
    "| `raw` | dict | Full Pydantic model dump |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a1b2c3d4-0011",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Print all detected candidates with their rationale and spans\n",
    "for c in candidates:\n",
    "    if c[\"answer\"] != \"Sí\":\n",
    "        continue\n",
    "    print(f\"{'='*60}\")\n",
    "    print(f\"Technique : {c['model']}\")\n",
    "    print(f\"Answer    : {c['answer']}  (ratio={c.get('ratio', '-'):.2f}, confidence={c['confidence']:.2f})\")\n",
    "    print(f\"Rationale : {c['rationale_summary']}\")\n",
    "    print(f\"Spans     : {c['span']}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1b2c3d4-0012",
   "metadata": {},
   "source": [
    "---\n",
    "## 5. Run the judge (`RunJuez`)\n",
    "\n",
    "`RunJuez` takes the candidates list and runs a three-step LLM evaluation:\n",
    "\n",
    "1. **Judge** — scores each candidate against the DEFAULT_RUBRIC (consistency × confidence × yes-preference) and selects which techniques to include.\n",
    "2. **Synthesizer** — generates a natural-language final answer listing all confirmed techniques.\n",
    "3. **Post-hoc selector** — maps the synthesis back to specific model names with per-technique evidence.\n",
    "\n",
    "The result dict keys:\n",
    "\n",
    "| Key | Description |\n",
    "|-----|-------------|\n",
    "| `judged` | Rubric scores for each candidate and list of selected indices |\n",
    "| `topk` | Selected candidate dicts (include=True in rubric) |\n",
    "| `synthesis` | Final consolidated answer + brief reasoning |\n",
    "| `selected_models_posthoc` | Detected technique names + per-model evidence |\n",
    "| `elapsed_sec` | Wall-clock time for the judge pipeline |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a1b2c3d4-0013",
   "metadata": {},
   "outputs": [],
   "source": [
    "result = RunJuez(TEXT, candidates, confidence_threshold=0.9)\n",
    "\n",
    "print(f\"Judge completed in {result['elapsed_sec']:.1f}s\")\n",
    "print(f\"\\nFinal answer:\\n{result['synthesis']['final_answer']}\")\n",
    "print(f\"\\nReasoning:\\n{result['synthesis']['brief_reasoning']}\")\n",
    "print(f\"\\nSelected models: {result['selected_models_posthoc']['selected_models']}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1b2c3d4-0014",
   "metadata": {},
   "source": [
    "---\n",
    "## 6. Visualizations\n",
    "\n",
    "### 6.1 Rubric scores table\n",
    "\n",
    "Shows per-technique scores (consistency, confidence, yes-preference) and whether the technique was included."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a1b2c3d4-0015",
   "metadata": {},
   "outputs": [],
   "source": [
    "show_rubric_table_from_result(\n",
    "    result,\n",
    "    candidates=candidates,\n",
    "    title=\"Rubric scores by technique\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1b2c3d4-0016",
   "metadata": {},
   "source": [
    "### 6.2 Synthesis report card\n",
    "\n",
    "A formatted HTML card summarizing all detected techniques, the final answer, and per-technique evidence."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a1b2c3d4-0017",
   "metadata": {},
   "outputs": [],
   "source": [
    "html_card = synthesis_report(\n",
    "    TEXT,\n",
    "    judged=result[\"judged\"],\n",
    "    topk=result[\"topk\"],\n",
    "    synthesis=result[\"synthesis\"],\n",
    "    selection=result[\"selected_models_posthoc\"],\n",
    "    display_inline=True\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1b2c3d4-0018",
   "metadata": {},
   "source": [
    "### 6.3 Highlighted span visualization\n",
    "\n",
    "Color-codes the original text by technique. Each color corresponds to a detected propaganda technique. Overlapping regions show a gradient of multiple colors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a1b2c3d4-0019",
   "metadata": {},
   "outputs": [],
   "source": [
    "selection = result[\"selected_models_posthoc\"]\n",
    "\n",
    "html_spans = visualize_spans(\n",
    "    TEXT,\n",
    "    candidates,\n",
    "    selection,\n",
    "    display_inline=True,\n",
    "    show_legend=\"bottom\",\n",
    "    show_badges=False,\n",
    "    boxed=True,\n",
    "    title=\"Detected spans\",\n",
    "    icon=\"⚖️\",\n",
    "    accent_color=\"#111827\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1b2c3d4-0020",
   "metadata": {},
   "source": [
    "---\n",
    "## 7. Running on multiple texts\n",
    "\n",
    "To analyze a batch of texts, simply loop over them. Each call to `run_consistency` + `RunJuez` is independent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a1b2c3d4-0021",
   "metadata": {},
   "outputs": [],
   "source": [
    "TEXTS = {\n",
    "    \"false_dilemma\": (\n",
    "        \"Either you support our policy completely or you want the country to fail. \"\n",
    "        \"There is no middle ground.\"\n",
    "    ),\n",
    "    \"appeal_to_authority\": (\n",
    "        \"Dr. Johnson, one of the world's leading economists, confirmed that \"\n",
    "        \"this tax cut will definitely grow the economy. Therefore it's true.\"\n",
    "    ),\n",
    "    \"name_calling\": (\n",
    "        \"These radical socialist extremists want to destroy everything we've built. \"\n",
    "        \"Don't listen to their dangerous, un-American agenda.\"\n",
    "    ),\n",
    "}\n",
    "\n",
    "results_batch = {}\n",
    "for label, text in TEXTS.items():\n",
    "    print(f\"Analyzing: {label}...\")\n",
    "    cands = run_consistency(analisis, text, trials=3, threshold=0.67)  # faster: 3 trials\n",
    "    res = RunJuez(text, cands)\n",
    "    selected = res[\"selected_models_posthoc\"][\"selected_models\"]\n",
    "    results_batch[label] = selected\n",
    "    print(f\"  → {selected}\")\n",
    "\n",
    "print(\"\\nBatch complete.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1b2c3d4-0022",
   "metadata": {},
   "source": [
    "---\n",
    "## 8. Using a subset of techniques\n",
    "\n",
    "You can instantiate `Analisis` with only the detectors you need, which speeds up the pipeline significantly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a1b2c3d4-0023",
   "metadata": {},
   "outputs": [],
   "source": [
    "from propaganda_pipeline import FalseDilemmaRunner, StrawmanRunner, FearPrejudiceRunner\n",
    "\n",
    "# Only run three specific detectors\n",
    "analisis_subset = Analisis(techniques=[FalseDilemmaRunner, StrawmanRunner, FearPrejudiceRunner])\n",
    "\n",
    "cands_subset = analisis_subset.run(TEXT)\n",
    "for c in cands_subset:\n",
    "    print(f\"{c['model']:25s}  answer={c['answer']}  confidence={c['confidence']:.2f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1b2c3d4-0024",
   "metadata": {},
   "source": [
    "---\n",
    "## 9. Accessing raw LLM output\n",
    "\n",
    "Each candidate's `raw` field contains the full Pydantic model dump from the LLM, including all technique-specific fields."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a1b2c3d4-0025",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Find the False Dilemma candidate and inspect its raw output\n",
    "fd_candidate = next((c for c in candidates if c[\"model\"] == \"FALSE_DILEMMA\"), None)\n",
    "\n",
    "if fd_candidate:\n",
    "    raw = fd_candidate[\"raw\"]\n",
    "    print(f\"is_fd                : {raw.get('is_fd')}\")\n",
    "    print(f\"options_extracted    : {raw.get('options_extracted')}\")\n",
    "    print(f\"third_options_possible: {raw.get('third_options_possible')}\")\n",
    "    print(f\"erases_continuum     : {raw.get('erases_continuum')}\")\n",
    "    print(f\"justification        : {raw.get('justification')}\")\n",
    "    print(f\"confidence           : {raw.get('confidence')}\")\n",
    "else:\n",
    "    print(\"FALSE_DILEMMA not in candidates list.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1b2c3d4-0026",
   "metadata": {},
   "source": [
    "---\n",
    "## 10. Quick reference — API summary\n",
    "\n",
    "```python\n",
    "# 1. Configure\n",
    "cfg = Configuracion(model_name=\"gpt-4o-2024-08-06\", api_key=\"...\")\n",
    "cfg.setup()\n",
    "\n",
    "# 2. Detect\n",
    "analisis = Analisis()                                # all 23 detectors\n",
    "analisis = Analisis(techniques=[SloganRunner, ...])  # subset\n",
    "\n",
    "candidates = analisis.run(TEXT)                      # single run\n",
    "candidates = run_consistency(                        # multi-trial\n",
    "    analisis, TEXT, trials=5, threshold=1.0\n",
    ")\n",
    "\n",
    "# 3. Judge\n",
    "result = RunJuez(TEXT, candidates, confidence_threshold=0.9)\n",
    "\n",
    "# result keys:\n",
    "#   result[\"judged\"]                  → rubric scores\n",
    "#   result[\"topk\"]                    → included candidates\n",
    "#   result[\"synthesis\"]               → final_answer, brief_reasoning\n",
    "#   result[\"selected_models_posthoc\"] → selected_models, evidence, confidence\n",
    "#   result[\"elapsed_sec\"]             → timing\n",
    "\n",
    "# 4. Visualize\n",
    "selection = result[\"selected_models_posthoc\"]\n",
    "\n",
    "show_rubric_table_from_result(result, candidates=candidates, title=\"...\")\n",
    "\n",
    "synthesis_report(\n",
    "    TEXT,\n",
    "    judged=result[\"judged\"],\n",
    "    topk=result[\"topk\"],\n",
    "    synthesis=result[\"synthesis\"],\n",
    "    selection=selection,\n",
    "    display_inline=True\n",
    ")\n",
    "\n",
    "visualize_spans(\n",
    "    TEXT, candidates, selection,\n",
    "    display_inline=True, show_legend=\"bottom\",\n",
    "    boxed=True, title=\"Detected spans\"\n",
    ")\n",
    "```"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.19"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}