Where AI breaks: testing internal contradictions in language models

Methods

Summary

This project investigates whether advanced AI systems can maintain logical consistency when faced with internal contradictions. Using GEP² (Generator of Epistemic Prompts and Protocols), we simulate structurally tense interactions between two AI models—one generating, the other validating. Through carefully crafted paradoxes, conflicting premises, and symbolic abstractions, we test whether the reasoning of AI remains coherent under pressure. Our approach emphasizes structure over utility, using custom infrastructure and open reporting to ensure reproducibility without commercial interference. Full methodology and test samples will be published for peer replication.

Challenges

One major challenge is that commercial AI APIs often suppress or censor high-friction prompts, introducing bias and limiting test depth. Another is that symbolic contradictions may be misread as performance errors, distorting the purpose of the experiment. To address these, we are using a local high-performance computing setup and logging all outputs independently to ensure integrity. The complexity of crafting non-utilitarian yet valid prompts also presents a challenge, mitigated through the GEP² framework and iterative refinement with a dedicated validation model.

Pre Analysis Plan

We will analyze outputs from paired model interactions by evaluating internal coherence, logical stability, and contradiction management. Each response will be scored based on structural integrity: alignment between premises and conclusions, absence of evasive reasoning, and response reproducibility. We will use a symbolic coherence index (SCI) developed within GEP² to track consistency across different AI pairs and test scenarios. All results will be documented in a public report with annotated outputs and statistical summaries comparing model behavior under pressure. Multiple outcomes (e.g., evasion, contradiction, collapse) will be categorized and mapped to specific prompt types.