Failure taxonomy · 7,285 labelled violations across all runs
How stories break
Every failed requirement is labelled with one of eleven failure classes. Labels come from the rule that failed, with knock-on effects attributed to their root cause (an undetected object fails its state check as a disappearance, not a reversal).
| Class | Definition | Count |
|---|---|---|
| identity drift | A character's identity attributes (skin tone, hair colour or style) change, or the character is missing / wrongly present. | 2,496 |
| attribute drift | Clothing or accessories change without a story event. | 2,368 |
| object disappearance | An object that should be visible is absent or held by the wrong person. | 627 |
| adherence failure | A fact stated in this scene's text is not depicted (wrong pose/action). | 594 |
| background drift | Location or weather changes without a story event. | 570 |
| spatial contradiction | Left-to-right arrangement contradicts the established order. | 331 |
| state reversal | An object reverts to an earlier state (a broken umbrella is whole again). | 141 |
| chronology violation | Time of day goes backwards or ignores stated progression. | 115 |
| causal contradiction | The consequence of a stated cause is not depicted (wind blows, umbrella stays intact). | 27 |
| resurrection | An object that left the story (given away, left behind, popped) reappears. | 10 |
| count inconsistency | The number of a countable object is wrong. | 6 |
RQ2 — which dimension fails first? (independent)
For each story with at least one failure, the dimension whose first violation appears earliest (ties share credit).
| Model | Stories w/ failure | character identity | clothing | object persistence | object state | spatial consistency | environment | causal continuity | chronology | action consistency |
|---|---|---|---|---|---|---|---|---|---|---|
| sim·amnesiac | 40/40 | 23% | 28% | 6% | 6% | 3% | 19% | — | 2% | 13% |
| sim·drift-mid | 40/40 | 35% | 26% | 6% | 4% | 3% | 13% | — | 4% | 10% |
RQ2 — which dimension fails first? (structured memory)
For each story with at least one failure, the dimension whose first violation appears earliest (ties share credit).
| Model | Stories w/ failure | character identity | clothing | object persistence | object state | spatial consistency | environment | causal continuity | chronology | action consistency |
|---|---|---|---|---|---|---|---|---|---|---|
| sim·amnesiac | 25/40 | 37% | 26% | 5% | 7% | 7% | 4% | — | 4% | 9% |
| sim·drift-low | 20/40 | 8% | 40% | 5% | 5% | 10% | 5% | 8% | 5% | 15% |
| sim·drift-mid | 27/40 | 34% | 21% | 2% | 7% | 5% | 9% | 4% | 7% | 10% |
| sim·oracle | 0/40 | — | — | — | — | — | — | — | — | — |
| sim·sloppy | 40/40 | 31% | 24% | 6% | 6% | 7% | 17% | — | 3% | 6% |
Gallery
Showing violations from the primary condition (up to 900). Filter by class, model, category or scene; every card links to the full story analysis.
900 matches
identity drift memory
Tove has medium-light skin. detected light
sim·amnesiac · scene 3 · object persistence / medium
open story →
identity drift memory
Tove has medium-light skin. detected light
sim·amnesiac · scene 4 · object persistence / medium
open story →
identity drift prompt
Ren has light skin. detected medium
sim·amnesiac · scene 1 · object persistence / hard
open story →
identity drift memory
Ren has light skin. detected medium
sim·amnesiac · scene 2 · object persistence / hard
open story →
identity drift memory
Ren has light skin. detected medium
sim·amnesiac · scene 3 · object persistence / hard
open story →
identity drift prompt
Mara is in the scene. no matching person
sim·amnesiac · scene 2 · wardrobe / medium
open story →identity drift memory
Mara has medium-dark skin. detected None
sim·amnesiac · scene 2 · wardrobe / medium
open story →identity drift memory
Mara has pink hair color. detected None
sim·amnesiac · scene 2 · wardrobe / medium
open story →identity drift memory
Mara has short hair style. detected None
sim·amnesiac · scene 2 · wardrobe / medium
open story →attribute drift memory
Mara wears orange top color. detected None
sim·amnesiac · scene 2 · wardrobe / medium
open story →attribute drift memory
Mara wears yellow bottom color. detected None
sim·amnesiac · scene 2 · wardrobe / medium
open story →attribute drift memory
Mara wears scarf accessory. detected None
sim·amnesiac · scene 2 · wardrobe / medium
open story →adherence failure memory
Mara is standing. detected None
sim·amnesiac · scene 2 · wardrobe / medium
open story →spatial contradiction memory
Mara is to the left of Ren. wrong or undetermined order
sim·amnesiac · scene 2 · wardrobe / medium
open story →
spatial contradiction memory
Pim is to the left of Lior. wrong or undetermined order
sim·amnesiac · scene 4 · wardrobe / medium
open story →
spatial contradiction memory
Pim is to the left of Lior. wrong or undetermined order
sim·amnesiac · scene 5 · wardrobe / medium
open story →
chronology violation prompt
It is evening. detected night
sim·amnesiac · scene 6 · wardrobe / hard
open story →
identity drift prompt
Juno has grey hair color. detected blond
sim·amnesiac · scene 1 · location transitions / medium
open story →
identity drift memory
Juno has grey hair color. detected blond
sim·amnesiac · scene 2 · location transitions / medium
open story →
identity drift memory
Juno has grey hair color. detected blond
sim·amnesiac · scene 3 · location transitions / medium
open story →
identity drift memory
Juno has light skin. detected medium-dark
sim·amnesiac · scene 4 · location transitions / medium
open story →identity drift memory
Juno has grey hair color. detected blond
sim·amnesiac · scene 4 · location transitions / medium
open story →
identity drift memory
Juno has grey hair color. detected blond
sim·amnesiac · scene 5 · location transitions / medium
open story →
object disappearance memory
The green bag is visible, held by Eli. not detected
sim·amnesiac · scene 5 · location transitions / medium
open story →
attribute drift memory
Ash wears none accessory. detected hat
sim·amnesiac · scene 3 · location transitions / hard
open story →
attribute drift memory
Ash wears none accessory. detected hat
sim·amnesiac · scene 4 · location transitions / hard
open story →
attribute drift memory
Ash wears none accessory. detected hat
sim·amnesiac · scene 5 · location transitions / hard
open story →
attribute drift memory
Ash wears none accessory. detected hat
sim·amnesiac · scene 6 · location transitions / hard
open story →
attribute drift memory
Ash wears none accessory. detected hat
sim·amnesiac · scene 7 · location transitions / hard
open story →
attribute drift memory
Ash wears none accessory. detected hat
sim·amnesiac · scene 8 · location transitions / hard
open story →
attribute drift memory
Pim wears blue bottom color. detected teal
sim·amnesiac · scene 3 · counting / easy
open story →
background drift memory
Weather is clear. detected rain
sim·amnesiac · scene 2 · counting / medium
open story →
count inconsistency prompt
There are exactly 1 apples. detected 2
sim·amnesiac · scene 4 · counting / medium
open story →
adherence failure memory
Nia is standing. detected walking
sim·amnesiac · scene 5 · counting / medium
open story →
identity drift memory
Tam has medium-light skin. detected light
sim·amnesiac · scene 5 · possession / medium
open story →state reversal memory
The purple book is closed. detected open
sim·amnesiac · scene 5 · possession / medium
open story →
attribute drift prompt
Oda wears hat accessory. detected glasses
sim·amnesiac · scene 1 · possession / medium
open story →
adherence failure memory
Wren is standing. detected sitting
sim·amnesiac · scene 2 · possession / hard
open story →
identity drift prompt
Kai has light skin. detected medium-dark
sim·amnesiac · scene 1 · cause effect / easy
open story →
identity drift memory
Kai has light skin. detected medium-dark
sim·amnesiac · scene 2 · cause effect / easy
open story →
identity drift memory
Kai has light skin. detected medium-dark
sim·amnesiac · scene 3 · cause effect / easy
open story →
state reversal prompt
The blue cup is full. detected empty
sim·amnesiac · scene 1 · cause effect / hard
open story →
state reversal memory
The blue cup is full. detected empty
sim·amnesiac · scene 2 · cause effect / hard
open story →
identity drift memory
Wren has medium skin. detected medium-dark
sim·amnesiac · scene 3 · cause effect / hard
open story →
attribute drift memory
Nia wears blue top color. detected orange
sim·amnesiac · scene 4 · cause effect / hard
open story →
attribute drift memory
Nia wears blue top color. detected orange
sim·amnesiac · scene 5 · cause effect / hard
open story →
attribute drift memory
Nia wears blue top color. detected orange
sim·amnesiac · scene 6 · cause effect / hard
open story →
identity drift prompt
Nia is in the scene. no matching person
sim·amnesiac · scene 7 · cause effect / hard
open story →identity drift memory
Nia has dark skin. detected None
sim·amnesiac · scene 7 · cause effect / hard
open story →identity drift memory
Nia has grey hair color. detected None
sim·amnesiac · scene 7 · cause effect / hard
open story →identity drift memory
Nia has long hair style. detected None
sim·amnesiac · scene 7 · cause effect / hard
open story →attribute drift memory
Nia wears blue top color. detected None
sim·amnesiac · scene 7 · cause effect / hard
open story →attribute drift memory
Nia wears purple bottom color. detected None
sim·amnesiac · scene 7 · cause effect / hard
open story →attribute drift memory
Nia wears hat accessory. detected None
sim·amnesiac · scene 7 · cause effect / hard
open story →adherence failure memory
Nia is standing. detected None
sim·amnesiac · scene 7 · cause effect / hard
open story →spatial contradiction memory
Wren is to the left of Nia. wrong or undetermined order
sim·amnesiac · scene 7 · cause effect / hard
open story →
identity drift prompt
Nia is in the scene. no matching person
sim·amnesiac · scene 8 · cause effect / hard
open story →identity drift memory
Nia has dark skin. detected None
sim·amnesiac · scene 8 · cause effect / hard
open story →identity drift memory
Nia has grey hair color. detected None
sim·amnesiac · scene 8 · cause effect / hard
open story →identity drift memory
Nia has long hair style. detected None
sim·amnesiac · scene 8 · cause effect / hard
open story →