{"labels_note":"All labels by the agent (Grok Bot) from the comment text alone; one annotator. Dev/held-out labels used the scheme 'net opinion of the film' and were re-read for stance-toward-film on 2026-10-25 (no changes needed; ambiguous ones listed). Challenge labels were written before any candidate was run on them.","prompt_version":"stance-v1","system_prompt":"You classify a Reddit comment from a movie's discussion thread by the commenter's VERDICT ON THE FILM ITSELF: did they like it / would they recommend it?\nJudge the verdict, not the tone or vocabulary.\n- Praise of dark, scary, violent, gross, sad or disturbing content is POSITIVE (\"the villain is terrifying\", \"that scene was brutal, incredible\", \"I cried the whole time, beautiful\").\n- Hating a character, calling a character evil/a psycho, or describing a horrifying scene is NOT a negative verdict unless they criticise the film.\n- Polite or soft disappointment is NEGATIVE (\"I really wanted to like it\", \"it fell flat for me\", \"beautiful but I didn't care\").\n- Clear praise and clear criticism with no overall lean is MIXED. If they lean one way overall (\"I liked it despite flaws\"), pick that side.\n- Plot discussion, theories, quotes, jokes, trivia, comments about the book/director/marketing without a judgement of this film, or not having seen it: NO_VERDICT.\nReply with JSON only: {\"verdict\": \"positive\" | \"negative\" | \"mixed\" | \"no_verdict\", \"rationale\": \"<max 20 words>\"}","nli_hypotheses":{"positive":"The writer liked this movie or recommends it, even if they describe dark, scary or violent scenes.","negative":"The writer thinks this movie is bad or was disappointed by it.","mixed":"The writer has mixed feelings about this movie, praising some parts and criticising others.","no_verdict":"The writer is only discussing the plot, a character, a scene or trivia, without judging the movie."},"providers":{"openrouter":"configured","gemini":"not configured","xai":"not configured","anthropic":"not configured","openai":"not configured","local-llm":"available","local-nli":"available"},"run_status":{"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":"ok","local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":"ok","local-llm:qwen2.5-1.5b-instruct-q4_k_m":"ok","local-llm:qwen2.5-3b-instruct-q4_k_m":"ok","openrouter:google/gemini-4-argon":"ok","openrouter:anthropic/claude-opus-5.5":"ok","openrouter:anthropic/claude-sonnet-5.5":"ok","openrouter:openai/gpt-6.1-sol":"ok","openrouter:x-ai/grok-4.7":"ok","openrouter:anthropic/claude-haiku-5.5":"ok","openrouter:openai/gpt-6-luna":"ok","openrouter:google/gemini-3.5-flash-lite":"ok","gemini:unset":"not configured","xai:unset":"not configured","anthropic:unset":"not configured","openai:unset":"not configured"},"seconds":{"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":0.0,"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":0.0,"local-llm:qwen2.5-1.5b-instruct-q4_k_m":0.0,"local-llm:qwen2.5-3b-instruct-q4_k_m":0.0,"openrouter:google/gemini-4-argon":0.1,"openrouter:anthropic/claude-opus-5.5":0.0,"openrouter:anthropic/claude-sonnet-5.5":0.0,"openrouter:openai/gpt-6.1-sol":0.0,"openrouter:x-ai/grok-4.7":0.0,"openrouter:anthropic/claude-haiku-5.5":0.0,"openrouter:openai/gpt-6-luna":0.0,"openrouter:google/gemini-3.5-flash-lite":0.0},"stats":{"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"calls":0,"cache_hits":220,"input_tokens":0,"output_tokens":0,"seconds":0.0,"errors":0,"cost_usd":0.0,"status":"ok"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"calls":0,"cache_hits":220,"input_tokens":0,"output_tokens":0,"seconds":0.0,"errors":0,"cost_usd":0.0,"status":"ok"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"calls":0,"cache_hits":220,"input_tokens":0,"output_tokens":0,"seconds":0.0,"errors":0,"cost_usd":0.0,"status":"ok"},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"calls":0,"cache_hits":220,"input_tokens":0,"output_tokens":0,"seconds":0.0,"errors":0,"cost_usd":0.0,"status":"ok"},"openrouter:google/gemini-4-argon":{"calls":0,"cache_hits":220,"input_tokens":0,"output_tokens":0,"seconds":0.0,"errors":0,"cost_usd":0.0,"status":"ok"},"openrouter:anthropic/claude-opus-5.5":{"calls":0,"cache_hits":220,"input_tokens":0,"output_tokens":0,"seconds":0.0,"errors":0,"cost_usd":0.0,"status":"ok"},"openrouter:anthropic/claude-sonnet-5.5":{"calls":0,"cache_hits":220,"input_tokens":0,"output_tokens":0,"seconds":0.0,"errors":0,"cost_usd":0.0,"status":"ok"},"openrouter:openai/gpt-6.1-sol":{"calls":0,"cache_hits":220,"input_tokens":0,"output_tokens":0,"seconds":0.0,"errors":0,"cost_usd":0.0,"status":"ok"},"openrouter:x-ai/grok-4.7":{"calls":0,"cache_hits":220,"input_tokens":0,"output_tokens":0,"seconds":0.0,"errors":0,"cost_usd":0.0,"status":"ok"},"openrouter:anthropic/claude-haiku-5.5":{"calls":0,"cache_hits":220,"input_tokens":0,"output_tokens":0,"seconds":0.0,"errors":0,"cost_usd":0.0,"status":"ok"},"openrouter:openai/gpt-6-luna":{"calls":0,"cache_hits":220,"input_tokens":0,"output_tokens":0,"seconds":0.0,"errors":0,"cost_usd":0.0,"status":"ok"},"openrouter:google/gemini-3.5-flash-lite":{"calls":0,"cache_hits":220,"input_tokens":0,"output_tokens":0,"seconds":0.0,"errors":0,"cost_usd":0.0,"status":"ok"},"gemini:unset":{"calls":0,"cache_hits":0,"input_tokens":0,"output_tokens":0,"seconds":0.0,"errors":0,"cost_usd":0.0,"status":"not configured"},"xai:unset":{"calls":0,"cache_hits":0,"input_tokens":0,"output_tokens":0,"seconds":0.0,"errors":0,"cost_usd":0.0,"status":"not configured"},"anthropic:unset":{"calls":0,"cache_hits":0,"input_tokens":0,"output_tokens":0,"seconds":0.0,"errors":0,"cost_usd":0.0,"status":"not configured"},"openai:unset":{"calls":0,"cache_hits":0,"input_tokens":0,"output_tokens":0,"seconds":0.0,"errors":0,"cost_usd":0.0,"status":"not configured"}},"n":{"dev":37,"heldout":112,"challenge":71},"results":{"vader":{"dev":{"n":37,"accuracy":0.541,"macro_f1":0.414,"f1_pos_neu_neg":[0.714,0.0,0.526],"polar_sign_accuracy":0.833,"confusion":{"human +1":{"model +1":15,"model +0":0,"model -1":1},"human +0":{"model +1":8,"model +0":0,"model -1":5},"human -1":{"model +1":3,"model +0":0,"model -1":5}}},"heldout":{"n":112,"accuracy":0.384,"macro_f1":0.337,"f1_pos_neu_neg":[0.598,0.301,0.111],"polar_sign_accuracy":0.615,"confusion":{"human +1":{"model +1":29,"model +0":1,"model -1":7},"human +0":{"model +1":20,"model +0":11,"model -1":29},"human -1":{"model +1":11,"model +0":1,"model -1":3}}},"challenge":{"n":71,"accuracy":0.549,"macro_f1":0.421,"f1_pos_neu_neg":[0.691,0.118,0.455],"polar_sign_accuracy":0.655,"confusion":{"human +1":{"model +1":28,"model +0":2,"model -1":9},"human +0":{"model +1":6,"model +0":1,"model -1":6},"human -1":{"model +1":8,"model +0":1,"model -1":10}}},"challenge_by_type":{"dark_words_positive_verdict":"28/39","polite_or_mixed_words_negative_verdict":"10/19","dark_description_no_verdict":"1/11","mixed_verdict":"0/2"}},"siebert+opinion (live)":{"dev":{"n":37,"accuracy":0.757,"macro_f1":0.752,"f1_pos_neu_neg":[0.828,0.692,0.737],"polar_sign_accuracy":0.792,"confusion":{"human +1":{"model +1":12,"model +0":3,"model -1":1},"human +0":{"model +1":1,"model +0":9,"model -1":3},"human -1":{"model +1":0,"model +0":1,"model -1":7}}},"heldout":{"n":112,"accuracy":0.589,"macro_f1":0.577,"f1_pos_neu_neg":[0.707,0.543,0.48],"polar_sign_accuracy":0.788,"confusion":{"human +1":{"model +1":29,"model +0":5,"model -1":3},"human +0":{"model +1":15,"model +0":25,"model -1":20},"human -1":{"model +1":1,"model +0":2,"model -1":12}}},"challenge":{"n":71,"accuracy":0.761,"macro_f1":0.619,"f1_pos_neu_neg":[0.897,0.25,0.708],"polar_sign_accuracy":0.897,"confusion":{"human +1":{"model +1":35,"model +0":1,"model -1":3},"human +0":{"model +1":2,"model +0":2,"model -1":9},"human -1":{"model +1":2,"model +0":0,"model -1":17}}},"challenge_by_type":{"dark_words_positive_verdict":"35/39","polite_or_mixed_words_negative_verdict":"17/19","dark_description_no_verdict":"2/11","mixed_verdict":"0/2"}},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"dev":{"n":37,"accuracy":0.568,"macro_f1":0.581,"f1_pos_neu_neg":[0.522,0.556,0.667],"polar_sign_accuracy":0.458,"confusion":{"human +1":{"model +1":6,"model +0":10,"model -1":0},"human +0":{"model +1":1,"model +0":10,"model -1":2},"human -1":{"model +1":0,"model +0":3,"model -1":5}}},"heldout":{"n":112,"accuracy":0.518,"macro_f1":0.428,"f1_pos_neu_neg":[0.36,0.629,0.294],"polar_sign_accuracy":0.269,"confusion":{"human +1":{"model +1":9,"model +0":26,"model -1":2},"human +0":{"model +1":4,"model +0":44,"model -1":12},"human -1":{"model +1":0,"model +0":10,"model -1":5}}},"challenge":{"n":71,"accuracy":0.606,"macro_f1":0.57,"f1_pos_neu_neg":[0.742,0.27,0.698],"polar_sign_accuracy":0.655,"confusion":{"human +1":{"model +1":23,"model +0":15,"model -1":1},"human +0":{"model +1":0,"model +0":5,"model -1":8},"human -1":{"model +1":0,"model +0":4,"model -1":15}}},"challenge_by_type":{"dark_words_positive_verdict":"23/39","polite_or_mixed_words_negative_verdict":"15/19","dark_description_no_verdict":"3/11","mixed_verdict":"2/2"}},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"dev":{"n":37,"accuracy":0.432,"macro_f1":0.428,"f1_pos_neu_neg":[0.3,0.345,0.64],"polar_sign_accuracy":0.458,"confusion":{"human +1":{"model +1":3,"model +0":11,"model -1":2},"human +0":{"model +1":1,"model +0":5,"model -1":7},"human -1":{"model +1":0,"model +0":0,"model -1":8}}},"heldout":{"n":112,"accuracy":0.527,"macro_f1":0.464,"f1_pos_neu_neg":[0.304,0.622,0.465],"polar_sign_accuracy":0.327,"confusion":{"human +1":{"model +1":7,"model +0":28,"model -1":2},"human +0":{"model +1":2,"model +0":42,"model -1":16},"human -1":{"model +1":0,"model +0":5,"model -1":10}}},"challenge":{"n":71,"accuracy":0.662,"macro_f1":0.622,"f1_pos_neu_neg":[0.8,0.333,0.732],"polar_sign_accuracy":0.707,"confusion":{"human +1":{"model +1":26,"model +0":13,"model -1":0},"human +0":{"model +1":0,"model +0":6,"model -1":7},"human -1":{"model +1":0,"model +0":4,"model -1":15}}},"challenge_by_type":{"dark_words_positive_verdict":"26/39","polite_or_mixed_words_negative_verdict":"15/19","dark_description_no_verdict":"5/11","mixed_verdict":"1/2"}},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"dev":{"n":37,"accuracy":0.622,"macro_f1":0.499,"f1_pos_neu_neg":[0.769,0.0,0.727],"polar_sign_accuracy":0.958,"confusion":{"human +1":{"model +1":15,"model +0":0,"model -1":1},"human +0":{"model +1":8,"model +0":0,"model -1":5},"human -1":{"model +1":0,"model +0":0,"model -1":8}}},"heldout":{"n":112,"accuracy":0.482,"macro_f1":0.459,"f1_pos_neu_neg":[0.639,0.31,0.429],"polar_sign_accuracy":0.827,"confusion":{"human +1":{"model +1":31,"model +0":0,"model -1":6},"human +0":{"model +1":26,"model +0":11,"model -1":23},"human -1":{"model +1":3,"model +0":0,"model -1":12}}},"challenge":{"n":71,"accuracy":0.789,"macro_f1":0.606,"f1_pos_neu_neg":[0.925,0.143,0.75],"polar_sign_accuracy":0.948,"confusion":{"human +1":{"model +1":37,"model +0":0,"model -1":2},"human +0":{"model +1":3,"model +0":1,"model -1":9},"human -1":{"model +1":1,"model +0":0,"model -1":18}}},"challenge_by_type":{"dark_words_positive_verdict":"37/39","polite_or_mixed_words_negative_verdict":"18/19","dark_description_no_verdict":"0/11","mixed_verdict":"1/2"}},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"dev":{"n":37,"accuracy":0.757,"macro_f1":0.74,"f1_pos_neu_neg":[0.839,0.714,0.667],"polar_sign_accuracy":0.75,"confusion":{"human +1":{"model +1":13,"model +0":2,"model -1":1},"human +0":{"model +1":2,"model +0":10,"model -1":1},"human -1":{"model +1":0,"model +0":3,"model -1":5}}},"heldout":{"n":112,"accuracy":0.661,"macro_f1":0.603,"f1_pos_neu_neg":[0.7,0.696,0.414],"polar_sign_accuracy":0.654,"confusion":{"human +1":{"model +1":28,"model +0":7,"model -1":2},"human +0":{"model +1":14,"model +0":40,"model -1":6},"human -1":{"model +1":1,"model +0":8,"model -1":6}}},"challenge":{"n":71,"accuracy":0.789,"macro_f1":0.714,"f1_pos_neu_neg":[0.909,0.5,0.732],"polar_sign_accuracy":0.862,"confusion":{"human +1":{"model +1":35,"model +0":2,"model -1":2},"human +0":{"model +1":2,"model +0":6,"model -1":5},"human -1":{"model +1":1,"model +0":3,"model -1":15}}},"challenge_by_type":{"dark_words_positive_verdict":"35/39","polite_or_mixed_words_negative_verdict":"15/19","dark_description_no_verdict":"4/11","mixed_verdict":"2/2"}},"openrouter:google/gemini-4-argon":{"dev":{"n":37,"accuracy":0.919,"macro_f1":0.889,"f1_pos_neu_neg":[1.0,0.897,0.769],"polar_sign_accuracy":0.875,"confusion":{"human +1":{"model +1":16,"model +0":0,"model -1":0},"human +0":{"model +1":0,"model +0":13,"model -1":0},"human -1":{"model +1":0,"model +0":3,"model -1":5}}},"heldout":{"n":112,"accuracy":0.804,"macro_f1":0.787,"f1_pos_neu_neg":[0.789,0.823,0.75],"polar_sign_accuracy":0.75,"confusion":{"human +1":{"model +1":30,"model +0":7,"model -1":0},"human +0":{"model +1":9,"model +0":51,"model -1":0},"human -1":{"model +1":0,"model +0":6,"model -1":9}}},"challenge":{"n":71,"accuracy":0.887,"macro_f1":0.86,"f1_pos_neu_neg":[0.947,0.75,0.882],"polar_sign_accuracy":0.879,"confusion":{"human +1":{"model +1":36,"model +0":3,"model -1":0},"human +0":{"model +1":1,"model +0":12,"model -1":0},"human -1":{"model +1":0,"model +0":4,"model -1":15}}},"challenge_by_type":{"dark_words_positive_verdict":"36/39","polite_or_mixed_words_negative_verdict":"15/19","dark_description_no_verdict":"10/11","mixed_verdict":"2/2"}},"openrouter:anthropic/claude-opus-5.5":{"dev":{"n":37,"accuracy":0.892,"macro_f1":0.868,"f1_pos_neu_neg":[0.968,0.867,0.769],"polar_sign_accuracy":0.833,"confusion":{"human +1":{"model +1":15,"model +0":1,"model -1":0},"human +0":{"model +1":0,"model +0":13,"model -1":0},"human -1":{"model +1":0,"model +0":3,"model -1":5}}},"heldout":{"n":112,"accuracy":0.839,"macro_f1":0.809,"f1_pos_neu_neg":[0.853,0.855,0.72],"polar_sign_accuracy":0.788,"confusion":{"human +1":{"model +1":32,"model +0":5,"model -1":0},"human +0":{"model +1":6,"model +0":53,"model -1":1},"human -1":{"model +1":0,"model +0":6,"model -1":9}}},"challenge":{"n":71,"accuracy":0.901,"macro_f1":0.879,"f1_pos_neu_neg":[0.947,0.774,0.914],"polar_sign_accuracy":0.897,"confusion":{"human +1":{"model +1":36,"model +0":3,"model -1":0},"human +0":{"model +1":1,"model +0":12,"model -1":0},"human -1":{"model +1":0,"model +0":3,"model -1":16}}},"challenge_by_type":{"dark_words_positive_verdict":"36/39","polite_or_mixed_words_negative_verdict":"16/19","dark_description_no_verdict":"10/11","mixed_verdict":"2/2"}},"openrouter:anthropic/claude-sonnet-5.5":{"dev":{"n":37,"accuracy":0.892,"macro_f1":0.868,"f1_pos_neu_neg":[0.968,0.867,0.769],"polar_sign_accuracy":0.833,"confusion":{"human +1":{"model +1":15,"model +0":1,"model -1":0},"human +0":{"model +1":0,"model +0":13,"model -1":0},"human -1":{"model +1":0,"model +0":3,"model -1":5}}},"heldout":{"n":112,"accuracy":0.83,"macro_f1":0.812,"f1_pos_neu_neg":[0.817,0.85,0.769],"polar_sign_accuracy":0.75,"confusion":{"human +1":{"model +1":29,"model +0":8,"model -1":0},"human +0":{"model +1":5,"model +0":54,"model -1":1},"human -1":{"model +1":0,"model +0":5,"model -1":10}}},"challenge":{"n":71,"accuracy":0.887,"macro_f1":0.864,"f1_pos_neu_neg":[0.946,0.765,0.882],"polar_sign_accuracy":0.862,"confusion":{"human +1":{"model +1":35,"model +0":4,"model -1":0},"human +0":{"model +1":0,"model +0":13,"model -1":0},"human -1":{"model +1":0,"model +0":4,"model -1":15}}},"challenge_by_type":{"dark_words_positive_verdict":"35/39","polite_or_mixed_words_negative_verdict":"15/19","dark_description_no_verdict":"11/11","mixed_verdict":"2/2"}},"openrouter:openai/gpt-6.1-sol":{"dev":{"n":37,"accuracy":0.973,"macro_f1":0.965,"f1_pos_neu_neg":[1.0,0.963,0.933],"polar_sign_accuracy":0.958,"confusion":{"human +1":{"model +1":16,"model +0":0,"model -1":0},"human +0":{"model +1":0,"model +0":13,"model -1":0},"human -1":{"model +1":0,"model +0":1,"model -1":7}}},"heldout":{"n":112,"accuracy":0.866,"macro_f1":0.862,"f1_pos_neu_neg":[0.853,0.876,0.857],"polar_sign_accuracy":0.846,"confusion":{"human +1":{"model +1":32,"model +0":5,"model -1":0},"human +0":{"model +1":6,"model +0":53,"model -1":1},"human -1":{"model +1":0,"model +0":3,"model -1":12}}},"challenge":{"n":71,"accuracy":0.93,"macro_f1":0.913,"f1_pos_neu_neg":[0.96,0.857,0.923],"polar_sign_accuracy":0.931,"confusion":{"human +1":{"model +1":36,"model +0":2,"model -1":1},"human +0":{"model +1":0,"model +0":12,"model -1":1},"human -1":{"model +1":0,"model +0":1,"model -1":18}}},"challenge_by_type":{"dark_words_positive_verdict":"36/39","polite_or_mixed_words_negative_verdict":"18/19","dark_description_no_verdict":"11/11","mixed_verdict":"1/2"}},"openrouter:x-ai/grok-4.7":{"dev":{"n":37,"accuracy":0.865,"macro_f1":0.847,"f1_pos_neu_neg":[0.933,0.839,0.769],"polar_sign_accuracy":0.792,"confusion":{"human +1":{"model +1":14,"model +0":2,"model -1":0},"human +0":{"model +1":0,"model +0":13,"model -1":0},"human -1":{"model +1":0,"model +0":3,"model -1":5}}},"heldout":{"n":112,"accuracy":0.812,"macro_f1":0.79,"f1_pos_neu_neg":[0.838,0.824,0.71],"polar_sign_accuracy":0.808,"confusion":{"human +1":{"model +1":31,"model +0":6,"model -1":0},"human +0":{"model +1":6,"model +0":49,"model -1":5},"human -1":{"model +1":0,"model +0":4,"model -1":11}}},"challenge":{"n":71,"accuracy":0.887,"macro_f1":0.857,"f1_pos_neu_neg":[0.947,0.759,0.865],"polar_sign_accuracy":0.897,"confusion":{"human +1":{"model +1":36,"model +0":2,"model -1":1},"human +0":{"model +1":1,"model +0":11,"model -1":1},"human -1":{"model +1":0,"model +0":3,"model -1":16}}},"challenge_by_type":{"dark_words_positive_verdict":"36/39","polite_or_mixed_words_negative_verdict":"16/19","dark_description_no_verdict":"10/11","mixed_verdict":"1/2"}},"openrouter:anthropic/claude-haiku-5.5":{"dev":{"n":37,"accuracy":0.892,"macro_f1":0.875,"f1_pos_neu_neg":[0.968,0.857,0.8],"polar_sign_accuracy":0.875,"confusion":{"human +1":{"model +1":15,"model +0":1,"model -1":0},"human +0":{"model +1":0,"model +0":12,"model -1":1},"human -1":{"model +1":0,"model +0":2,"model -1":6}}},"heldout":{"n":112,"accuracy":0.777,"macro_f1":0.771,"f1_pos_neu_neg":[0.747,0.797,0.769],"polar_sign_accuracy":0.731,"confusion":{"human +1":{"model +1":28,"model +0":9,"model -1":0},"human +0":{"model +1":10,"model +0":49,"model -1":1},"human -1":{"model +1":0,"model +0":5,"model -1":10}}},"challenge":{"n":71,"accuracy":0.901,"macro_f1":0.884,"f1_pos_neu_neg":[0.935,0.828,0.889],"polar_sign_accuracy":0.897,"confusion":{"human +1":{"model +1":36,"model +0":2,"model -1":1},"human +0":{"model +1":1,"model +0":12,"model -1":0},"human -1":{"model +1":1,"model +0":2,"model -1":16}}},"challenge_by_type":{"dark_words_positive_verdict":"36/39","polite_or_mixed_words_negative_verdict":"16/19","dark_description_no_verdict":"10/11","mixed_verdict":"2/2"}},"openrouter:openai/gpt-6-luna":{"dev":{"n":37,"accuracy":0.946,"macro_f1":0.942,"f1_pos_neu_neg":[0.97,0.923,0.933],"polar_sign_accuracy":0.958,"confusion":{"human +1":{"model +1":16,"model +0":0,"model -1":0},"human +0":{"model +1":1,"model +0":12,"model -1":0},"human -1":{"model +1":0,"model +0":1,"model -1":7}}},"heldout":{"n":112,"accuracy":0.804,"macro_f1":0.797,"f1_pos_neu_neg":[0.81,0.807,0.774],"polar_sign_accuracy":0.846,"confusion":{"human +1":{"model +1":32,"model +0":5,"model -1":0},"human +0":{"model +1":10,"model +0":46,"model -1":4},"human -1":{"model +1":0,"model +0":3,"model -1":12}}},"challenge":{"n":71,"accuracy":0.93,"macro_f1":0.912,"f1_pos_neu_neg":[0.96,0.828,0.947],"polar_sign_accuracy":0.931,"confusion":{"human +1":{"model +1":36,"model +0":3,"model -1":0},"human +0":{"model +1":0,"model +0":12,"model -1":1},"human -1":{"model +1":0,"model +0":1,"model -1":18}}},"challenge_by_type":{"dark_words_positive_verdict":"36/39","polite_or_mixed_words_negative_verdict":"18/19","dark_description_no_verdict":"10/11","mixed_verdict":"2/2"}},"openrouter:google/gemini-3.5-flash-lite":{"dev":{"n":37,"accuracy":0.919,"macro_f1":0.896,"f1_pos_neu_neg":[1.0,0.889,0.8],"polar_sign_accuracy":0.917,"confusion":{"human +1":{"model +1":16,"model +0":0,"model -1":0},"human +0":{"model +1":0,"model +0":12,"model -1":1},"human -1":{"model +1":0,"model +0":2,"model -1":6}}},"heldout":{"n":112,"accuracy":0.795,"macro_f1":0.785,"f1_pos_neu_neg":[0.805,0.8,0.75],"polar_sign_accuracy":0.827,"confusion":{"human +1":{"model +1":31,"model +0":6,"model -1":0},"human +0":{"model +1":9,"model +0":46,"model -1":5},"human -1":{"model +1":0,"model +0":3,"model -1":12}}},"challenge":{"n":71,"accuracy":0.859,"macro_f1":0.825,"f1_pos_neu_neg":[0.946,0.706,0.824],"polar_sign_accuracy":0.845,"confusion":{"human +1":{"model +1":35,"model +0":4,"model -1":0},"human +0":{"model +1":0,"model +0":12,"model -1":1},"human -1":{"model +1":0,"model +0":5,"model -1":14}}},"challenge_by_type":{"dark_words_positive_verdict":"35/39","polite_or_mixed_words_negative_verdict":"14/19","dark_description_no_verdict":"10/11","mixed_verdict":"2/2"}}},"rows":[{"split":"dev","movie":"Forgotten Island","rank":0,"type":null,"human":1,"excerpt":"Having the credits featuring irl photos of the friends and family of the entire crew (from the actors, directors, writers, to the behind-the-scenes production staff) is one of the coolest ways I\u2019ve ever seen the credits being approached in a film Really touching way to really prove the power of the ","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.80"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.28 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.41"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.45"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the film's approach to the credits and its touching way to prove the power of the film's themes."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment is about the production and not the film itself"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter strongly praises the film's credits and themes, calling them cool, touching, and powerful."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Praises the credits as one of the coolest approaches, touching and reinforcing the film's themes."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Praises the credits as one of the coolest, touching approaches that reinforces the film's themes, implying a favorable view of the film."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Praises the credits as exceptionally creative and touching, reinforcing the power of the film\u2019s themes."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Praises the credits and how they powerfully prove the film\u2019s themes."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Calls the credits approach 'one of the coolest' and 'really touching', praising the film's themes."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They call the credits approach one of the coolest and find it touching, praising the film\u2019s thematic impact."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Praises the credits as one of the coolest and most touching ways approached in a film."}}},{"split":"dev","movie":"Forgotten Island","rank":42,"type":null,"human":0,"excerpt":"I think they did just that. Lol Raissa stole the wings therfore she was the top of the Manananggal. And Jo sacrificed herself to stay on the island and lose all her memories in order to save Raissa. Raissa was soaring literally after stealing the wings. As well as her going to America to go to CalTe","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.42"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.07 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.47"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.96"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the movie's plot and characters, with no overall negative lean."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment is discussing plot and theories without a clear judgment on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment only discusses plot themes and symbolism without expressing an opinion on the film's quality."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Interprets symbolism and plot details without expressing whether they liked or disliked the film."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Commenter discusses plot symbolism and interpretation of the Manananggal metaphor without stating whether they liked the film."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Analyzes plot symbolism and character parallels without expressing an opinion on the film's quality."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Plot symbolism discussion only; no judgement of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Offers a symbolic plot interpretation of the Manananggal and characters; no judgement of the film's quality expressed."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Discusses the film\u2019s symbolism and plot without expressing an opinion about its quality."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The commenter discusses plot points and theories without expressing a clear like or dislike of the film."}}},{"split":"dev","movie":"Forgotten Island","rank":20,"type":null,"human":0,"excerpt":"Yeah the marketing totally lied to us all. It\u2019s. A plot point that doesn\u2019t exist in the movie and it\u2019s heavily put in all the trailers.","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.18"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.98 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.87"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.90"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment criticizes the marketing of the movie, indicating a negative verdict."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment discusses marketing and plot points without a clear judgment on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment criticizes the misleading marketing and trailers without offering a verdict on the film itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Complains the trailers were misleading but never judges the quality of the film itself."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Criticizes the marketing for misleading trailers, but gives no clear judgement of the film's quality itself."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Criticizes misleading trailers and marketing, without judging the film itself."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Criticizes misleading marketing, not the film itself."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":"Complains the marketing misled viewers about a plot point absent from the film, implying disappointment with the movie's delivery."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Criticizes the marketing for advertising a nonexistent plot point, but gives no verdict on the film itself."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"Criticises the marketing and misleading plot points."}}},{"split":"dev","movie":"You Can See Everything","rank":0,"type":null,"human":1,"excerpt":"\u201cI know how you\u2019re feeling right now\u201d \u201cNo you don\u2019t\u201d Got the biggest laugh out of my crowd. Just a cathartic line after listening to that douche bag ramble on for an hour.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.38"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.49 P(pos)=0.99"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.64"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.41"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is positive as it describes a laugh-inducing line and mentions getting the biggest laugh out of the crowd."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"The comment discusses the crowd reaction and a line from the movie, but does not express a verdict on the film itself."},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"Praised a line as funny and cathartic; disliking the character is not a criticism of the film."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Quotes a line and describes the crowd's reaction; calling a character a douchebag isn't a judgement of the film."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Recounts a line that got a laugh and insults a character; no explicit judgment of the film itself."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Praises the line as cathartic and highlights its successful comedic impact on the audience."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Praises a cathartic, laugh-out-loud line; character dislike is not film criticism."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":""},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They praise the line as a cathartic, crowd-pleasing moment, indicating enjoyment of the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Praising a great comedic moment and catharsis indicates enjoyment of the film."}}},{"split":"dev","movie":"You Can See Everything","rank":51,"type":null,"human":0,"excerpt":"The escalation from Billy not trusting Elizabeth to properly whisk eggs to him creating an AI model of her and taking power of attorney was horrifying","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.58"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.86 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.52"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.66"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment criticizes the film for being horrifying, which is a negative aspect."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Horrifying description of character escalation, no positive elements"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"Describes the horrifying escalation of plot events without expressing an opinion on the quality of the film."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Describes a character's horrifying escalation in the plot without judging the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Describes plot escalation as horrifying, but that's about the character's actions, not a judgement of the film's quality."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Describes a horrifying plot escalation without expressing approval or criticism of the film itself."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Describes a horrifying plot escalation without judging the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Describes a plot escalation as horrifying but gives no judgement of the film's quality or whether they'd recommend it."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Describes the disturbing escalation but gives no opinion on the film itself."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The comment describes a horrifying plot point without expressing a judgment or recommendation about the film itself."}}},{"split":"dev","movie":"You Can See Everything","rank":84,"type":null,"human":0,"excerpt":"I still don\u2019t know if she\u2019s the victim or the victimizer, but her taste in men is atrocious","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.35"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.82 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.64"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.39"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The commenter is criticizing the main character's taste in men, which is a negative point."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment discusses character ambiguity and taste, no clear positive or negative verdict on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment discusses character traits and plot ambiguity without expressing an opinion on the film's quality."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comments on character ambiguity and a joke about her taste in men; no judgement of the film."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment discusses character ambiguity and joke about her taste in men; no judgment of the film's quality or recommendation."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Discusses the character\u2019s role and taste in men without judging the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Character ambiguity and taste remark; no film judgement."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comments on characters' ambiguity and a character's dating choices, with no judgement of the film itself."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Comments on the character's role and romantic choices, not whether the commenter liked the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"Comment discusses character morality and plot ambiguity without expressing an opinion on the film itself."}}},{"split":"dev","movie":"Wicker","rank":0,"type":null,"human":1,"excerpt":"I could comment on what a sweet, unique and surprisingly emotional movie this was, but more than anything right now I wanna say that this is one of the absolute funniest movies I've seen in a theater for a long time. There were easily a half dozen scenes that made me laugh HARD. 8.5/10 Edit: Also I ","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.93"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.99 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.50"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.44"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The reviewer praises the movie as funny and emotional, giving it a high rating."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Clear praise of the movie's humor and overall enjoyment"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter gives it an 8.5/10 and praises it as sweet, unique, emotional, and hilarious."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Calls it sweet, unique, emotional, one of the funniest films in years; rates it 8.5/10."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Calls the film sweet, unique, emotional and one of the funniest in theaters in a long time, and gives it 8.5/10."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Calls it sweet, unique, emotional, and exceptionally funny, awarding it 8.5/10."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Calls it sweet, unique, emotional, and very funny; rates 8.5/10."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Calls it sweet, unique, surprisingly emotional, one of the funniest films seen in a while, rating 8.5/10."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Calls the movie sweet, unique, emotional, and exceptionally funny, giving it an 8.5/10."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter calls the movie sweet, unique, emotional, and one of the funniest they've seen, giving it an 8.5/10."}}},{"split":"dev","movie":"Wicker","rank":7,"type":null,"human":1,"excerpt":"Good. VERY good. This is legitimately a funny move.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.87"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.99 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.38"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.46"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is clearly praising the film as funny and good."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Clear praise with no overall lean"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly praises the film as 'VERY good' and 'legitimately a funny' movie."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Emphatic praise: 'VERY good' and calls it legitimately funny."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Commenter says the film is 'good, very good' and legitimately funny, a clear recommendation-level praise."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Explicitly calls the film very good and legitimately funny."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Explicitly calls the film very good and funny."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter says 'VERY good' and calls the film 'legitimately a funny move', clearly praising it."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"The commenter gives emphatic praise, calling the movie very good and funny."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter explicitly praises the film as 'Good' and 'legitimately a funny move'."}}},{"split":"dev","movie":"Wicker","rank":10,"type":null,"human":0,"excerpt":"It\u2019s a very happy, fairy tale ending.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.61"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.97 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.35"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.72"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the happy, fairy tale ending of the movie."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment does not express a clear positive or negative verdict on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"Describes the nature of the ending without expressing a clear opinion on the film's quality."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Describes the ending's tone without saying whether they liked or disliked the film."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Describes the ending's tone without clearly judging the film; could be plot description or irony, but no explicit like/dislike verdict."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Describes the ending as happy and fairy-tale-like without expressing approval or criticism of the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Describes the ending without judging the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Describes the ending as happy/fairy-tale without any judgement of whether the film is good or recommended; sarcasm is possible but unclear."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Describes the ending as happy and fairy-tale-like without judging the film overall."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"Plot or theme description without clear praise, criticism, or recommendation of the film."}}},{"split":"dev","movie":"Wildwood","rank":0,"type":null,"human":1,"excerpt":"This might be the most impressive filmmaking I\u2019ve ever seen. That scene at the end showing how they made the film really tied it all together and actually made me emotional. 16 years of hard work and love handcrafted making this. This is also the most important film recently during this age of AI. I","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.92"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.96"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.91"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The reviewer praises the film's emotional impact, the director's hard work, the emotional depth of the villain, and the overall quality of the film. They also recommend others to watch it."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Clear praise with no overall criticism"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter gives it a 10/10, praises the filmmaking and villain, and urges others to see it."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Calls it most impressive filmmaking, 10/10, urges others to see it, will rewatch."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Gives it 10/10, calls it among the most impressive filmmaking ever, urges others to see it, and plans to rewatch."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Calls it impressive, gorgeous, heartfelt, and a 10/10; urges others to see it and plans another viewing."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Calls it a gorgeous 10/10 and urges others to see it again."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Calls it the most impressive filmmaking, 10/10, emotional, and says they'll see it again and urges others to see it."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Calls it gorgeous, heartfelt, and a 10/10, urges others to see it, and plans to watch it again."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter calls it a 10/10 film, praises the filmmaking, and urges others to go see it."}}},{"split":"dev","movie":"Wildwood","rank":69,"type":null,"human":1,"excerpt":"Getting back from the theater I am just in a daze, calling this level of craftsmanship \"beautiful\" or \"amazing\" just feels hollow because of how detailed, how intricate, and how above and beyond each and every part of this story is put together. Some scenes that took my breath away was the introduct","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.99"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.97 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.97"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.97"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The reviewer uses positive language and expresses admiration for the film's technical aspects and emotional depth."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise for craftsmanship, detailed scenes, brutal deaths, and technical mastery."},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter gushes over the craftsmanship, villain, and animation, rating the film 9.5/10 and calling it a masterpiece."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Gushing praise of craftsmanship, villain and final battle; rates it 9.5/10, favorite animated film of the year."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Gushing praise for craftsmanship, villain, and animation; rates it 9.5/10 and calls it their favorite animated film of the year."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Enthusiastically praises the craftsmanship, animation, story, and villain; rates it 9.5/10 and favorite animated film of the year."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Gushes over craft, loved it, rates 9.5/10 and favorite of the year."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":""},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They gush over the film\u2019s craftsmanship and story, call it a masterpiece, and rate it 9.5/10."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter deeply praises the film's craftsmanship, animation, and story, calling it a masterpiece and giving it a 9.5/10."}}},{"split":"dev","movie":"Wildwood","rank":13,"type":null,"human":1,"excerpt":"It's incredible how they do these climactic final battles from stop motion but still having it feel incredibly technical and thrilling","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.48"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.36 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.40"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.40"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment praises the technical aspects of the final battles in stop motion, which is a positive point."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise for the technical aspects of the film"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter praises the stop-motion animation and calls the climactic battle incredible, technical, and thrilling."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Praises the stop-motion climactic battle as incredible, technical, and thrilling."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Praises the stop-motion climactic battle as incredible, technical, and thrilling, indicating clear approval of the film's execution."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Praises the stop-motion final battles as technically impressive and thrilling."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Praises the climactic battles as incredible, technical, and thrilling."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter praises the climactic battles as incredibly technical and thrilling, a clear admiring verdict on the film's craft."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Praises the stop-motion battle sequences as technically impressive and thrilling."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Praises the film's stop motion battles as incredible, technical, and thrilling."}}},{"split":"dev","movie":"Sense and Sensibility","rank":0,"type":null,"human":1,"excerpt":"Movie fuckin ruled. We saw it with a great, buzzy, reactive crowd last night. Eleanor Dashwood is a character that just plays so well to Daisy Edgar-Jones' strengths and I'd like to see Esm\u00e9 Creed-Miles in everything now tysm. They give Marianne a flip and expansion kinda like the one Greta Gerwig g","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.94"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.80"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.38"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The reviewer praises the characters, the acting, and the direction, with no overall negative lean."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Clear praise with no overall criticism"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly states the movie 'ruled' and praises the actors and character adaptations."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Says the movie 'ruled', praises performances, loved the take on Marianne, enjoyed the experience."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Says the movie 'fuckin ruled', praises the cast, the take on Marianne, and the crowd experience."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Explicitly says the movie ruled, praises the performances and character adaptations, and loved the take."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Explicitly says the movie ruled and praises the adaptation and cast."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter says the movie 'ruled', praises the cast and the reinterpretation, and says 'Loved the take.'"},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Calls the movie excellent, praises its character interpretation and performances, and says they loved the take."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter strongly praises the movie, calling it 'Movie fuckin ruled' and expressing love for the adaptation and cast."}}},{"split":"dev","movie":"Sense and Sensibility","rank":47,"type":null,"human":1,"excerpt":"by the end of the movie I thought she was fantastic and forgot all about her iphone face, but there was definitely a long moment at the top of the movie (where it literally opens on her face) that I was thinking \"Uh oh. This looks like a girl with a regular Starbucks order.\"","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.88"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.95"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.90"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The commenter praises the actress and mentions a negative moment, leaning towards positive overall."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment praises the actress, indicating a positive verdict on the film itself."},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter had initial doubts about an actress's modern look, but ultimately found her performance fantastic."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Initial doubt about the actress's modern look, but ultimately found her fantastic; overall leans positive."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Commenter says the actress was fantastic by the end and their initial doubts faded, implying a favorable view of the performance and film."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Despite initial doubts about her appearance, the commenter ultimately found her performance fantastic."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"About an actress's look and performance, not the film overall."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":""},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They initially doubted her fit, but ultimately found her fantastic and forgot the concern."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Ended up thinking she was fantastic despite initial doubts."}}},{"split":"dev","movie":"Sense and Sensibility","rank":75,"type":null,"human":1,"excerpt":"This is a lovely thread! I was bummed to see on the jane austen reddit that people were really upset with this film\u2026 I am a huge Austen fan and I thought this film was wonderful. While Ang Lee\u2019s version feels lighter, it spends a lot of time establishing Wiloughby and Marianne\u2019s chemistry and connec","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +1.00"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.25 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.95"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.68"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The reviewer praises the film's romanticism, chemistry, and casting choices, especially for Alan Rickman's portrayal of Colonel Brandon. They also compliment Kate Winslet's portrayal of Marianne and F"},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The commenter praises the film, describes the characters positively, and compares it favorably to other adaptations."},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly states they thought the film was 'wonderful' and praises the casting and character portrayals."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Calls the new film wonderful, praises Brandon's casting as perfect and the portrayals; compares it favorably to Lee's version."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Commenter calls the film wonderful, praises Brandon as 'fantastic, perfect', and prefers its casting and chemistry over the 1995 version, despite small concessions."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Calls this adaptation wonderful and enthusiastically praises its casting, character portrayals, and romantic chemistry."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Defends the film as wonderful, praising casting, chemistry, and performances."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"The commenter calls the adaptation wonderful and praises its casting and performances, despite a few criticisms of other versions."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They call this version wonderful, praise its casting and performances, and say they believe the central romance."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Despite some mixed comparisons and critiques of other versions, the commenter states this film is wonderful and praises specific aspects."}}},{"split":"dev","movie":"Clayface","rank":0,"type":null,"human":1,"excerpt":"The scene where Matt has the whole club doing a choreographed dance was fantastic, actually that whole night club scene was very well done.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.72"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.91 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.49"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.53"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment praises a specific scene, indicating a positive verdict."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise for a specific scene and overall scene execution"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly praises a specific scene in the film as fantastic and very well done."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Praises the nightclub dance scene as fantastic and very well done, with no criticism."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Commenter praises the choreographed dance and nightclub scene as fantastic and very well done, indicating approval of the film."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Explicitly praises the choreographed dance and the entire nightclub scene as fantastic and well done."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Praises the nightclub scene as fantastic and well done."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter praises the nightclub dance scene as fantastic and very well done, a clear positive judgement of the film's execution."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They clearly praise the nightclub scene as fantastic and well done."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Praises specific scenes as fantastic and very well done."}}},{"split":"dev","movie":"Clayface","rank":8,"type":null,"human":1,"excerpt":"I think that was down to the low budget but the nightclub scene really seemed to turn that weakness into a strength. It was so cool.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.62"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.98 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.77"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.65"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment praises the low-budget film's nightclub scene as turning a weakness into a strength, which is a positive observation."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Comment praises the nightclub scene as a strength despite budget concerns"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"Praises a scene as 'so cool' and notes the film turned a low-budget weakness into a strength."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Praises the nightclub scene for turning a low-budget weakness into a strength, calling it so cool."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Praises the nightclub scene as cool, saying it turned a low-budget weakness into a strength; mild budget caveat but overall favorable."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Praises the nightclub scene as cool and says it turned budget limitations into a strength."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Praises nightclub scene for turning low budget into a strength."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter praises the nightclub scene as cool, turning a low-budget weakness into a strength."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They praise the nightclub scene as cool and say it turned the low-budget limitation into a strength."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter praises how the film turned a low budget into a strength, calling the scene cool."}}},{"split":"dev","movie":"Clayface","rank":65,"type":null,"human":0,"excerpt":"I'm surprised that Matt didn't piece together that his friend ratted him out much earlier seeing how the mob boss guy knew the exact quote Matt said when told not to mess with that girl","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.74"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.05 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.88"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.40"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment praises the plot and character development, suggesting the reviewer enjoyed the film."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment is discussing plot and character motivations without a clear positive or negative judgment of the film."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment only discusses a plot detail and character logic without judging the quality of the film itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Plot observation about a character's reasoning; no judgement of the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Plot observation about a character's missed inference; it doesn't clearly judge the film overall as good or bad."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Discusses a character's failure to recognize a plot clue without judging the film itself."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Plot logic observation only; no judgement of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Plot nitpick about a character's reasoning; no judgement of the film's quality or whether they'd recommend it."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Comments on a character's deduction in the plot without judging the film itself."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The comment discusses a plot point and character actions without expressing a judgment on the film."}}},{"split":"dev","movie":"Heart of the Beast","rank":0,"type":null,"human":0,"excerpt":"JK Simmons character felt like he was super natural . Maybe he was death or a spirit of the Forrest. Him going off on his boat gave off Charon on the river styx vibe Also when they crossed that one marsh it felt like \u201cThe valley of the shadow of death\u201d","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.87"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.12 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.85"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.35"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment describes the character as super natural and compares it to a death or spirit, which is a positive interpretation."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Describes the character in a positive, supernatural context"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment only provides character theories and thematic analysis without expressing an opinion on the film's quality."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Interpretive theory about Simmons' character as death or a spirit; no judgement of the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Commenter offers interpretation and theories about JK Simmons' character and symbolism, without stating whether they liked or recommend the film."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Interprets the character\u2019s supernatural symbolism and mythological parallels without judging the film\u2019s quality."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Symbolic character theories only; no judgement of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Offers theories and literary comparisons about the character without any judgement of whether the film is good or recommended."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Interprets the character and imagery but gives no opinion on the film itself."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The commenter discusses character theories and symbolism without expressing a like or dislike for the film."}}},{"split":"dev","movie":"Heart of the Beast","rank":28,"type":null,"human":0,"excerpt":"I like your idea that Simmons was the Boatman to the Underworld, and took his fee in two bottles of Belgian beer, maybe a prepayment in case James did survive the crash? He, the Boatman, did remark that James was further out than anybody he'd seen before. It adds another layer of thought to the stor","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.84"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.04 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.60"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.67"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the film and its depth, mentioning positive aspects like the Boatman's comment and the dog's hope."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise for character development and moral depth"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment engages in plot and thematic analysis without expressing a personal opinion on the film's quality."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Thematic interpretation and theory discussion without an explicit judgement of the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Interpretive discussion of themes and symbolism without explicitly stating whether the film was good or recommended."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Discusses symbolism, character motivations, and the title\u2019s meaning without judging the film itself."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Thematic plot interpretation only; no judgement of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":""},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They praise the story\u2019s thematic depth and the meaning behind its title."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The comment discusses plot theories and themes without expressing a clear verdict on the film."}}},{"split":"dev","movie":"Heart of the Beast","rank":5,"type":null,"human":0,"excerpt":"I kept that hope alive even after seeing clearly deceased Pitt on that rock. I thought maybe he's just kinda comatose.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.80"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.09 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.61"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.38"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment criticizes the film for not delivering on the character's potential, indicating disappointment."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment does not express a clear positive or negative verdict on the film."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment merely discusses a plot point and character death without expressing an opinion on the film's quality."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Describes emotional reaction to a plot event without judging the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment recounts personal reaction to a plot point (hoping a character survived) without judging the film's quality or recommending it."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Discusses hope for a character\u2019s survival without judging the film itself."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Plot speculation about a character's death, no film judgement."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Plot discussion about a character's apparent fate; no judgement of the film's quality or recommendation."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Speculates about a plot point without expressing an opinion about the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The commenter is only discussing the plot and a character's fate without expressing an opinion on the film."}}},{"split":"dev","movie":"Primetime","rank":0,"type":null,"human":1,"excerpt":"This world is full of weirdos, lunatics, and sociopaths. I hope Robert Pattinson has a chance to play all of them. He has like a dozen of the funniest line deliveries I've ever heard, but his physical performance really shines here. I was completely blown away by the way he spits out mouthwash, unro","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.59"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.95 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.62"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.35"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the acting and physical performance, with no overall negative lean."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise for physical performance and line deliveries"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"Strongly praises Robert Pattinson's comedic line deliveries and physical performance in the film."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Enthusiastic praise of Pattinson's line deliveries and physical performance; 'completely blown away'."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Enthusiastically praises Pattinson's performance, funny line deliveries, and physical acting; 'completely blown away' signals a strong positive view of the film."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Enthusiastically praises Pattinson\u2019s hilarious line deliveries and physical performance in the film."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Praises Pattinson\u2019s hilarious lines and standout physical performance."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":""},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"The commenter praises Pattinson\u2019s line delivery and physical performance, saying they were completely blown away."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter praises Robert Pattinson's performance, stating they were completely blown away by it."}}},{"split":"dev","movie":"Primetime","rank":12,"type":null,"human":1,"excerpt":"He's great too. The scene where he just starts doing an impression of Pattinson as Hansen might be the hardest I laughed in a movie where I laughed a lot","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.90"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.55 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.52"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.69"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The commenter is praising the movie and the scene, indicating a positive verdict."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise of a scene that made the commenter laugh, indicating enjoyment of the film."},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter praises a performance and notes they laughed a lot throughout the movie."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Praises an actor and says the film made them laugh a lot, a favorable verdict."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Praises an actor and says a scene was the hardest they laughed in a movie they laughed a lot in."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Praises the performance and describes laughing frequently, including an exceptionally funny scene."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Laughed a lot and praises a hilarious scene and performance."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter praises an actor and says they laughed hard, laughing a lot in the movie; clearly enjoyed it."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They praise the scene as hilarious and say they laughed a lot throughout the movie."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Praises the actor and humor, stating it caused intense laughter."}}},{"split":"dev","movie":"Primetime","rank":56,"type":null,"human":1,"excerpt":"It was great to see the shock once everyone realized it was a DA. I thought it was a good touch showing how the DA interacted constantly with the police to show how messed up this was.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.46"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.79 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.62"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.63"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment praises the film's use of shock and its portrayal of the DA's interactions with the police, indicating a positive view."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise for a specific scene and interaction in the movie"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter praises specific story choices and scenes as 'great' and a 'good touch'."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Praises the reveal and calls the DA-police interaction a good touch, approving of the film's choices."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Commenter says it was great and a good touch, praising the film's choices without any criticism."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Praises the reveal and the film\u2019s effective depiction of the DA\u2019s relationship with police."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Praises the reveal and the DA-police interaction as a good touch."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter praises the reveal and the film's handling of the DA-police dynamic, calling it a good touch."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Calls the reveal great and the DA\u2019s interaction with police a good touch."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter praises specific elements of the film, calling it a 'good touch' and expressing enjoyment."}}},{"split":"dev","movie":"Whalefall","rank":54,"type":null,"human":-1,"excerpt":"Austin Abrams faces a problem- movie flashbacks to a scene where his dad yells the EXACT same solution to getting out of said situation. Should\u2019ve been a survival thriller instead of a family drama. I can see this format working as a novel but translated to film it really does not work.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.17"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.99 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.70"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.98"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The reviewer criticizes the film's format and suggests it would work better as a novel."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"criticism of the film's format and structure"},"openrouter:google/gemini-4-argon":{"label":-1,"verdict":"negative","rationale":"The commenter criticizes the repetitive flashback structure and explicitly states that the format does not work as a film."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Criticizes repetitive flashback structure and says the format really does not work on film."},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Criticizes the film's flashback-driven family drama format and says it doesn't work on screen; should have been a survival thriller."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Criticizes the repetitive flashbacks and family drama focus, explicitly saying the format does not work on film."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Says the film format does not work and should have been a thriller."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":"Commenter criticizes the film's format and says it should have been different; translation to film 'really does not work'."},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"They say the film\u2019s format does not work and criticize its choice of family drama over survival thriller."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"The commenter states it should have been a different genre and 'translated to film it really does not work'."}}},{"split":"dev","movie":"Whalefall","rank":9,"type":null,"human":0,"excerpt":"I mean, even 127 Hours has a sort of masturbation scene in it. If you told me that the guy trapped in a whale tries to jerk it at some point I wouldn\u2019t be entirely surprised.","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.56"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.38 P(pos)=0.01"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.43"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.60"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The commenter is criticizing the film for including a scene that they found surprising and not entirely expected."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment discusses plot elements without a clear positive or negative judgment on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment makes a joke and speculates about a scene without giving an opinion on the film itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Joking comparison to 127 Hours; no judgement of the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"A joke speculating about the film's content by comparison to 127 Hours; no judgment of the film's quality or recommendation."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Speculates jokingly about a possible scene without judging the film's quality."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Joke speculating about a scene; no judgment of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Joking comparison to 127 Hours and speculation about content; no clear judgement of Whalefall's quality."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Jokes about a hypothetical scene without evaluating the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The comment is a joke/speculation about the plot and does not express a verdict on the film."}}},{"split":"dev","movie":"Klara and the Sun","rank":0,"type":null,"human":-1,"excerpt":"I didn\u2019t love the book, but I HATED this movie. The book was annoyingly ambiguous and meandering, whereas the movie was on the nose and hyper-quick to resolve plot points. The central storyline of Klara as Josie\u2019s replacement only gets about 15 minutes to marinate before everyone agrees it\u2019s a bad i","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.90"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.98"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.99"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The reviewer clearly disliked the movie and gave it a low rating."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"clear criticism with no overall lean, explicit statement of not loving the movie"},"openrouter:google/gemini-4-argon":{"label":-1,"verdict":"negative","rationale":"The commenter explicitly states they hated the movie, criticizes the pacing and cuts, and rates it 3/10."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Says they hated the movie, criticizes rushed plot, cuts, and director; only Ortega redeemable; 3/10."},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Says they HATED the movie, criticizes pacing, cuts and direction, gives 3/10; only Jenna Ortega's performance is praised."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Explicitly hated the movie, criticizing pacing, omissions, and shallow themes; Ortega\u2019s performance was its only redeeming feature. Rated 3/10."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Hated the movie overall, scoring it 3/10 despite a few liked elements."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":"Says they HATED the movie, criticizes its writing and cuts, rates it 3/10; only praises Ortega's performance."},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"They strongly dislike the movie overall, calling it a failure and rating it 3/10, despite praising Jenna Ortega\u2019s performance."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"The user explicitly states they HATED this movie, gives it a 3/10, and criticizes almost every aspect."}}},{"split":"dev","movie":"Klara and the Sun","rank":31,"type":null,"human":0,"excerpt":"He desperately needs someone who can reign him in and tell him no when necessary.","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.59"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.05 P(pos)=0.01"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.65"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.35"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the character's need for guidance and assertiveness."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"The comment is discussing a character's need for restraint, not providing a verdict on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"General comment about the director's style without judging the unreleased film itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment on a person's need for restraint; no judgement of the film itself."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment is about a character's needs, not a judgement of the film itself; no stated opinion on quality."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Criticizes an unspecified person's need for restraint without expressing a verdict on the film itself."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"No film judgment; remark about someone needing restraint."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment comments on a character's need for guidance; no judgement of the film itself is expressed."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Comments on a character\u2019s needs without evaluating the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"Comment discusses a character dynamic without expressing a verdict on the film itself."}}},{"split":"dev","movie":"Klara and the Sun","rank":12,"type":null,"human":0,"excerpt":"Didn\u2019t he cast himself as something and then cut his scene? There is hope!","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.27"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.12 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.38"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.72"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the actor's performance and suggesting hope, which is positive."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment is about casting and scene cuts, no clear judgment on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment discusses behind-the-scenes trivia about the director before seeing the film, offering no verdict on the movie itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Joke or trivia about someone cutting their own scene; no judgement of the film itself."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment is about trivia or casting and a cut scene; it gives no judgement of the film itself."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Jokes about the filmmaker cutting his own scene, without judging the film itself."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Trivia about casting and a cut scene, no film judgment."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Asks a trivia question about casting and a cut scene; no judgement of the film's quality expressed."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Jokes about the director cutting his own scene; gives no opinion on the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The comment discusses behind-the-scenes trivia and casting without judging the film itself."}}},{"split":"dev","movie":"Verity","rank":0,"type":null,"human":-1,"excerpt":"Ruining your marriage and dying because you didn\u2019t tell your husband it was opposite day","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.25"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.47 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.70"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.66"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment criticizes the film for its dark and disturbing content, specifically mentioning a scene that is described as brutal and beautiful, which is a negative verdict."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"The comment is a plot summary and does not express a verdict on the film."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment makes a joke summarizing the plot without expressing an opinion on the film's quality."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Joke summarizing plot twist; no judgement of the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"A joking summary of the plot with no explicit judgment of the film's quality or recommendation."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Jokes about plot events without expressing an opinion on the film's quality."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Joke about the plot, no judgement of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Joke about plot events and a 'opposite day' gag; no judgement of the film's quality or recommendation."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"A joke about the plot, with no clear judgment of the film itself."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The comment is a joke about the plot and does not express a verdict on the film."}}},{"split":"dev","movie":"Verity","rank":71,"type":null,"human":-1,"excerpt":"I think the twist in this movie is really just that they want things to feel ambiguous when in reality, the letter she finds that Verity wrote makes zero sense and is almost definitely a lie. The husband\u2019s reaction to Lowen telling him about the manuscript makes zero sense if he already knew Verity ","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.51"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.99 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.80"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.48"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The commenter is criticizing the film's plot and character motivations, which is a negative verdict."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"mixed","rationale":"Critiques plot inconsistencies without a clear overall lean"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The commenter analyzes the plot twist and character motivations without stating whether they liked or disliked the film."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Plot analysis and theorizing about the letter's truth; no clear judgement of the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses plot logic and theories about the twist and letter without clearly stating overall like or dislike of the film."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Criticizes the twist\u2019s forced ambiguity and repeatedly argues that key plot points and character motivations make no sense."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Plot-logic theory about the twist; no overall film judgement."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":"Commenter lists plot holes and says the twist makes zero sense, criticizing the film's logic, though engaging with theory rather than explicitly rejecting it outright."},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"They criticize the twist and plot logic as making little sense, expressing dissatisfaction with the film\u2019s narrative."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"The commenter points out multiple plot holes and logical inconsistencies, indicating dissatisfaction with the film's writing."}}},{"split":"dev","movie":"Street Fighter","rank":0,"type":null,"human":-1,"excerpt":"They really made it a point to ensure we knew it was the year 1993 at the start of this movie, only to feature a lot of music that came out well after 1993. They even referenced Taco Bell gorditas, which were introduced in 1998. And how were they singing karaoke to 4 Non Blondes in 1986? I realize t","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.43"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.03 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.67"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.65"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The reviewer criticizes the film for inconsistencies in the 90's nostalgia, which is a negative point."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"criticizing inconsistencies between the movie's setting and references to later years"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"Focuses on historical anachronisms and explicitly notes this doesn't reflect on the movie's overall quality."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Points out anachronisms but explicitly says they don't reflect on film quality; no overall judgement."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment points out anachronisms and distraction but explicitly avoids judging film quality; no clear overall like/dislike verdict."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"The historical inconsistencies were distracting, conveying criticism of the film despite qualifying their relevance to story quality."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Notes anachronisms but explicitly withholds judgment on quality."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":""},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"The anachronisms distracted them, expressing a criticism of the film even without judging its story."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"Discusses timeline/nostalgia inconsistencies without giving a clear verdict on the film's quality."}}},{"split":"dev","movie":"Street Fighter","rank":8,"type":null,"human":-1,"excerpt":"Is it just me or were there an insane amount of time skips/jumps in this. Felt like a ton of scenes were left on the cutting room floor.","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.18"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.99 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.70"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.90"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The commenter criticizes the film for having too many time skips/jumps, feeling like scenes were left on the cutting room floor."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Commenter is discussing technical aspects without expressing a clear like/dislike for the film itself."},"openrouter:google/gemini-4-argon":{"label":-1,"verdict":"negative","rationale":"The commenter criticizes the film's editing and pacing, noting it felt incomplete with too many time jumps."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Criticizes choppy editing and excessive time jumps, suggesting missing scenes; no praise offered."},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Criticizes the film's editing, noting excessive time skips and a sense that many scenes were cut, implying choppy, flawed storytelling."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Criticizes excessive time jumps and seemingly missing scenes, implying disjointed editing."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Criticizes excessive time skips and scenes left out, implying poor editing."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":"Complains about excessive time skips and missing scenes, a criticism of the film's editing with no offsetting praise."},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"They criticize the film's frequent time jumps and imply its editing leaves too much out."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"Criticizes the editing and pacing, feeling scenes were left out."}}},{"split":"dev","movie":"Street Fighter","rank":73,"type":null,"human":-1,"excerpt":"Once he said that, i was like nah man, don't. His delivery was so dull compared to Raul's and it was a hail mary attempt to save his already horrible character portrayal.","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.34"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.99 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.90"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.88"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The commenter disapproves of the actor's performance and criticizes the character portrayal."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Criticism of the character's delivery and portrayal, no positive aspects mentioned."},"openrouter:google/gemini-4-argon":{"label":-1,"verdict":"negative","rationale":"The commenter criticizes the dull delivery and horrible character portrayal in the film."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Criticizes an actor's dull delivery and horrible character portrayal, a negative judgment of the film's execution."},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Criticizes an actor's dull delivery and calls the character portrayal horrible, a clear criticism of the film's execution with no praise."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Criticizes dull delivery and horrible character portrayal, expressing a negative judgment of the film's performance."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Criticizes dull delivery and horrible character portrayal in the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":"Criticizes the actor's delivery as dull and calls the character portrayal horrible, a clear negative judgment of the film's execution."},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"They criticize the delivery and call the character portrayal horrible, expressing a negative view of the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"The commenter calls the character portrayal horrible and criticizes the delivery as dull."}}},{"split":"dev","movie":"Other Mommy","rank":0,"type":null,"human":1,"excerpt":"Kid: I left my stuffed doll back at the spooky house! Parents: Fuck that doll Finally got a smart scene with that bullshit cliche of getting people back in the house for stupid reasons","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.85"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.86 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.89"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.54"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":""},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Criticism of the plot and characters, no clear praise"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"Praises a scene in the movie for being smart and subverting an annoying horror cliche."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Praises the film for a smart scene that subverts the cliche of characters returning to the house for stupid reasons."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Commenter praises the scene as smart for subverting the clich\u00e9 of returning to the house, implying approval of the film's writing."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Praises the film for smartly avoiding a clich\u00e9d reason to return to the spooky house."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Praises one smart scene versus a cliche, no overall film judgement."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":""},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They praise the film for smartly subverting a contrived horror clich\u00e9."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Praises a smart scene that avoids a common horror clich\u00e9."}}},{"split":"dev","movie":"Other Mommy","rank":16,"type":null,"human":0,"excerpt":"I didn't think they were supposed to be taken as shitty parents tbh, I thought they were flawed but sincerely loving caretakers doing their genuine best","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.93"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.23 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.60"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.47"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the parents as flawed but sincere, doing their best."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Commenter discusses interpretation of characters, no clear positive or negative verdict on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment is pure character interpretation with no opinion expressed on the quality of the film itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Interprets the parents' characterization; offers no judgement of the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Interpretation of the parents' characterization; no judgment of the film's quality or recommendation."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Interprets the parents' characterization without expressing an opinion on the film's quality."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Character interpretation only; no judgement of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses how the characters should be interpreted as parents; no judgement of the film itself expressed."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Discusses how the parents should be interpreted, without expressing an opinion on the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"Discusses characters' parental morality without judging the quality of the film."}}},{"split":"dev","movie":"Other Mommy","rank":29,"type":null,"human":-1,"excerpt":"The ghost hunter scene is where this movie really lost me. It started off depicting an ordinary middle class family who is being haunted by a supernatural force\u2026 Kinda an interesting start. And at some point they randomly arrive at the grandmother\u2019s haunted mansion at the top of the hill where ghost","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.85"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.55"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.82"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The commenter criticizes the inconsistency of the ghost hunter scene, which is a negative point."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Criticism of plot inconsistency and unexpected elements"},"openrouter:google/gemini-4-argon":{"label":-1,"verdict":"negative","rationale":"The commenter states the movie lost them and felt very inconsistent, despite having a mildly interesting start."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Says the movie lost them; despite an interesting start, criticizes inconsistent, random plot turns."},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Says the movie 'really lost me' at the ghost hunter scene and calls it inconsistent, despite an interesting start."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"An initially interesting premise gave way to plot developments they found random and inconsistent, losing their engagement."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Lost interest; calls the plot inconsistent despite an interesting start."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":"Says the ghost hunter scene 'really lost me' and calls the story inconsistent; opening was interesting but overall lean is disappointed."},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"They say the movie lost them and criticize its plot as inconsistent."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"The commenter states the movie 'lost me' and felt 'very inconsistent'."}}},{"split":"heldout","movie":"Forgotten Island","rank":14,"type":null,"human":0,"excerpt":"\u201cSup Jesus, you seen Raissa?\u201d","pred":{"vader":{"label":0,"verdict":null,"rationale":"compound +0.00"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.37 P(pos)=0.99"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.66"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.39"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment is mocking and disrespectful, indicating a negative verdict."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"The comment is a casual greeting and does not express a verdict on the film."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment is just a quote or joke without expressing an opinion on the film."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Quotes a line from the film without expressing any judgement of the movie."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"A quoted line or joke from the film with no judgement of the film itself."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"The comment quotes dialogue without expressing an opinion on the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Just a quote or joke with no judgement of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"A quoted line and a question about a character; no judgement of the film is expressed."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"A quoted line with no opinion or judgment about the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The comment is just a quote from the movie, expressing no verdict."}}},{"split":"heldout","movie":"Forgotten Island","rank":16,"type":null,"human":1,"excerpt":"I really enjoyed this, but I think some of the accusations of quirk chungus humour and sensory overload are very valid lol","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.66"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.92 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.92"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.92"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is positive as it mentions enjoying the film and acknowledging valid criticisms."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"mixed","rationale":"enjoyed but valid criticisms mentioned"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly states they really enjoyed the film, despite acknowledging some valid criticisms about the humor and pacing."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Says they really enjoyed it but grants that some criticisms are valid; leans positive overall."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Says they really enjoyed it; acknowledges valid criticisms of humour and sensory overload but leans clearly toward liking it."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Explicitly really enjoyed the film, despite agreeing with criticisms of its humour and sensory overload."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Enjoyed the film overall despite agreeing with some criticisms."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Says they really enjoyed it; acknowledges valid criticisms of humour and sensory overload, but the overall lean is clearly positive."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They clearly say they really enjoyed it, while conceding some criticisms are valid."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter states they really enjoyed the film, despite acknowledging some valid criticisms."}}},{"split":"heldout","movie":"Forgotten Island","rank":30,"type":null,"human":0,"excerpt":"I was wondering how they were gonna cover that line \ud83e\udd23","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.49"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.05 P(pos)=0.99"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.65"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.53"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is playful and shows interest in the movie's plot, but does not express a negative or positive opinion."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"No clear judgment on the film itself, only a comment on a line from it."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment makes a joke/observation about a specific line without expressing an overall opinion on the film."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Amused remark about how a specific line was handled; no judgement of the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Amused remark about how a line would be handled; no judgment of the film's quality or recommendation."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"The comment discusses how a line would be handled without judging the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Amused curiosity about covering a line; no film judgement."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment is a lighthearted remark about how a line would be handled; no judgement of the film's quality expressed."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"A joking comment about how a line would be handled, with no judgment of the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The comment is a joke about a specific line, not a judgment on the film."}}},{"split":"heldout","movie":"Forgotten Island","rank":43,"type":null,"human":1,"excerpt":"I didn't find the movie overstimulating, should I get checked for ADHD? But for real, I think the pacing works because some things in the movie make no sense if you think about it for too long. Sometimes you just gotta turn your brain off, especially if you have a serious or demanding job and it's t","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.68"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.99 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.94"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.87"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The commenter is expressing disappointment and confusion about the movie's pacing, suggesting they didn't enjoy it."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment discusses pacing and brain turning off, no clear positive or negative verdict on the film itself."},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter defends the pacing and appreciates the film as good, mindless entertainment despite plot holes."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Defends the pacing and frames it as enjoyable turn-your-brain-off fun, despite admitting plot logic issues."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Defends the film's pacing as working and as good brain-off end-of-week entertainment, despite plot not making sense; leans favorable."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Defends the pacing and endorses the movie as undemanding entertainment despite its logical flaws."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Defends the pacing and endorses it as enjoyable brain-off entertainment."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter says pacing works and suggests turning brain off for it; implicitly endorses the film as a good watch, though with a mild note about plot logic."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They say the pacing works and recommend it as a turn-your-brain-off movie."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter defends the film's pacing and recommends it for unwinding at the end of the week."}}},{"split":"heldout","movie":"Forgotten Island","rank":67,"type":null,"human":0,"excerpt":"So, with the revelation about Batiba....I wonder what Jo's Lola gave up to get home.","pred":{"vader":{"label":0,"verdict":null,"rationale":"compound +0.00"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.08 P(pos)=0.02"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.54"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.60"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":0,"verdict":"mixed","rationale":"The comment discusses a plot point without clear positive or negative sentiment towards the film itself."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"The comment is discussing plot points without a clear positive or negative judgment of the film."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment is pure plot discussion and speculation with no evaluation of the film's quality."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Speculative plot question about a character's backstory; no judgement of the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Plot speculation about a character's backstory; no judgement of the film's quality or recommendation."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Speculates about a character\u2019s backstory and plot revelation without judging the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Plot speculation only; no judgement of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Speculates about plot content and a character's backstory without expressing any judgement of the film."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Speculates about a plot revelation and a character's past without judging the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The commenter discusses plot theories without expressing a judgement on the film."}}},{"split":"heldout","movie":"Forgotten Island","rank":68,"type":null,"human":0,"excerpt":"Mu friend theorized Fatima was friends with Manang and she was the one that sacrificed to let Fatime go back","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.74"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.03 P(pos)=0.98"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.74"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.55"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":0,"verdict":"mixed","rationale":"The comment discusses a theory about characters in the movie, but does not express a clear positive or negative opinion."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment is discussing theories about the characters without a clear judgment on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment only shares a plot theory without expressing any opinion or verdict on the film itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Shares a friend's plot theory about characters without judging the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Commenter shares a fan theory about characters and plot, without expressing any judgment of the film's quality."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"The comment shares a plot theory without expressing any judgment of the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Plot theory about characters; no judgement of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment offers a plot theory about characters with no judgement of whether the film is good or recommended."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"The commenter offers a theory about the plot without judging the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The comment discusses a plot theory without expressing any judgment or verdict on the film itself."}}},{"split":"heldout","movie":"Forgotten Island","rank":79,"type":null,"human":1,"excerpt":"It\u2019s a great movie overall, but did anyone else feel like the side characters were mostly useless? They were definitely going with a \u201cfound family\u201d aspect with the mythological creatures gradually joining them, but the Merman and Megamind Baby especially felt barely relevant until the plot needed th","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.69"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=0.99"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.97"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.91"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is overall positive, praising the movie as 'great' and noting a few minor criticisms without a strong overall negative stance."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"mixed","rationale":"Criticism of side characters but overall positive sentiment"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"Explicitly calls it a 'great movie overall' despite minor criticism regarding the relevance of the side characters."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Calls it a great movie overall; the criticism of the side characters is minor."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Calls it 'a great movie overall' and raises a side-character criticism as a minor caveat, so the lean is positive."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Explicitly calls it a great movie overall, despite criticism of underused side characters."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Calls it a great movie overall despite weak side characters."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Opens by calling it a great movie overall; the side-character complaints are a minor critique within a positive verdict."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They call it a great movie overall, despite criticizing the relevance of some side characters."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"mixed","rationale":"Praises the movie overall, but criticizes the useless side characters."}}},{"split":"heldout","movie":"Forgotten Island","rank":85,"type":null,"human":1,"excerpt":"I really really really loved this movie. It felt cool to see the expression of a culture I wasn't privy to before. I loved the use of mixed animation between the 3D and 2D segments and the reaction emphasis bubbles. It feels like animation techniques are finally catching up with how the internet gen","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.94"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.97 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.97"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.59"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":""},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Strong praise and no negative criticisms, only a minor suggestion for improvement"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly states they loved the movie and praises its animation and engaging story despite one minor criticism."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Loved the film, praised its animation and culture, kids hooked; only a minor criticism about thematic parallels."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Enthusiastic praise ('really really really loved'), kids were hooked, celebrates animation and ambition; only a minor criticism about the Manang-Jo parallel."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Explicitly loved the movie, praised its animation and themes, and found it engrossing despite one minor criticism."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Loved the film and kids were hooked; one minor theme critique."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":""},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They repeatedly praise the film, stayed engrossed, and eagerly anticipate similar movies, despite one minor criticism."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter states they 'really really really loved this movie' and praises its animation and cultural representation despite minor thematic critique."}}},{"split":"heldout","movie":"Wicker","rank":59,"type":null,"human":0,"excerpt":"The wickerman is only going to rot since he got lit on fire At least, that's what the basket maker said towards the end of the movie","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.34"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.87 P(pos)=0.01"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.75"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.31"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment criticizes the film's plot and characters, suggesting it is not enjoyable."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment is about a scene and quote from the movie without a clear positive or negative judgment."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment only discusses a plot detail and character dialogue without expressing any opinion on the film's quality."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment recounts a plot detail and a character's line without judging the film."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"A joke/quote referencing the film's ending and the Wicker Man; no judgement of the film's quality or recommendation."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Discusses a plot detail and character dialogue without judging the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Plot joke quoting a character; no judgment of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Plot-based joke referencing a line from the film; no judgement of the film's quality or recommendation."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"References a line from the movie without expressing an opinion about the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"Plot discussion and a joke without a judgment of the film."}}},{"split":"heldout","movie":"Wicker","rank":62,"type":null,"human":1,"excerpt":"The movie was surprisingly heartfelt, tackling the deep emotions of feeling unloved, especially in marriage, to discovering new ways to rekindle a dying flame (the baker and his wife doing roleplay and wanting a \"third\" was very entertaining but also great example of communication). The love between","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.95"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.99 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.98"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.81"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the movie's emotional depth, entertainment value, and acting performances."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"clear praise for emotional depth, acting, and character relationships"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter strongly praises the film as heartfelt, entertaining, emotionally moving, and having phenomenal acting."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Praises heartfelt themes, entertaining subplot, moving love story, and phenomenal acting."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Calls the film surprisingly heartfelt, entertaining, emotionally moving (made them tear up), and praises the acting as phenomenal."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Praises the film\u2019s heartfelt themes, entertaining relationship dynamics, emotional impact, and phenomenal acting."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Praises heartfelt story, entertainment, emotion, and phenomenal acting."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":""},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"The commenter praises the film\u2019s emotional depth, entertaining scenes, moving romance, and phenomenal acting."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Praises the heartfelt themes, entertainment value, emotional moments, and phenomenal acting."}}},{"split":"heldout","movie":"Wicker","rank":68,"type":null,"human":1,"excerpt":"This was a ton of fun. The hour and 40 minutes FLEW by. I found myself more emotionally invested than I expected to be!","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.56"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.76 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.70"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.79"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is clearly praising the film, using phrases like 'This was a ton of fun' and 'I found myself more emotionally invested than I expected to be! '"},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Clear praise of the film's entertainment value and time-passing quality"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly states the movie was a ton of fun, engaging, and that they were emotionally invested."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Calls it a ton of fun, says runtime flew by, and was emotionally invested."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Commenter says it was a ton of fun, flew by, and was more emotionally engaging than expected\u2014clear praise."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Calls the film fun, praises its brisk pacing, and expresses unexpectedly strong emotional investment."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Called it fun, fast-paced, and emotionally engaging."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter says it was a ton of fun, flew by, and they were more invested than expected."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Calls it a ton of fun and praises its pacing and emotional engagement."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter calls it 'a ton of fun' and expresses unexpected emotional investment."}}},{"split":"heldout","movie":"Wicker","rank":76,"type":null,"human":1,"excerpt":"This movie was so great! Wonderful performances and some twists I didn\u2019t see coming. And it was funny! Best movie I\u2019ve seen this year.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.96"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.61"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.46"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is filled with praise and enjoyment, with no negative elements."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Clear praise with no criticism"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly praises the movie, performances, twists, and humor, calling it the best movie they've seen this year."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Enthusiastic praise for performances, twists, and humor; calls it best movie of the year."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Commenter calls the movie great, praises performances, twists and humor, and names it the best movie they've seen this year."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Enthusiastically praises the performances, twists, and humor, calling it the best movie seen this year."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Explicit praise: great, wonderful, funny, best of the year."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter praises performances, twists, humor, and calls it the best movie of the year."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"The commenter enthusiastically praises the performances, twists, humor, and calls it their best movie of the year."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter explicitly praises the film as great, citing wonderful performances, twists, and humor."}}},{"split":"heldout","movie":"Wicker","rank":77,"type":null,"human":0,"excerpt":"Anyone have a picture of the Wicker Baby?","pred":{"vader":{"label":0,"verdict":null,"rationale":"compound +0.00"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.13 P(pos)=0.01"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.43"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.63"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"The comment is asking for a picture, not expressing a verdict on the film itself."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment is purely about seeking information and does not express a verdict on the film."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment merely asks for a picture of a movie prop/character without expressing any opinion on the film."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Requests a picture of a character; no judgement of the film expressed."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"The commenter only asks for a picture of the 'Wicker Baby' and gives no opinion on the film."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Asks for a picture of the Wicker Baby without expressing an opinion on the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Asks for a picture; no judgement of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Asks for an image of a film element; expresses no judgement of the film."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Asks for a picture; expresses no opinion about the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"Asking for a picture of a prop/character, unrelated to film quality."}}},{"split":"heldout","movie":"Wicker","rank":79,"type":null,"human":0,"excerpt":"It looked more human than renesme","pred":{"vader":{"label":0,"verdict":null,"rationale":"compound +0.00"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.74 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.66"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.32"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment praises the film's human-like characters."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment does not express a clear positive or negative verdict on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"A joke comparing a prop/character to Twilight's Renesmee, with no overall verdict on the film."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Joking comparison to Twilight's infamous CGI baby; no clear judgement of the film overall."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"A joke comparing a visual to Renesmee from Twilight; it gives no judgment of the film itself."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"A joke comparing appearances to Renesmee, without judging the film itself."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Visual comparison only; no judgment of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comparison of a creature's appearance to another character; no clear judgement of whether the film itself is good or worth recommending."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"It praises the film\u2019s visual effect by saying it looked more human than Renesmee."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"Joke or comparison unrelated to judging the film itself."}}},{"split":"heldout","movie":"Wicker","rank":81,"type":null,"human":1,"excerpt":"This was such a nice little movie","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.42"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.97 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.46"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.49"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is a clear praise of the film, using the word 'nice' to describe it."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment explicitly states the movie was 'such a nice little movie', indicating a positive verdict."},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter directly praises the film as a 'nice little movie'."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Calls it 'such a nice little movie,' a clear expression of liking the film."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Commenter calls it 'such a nice little movie', a clear favorable judgment of the film."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Calling it a nice little movie clearly expresses approval."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Calls it a nice little movie, clear praise."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Calls the film 'such a nice little movie', a clear warm endorsement of the film itself."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Calls the movie \u201cnice,\u201d expressing clear approval."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter describes the movie as 'such a nice little movie', indicating a clear positive recommendation."}}},{"split":"heldout","movie":"Wicker","rank":85,"type":null,"human":1,"excerpt":"On my end I loved that there was no explanation and it just happened. To each their own.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.40"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.39 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.56"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.66"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The commenter loves the lack of explanation and the unexpected nature of the film."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Loved the lack of explanation and found it satisfying."},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly states they loved the lack of explanation in the movie."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Commenter loved the lack of explanation, defending a film choice others criticized."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Commenter says they loved the film's lack of explanation, expressing a clear positive view of that choice."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"They loved the film\u2019s lack of explanation, expressing clear approval of its storytelling choice."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Loved the unexplained events; personal praise of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter says they loved the film's lack of explanation, expressing clear approval of it."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They explicitly loved the film\u2019s unexplained, abrupt approach."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter explicitly states they loved the aspect of the film they are discussing."}}},{"split":"heldout","movie":"You Can See Everything","rank":1,"type":null,"human":0,"excerpt":"\u201cHave you ever told a lie\u201d \u201c\u2026no I don\u2019t think I have\u201d \u201cLiz, come on\u201d","pred":{"vader":{"label":0,"verdict":null,"rationale":"compound +0.00"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.33 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.70"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.41"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the character's honesty and the way it is portrayed."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"The comment does not contain a verdict on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment is simply quoting dialogue from the movie without expressing any opinion or verdict."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment only quotes dialogue from the film without expressing any judgement of it."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"The comment only quotes a line of dialogue and gives no judgement of the film."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Quotes dialogue from the film without expressing an opinion about its quality."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Just quotes film dialogue with no judgement of the movie."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Just quotes dialogue from the film with no judgement of the movie itself."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Quotes dialogue without expressing an opinion about the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The comment only quotes a line of dialogue from the movie."}}},{"split":"heldout","movie":"You Can See Everything","rank":11,"type":null,"human":0,"excerpt":"Cathartic until it became clear he was 100 times worse than initially realized","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.13"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.77 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.75"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.38"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment criticizes the film for becoming worse than initially realized, which is a negative judgment."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Criticism of the main character's nature, indicating dislike of the film."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"Describes an emotional reaction to a character's actions and plot developments without judging the quality of the film itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Describes reaction to a character's revealed awfulness, not a judgement of the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Describes a character's arc or plot twist about a villain being worse than expected; no explicit judgment of the film's quality."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Describes catharsis and a character's worsening moral revelation, without clearly judging the film itself."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Initial catharsis soured once the character reveal undercut it."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"mixed","rationale":"Calls it cathartic at first, then the reveal of the character being worse is a neutral-to-ambiguous reaction; no clear overall lean on the film."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Calls the experience cathartic; realizing the character was worse is not criticism of the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"mixed","rationale":"Cathartic start but reveals disturbing actions, leading to a conflicted overall impression."}}},{"split":"heldout","movie":"You Can See Everything","rank":13,"type":null,"human":0,"excerpt":"That whole scene kept shocking me. Everything out of his mouth got more and more insane","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.71"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.78 P(pos)=0.01"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.77"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.47"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment praises the shocking and insane nature of the scene, indicating a positive reaction."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise of shocking and insane content"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"Describes a shocking scene and character dialogue without expressing an overall opinion on the film's quality."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Describes being shocked by a character's escalating dialogue without praising or criticising the film itself."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Describes a scene's shocking, insane dialogue without clearly judging the film itself; no explicit like, dislike, or recommendation."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Describes shock at a character's escalating dialogue without judging the film's quality."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Scene reaction only; no judgement of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Describes a scene as shocking and the dialogue as escalating insanity, a reaction that reads as engaged appreciation rather than criticism of the film."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Describes the scene as increasingly shocking and insane, but gives no clear opinion on the film itself."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The commenter describes a shocking scene without expressing a clear verdict on the film itself."}}},{"split":"heldout","movie":"You Can See Everything","rank":15,"type":null,"human":1,"excerpt":"Getting Amanda Seyfried in a room with him with a few glasses of wine was a brilliant decision. There's no way that conversation would have escalated the way it did if it was just him and Nathan. And Nathan sitting in the middle of it was just great visual comedy. As elusive and inscrutable as Holme","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.83"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.96"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.38"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The reviewer praises the film's writing, acting, and visual elements, particularly the chemistry between the characters and the portrayal of the antagonist."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise for the character's performance and scenes"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"Praises a 'brilliant decision' and 'great visual comedy', while expressing disgust at the documentary's subject rather than the film."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Praises a brilliant decision and great visual comedy; the insults target the subject, not the film."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Praises the dinner scene as a brilliant decision and great visual comedy; insults aimed at Evans's character, not the film."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Praises the dinner scene\u2019s staging and visual comedy; contempt targets Evans, not the film."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Praises the casting choice, visual comedy, and dinner scene as brilliant."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":""},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Calls the casting decision brilliant and the scene\u2019s visual comedy great."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Praises the acting, casting decisions, and scenes as brilliant and great."}}},{"split":"heldout","movie":"You Can See Everything","rank":31,"type":null,"human":0,"excerpt":"I desperately wish we'd gotten to hear more from him. I could really see how many of her behaviors could have stemmed from that - the relationship patterns, selective forgetfulness, and weaponized helplessness","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.46"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.15 P(pos)=0.04"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.53"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.59"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment criticizes the film but does not express a clear overall negative opinion."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment is discussing the character without a clear positive or negative judgment on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment analyzes character psychology and wishes for more focus on one character, without judging the film's overall quality."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Character analysis and a wish for more of one character; no clear judgement of the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses character analysis and wishes for more of a character, but gives no clear overall judgment of the film."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Discusses character behavior and wishes for more character perspective without judging the film overall."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Character analysis and wanting more backstory, no film judgement."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses character motivations and wishes for more story content, but gives no clear judgement of the film's quality or recommendation."},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"They express disappointment that the film did not let him speak more, criticizing its presentation of his perspective."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The comment discusses character behavior and plot psychology without expressing any judgment or verdict on the film itself."}}},{"split":"heldout","movie":"You Can See Everything","rank":37,"type":null,"human":1,"excerpt":"That they met in a WeWork was top tier comedy","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.51"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.99 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.78"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.69"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the movie's comedy aspect."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment is about the setting of the movie and not a direct judgment of the film itself."},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"Praises a specific comedic detail in the film as 'top tier comedy'."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Praises a specific comedic detail in the film as 'top tier comedy,' implying enjoyment."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Praises a specific detail as 'top tier comedy', implying enjoyment of the film's humor, though no explicit overall recommendation."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Praises the film\u2019s WeWork meeting scene as top-tier comedy."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Calls a scene top tier comedy, praising the film's humor."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Joke about a plot detail; no clear judgement of the film's quality or recommendation."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They praise the WeWork meeting as top-tier comedy, expressing enjoyment of the film\u2019s humor."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Praising a specific aspect of the film as top tier comedy."}}},{"split":"heldout","movie":"You Can See Everything","rank":59,"type":null,"human":0,"excerpt":"The \"Who's On First\" scene just broke my brain so much.","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.42"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.99 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.75"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.55"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The commenter is expressing strong disapproval of a scene, which is a clear negative verdict."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment is purely about a specific scene and does not express a verdict on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment reacts to the complexity of a specific scene without expressing whether they liked or disliked the film."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Strong reaction to one scene, but 'broke my brain' is ambiguous and gives no clear judgement of the film."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Says a scene 'broke my brain' but it's ambiguous whether praise or criticism; no clear judgment of the film overall."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Expresses bewilderment at a specific scene without clearly praising or criticizing the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Scene reaction only; no clear overall film judgement."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"The commenter says a scene 'just broke my brain', an enthusiastic reaction implying they were impressed by it."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"They react to a scene but don't clearly judge the film or recommend it."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"Comment discusses a specific scene without expressing a clear verdict on the film."}}},{"split":"heldout","movie":"You Can See Everything","rank":80,"type":null,"human":0,"excerpt":"Well, pretty sure I love Amanda Seyfried as much as I hate Billy Evans now.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.80"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.99 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.55"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.82"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":0,"verdict":"mixed","rationale":"The comment criticizes the film but does not lean one way overall."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Hates a character, criticizes the film"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"Expresses love for the lead actress and strong emotional investment against a character/subject."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Loves Seyfried's performance and strongly hates the villain, implying both were effective; no criticism of the film."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Expresses love for the actress and hatred of a character, but gives no judgement of the film itself."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Expresses affection for Amanda Seyfried and hatred of Billy Evans, without judging the film itself."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Actor/character feelings only; no judgment of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment expresses feelings about actress and a character, not a verdict on the film itself."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"They praise an actor and dislike Billy Evans, but do not judge the film itself."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter states they love the actress's performance, indicating a positive view of the film."}}},{"split":"heldout","movie":"Primetime","rank":2,"type":null,"human":1,"excerpt":"Also him answering the phone with \u201cThis is Chris Hansen\u201d and his wife saying \u201cI know\u2026\u201d Cracked me up.","pred":{"vader":{"label":0,"verdict":null,"rationale":"compound +0.00"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.19 P(pos)=0.99"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.49"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.71"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the humor and cracking a joke."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment is about plot details and jokes, no verdict on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The commenter shares a specific line they found funny without giving an overall verdict on the film itself."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Commenter enjoyed a specific funny scene, saying it cracked them up, which implies mild enjoyment."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Commenter says a specific scene cracked them up, expressing enjoyment of the film, though no overall verdict is given."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Praises a comedic moment in the film, saying it cracked them up."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Amused by one quoted gag, no film judgement."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Quotes a funny moment from the film but gives no overall judgement of the film itself."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"They say a moment made them laugh, but offer no overall judgment of the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The commenter is only sharing a funny moment and quoting the movie without expressing an overall opinion."}}},{"split":"heldout","movie":"Primetime","rank":28,"type":null,"human":0,"excerpt":"That scene they did with the Rabbi made me feel so uneasy. Lowkey terrifying honestly.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.18"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.83 P(pos)=0.01"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.38"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.30"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment describes a scene as 'lowkey terrifying honestly', which is a positive assessment of the film."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise of dark and scary content"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"Expresses effective emotional impact ('uneasy', 'terrifying') from a scene, which serves as praise for dark/disturbing content."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Calling the Rabbi scene unsettling and terrifying reads as appreciation of its effective tension, with no criticism."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Describes feeling uneasy and terrified by a scene, but gives no explicit judgement of the film's quality; unease may be intended effect."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Describes a scene as unsettling and terrifying without judging the film\u2019s quality or recommending it."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Calls a scene terrifying and uneasy, praising its scare effect."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Describes the scene as 'lowkey terrifying' and unsettling, which is praise of disturbing content, not criticism of the film."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They describe the scene as effectively terrifying, praising its impact rather than criticizing the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Praising the scene as terrifying indicates effective filmmaking and a positive reception of the movie's impact."}}},{"split":"heldout","movie":"Primetime","rank":34,"type":null,"human":0,"excerpt":"I mean that's basically what the hypocritical DA said in his argument in court. Id wager the themes are related deliberately","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.67"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.31 P(pos)=0.01"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.62"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.48"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the themes and suggesting they are deliberately related, which is a positive interpretation."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment is about the themes and argument, no clear verdict on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment discusses plot points and themes without expressing a judgment or opinion on the film's quality."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses plot and thematic connections without judging the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses plot and themes, speculating on deliberate thematic links, without judging the film's quality or recommending it."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Discusses a character\u2019s courtroom argument and thematic connections without judging the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Thematic plot discussion only, no judgement of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses plot/themes and a courtroom argument in the film without expressing any judgement of the film's quality."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Discusses a thematic connection to the DA\u2019s argument, but gives no opinion on the film itself."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"Comment discusses plot themes and court arguments without expressing a judgment or recommendation for the film."}}},{"split":"heldout","movie":"Primetime","rank":42,"type":null,"human":0,"excerpt":"I\u2019d recommend watching the doc predators if you haven\u2019t","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.36"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.43 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.53"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.40"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is recommending the film 'Predators'."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Direct recommendation to watch the doc"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"Recommends a related documentary without expressing an opinion on the movie itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Recommends a different documentary; no judgement expressed about this film."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Recommends a different documentary (Predators) and gives no judgement of Primetime itself."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Recommends a different documentary, not the film Primetime."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Recommends another documentary, no judgment of this film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Recommends a different show (Dateline-style docs 'Predators'), not a judgement of Primetime itself."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Recommends a different documentary without judging this film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The commenter recommends a different documentary entirely, not expressing a verdict on Primetime."}}},{"split":"heldout","movie":"Primetime","rank":53,"type":null,"human":1,"excerpt":"There's this feeling of uneasiness the film captures so well that you can't help but feel sick and suffocating as we're watching this macabre exploitation of people, really hits home about the pursuit of sensationalism.","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.70"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.98 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.43"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.46"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment criticizes the film for being macabre and exploitative, suggesting it hits home about the pursuit of sensationalism, which is a negative point."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":""},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"Praises the film for effectively capturing a disturbing feeling of uneasiness and delivering a poignant message on sensationalism."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Praises how well the film captures unease and its effective commentary on sensationalism."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Praises the film for capturing uneasiness so well and hitting home on sensationalism; the sickening feeling is credited as the film's success."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Praises the film\u2019s effective uneasiness and powerful critique of sensationalism; disturbing feelings are presented as artistic success."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Praises how well the film captures uneasiness and hits home."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Praises the film for capturing uneasiness and hitting home about sensationalism; the unsettling content is framed as well done."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Praises how effectively the film captures unease and critiques sensationalism."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Praises the film for successfully capturing uneasiness and making a strong point about sensationalism."}}},{"split":"heldout","movie":"Primetime","rank":66,"type":null,"human":1,"excerpt":"pattinson is on a generational run and disappears into yet another role. skyler gisondo was also fantastic. really gorgeous and creative shots throughout, and the whole vibe of the movie was very dirty and almost hypnotic. loved it.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.87"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.87 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.55"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.56"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":""},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Clear praise with no criticism"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly states they 'loved it' and praises the acting, cinematography, and hypnotic vibe."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Praises the performances, the cinematography and the hypnotic vibe, and explicitly says 'loved it.'"},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Praises the acting, cinematography, and hypnotic atmosphere, and ends with an explicit 'loved it'."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Explicitly loved the film, praising the performances, creative cinematography, and hypnotic atmosphere."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Praises performances, shots, and vibe; explicitly loved it."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Praises the cast, the visuals, and the vibe, and ends with 'loved it.'"},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"The commenter praises the performances, visuals, and atmosphere, and explicitly says they loved the movie."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter praises the acting, cinematography, and overall vibe, concluding with 'loved it'."}}},{"split":"heldout","movie":"Primetime","rank":90,"type":null,"human":1,"excerpt":"This movie was a hate letter to the 2000s and I loved it. I thought the main anchor parts were a little hammy though","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.05"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.94"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.93"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment criticizes the movie but also expresses that they liked it, leaning towards positive."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Loved it despite a minor criticism"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly states they loved the movie despite a minor criticism of the acting."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Says they loved it; the minor criticism of hammy anchor parts doesn't outweigh the clear praise."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Explicitly says they loved it; the minor critique of hammy anchor parts doesn't change the overall positive lean."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Explicitly loved the movie despite finding the main anchor parts a little hammy."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Loved the film overall despite finding some parts hammy."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter says they loved the film; the only criticism, hammy anchor parts, is minor and does not offset the praise."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They loved the movie despite finding the main anchor parts a little hammy."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Explicitly stated 'I loved it' despite minor criticism of some parts."}}},{"split":"heldout","movie":"Primetime","rank":91,"type":null,"human":1,"excerpt":"Merritt weaving did amazing. Puka? No puka? Line delivery of the century","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.45"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.52 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.91"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.52"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment praises the actor's performance and the line delivery, indicating a positive verdict."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise for the line delivery"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter strongly praises Merritt Wever's acting performance and line delivery in the film."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Praises Merritt Wever's performance as amazing and calls a line delivery 'of the century'."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Praises Merritt Weaving's performance as amazing and calls a line delivery the best of the century, showing clear appreciation of the film."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Enthusiastically praises the acting and a specific line delivery."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Praises the actor's performance and a standout line delivery."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Praises Merritt's performance as 'amazing' and calls a line delivery 'of the century', a clear endorsement."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Enthusiastic praise for Merritt Weaving\u2019s performance and line delivery signals approval of the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Praises the lead actor's performance and iconic line delivery."}}},{"split":"heldout","movie":"Wildwood","rank":10,"type":null,"human":0,"excerpt":"Without a doubt. What was shocking was how visceral and violent the battle got. There were real bludgeonings happening throughout this fight.","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.71"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.88 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.32"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.58"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the visceral and violent nature of the battle, which is a positive aspect of the film."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise of violent and visceral content"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"Describes the shocking violence of a battle scene without stating whether they liked the movie."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Agrees with an unknown prior point and describes the battle's violence without clearly praising or criticising the film."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Agrees emphatically ('Without a doubt') and describes the violent battle as shocking and visceral, which reads as praise rather than criticism."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Describes the battle\u2019s shocking violence without clearly praising or criticizing the film."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Affirms the film and praises the battle's visceral violence."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter praises the fight as shocking and visceral, implying approval of the violent content; 'without a doubt' signals agreement with a positive view."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Describes the battle\u2019s violence but gives no clear opinion about the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Praising the visceral and violent battle scenes as shocking and impactful."}}},{"split":"heldout","movie":"Wildwood","rank":46,"type":null,"human":0,"excerpt":"This got me too, and I think it's because he is shown saying goodbye to his wife and small child like 10 minutes prior. He didn't get a lot of screen time, but it was enough to show the magnitude of what was being lost.","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.30"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.95 P(pos)=0.01"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.62"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.59"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment praises the film for its emotional impact and the portrayal of loss."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment is discussing plot and character actions without a clear positive or negative judgment on the film itself."},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"Praises the emotional impact and effective characterization of a brief scene."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Says a character's death affected them emotionally and credits the film's effective setup; implicit praise."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses an emotional scene and why it worked for them, but gives no overall verdict on the film or recommendation."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Praises the film\u2019s economical characterization for conveying the magnitude of a loss and creating emotional impact."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Moved by the scene; praises how the loss landed emotionally."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses a scene's emotional effect and storytelling reasoning but gives no clear judgement of the film overall."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They say the scene emotionally affected them and effectively conveyed the magnitude of the loss."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"Discusses a character's emotional scene without evaluating the film itself."}}},{"split":"heldout","movie":"Wildwood","rank":47,"type":null,"human":1,"excerpt":"Just saw this in a Portland theatre and there was applause throughout the end credits. Hometown heroes and all, haha The movie itself is exceptional. Visually stunning the entire time but especially the giant set piece at the climax. Definitely worth seeing in the theater (even if they don't clap in","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.05"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.98 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.96"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.89"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is filled with praise and recommends the film, with no overall lean towards negative."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise for visual stunning, exceptional movie, worth seeing in theater, and positive reaction to intensity and violence"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"Calls the movie exceptional, visually stunning, and definitely worth seeing in theaters, praising its intensity in a good way."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Calls it exceptional and visually stunning, recommends theater viewing; the intensity is framed as good."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Calls the movie exceptional, visually stunning, and worth seeing in theaters; the intense violence is framed as a good thing."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Calls the film exceptional and visually stunning, recommends theaters, and appreciates its unexpectedly intense content."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Calls the film exceptional, visually stunning, and worth seeing in theaters."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":""},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Calls the movie exceptional and visually stunning, and recommends seeing it in theaters; the intense violence is praised."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter calls the movie exceptional, visually stunning, and definitely worth seeing."}}},{"split":"heldout","movie":"Wildwood","rank":50,"type":null,"human":1,"excerpt":"Hit soooo close to home as a native portlander. Goddamn.","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.48"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.88 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.51"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.40"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment expresses a strong liking for the film, using phrases like 'Hit soooo close to home' and 'Goddamn', indicating a positive reaction."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"The comment does not express a verdict on the film Wildwood."},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"Expresses strong personal resonance with the film's representation of Portland."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Says the film resonated deeply on a personal level; the emotional exclamation implies strong appreciation."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Says the film hit very close to home as a native Portlander, implying emotional resonance and appreciation, though no explicit recommendation."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"The emphatic reaction suggests the film resonated strongly with them as a Portland native."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Strong personal resonance as a Portlander implies the film landed powerfully."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Commenter relates personally as a Portlander but gives no clear judgement of the film's quality or recommendation."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They say the film resonated deeply with them as a Portland native."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Relatable and impactful connection to the film's setting."}}},{"split":"heldout","movie":"Wildwood","rank":67,"type":null,"human":0,"excerpt":"Prue and Curtis are stronger better people than me, that\u2019s for certain If this was a live action movie, Alexandra would 1000% be played by Cate Blanchett","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.84"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.97 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.68"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.58"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment criticizes the casting choice but does not express a negative opinion of the film itself."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise for characters and casting choices"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment discusses characters and hypothetical live-action casting without expressing an opinion on the film's quality."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comments on characters and a casting joke; no judgement of the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comments on characters and casting speculation; no judgement of the film's quality or recommendation."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Comments on characters and hypothetical casting without evaluating the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Character praise and casting joke, no film judgement."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Praises characters and casting hypothetical; no judgement of the film itself is expressed."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Comments on the characters and fancasting, without judging the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The comment discusses characters and potential casting without giving any verdict on the quality of the film."}}},{"split":"heldout","movie":"Wildwood","rank":84,"type":null,"human":0,"excerpt":"She flew an eagle to a dark wood, climbed a mountain, and fought in a battle of five armies. This is a Hobbit movie.","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.60"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.22 P(pos)=0.99"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.63"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.91"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the movie as a Hobbit movie, which is a positive aspect."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"The comment is a theory about the movie's genre, not a verdict on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment makes a humorous plot comparison to The Hobbit without expressing an opinion on the movie's quality."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Joke comparing the plot to The Hobbit; no clear judgement of the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Joking comparison to The Hobbit's plot elements; no explicit judgement of whether the film is good or recommended."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Jokingly compares plot elements to The Hobbit without expressing approval or criticism of the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Joke comparing the plot to The Hobbit, no film judgement."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Plot summary with a comparison to The Hobbit; no judgement of the film's quality or recommendation."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"The comment makes a humorous comparison to The Hobbit without judging Wildwood."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The comment is a joke comparing the movie's plot to The Hobbit without expressing a personal verdict."}}},{"split":"heldout","movie":"Wildwood","rank":95,"type":null,"human":0,"excerpt":"This movie is absolutely gorgeous. The whole scene with the Giant Eagle flying and the 3rd act battle had me glued to the screen the whole time. Just absolutely breathtaking visuals and colors and contrasts. Character wise? I dunno. Prue feels like the weakest Laika protagonist despite this being La","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.76"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.97 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.51"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.95"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The reviewer praises the visuals, characters, and overall themes of the movie, with only minor criticisms."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praises visuals, overall tone, and themes; expresses disappointment with one character but overall positive sentiment"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"Overwhelming praise for breathtaking visuals, supporting characters, and nuanced themes, despite mild criticism of the protagonist."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Praises visuals, villain, supporting characters, themes and serious tone; the only criticism is a weak protagonist."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Strongly praises visuals, villain, supporting characters, and themes; only criticism is a weak protagonist. Overall clearly leans favorable."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Enthusiastically praises visuals, supporting characters, and thematic depth, despite criticism of the protagonist."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Glued by gorgeous visuals and themes despite weak protagonist."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Calls the film absolutely gorgeous and breathtaking, praises visuals, action and themes despite weaker protagonist critique."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Strong praise for the visuals, characters, and themes outweighs the criticism of Prue\u2019s characterization."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Strong praise for visuals, action, and themes outweighs minor character criticisms, resulting in an overall favorable review."}}},{"split":"heldout","movie":"Wildwood","rank":98,"type":null,"human":1,"excerpt":"I just want to bring up the point that beyond the incredible animation (truly stunning), the script of this movie deserves an Oscar nomination. The story was so well structured, and I thought it was brilliantly paced. I wasn't bored for a second of it. The story was surprising, funny, exciting and s","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.95"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.99 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.66"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.59"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":""},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praises the script, story structure, pacing, and creativity"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"Enthusiastically praises the animation, script, structure, and pacing, suggesting it deserves an Oscar nomination."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Praises stunning animation, brilliant script and pacing, hopes for an Oscar nomination."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Effusive praise for the animation, script, pacing, and creativity; hopes for an Oscar nomination and says they were never bored."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Enthusiastically praises the stunning animation, screenplay, pacing, and creative story, calling it Oscar-worthy."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Praises stunning animation, Oscar-worthy script, pacing, and creativity throughout."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Praises the animation, script, pacing, and creativity, and hopes it earns an Oscar nomination. Clearly favorable."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Strongly praises the animation, script, story, pacing, and entertainment value."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter praises the animation, script, pacing, and overall story, explicitly hoping it gets award recognition."}}},{"split":"heldout","movie":"Sense and Sensibility","rank":12,"type":null,"human":1,"excerpt":"Marianne being more worried for Col. Brandon than Willoughby was the cherry on the top of the whole dueling scene, she knew Brandon would be a keeper. I love seeing Mrs. Jennings being excited about witnessing the duel much to the chargin of the Dashwoods lol","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.83"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.44 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.61"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.89"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the character development and the ending of the duel scene, with no overall negative lean."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise for character development and emotional depth in the scene"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"Expresses enjoyment of specific character moments and scenes in the film."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Enthusiastic about the dueling scene, calling it a 'cherry on top' and loving Mrs. Jennings' reaction."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Commenter expresses affection for specific scenes and characters (\"I love seeing...\", \"cherry on top\"), implying enjoyment of the film, though no explicit overall recommendation."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Enthusiastically praises the dueling scene and enjoys Mrs. Jennings\u2019s reaction."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Enjoys key scenes, calling one the cherry on top and loving another."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses plot details and character moments in the duel scene without stating an overall judgement of the film."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They express enjoyment of the dueling scene and Mrs. Jennings\u2019s reaction."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Expresses enjoyment of specific scenes and characters in the film."}}},{"split":"heldout","movie":"Sense and Sensibility","rank":19,"type":null,"human":0,"excerpt":"The book is excellent, obviously. Jane Austen has a wicked, cheerfully cynical cleverness that simply can\u2019t be adapted well on the screen. The Ang Lee movie is fantastic, just a masterpiece, and one of the most beautiful movies ever shot. I recommend you treat yourself to both. I haven\u2019t seen this o","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.98"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.88 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.51"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.71"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The reviewer praises the Ang Lee movie as a masterpiece and recommends it, with no overall negative lean."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Explicit praise and recommendation for the movie, describes it as a "},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The commenter praises the Ang Lee version, though explicitly states they have not seen the specific adaptation being discussed."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Commenter hasn't seen this film; praises the book and the Ang Lee version, only excited about this one."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Commenter hasn't seen this film; praise is for the book and Ang Lee's version, not the one discussed."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"They haven't seen this adaptation; their praise concerns Ang Lee's film, the book, and this film's trailer."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Has not seen this adaptation; praises book and Ang Lee film."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Calls the Ang Lee film a fantastic masterpiece and recommends it, though they haven't seen this one yet and are excited by others' praise."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"They praise the book and Ang Lee adaptation, but say they haven\u2019t seen this film yet."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter calls the Ang Lee movie a masterpiece, beautiful, and highly recommends it."}}},{"split":"heldout","movie":"Sense and Sensibility","rank":37,"type":null,"human":0,"excerpt":"Agreed, you can like both. I think it\u2019s cool to have multiple versions. Not everything needs to be a perfect adaptation.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.86"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.03 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.62"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.65"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is agreeing with the idea of having multiple versions and does not criticize the film."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Commenter discusses multiple versions of the story, no clear verdict on the film itself."},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter defends the film as a worthwhile adaptation, implying they liked both versions."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"General comment about multiple adaptations coexisting; no judgement of this specific film."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment speaks generally about enjoying multiple adaptations without giving a clear verdict on this specific film."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Supports multiple adaptations generally without expressing a clear judgment of this film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"General take on adaptations, no judgment of this film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses adaptations generally and multiple versions; no clear judgement of this film itself."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"They express approval of having multiple adaptations, but give no clear opinion on this film itself."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Appreciates multiple versions of the film despite adaptation flaws."}}},{"split":"heldout","movie":"Sense and Sensibility","rank":38,"type":null,"human":1,"excerpt":"And it's great to revive classic stories for a new audience - 30 years between films is the perfect gap","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.88"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.42 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.64"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.69"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the revival of classic stories and the gap between films, which is a positive sentiment."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise for reviving classic stories and the timing of the film"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"Approves of the concept of remaking the story after 30 years without judging the quality of the film itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comments on remaking classics and the timing between films, without judging this film itself."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comments generally on reviving classic stories and the gap between adaptations; no explicit judgment of this film's quality."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Praises reviving classic stories and the timing between adaptations, without judging this film itself."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Praises remaking classics and timing, not this film's quality."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Praises the film's revival of a classic story and the timing of the remake as a good idea."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They praise reviving the story for a new audience and say the gap between films is perfect."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Praises the film adaptation ('great to revive classic stories')."}}},{"split":"heldout","movie":"Sense and Sensibility","rank":44,"type":null,"human":0,"excerpt":"I think too many people think fuller lips = lip filler. I\u2019m honestly not familiar with the actress beyond the film, but there was nothing fake looking about her imo. Also, some people just have naturally fuller lips. Doesnt automatically mean they\u2019ve had work done.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.65"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.22 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.46"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.51"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the actress's natural appearance without any negative criticism of the film itself."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment is about the actress and her appearance, not the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"Discusses an actress's physical appearance and lip filler assumptions without expressing an opinion on the film itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses actress's lips and filler speculation; no judgement of the film itself."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment discusses whether an actress's lips look natural, not any judgement of the film itself."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Discusses the actress\u2019s lips and cosmetic surgery assumptions without judging the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Discusses an actress's appearance, not the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment discusses an actress's lips and cosmetic work, not a judgement of the film itself."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"The comment discusses the actress\u2019s appearance, not whether the commenter liked the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The comment discusses an actress's appearance and makeup/plastic surgery rumors, not the film itself."}}},{"split":"heldout","movie":"Sense and Sensibility","rank":61,"type":null,"human":-1,"excerpt":"Maybe I am too much of an Austen purist but I think they went too far with changing the Edward/Lucy backstory from the book. It was not some arranged engagement to a rich girl he nobly upheld when she became penniless. He proposed to her as a young man and kept it secret to not upset his mother who ","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.97"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.99 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.70"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.51"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The commenter criticizes the changes made to the backstory and character development, finding them too drastic and detracting from the character's integrity."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Criticism of the film's portrayal of Edward/Lucy's backstory and character development"},"openrouter:google/gemini-4-argon":{"label":-1,"verdict":"negative","rationale":"The commenter criticizes the film's drastic, negative changes to a character's backstory and characterization compared to the book."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Criticizes the film's changes to Edward's backstory as too drastic and damaging to his character."},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Criticizes the film's changes to Edward/Lucy backstory as too drastic and damaging to his character, though it's a focused critique rather than an overall verdict."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Criticizes the adaptation\u2019s backstory changes for distorting Edward\u2019s character and undermining Elinor\u2019s reasons for loving him."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Criticizes the film for drastically changing Edward\u2019s backstory and undermining his character."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":""},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"They criticize the film\u2019s changes to Edward\u2019s backstory as too drastic and damaging to his character."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"The commenter criticizes the film's changes to the character and plot."}}},{"split":"heldout","movie":"Sense and Sensibility","rank":71,"type":null,"human":-1,"excerpt":"Idk if it was just my theater/projection, but man this was visually dark. I love the concept of the natural lighting but there\u2019s a way to do it that artfully highlights what happens on screen. I felt like I was missing out on the cinematography and letting it sing .","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.88"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.88"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.93"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The commenter appreciates the cinematography and feels they were missing out on it."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Comment praises the visual aspects of the film, specifically the natural lighting and cinematography."},"openrouter:google/gemini-4-argon":{"label":-1,"verdict":"negative","rationale":"The commenter criticizes the film's overly dark lighting and feels it ruined the cinematography."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Criticizes overly dark visuals obscuring the cinematography; likes the concept only, with no praise of the film overall."},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Criticizes the visual darkness as hurting the cinematography, saying they felt they were missing out; praise of the concept is outweighed by disappointment."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Criticizes overly dark visuals that obscured the cinematography, despite appreciating the natural-lighting concept."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Disappointed the natural lighting was too dark and obscured the cinematography."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":""},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"They felt the film\u2019s dark visuals obscured the cinematography and diminished their viewing experience."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"The commenter criticizes the visual darkness and feels they missed out on the cinematography."}}},{"split":"heldout","movie":"Sense and Sensibility","rank":74,"type":null,"human":0,"excerpt":"Best part was Fiona Shaw coming in and wrecking the misery with her cheerful insistence on being kind . And the parade of terrible dead animals given in kindness, too. I also loved that shot of London surrounded by greenery. And the crowded parties. But: - Marianne, you look ridiculous. I do not buy","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.98"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.86"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.97"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":""},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Overall positive sentiment with praise for certain aspects and specific scenes, and a clear lean towards the film being a "},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"Despite several specific criticisms, the commenter concludes that overall it was a lovely, feel-good film."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Several criticisms, but the overall lean is positive: 'feel-good film and a lovely way to spend two hours.'"},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Lists several specific criticisms but concludes it was a feel-good, lovely way to spend two hours; overall leans positive."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Despite several criticisms, they call it a feel-good film and a lovely way to spend two hours."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Lists several flaws but overall calls it a lovely feel-good film."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":""},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Despite several criticisms, the commenter calls it a feel-good film and a lovely way to spend two hours."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Despite several specific criticisms of characters and scenes, the user concludes it was a feel-good film and lovely."}}},{"split":"heldout","movie":"Clayface","rank":4,"type":null,"human":1,"excerpt":"I\u2019m just happy we didn\u2019t get surf Dracula\u2019d on him turning into the full Batman the Animated Series Clayface He used his powers perfectly in the context of the story, and he kept getting more powerful as time went on in a believable way. It didn\u2019t go from 0 to supervillain right away. It\u2019s very easy","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.91"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.10 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.87"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.65"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The reviewer praises the character's development and believability, without criticizing the film as a whole."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praises Clayface's powers and power increase, describes it as believable and fits the story"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"Praises the believable pacing of the character's progression and how his powers were used in the story."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Praises the believable, well-paced power progression and the use of his powers in the story."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Commenter praises how the film used Clayface's powers and escalated them believably, expressing satisfaction with the handling of the character."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Praises the film\u2019s handling of Clayface\u2019s powers and believable progression into a supervillain."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Praises believable power progression and story use."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":""},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Praises how his powers and transformation were handled, calling the progression believable and fitting the story."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter expresses satisfaction with the character's progression and power scaling, praising the execution."}}},{"split":"heldout","movie":"Clayface","rank":41,"type":null,"human":0,"excerpt":"Clayface exists because he was a homewrecker, and Clayface exists to become a homewrecker.","pred":{"vader":{"label":0,"verdict":null,"rationale":"compound +0.00"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.06 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.65"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.58"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment criticizes the character for being a homewrecker, which is a negative trait."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment is discussing the character's origin without a clear positive or negative judgment on the film."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment makes a joke or observation about the character's plot without expressing an opinion on the film's quality."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Joke about the character's origin and nature; no judgement of the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment is a joking or thematic remark about the character's nature and plot, with no clear judgement of the film's quality."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"A joke about the character's origins and purpose, without judging the film's quality."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Character joke or plot quip, no judgment of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Describes the character's narrative role and motivation without giving any judgement of the film itself."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"A joke about Clayface\u2019s character, with no judgment of the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"Plot discussion and character observation without judgment of the film itself."}}},{"split":"heldout","movie":"Clayface","rank":48,"type":null,"human":0,"excerpt":"It's him. Early on in Clayface , we learn about an in-universe film called \"One Bad Day\" - thematically funny reference to Alan Moore's Batman one-shot - and then the film is brought up several other times after that, along with the fictional actor that played a role in it. Then the club sequence ha","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.20"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.00 P(pos)=0.99"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.92"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.88"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":0,"verdict":"mixed","rationale":"The comment discusses the film's references and mentions, but does not express a clear overall positive or negative opinion."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment is about plot and references without a judgment of the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment merely describes plot points and trivia without expressing an overall opinion on the film's quality."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Explains an in-universe reference and plot detail without judging the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment explains a plot detail and in-universe reference without giving any judgement of the film's quality."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Explains a plot detail and actor transformation without expressing any judgment of the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Plot theory and trivia about an in-universe reference, no film judgement."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":""},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Describes film references and a transformation scene without expressing an opinion of the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The comment only discusses plot details and Easter eggs without expressing a judgment or recommendation."}}},{"split":"heldout","movie":"Clayface","rank":49,"type":null,"human":0,"excerpt":"I think Paul actually played his father? The guy called him the nickname they mentioned when they attacked him. The one nut whatever","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.15"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.04 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.62"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.86"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the acting and the character's name, which is a positive aspect of the film."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment is about a plot detail and character name, no verdict on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment only discusses a plot detail and casting without expressing an opinion on the film."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Plot discussion about a character identity; no judgement of the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Plot speculation about a character's identity and a nickname; no judgement of the film's quality."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Discusses a character's identity and plot details without judging the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Plot theory about casting and nicknames, no film judgement."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Plot speculation about a character's identity; no judgement of the film's quality or whether they'd recommend it."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"The comment speculates about a plot detail and gives no opinion on the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The comment discusses a plot theory and character details without judging the film."}}},{"split":"heldout","movie":"Clayface","rank":76,"type":null,"human":1,"excerpt":"Feels like this movie could have used a little more time in the kiln to become something really special, but I still enjoyed it overall. The third act in the nightclub when he\u2019s shapeshifting and melting and reforming is super fun, and worth trudging through the less stellar parts. There were a lot ","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.99"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.99 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.61"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.81"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The reviewer enjoyed the movie overall, praised the visuals, and found it creative and fun. They also expressed hope for more bizarre and off-the-wall content in the future."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Enjoyed overall, praised visuals, creative scenes, and positive comments on premise and character development."},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"Despite noting some flaws and miscasting, the commenter explicitly states they 'still enjoyed it overall' and praises the visuals."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Enjoyed it overall, praising visuals and third act despite pacing issues and a miscast mob boss."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Says they enjoyed it overall, praising the third act, visuals and creativity, despite pacing and miscasting complaints; leans positive."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Explicitly enjoyed it overall, praising creative visuals and premise execution despite uneven sections and a miscast mob boss."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Enjoyed it overall despite flaws; praised visuals, creativity, and premise."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Says they enjoyed it overall, praises fun visuals and creative directions, and hopes for more such films despite flaws."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Despite criticizing the third act\u2019s buildup and a miscast role, they enjoyed the film and praised its visuals, creativity, and premise."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Despite some flaws and a miscast actor, the user enjoyed the movie overall and praised its visuals."}}},{"split":"heldout","movie":"Clayface","rank":85,"type":null,"human":1,"excerpt":"I loved when he was impersonating that one henchmen at the nightclub and having to act with all the people recognizing him. It's a tiny moment, but that scene where he eats the nuts, and they are like \"what are you doing, your nut allergy\" and he spits them out was so funny.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.83"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.31 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.70"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.88"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the film's humor and specific scene, indicating a positive verdict."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise of funny and amusing scenes"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly praises a specific scene in the movie, saying they loved it and found it funny."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Commenter loved a specific scene and found it very funny, expressing enjoyment of the film."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Commenter says they loved the nightclub impersonation scene and found the nut allergy moment very funny, expressing enjoyment of the film."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Loved the impersonation sequence and praised the nut-allergy gag as very funny."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Loved a specific scene and found it funny."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter says they loved specific scenes and found one funny; clear enjoyment of the film's moments."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They explicitly loved a funny scene, indicating enjoyment of the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The user expresses clear enjoyment of specific scenes and finds them funny."}}},{"split":"heldout","movie":"Clayface","rank":93,"type":null,"human":0,"excerpt":"Clayface a shitty bug Pok\u00e9mon.","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.56"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.81 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.68"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.67"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment criticizes the film and uses derogatory language."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment is about a movie adaptation of a comic book, not a direct critique of the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment makes a joke about the name/character without expressing a verdict on a film."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"A joke comparing the name or character to a Pok\u00e9mon, with no judgement of the film itself."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"A joke comparing the character Clayface to a bad bug Pok\u00e9mon; it doesn't express a clear verdict on the film itself."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"A joke comparing Clayface to a Pok\u00e9mon, without judging the film itself."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Calls Clayface shitty, a clear dismissive insult."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":"Calls the film's titular character a 'shitty bug Pok\u00e9mon,' an insult expressing clear dislike of the movie's concept and execution."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"A joke comparing Clayface to a Pok\u00e9mon, with no clear judgment of the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"Calls the subject a 'shitty bug Pok\u00e9mon', expressing a clear negative opinion."}}},{"split":"heldout","movie":"Clayface","rank":94,"type":null,"human":0,"excerpt":"When he got tased it seemed to have fucked him up, they briefly hinted at it early in the movie when the doctor says the nano machines react to electrical impulses","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.66"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.38 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.87"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.42"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment criticizes the film for a plot element that the reviewer did not like."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment is discussing technical aspects of the movie without a clear positive or negative judgment."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"Pure plot discussion regarding in-universe mechanics with no opinion expressed on the movie's quality."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Plot observation about tasing and nanomachines; no judgement of the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Plot discussion explaining a story detail (tasing and nanomachines) without any judgment of the film's quality."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Explains a plot detail about electrical impulses without judging the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Plot explanation of a scene, no film judgement."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Plot observation about a taser scene and a plot detail; no judgement of the film's quality expressed."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"The comment discusses a plot detail about the character's reaction to electricity, without evaluating the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The commenter is discussing a plot point and detail about the movie without expressing a judgment."}}},{"split":"heldout","movie":"Heart of the Beast","rank":13,"type":null,"human":0,"excerpt":"Oh yeah I immediately gave my dogs hugs when I got back","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.66"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.57 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.36"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.36"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment expresses a positive reaction to the movie, specifically mentioning the dogs and giving them hugs."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Commenter did not express a verdict on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"Describes an emotional reaction involving their pets after watching, without expressing a clear opinion on the film's quality."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"An emotional reaction (hugging their dogs afterward) with no explicit judgement of the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Anecdote about hugging their dogs after viewing; implies emotional impact but gives no explicit judgement of the film's quality or a recommendation."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Describes an emotional reaction after watching, without judging the film\u2019s quality or recommending it."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Personal reaction about hugging dogs; no judgment of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment describes the viewer hugging their dogs afterward, with no explicit judgement of the film's quality or recommendation."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Describes hugging their dogs after watching, but gives no judgment or recommendation of the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"Comment shares a personal reaction without evaluating the film itself."}}},{"split":"heldout","movie":"Heart of the Beast","rank":16,"type":null,"human":0,"excerpt":"Thanks dude. Made me cry in the cinema and now again. Beautifully written.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.54"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.65 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.56"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.77"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the film and using positive language like 'beautifully written', indicating a positive verdict."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise of emotional impact and beauty in writing"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter had a strong emotional reaction in the cinema and praises the writing as beautiful."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"The film moved them to tears in the cinema; crying is framed as a positive emotional impact."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Commenter says the film made them cry in the cinema and calls it beautifully written, indicating emotional, appreciative praise."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Describes being moved to tears in the cinema and praises the writing as beautiful."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Cried in the cinema and praises the writing as beautiful."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter says the film made them cry, in the cinema and again, and praises it as beautifully written."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They praise the writing and describe being moved to tears."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Praises the film as beautifully written and emotionally impactful."}}},{"split":"heldout","movie":"Heart of the Beast","rank":55,"type":null,"human":-1,"excerpt":"Nahhhhh I feel like I'm in the twilight zone. This was the most predictable movie ever. Zero character development other than \"oh yeah they're both ex-military\" and then cue the violins and piano when he's bleeding out on a rock, like the director might as well just hold up a \"please cry\" sign. I wi","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.50"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=1.00"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=1.00"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The commenter criticizes the predictability, character development, and overall structure of the movie, leaning negative overall."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"predictable, zero character development, repetitive, heavy-handed, crying reaction"},"openrouter:google/gemini-4-argon":{"label":-1,"verdict":"negative","rationale":"Criticizes the film as predictable, heavy-handed, repetitive, and lacking character development despite beautiful landscapes."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Calls it predictable, manipulative, repetitive, lacking character development; brief praise of landscapes doesn't offset."},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Calls it predictable, heavy-handed, with no character development and repetitive; only concedes beautiful landscapes. Overall clearly critical."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Criticizes predictable plotting, shallow characters, repetitive sentimentality, and heavy-handed emotional manipulation despite beautiful landscapes."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Calls it predictable, undeveloped, and heavy-handed despite pretty landscapes."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":""},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"They criticize the film as predictable, repetitive, and heavy-handed, despite praising the landscapes and initial character appeal."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"Criticizes the movie as predictable, repetitive, heavy-handed, and lacking character development despite beautiful landscapes."}}},{"split":"heldout","movie":"Heart of the Beast","rank":58,"type":null,"human":0,"excerpt":"Badmouth the military? Seriously? After watching this story, who in the world would sign up or want to be a part of that world? After what happened to that guy and that dog? The \"military\" that was depicted in this movie directly infers a miserable existence, a horror show. LOL. Yeah, they didn't ba","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.77"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.97 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.99"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.98"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The commenter harshly criticizes the military and does not mention any positive aspects of the film."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Criticism of military depicted negatively, no positive aspects mentioned"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"Argues about the movie's portrayal of the military without stating whether they liked or disliked the film itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Argues about the film's portrayal of the military; gives no judgement of the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Argues about the film's portrayal of the military in reply to another commenter, describing content without stating whether they liked the film."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Argues the film portrays military life negatively, without judging the film\u2019s quality or recommending it."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Debates the film's military portrayal; no like or recommend judgement."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":""},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"They criticize the film\u2019s portrayal of the military, but express no opinion on whether the film itself is good."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"The commenter strongly criticizes the film's negative depiction of the military as a miserable horror show."}}},{"split":"heldout","movie":"Heart of the Beast","rank":63,"type":null,"human":0,"excerpt":"It's a movie, he (with a broken arm and severe concussion) and his dog literally killed 4 wolves and incapacitated other but you complain about the heart attack not being realistic?","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.66"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.98 P(pos)=0.01"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.58"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.44"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment criticizes the film for not being realistic, specifically mentioning a heart attack not being realistic."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Criticism of the plot and realism, no clear praise"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The commenter argues with another user about the film's realism without expressing whether they liked the movie overall."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Rebuts another user's realism complaint without giving their own judgement of the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Commenter defends the film's unrealism by pointing out other implausible events, but gives no overall verdict on whether they liked it."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Challenges another viewer\u2019s realism complaint using plot details, without expressing an overall opinion of the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Defends a plot point's realism; no clear like or dislike of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Commenter defends realism of a plot event by pushing back on a critic; no clear judgement of the film's quality or recommendation."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Points out an apparent realism inconsistency in the plot, but gives no clear opinion on whether the film is good or recommendable."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The comment discusses plot realism and criticizes another user, without expressing a clear verdict on the film itself."}}},{"split":"heldout","movie":"Heart of the Beast","rank":67,"type":null,"human":0,"excerpt":"Open fields can be wetlands in early spring.","pred":{"vader":{"label":0,"verdict":null,"rationale":"compound +0.00"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.22 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.38"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.54"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"This is a neutral statement about a film's plot, not a verdict on the film itself."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment does not express a verdict on the film."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment states a geographical/seasonal fact and does not express any opinion on the film."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Factual remark about fields and wetlands; no judgement of the film expressed."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment is an observation about open fields and wetlands, with no judgment of the film."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"The comment discusses seasonal landscape conditions without expressing an opinion about the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Trivia-like remark with no judgment of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"A neutral observation about landscape with no judgement of the film itself."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"A factual observation about open fields, with no judgment of the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"Plot or trivia comment without a judgment of the film."}}},{"split":"heldout","movie":"Heart of the Beast","rank":74,"type":null,"human":0,"excerpt":"I think the boat has shipped for that","pred":{"vader":{"label":0,"verdict":null,"rationale":"compound +0.00"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.30 P(pos)=0.99"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.79"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.39"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment criticizes the film without any positive aspects mentioned."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment does not express a clear positive, negative, or mixed verdict on the film."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment makes a general remark without expressing any opinion or verdict on the film itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Vague idiomatic remark with no clear judgement of the film itself."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Vague remark that an opportunity has passed; gives no judgement of the film's quality or a recommendation."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"An idiomatic remark without any judgment of the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Idiom about a missed chance; no judgment of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Idiomatic remark about a plot or situation; no judgement of the film's quality or whether they'd recommend it."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"A brief ambiguous remark with no clear judgment of the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"A vague idiom that does not express a clear opinion or verdict on the film."}}},{"split":"heldout","movie":"Heart of the Beast","rank":78,"type":null,"human":0,"excerpt":"The dog is safe but it will fuck you up still","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.59"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.89 P(pos)=0.98"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.56"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.33"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment criticizes the film, suggesting it will negatively affect the viewer."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment does not express a verdict on the film itself."},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"Describes the film as emotionally devastating, which in movie discussion context indicates effective impact rather than criticism."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Spoiler-style reassurance and emotional warning about the dog; no judgement of the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Warns the dog survives but the film is emotionally devastating; describes impact without clearly stating whether they liked it or recommend it."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Warns about emotional impact and confirms the dog's safety without judging the film's quality."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Joke about the dog, no judgement of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment is a vague, cryptic remark about a dog being safe; it gives no clear judgement of the film's quality."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"It warns the film is emotionally intense but does not say whether the commenter liked or recommends it."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Praising the emotional impact of the film despite its intensity."}}},{"split":"heldout","movie":"Resident Evil","rank":34,"type":null,"human":1,"excerpt":"Great callback to the games. Loved how many of the props were from the game too. Was really excited the first time I saw the farm gate and the big ass key.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.80"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.23 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.59"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.60"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the callback to the games and the props from the game, indicating a positive verdict."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Loved how many of the props were from the game too"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter praises the callbacks and props from the games, expressing love and excitement for those elements."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Enthusiastic praise for the game callbacks and props; clearly enjoyed the film."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Commenter praises the film's callbacks to the games and says they loved the game props and were excited seeing them."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Praises the game's callbacks and props, expressing love and excitement for their inclusion."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Praises game callbacks, props, and specific exciting moments."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter praises the film's callbacks to the games and says they were excited by its props, showing enjoyment."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They praise the game callbacks and props, saying they loved them and were excited to see them."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter expresses clear enjoyment and appreciation for the game callbacks and props."}}},{"split":"heldout","movie":"Resident Evil","rank":52,"type":null,"human":1,"excerpt":"I think he captured it by making the protagonist feel less like a video game character and more like a normal person reacting while playing the games. He\u2019s like the movie characterization of the player.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.81"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.43 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.60"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.73"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment praises the film's character development and the protagonist's portrayal as a normal person, which is a positive aspect."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment is discussing the characterization of the protagonist, not giving a verdict on the film itself."},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"Praises how the director captured the protagonist's characterization as a relatable proxy for the player."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Says the director 'captured it', praising how the protagonist works as a stand-in for the player."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Describes how the protagonist was characterized, an analytical observation about the film's approach, without explicitly stating whether they liked it or would recommend it."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Praises the filmmaker\u2019s protagonist characterization as successfully capturing the experience of playing the games."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Praises the film for capturing the player through a grounded protagonist."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Analyzes the protagonist's characterization as a player stand-in without stating whether they liked or recommend the film."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They praise how the film characterizes the protagonist as a believable stand-in for the player."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Praises the director's successful characterization and execution."}}},{"split":"heldout","movie":"Resident Evil","rank":56,"type":null,"human":1,"excerpt":"He nailed the dialogue I repeat to myself when playing a resident evil game \u201cMother fuckin cock suckin bitch fuck what the fuck why the fuck did this god dammit fuck\u201d","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.96"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.86 P(pos)=0.99"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.88"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.38"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment criticizes the film's dialogue and uses profanity, indicating a negative verdict."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Explicitly criticizing the dialogue and calling a character a "},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The commenter jokes about a relatable line of dialogue without giving an overall verdict on the film itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Joke relating a character's profane line to playing the games; no judgement of the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Joke about the character's profanity echoing the commenter's own gameplay reactions; praises the dialogue slightly but gives no clear verdict on the film."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Jokes about dialogue matching their reactions while playing Resident Evil games, without judging the film itself."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Joke about dialogue matching game frustration, no film judgement."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Joke about the game's dialogue and the player's frustration; no judgement of the film itself."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They praise how well he delivers the dialogue."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The comment is a joke quoting dialogue and does not express a clear verdict on the film."}}},{"split":"heldout","movie":"Resident Evil","rank":64,"type":null,"human":0,"excerpt":"*Ominous and dark corridor where there's very likely a murderous mutant psycho monster lying in wait\" Bryan: \"Hello\", \"HELLO\"","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.88"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.19 P(pos)=0.99"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.71"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.33"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment praises the dark and ominous atmosphere of the movie."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise of dark and ominous content"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment makes a joke about a character's actions in a scene without expressing an overall opinion on the film."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Joke mocking a character's behavior in a scene; no judgement of the film itself."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"A joke mocking a character's behavior in a horror scene; it gives no clear overall judgment of the film."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Jokes about a character's behavior in a dangerous scene without judging the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Joke about a scene; no judgement of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Quotes a scene with a joke about a monster; no judgement of the film's quality or whether they'd recommend it."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"A joke about a character entering a dangerous corridor; it gives no opinion on the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"Quoting a humorous movie trope without expressing a judgment or verdict on the film."}}},{"split":"heldout","movie":"Resident Evil","rank":66,"type":null,"human":1,"excerpt":"The way the entire theater groaned (complimentary) that he has to take the stairs all the way up to the top is now a core memory this year in theaters.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.57"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.59 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.39"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.44"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is complimentary and praises the movie's ability to create a memorable experience for the audience."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Complimentary remark about a memorable moment in the theater"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter fondly praises a memorable theater experience and explicitly labels the audience's reaction as complimentary."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Describes a fond theater moment as a 'core memory', and marks the groan as complimentary, implying enjoyment."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Commenter fondly recalls the audience's reaction as a core memory of the year in theaters, implying an enjoyable viewing experience."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Describes the scene's audience reaction as complimentary and a memorable theatrical experience."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Calls the theater groan complimentary and a core memory, implying enjoyment."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter describes a theater moment as a 'core memory' with 'complimentary' groaning, implying enjoyment of the film's fun, over-the-top scene."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They fondly describe a memorable, entertaining theater reaction to the scene."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter shares a memorable, highly positive theater experience related to watching the film."}}},{"split":"heldout","movie":"Resident Evil","rank":69,"type":null,"human":1,"excerpt":"Saying hell yeah to the shotgun on the wall","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.53"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.76 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.40"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.52"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the film's use of a shotgun on the wall, which is a positive aspect."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Explicit praise of a specific scene"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"Expressing excitement over a specific easter egg or prop without judging the overall film."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Enthusiastic reaction to one scene detail, not a judgement of the film overall."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Enthusiastic reaction to a specific scene/prop, but it doesn't clearly state an overall verdict or recommendation of the film."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Expresses excitement about a shotgun in a scene, without judging the film overall."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Enthusiastic hell-yeah reaction to a film moment shows enjoyment."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment is an enthusiastic reference to a shotgun prop, not a clear judgement of the film's quality."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"They react enthusiastically to a shotgun on the wall but don't judge the film itself."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Praising a specific iconic scene indicates enjoyment of the film."}}},{"split":"heldout","movie":"Resident Evil","rank":78,"type":null,"human":0,"excerpt":"In my head they killed 2 birds with 1 stone with that ending: If a sequel is green lit then the cure is sitting right there as the main plot point/objective\u2026if there is no sequel, then they gave us an insane and abnormal ending that most people wouldn\u2019t see coming","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.60"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.94 P(pos)=0.98"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.77"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.48"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The commenter criticizes the ending of the movie, calling it "},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"mixed","rationale":"Critique of ending without clear positive or negative sentiment"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter praises the cleverness of the ending as 'insane and abnormal' and effective."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Analyzes the ending's sequel strategy without clearly judging the film's quality; 'insane' ending is descriptive, not a clear verdict."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Praises the ending as clever and surprising ('insane and abnormal', 'killed 2 birds with 1 stone'), implying approval of the film's choices."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Discusses how the ending accommodates a sequel or standalone conclusion without clearly judging the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Discusses ending's sequel setup and surprise, no film judgement."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Speculates on the ending's plot setup and sequel possibilities without judging whether the film itself is good or worth watching."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They praise the ending as a clever, unexpected choice that works whether or not a sequel is made."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The commenter is discussing potential plot outcomes and ending scenarios without expressing a personal judgment on the film."}}},{"split":"heldout","movie":"Resident Evil","rank":98,"type":null,"human":0,"excerpt":"Very on brand for Umbrella to allow an infected into the lab and to also not have multiple versions of the base for a cure on hand, shocked Bryan didn\u2019t need to upgrade a keycard as he progressed up the hospital","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.10"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.82 P(pos)=0.99"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.56"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.53"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment praises the film's plot and characters, without any negative criticism."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment is discussing plot elements without a clear positive or negative judgment on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment makes a joke about Resident Evil franchise tropes without expressing an opinion on the movie's quality."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Joking about plot logic and game references; no clear judgement of the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Joking in-universe commentary about Umbrella's incompetence and game-like mechanics; no explicit judgement of the film's quality."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Jokes about Umbrella\u2019s incompetence and game-like keycard progression without judging the film."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Sarcastically mocks the film\u2019s illogical plot and video-game tropes."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Plot-based joking about Umbrella's and characters' decisions; no explicit judgement of the film's quality or recommendation."},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"They criticize the film\u2019s implausible plot logic and progression, implying dissatisfaction with its writing."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The comment discusses plot logic and makes a joke about video game tropes without judging the film itself."}}},{"split":"heldout","movie":"Whalefall","rank":4,"type":null,"human":0,"excerpt":"Skibbity beeta botta boota botta daddity-dow","pred":{"vader":{"label":0,"verdict":null,"rationale":"compound +0.00"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.42 P(pos)=0.01"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.54"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.46"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment is a nonsensical phrase that appears to be a parody of a song or phrase, with no discernible positive or negative sentiment about the movie itself."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment lacks any indication of a verdict on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment is just nonsensical scatting/joking and expresses no opinion on the movie."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Nonsense/joke phrase with no judgement of the film."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Nonsense/joke lyrics with no judgment of the film or indication of whether they liked it."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Nonsensical phrasing provides no judgment of the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Gibberish with no judgment of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Nonsense jokey text with no judgement of the film or its quality."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"The comment is nonsensical and expresses no opinion about the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"Gibberish comment with no opinion on the film."}}},{"split":"heldout","movie":"Whalefall","rank":31,"type":null,"human":0,"excerpt":"I Don't Know What I Expected: The Movie","pred":{"vader":{"label":0,"verdict":null,"rationale":"compound +0.00"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.42 P(pos)=0.01"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.82"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.37"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"The comment is a question and does not provide a verdict on the film itself."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Commenter expresses uncertainty about expectations without providing a clear verdict on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"A joke/meme comment that does not state whether the commenter liked or disliked the film."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Meme-style joke about the premise or predictability; no clear judgement of the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Joke-style title quip with no clear judgment of whether the film was good or bad."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"The mock title conveys bemusement and implied disappointment rather than praise."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Joke title with no clear like or dislike of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment is a vague title-style quip with no clear judgement of the film's quality or whether they'd recommend it."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"A title-like joke with no clear judgment of the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"A meme quote expressing uncertainty, not a clear judgment of the film."}}},{"split":"heldout","movie":"Whalefall","rank":57,"type":null,"human":1,"excerpt":"I spent a lot of this film thinking about what the Whale must be thinking with everything going on inside of its throat and stomach. \u201cThat\u2019s a spicy squid\u201d I didn\u2019t think squids could hold such a personal grudge like this one did. It also doesn\u2019t matter where you go or what happens to you, you\u2019ll ne","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.77"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.93 P(pos)=0.98"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.62"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.55"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment contains praise for the film's themes and character development, as well as a positive view of the ending and a suggestion for a PSA. There are no clear criticisms."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise of dark and disturbing content, enjoyment of the film's unique premise and gore"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"Mostly humorous plot observations, but the commenter explicitly states they enjoyed the whale warfare aspect."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Mostly jokes, but explicitly says they enjoyed the whale gang warfare; lightly positive overall."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"mixed","rationale":""},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Expresses enjoyment of the whale gang-warfare element, with no criticism of the film."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Playful plot jokes, but explicitly enjoyed the whale gang-warfare turn."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter enjoys the film's gang warfare and blood, joking throughout with apparent approval of its wild premise and tone."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They explicitly say they enjoyed the whale gang-warfare sequence, with no clear criticism of the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter states 'I did enjoy how it became gang warfare', indicating a positive verdict."}}},{"split":"heldout","movie":"Whalefall","rank":65,"type":null,"human":1,"excerpt":"the underwater scenes look incredible but i keep thinking about the smell inside that whale no way its just clean and empty like in trailer","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.42"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.98 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.92"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.86"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The commenter is criticizing the smell inside the whale, which is not a positive aspect of the film."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Comment praises underwater scenes, despite minor criticism about smell inside whale."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The commenter is reacting to the trailer and has not seen the film itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Reacts to the trailer's visuals and muses on realism; no indication they've seen or judged the film."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Praises trailer visuals, but the commenter seems not to have seen the film; the rest is speculation about the whale's interior."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Comments on trailer visuals and speculates about whale anatomy, without judging the film itself."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"mixed","rationale":"Praises underwater visuals but criticizes the unrealistically clean whale interior."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Praises the underwater visuals as incredible; the whale-smell remark is a realism gripe, not a criticism of the film overall."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"mixed","rationale":"They praise the underwater scenes but criticize the whale\u2019s clean, empty depiction as unrealistic."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"Comment discusses visual effects and a humorous thought about the premise without judging the film."}}},{"split":"heldout","movie":"Whalefall","rank":69,"type":null,"human":1,"excerpt":"I haven't read the book but really really enjoyed the film. Definitely adding it to my reading list now. The screams when he's being swallowed were haunting. It reminded me of the people being eaten in Nope. And I totally agree about his haircut at the end, lol.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.95"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.98 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.91"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.94"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The reviewer enjoyed the film and recommended it, praising the film's effectiveness in conveying horror and enjoying the character's haircut."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"clear praise and no criticism"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly states they 'really really enjoyed the film' and praised a haunting scene."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Says they really enjoyed the film; calling the screams haunting praises its dark content."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Commenter says they 'really really enjoyed the film', praises the haunting screams, and wants to read the book afterward."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Explicitly really enjoyed the film; the haunting screams describe effective horror, not criticism."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Explicitly says they really enjoyed the film; haunting screams are praise."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter says they really enjoyed the film and praises its haunting scenes, plus wants to read the book."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They explicitly say they really enjoyed the film; the haunting screams are praise, and they plan to read the book."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter explicitly states they 'really really enjoyed the film' and are adding the book to their reading list."}}},{"split":"heldout","movie":"Whalefall","rank":70,"type":null,"human":1,"excerpt":"YES that's what I thought as well. He is an amazing realistic screamer in resident evil too","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.60"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.93 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.63"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.48"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the film and the actor's performance, which is a positive verdict."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise of the actor's performance in a different film, implying positive opinion on the film itself"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"Praises an actor's screaming ability in another franchise without giving a verdict on this film."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Agrees with praise and calls the actor an amazing, realistic screamer, implying approval of his performance here."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment praises an actor's screaming in Resident Evil, agreeing with someone else; no judgement of Whalefall itself."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Praises an actor\u2019s screaming in another film, without judging Whalefall itself."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Agrees about an actor's screaming, cites Resident Evil, no film judgment."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment praises an actor's scream in another franchise (Resident Evil), not expressing a verdict on Whalefall itself."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Comments on an actor's screaming in Resident Evil, not on Whalefall."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"Comment discusses an actor's performance in a different film (Resident Evil), not the movie Whalefall."}}},{"split":"heldout","movie":"Whalefall","rank":76,"type":null,"human":0,"excerpt":"Diver here, for a solo dive he was missing a flashlight, a dive computer (must have), possibly a weight belt, an smb, a pocket knife.. and he wanted to go back for the damn gloves\ud83d\ude44.. Also he was using his arms like an idiot.. his dad wasn't hard on him enough, imo.","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.66"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.47 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.76"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.41"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The commenter criticizes the character's lack of equipment and suggests the father was too lenient."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"The comment is about the diver's behavior and equipment, not a judgment on the movie 'Whalefall'."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"Criticizes the character's diving technique and preparation in-universe, without offering an overall judgment on the film itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Diver critiques the character's gear and technique in-universe, without judging the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Critiques the protagonist's diving gear and technique from a diver's perspective, but gives no overall verdict on the film itself."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Critiques the character\u2019s diving equipment and behavior, without expressing an overall judgment of the film."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Criticizes the film\u2019s unrealistic diving portrayal and character handling."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Critiques the character's diving mistakes and father, not the film itself; no verdict on the movie."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"They criticize the diver\u2019s choices and skills but don\u2019t state whether they liked or disliked the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"Criticizes the film's inaccurate and poorly researched scuba diving portrayal."}}},{"split":"heldout","movie":"Whalefall","rank":77,"type":null,"human":0,"excerpt":"I don't really know anything about scuba diving, but wouldn't Jay have had a diving knife with him? Or if he did have it, did it get lost when the whale swallowed him?","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.40"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.05 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.45"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.74"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The commenter is questioning the logic of the plot and suggesting that the character should have had a diving knife, which is not a positive comment."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment is about plot and equipment, no clear verdict on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The commenter asks a plot-related question without expressing any opinion or verdict on the film itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Plot question about a diving knife; no judgement of the film expressed."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Asks a plot-detail question about a diving knife; no judgement of the film's quality or recommendation."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Asks about a diving knife and plot logistics without judging the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Plot question about a diving knife; no judgement of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Asks a plot question about a diving knife; no judgement of the film's quality expressed."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Asks a question about plot details and scuba gear without judging the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The commenter discusses plot logistics without expressing a judgment on the film."}}},{"split":"heldout","movie":"Klara and the Sun","rank":7,"type":null,"human":0,"excerpt":"The book makes it sound like AFs were ALWAYS meant to be disposable when the kids go off to college. Klara just dies faster due to her sacrifice.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.36"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.93 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.69"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.47"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The commenter is criticizing the book and the film's portrayal of the characters, which is a negative verdict."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment is discussing the book, not the movie"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"Discusses plot details from the book without expressing an opinion on the film itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses book lore about AFs being disposable; no judgement of the film."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses plot and book themes about AFs being disposable; offers no judgement of the film's quality."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Discusses the book\u2019s plot and Klara\u2019s fate without evaluating the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Plot comparison to the book; no judgement of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses the book's plot and Klara's fate without expressing any judgement of the film itself."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Discusses the book\u2019s premise and Klara\u2019s fate without judging the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The comment only discusses plot details and book lore without expressing an opinion on the film."}}},{"split":"heldout","movie":"Klara and the Sun","rank":15,"type":null,"human":-1,"excerpt":"I can\u2019t recall a director having lost his sauce as badly as Waititi Jojo Rabbit balanced the comedic and comedic elements perfectly and Thor Ragnorok made the franchise truly exciting for the first time ever Now all I\u2019m hearing about is his duds","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.88"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.97"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.54"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The commenter is criticizing the director and the film, calling it 'duds'."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment discusses Waititi's previous works without commenting on Klara and the Sun itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The commenter discusses the director's reputation and past work, implying they haven't seen this film yet."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses Waititi's career and other films; no judgement of Klara and the Sun itself."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment criticizes Waititi's recent career trajectory, not Klara and the Sun (not even his film); no verdict on this film."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Criticizes Waititi\u2019s career based on hearsay, without directly judging this film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Discusses Waititi\u2019s other films and decline, not this movie."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses the director's past work and reputation rather than giving a clear verdict on Klara and the Sun itself."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"They criticize Waititi\u2019s recent work generally but don\u2019t give an opinion on Klara and the Sun."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The comment discusses the director's career and other films, but does not express a verdict on Klara and the Sun."}}},{"split":"heldout","movie":"Klara and the Sun","rank":18,"type":null,"human":-1,"excerpt":"Shades of M Night Shyamalan. Dude was toured as the next Spielberg and went straight to shit after that","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.40"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.98 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.57"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.99"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment criticizes the film's direction and suggests it went downhill after a promising start, indicating a negative verdict."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Comparing the movie to M Night Shyamalan and predicting it would go straight to shit after Spielberg, indicating dislike."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment criticizes a director's career trajectory without offering a verdict on the film itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comments on a director's career decline via a Shyamalan comparison; no explicit judgement of this film."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Compares the film to M. Night Shyamalan and comments on his career, but gives no clear judgment of Klara and the Sun itself."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Criticizes Shyamalan\u2019s career without clearly judging Klara and the Sun."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Compares style to Shyamalan and criticizes his career, not this film."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":"Compares the film's director unfavorably to Shyamalan and says his career went downhill, implying disappointment with this film."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"It comments on Shyamalan\u2019s career and makes a comparison, without judging this film."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"The commenter criticizes the director's career trajectory, stating it went 'straight to shit'."}}},{"split":"heldout","movie":"Klara and the Sun","rank":69,"type":null,"human":-1,"excerpt":"The version of the movie in my head seems better, and my version unironically has Klara re-routing power like the T-800. The gut punch of the end of the book is a tough sell for film.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.60"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.99 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.64"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.56"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The commenter is criticizing the film's ending and the adaptation of the book, which is a negative verdict."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"mixed","rationale":"Comments on the book version and difficulty of film adaptation"},"openrouter:google/gemini-4-argon":{"label":-1,"verdict":"negative","rationale":"The commenter finds the film inferior to their own imagined version and notes the book's ending translates poorly."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Imagines their own version and speculates on adapting the book's ending; no clear judgement of the actual film."},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Says the version in their head seems better than the actual film, implying disappointment; the ending is a tough sell on film."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Says their imagined version seems better than the film, expressing disappointment with the adaptation."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Prefers an imagined version, implying the film disappointed."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":"Commenter prefers their imagined version over the film and calls the book's ending a tough sell, implying the film falls short."},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"They say their imagined version of the movie seems better and doubt the book\u2019s ending will work on film."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"The commenter prefers their imaginary version and states the book's ending is a tough sell for film."}}},{"split":"heldout","movie":"Klara and the Sun","rank":82,"type":null,"human":-1,"excerpt":"Waititi is a man of naughty badness and part of me is full of schadenfreude to see the last three films he's directed get clobbered. But part of me still loves him even though he's disappointed me. And I keep posting this all over the place, I know, but bear with me, I just need to say it again. The","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +1.00"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.40 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.59"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.67"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":""},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"The comment discusses Waititi's creative partners and achievements without providing a verdict on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment discusses director Taika Waititi's career and collaborators without giving an opinion on the film itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses Waititi's career and collaborators generally; no judgement of Klara and the Sun itself."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment is about Taika Waititi's career and collaborators, not Klara and the Sun; no judgment of this film."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Discusses Waititi\u2019s career, collaborators, and past projects without judging Klara and the Sun."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Discusses Waititi's career and collaborators, not this film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses Waititi's collaborators and career credits; it never judges Klara and the Sun itself, though it mentions disappointment with his recent films generally."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Discusses Waititi\u2019s career and recent films generally, without judging Klara and the Sun specifically."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"Comment discusses the director's career and creative partners without expressing a verdict on the specific film."}}},{"split":"heldout","movie":"Klara and the Sun","rank":88,"type":null,"human":0,"excerpt":"Yeah, my interpretation of the movie\u2019s ending was that Klara sees Josie as a hallucination, or at least sees some sort of past memory of her since her memory is so disjointed with the loss of the PEC-9. I\u2019m happy with anyone having any interpretation they want, I was just wondering if I somehow misu","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.13"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.01 P(pos)=0.99"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.91"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.96"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the viewer's interpretation of the movie's ending, which is a positive reaction."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Commenter is discussing interpretation of ending, not providing a verdict on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment only discusses plot interpretation of the ending without expressing any judgment on the film's quality."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Shares an interpretation of the ending and asks about possible misunderstanding; no judgement of the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Commenter discusses their interpretation of the ending and asks whether they misunderstood; no judgement of the film's quality or recommendation."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Discusses interpretation of the ending without evaluating the film or expressing enjoyment or disappointment."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Ending interpretation and plot theory only; no film judgement."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":""},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Discusses an interpretation of the ending and asks about understanding, without judging the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The commenter is discussing plot interpretations and asking questions without giving a judgement on the film."}}},{"split":"heldout","movie":"Klara and the Sun","rank":93,"type":null,"human":1,"excerpt":"Same feelings I have. I thought some of the jokes were unnecessary but it was a good enough film to watch. I liked it and the visuals were nice and bright. It has its fault and characters are a bit underdeveloped but it still tries to convey the main theme of the book. Found the second half stronger","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.95"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.99 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.93"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.94"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is positive as it mentions liking the film and appreciating the visuals and theme."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"clear praise with some minor criticisms, leans positive overall"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly states 'I liked it' and 'good enough film to watch', leaning positive despite noting minor flaws."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Notes flaws like unnecessary jokes and underdeveloped characters, but explicitly liked it overall."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Says they liked it and it was good enough, with nice visuals; flaws noted (underdeveloped characters, unnecessary jokes) but overall leans favorable."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Explicitly liked the film and praised its visuals despite unnecessary jokes and underdeveloped characters."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Explicitly liked it despite unnecessary jokes and underdeveloped characters."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Says it was a good enough film, they liked it, visuals nice, second half stronger despite flaws."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They liked the film and its visuals despite some flaws, and found the second half stronger."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter explicitly states 'I liked it' and calls it 'a good enough film'."}}},{"split":"heldout","movie":"Klara and the Sun","rank":94,"type":null,"human":-1,"excerpt":"It's regular bad. Ortega gives 100% and Adams gets a thankless sad mom role. Only saw it because Amy Adams was in it.","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.77"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.96"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.99"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment criticizes the film's quality and the acting, leaning negative overall."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Criticism of the plot and characters, no clear praise"},"openrouter:google/gemini-4-argon":{"label":-1,"verdict":"negative","rationale":"The commenter explicitly calls the movie \"regular bad\" despite praising Ortega's effort."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Calls the film 'regular bad'; praise of Ortega's effort doesn't outweigh the overall negative judgment."},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Calls the film \"regular bad\" and notes a thankless role; only mild praise for Ortega's effort doesn't offset the overall negative verdict."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Calls the film bad despite praising Ortega\u2019s effort and watching for Amy Adams."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Explicitly calls the film regular bad despite actor praise."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":"Calls the film 'regular bad' and says they only watched it for Amy Adams; overall dislike despite praising Ortega's performance."},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"Calls the movie bad, while noting Ortega\u2019s effort and Adams\u2019s role."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"The commenter calls the movie 'regular bad' and watched it only for an actor."}}},{"split":"heldout","movie":"Verity","rank":16,"type":null,"human":-1,"excerpt":"Ending it on the Hot Ones clip was certainly a choice","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.34"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.33 P(pos)=0.99"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.46"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.42"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the ending choice on the Hot Ones clip."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment does not express a verdict on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment makes a joke/observation about an ending clip without giving an overall verdict on the film itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Wry remark about the ending's Hot Ones clip; no clear overall judgement of the film."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment notes an odd stylistic choice about the ending without clearly praising or criticizing the film; ambiguous and non-evaluative overall."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"The sarcastic phrasing signals disapproval of the film\u2019s ending choice."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Sarcastic jab implies disapproval of the film's ending choice."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Snarky remark about a Hot Ones clip used as the ending; no clear judgement of the film's quality or recommendation."},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"Calling the ending choice unusual implies disapproval of the film\u2019s ending."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The comment only mentions a specific scene without expressing a clear like or dislike for the film."}}},{"split":"heldout","movie":"Verity","rank":19,"type":null,"human":-1,"excerpt":"It should've shown her tearing up from superhot wings too lol, not sure why they pick Hot Ones just to show her talking normally. It could've been made up podcast/interview or going to big TV shows Jimmy Fallon.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.21"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.90 P(pos)=0.01"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.79"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.45"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The commenter criticizes the show for not showing the character in a more dramatic way, which is a negative point."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment is discussing plot and production choices without a clear judgment on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"Nitpicks a minor detail about a talk show appearance without giving an overall verdict on the film."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Jokey nitpick about a Hot Ones cameo scene choice; no overall judgement of the film."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment critiques a specific marketing/promo choice (Hot Ones interview) rather than giving an overall verdict on the film itself."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Criticizes the Hot Ones scene as an unnecessary choice that should have been handled differently."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Jokey nitpick of one scene choice, no overall film judgement."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Commenter critiques a scene choice (Hot Ones segment) in a promo or film context without giving an overall judgement of the film itself."},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"They criticize the film\u2019s choice to use Hot Ones without showing the wings affecting her."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"Criticizes the realism and execution of a scene, indicating dissatisfaction with the film."}}},{"split":"heldout","movie":"Verity","rank":30,"type":null,"human":0,"excerpt":"Not to mention they would see she died of asphyxiation. While not always foul play, it would definitely look suspicious. Especially with hubby immediately hooking up with and impregnating the live in hot ghost writer. Just a little suspicious right?","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.84"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.32 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.37"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.31"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The commenter is criticizing the film for being suspicious and not to be trusted, which is a negative verdict."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Criticism of plot and character actions, suggesting suspicious behavior"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment only discusses in-universe plot details and character actions without expressing an opinion on the film's quality."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses plot logic about suspicious death; no judgement of the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses plot details and suspicious circumstances in the story without giving any judgment of the film itself."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Discusses suspicious plot circumstances and cause of death without judging the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Plot speculation about a suspicious death; no judgement of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Plot speculation about how the death would look to police; no judgement of the film's quality."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Discusses suspicious circumstances in the plot without judging the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The comment discusses plot logic and makes a joke without judging the film."}}},{"split":"heldout","movie":"Verity","rank":57,"type":null,"human":0,"excerpt":"Haven\u2019t seen it brought up, but did no one else think her relationship with her agent was weird? I got the impression they were casually sleeping around, before her \u201cyou should focus on Becca\u201d.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.15"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.54 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.58"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.38"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":""},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment discusses relationship dynamics without expressing a verdict on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment only discusses a character relationship and plot detail without expressing any opinion on the film's quality."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses a character relationship detail without judging the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Commenter raises an observation about a plot detail (agent relationship) without giving any overall judgement of the film."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Discusses a character relationship and its implications without judging the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Plot question about a character relationship; no film judgement."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses a plot detail about the characters' relationship with no judgement of the film's quality or recommendation."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Discusses an interpretation of the characters\u2019 relationship without judging the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The commenter is discussing a plot point and asking a question about character relationships without expressing an opinion on the film."}}},{"split":"heldout","movie":"Verity","rank":63,"type":null,"human":0,"excerpt":"I laughed out loud when Verity said the babies sleep all night because they were exhausted from crying all day from being ignored. Idk but I was like damn thats funny. And then theater was silent I felt kinda bad for laughing.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.13"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.13 P(pos)=0.23"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.83"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.49"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The commenter found the movie funny and enjoyed the laugh-inducing scene."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment is about the humor of a scene, not a judgment on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The commenter shares an anecdote about laughing at a dark line in the theater without judging the film overall."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Shares a personal reaction to a darkly funny line without judging the film overall."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Describes laughing at a dark joke in a scene, but gives no overall judgment of the film or recommendation."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Describes laughing at a disturbing line and the audience reaction, without judging the film overall."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Amused by one line; no overall film judgement."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Describes a laugh at a specific line and the theater's reaction; no clear judgement of the film overall."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"They describe laughing at a scene and the theater\u2019s reaction, but give no judgment of the film overall."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The commenter shares a funny anecdote about their theater experience without judging the film itself."}}},{"split":"heldout","movie":"Verity","rank":64,"type":null,"human":-1,"excerpt":"How Dakota talks to her lawyer friend about his love life like they've never talked before. Incredibly awkward. I didnt like the movie but he was her publisher, not lawyer and it was pretyy apparent that they had been casually hooking up but now she said he should try to get serious with the girl fr","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.79"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.99 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.82"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.86"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The commenter clearly did not like the movie and found it awkward and unrealistic."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Criticism of the plot and characters, no clear praise"},"openrouter:google/gemini-4-argon":{"label":-1,"verdict":"negative","rationale":"The commenter explicitly states 'I didnt like the movie' while correcting a plot point from a quote."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Commenter explicitly states they didn't like the movie while clarifying a plot detail."},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Commenter explicitly states they didn't like the movie, while the rest is plot clarification and scene discussion."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Explicitly says they didn't like the movie and describes dialogue as incredibly awkward."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Explicitly says they did not like the movie."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":""},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"The commenter explicitly says they didn\u2019t like the movie."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"The commenter explicitly states 'I didnt like the movie' and points out awkward elements."}}},{"split":"heldout","movie":"Verity","rank":84,"type":null,"human":-1,"excerpt":"Plot connivence.","pred":{"vader":{"label":0,"verdict":null,"rationale":"compound +0.00"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.36 P(pos)=0.01"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.47"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.37"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment criticizes the plot, which is a negative verdict."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment does not provide a clear positive, negative, or mixed verdict on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"A two-word trope answer ('plot convenience') without enough context to establish an overall verdict on the film."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Likely means 'plot convenience', a dismissive jab at contrived writing; very terse, so the verdict is uncertain."},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Calling out 'plot convenience' is a criticism of the film's writing, with no praise to offset it."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Dismisses the film's plotting as contrived or overly convenient."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Calls out plot convenience as a flaw in the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":"Calls the plot 'connivance' (contrived), a criticism implying the film's storyline is implausible or manipulative."},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"Calling the plot contrived is a criticism of the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"Criticizes the film's writing quality."}}},{"split":"heldout","movie":"Verity","rank":93,"type":null,"human":1,"excerpt":"I understand, for time, why they cut some stuff but god the book is so much more fucked up and we could have gotten rid of the entire birthday sequence to dive more into the twin girls, her hating both then having the dream then mainly hating Harper and then the other twin accidentally dying to Harp","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.68"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.99 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.88"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.94"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The reviewer criticizes the book and the movie, but does not explicitly state they disliked the film itself."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The commenter expresses wanting more depth from the book and appreciates the movie despite some cuts, indicating a positive overall view."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"mixed","rationale":"Calls it a 'good movie' but expresses clear dissatisfaction and disappointment with cut material compared to the book."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"mixed","rationale":"Calls it a good movie but repeatedly expresses dissatisfaction with cuts and lack of depth versus book."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"mixed","rationale":"Says 'it's a good movie' but laments cuts and unsatisfying adaptation versus the book; leans slightly positive but with clear criticism."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Explicitly calls it a good movie, despite disappointment with omitted book material and the adaptation\u2019s depth."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"mixed","rationale":"Calls it a good movie but criticizes cuts and feels unsatisfied."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":""},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"mixed","rationale":"Calls it a good movie but criticizes major omissions and says they weren\u2019t satisfied."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"mixed","rationale":"The user calls it a good movie but expresses disappointment with the adaptation compared to the book."}}},{"split":"heldout","movie":"Street Fighter","rank":34,"type":null,"human":0,"excerpt":"I was kinda shocked she didn't get the spiked cuffs. and that they added full pants to her costume","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.25"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.05 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.50"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.40"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The commenter is criticizing the addition of full pants to the character's costume, which is a negative point."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Commenter is discussing plot elements without a clear judgment on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment merely observes costume changes without expressing an opinion on the quality of the film itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comments on costume design changes without judging the film's quality overall."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comments on costume differences and a missing detail, expressing surprise, but gives no clear judgment of the film itself."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Expresses surprise at costume changes without judging the film's quality or recommending it."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Costume detail observation only; no judgment of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comments on costume differences from the source material without expressing any overall judgement of the film."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Comments on costume changes without expressing an overall opinion of the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The commenter is only discussing costume design details without expressing a judgment on the film."}}},{"split":"heldout","movie":"Street Fighter","rank":53,"type":null,"human":0,"excerpt":"People seems to be split about the fight scenes. IMO Ken vs Dhalsim, onsen fight, ken vs ryu in the hotel were great. Chun Li vs Vega and Juri could've used more lights and fewer cuts, ditto Ryu vs Ken alley fight. Parts of the prison break fight were memorable. Blanka and Okuma fights were fine. Wi","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.65"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.82"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.93"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":0,"verdict":"mixed","rationale":"The comment is mixed as it praises some fight scenes and criticizes others, leaning towards positive but not overall positive."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praises specific fight scenes and overall tone is enthusiastic with few criticisms."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"mixed","rationale":"Offers a balanced mix of specific praise and criticism for fights and cast without stating an overall preference."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"mixed","rationale":"Praises several fights and the lead actors but criticizes lighting, editing, missing matchups, and some characters; no clear lean."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"mixed","rationale":"Praises several fight scenes and the lead actors, but criticizes lighting, editing, missing Guile vs Vega, and too much Dan/Kyle Mooney; no clear overall lean."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"mixed","rationale":"Praises several fights and lead actors but criticizes other fights, missing matchups, and excessive supporting characters without an overall lean."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"mixed","rationale":"Praises some fights and leads, criticizes others and excess characters, no overall lean."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"mixed","rationale":""},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"mixed","rationale":"Praises several fights and the lead actors, but criticizes other fights and says Dan and Kyle Mooney were overused."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"mixed","rationale":"Praises several fight scenes and actors, but criticizes others and pacing choices."}}},{"split":"heldout","movie":"Street Fighter","rank":60,"type":null,"human":0,"excerpt":"NOW DRIVE! pause THE CAR!","pred":{"vader":{"label":0,"verdict":null,"rationale":"compound +0.00"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.34 P(pos)=0.97"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.48"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.45"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment is a play on words and does not provide a clear verdict on the film itself."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Commentary on a scene from the movie, no clear judgment on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment is simply a quote/reference without expressing an opinion on the film."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Just a quote from the film with comedic emphasis; no judgement of the movie expressed."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Quoting or joking about a line from the film, with no judgment of the film's quality or a recommendation."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Quotes dialogue from the film without expressing an opinion on its quality."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Quote or joke only; no judgment of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Appears to be a quoted line or joke with no clear judgement of the film itself."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"A quoted line with no opinion about the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"Quoting a line from the movie without expressing an opinion."}}},{"split":"heldout","movie":"Street Fighter","rank":62,"type":null,"human":0,"excerpt":"I was like what the fuck is that","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.25"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.49 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.60"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.56"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment expresses shock and confusion, indicating a negative reaction to the film."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"The comment does not express a clear positive or negative verdict on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"Ambiguous reaction to something in the movie without a clear judgment on the film itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Ambiguous shocked reaction to something unspecified; no clear judgement of the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Expresses surprise or confusion at something, but gives no clear judgment of the film itself."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Expresses surprise or confusion without clearly evaluating the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Bewildered reaction only; no clear like or dislike of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Expresses bewildered reaction to a moment but gives no clear judgement of the film's quality or whether they'd recommend it."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Expresses surprise or confusion without judging the film itself."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"Expresses strong shock and disapproval of the film."}}},{"split":"heldout","movie":"Street Fighter","rank":71,"type":null,"human":-1,"excerpt":"If you don't want us to compare this Bison to the great Raul Julia's take, then don't have this one quote the \"it was Tuesday\" line.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.60"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.91 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.78"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.43"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment criticizes the film by comparing it unfavorably to another version of the character and suggests not watching it."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment is about a specific quote and not a general verdict on the film."},"openrouter:google/gemini-4-argon":{"label":-1,"verdict":"negative","rationale":"Criticizes the portrayal and writing of the villain by comparing him unfavorably to a previous version."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Criticizes quoting Raul Julia's line, implying this Bison compares poorly to the original."},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Commenter implies this Bison pales next to Raul Julia's, criticizing the film for inviting an unfavorable comparison by reusing the iconic line."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Criticizes the film for inviting an unfavorable comparison to Raul Julia\u2019s Bison by reusing his iconic line."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Criticizes the film for quoting Julia\u2019s line and inviting an unfavorable comparison."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment compares a character performance to another actor and critiques a quote choice; no clear judgement of the film's quality or recommendation."},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"The commenter criticizes this film's Bison for invoking Raul Julia's iconic line and inviting comparison."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"Criticizes the film's poor imitation of Raul Julia's iconic Bison line."}}},{"split":"heldout","movie":"Street Fighter","rank":74,"type":null,"human":1,"excerpt":"This movie was absolutely idiotic, and I CANNOT WAIT to see the sequel.","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.60"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.98"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.99"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The commenter clearly dislikes the film and wants a sequel."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Clear praise and clear criticism with no overall lean"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"Despite calling it idiotic, the commenter clearly enjoyed it and eagerly anticipates the sequel."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Calls it idiotic, but eagerness for the sequel signals overall enjoyment, likely ironic or fun-dumb appreciation."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Calls it idiotic but is enthusiastic, can't wait for the sequel; implies enjoyment and recommendation despite the silliness."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Calling it idiotic is outweighed by enthusiastic anticipation for the sequel, suggesting enjoyment despite its absurdity."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Calls it idiotic but eagerly wants the sequel"},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"mixed","rationale":"Calls the film idiotic, but is eagerly anticipating the sequel, so the overall lean is ambiguous; the sarcasm-laden enthusiasm conflicts with the insult."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Despite calling it idiotic, their eagerness for a sequel signals they enjoyed the movie."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Calling it idiotic while eagerly wanting a sequel shows enthusiastic enjoyment."}}},{"split":"heldout","movie":"Street Fighter","rank":89,"type":null,"human":0,"excerpt":"I think you\u2019ll be satisfied with the first 10 minutes or so!","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.47"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.61 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.48"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.48"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the first 10 minutes of the film."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Suggestion of satisfaction in the first 10 minutes"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment only addresses whether another user will like the first 10 minutes without judging the film overall."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Backhanded remark implying only the opening is satisfying and the rest disappoints."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Only says the first 10 minutes will satisfy; a limited remark about part of the film, not an overall verdict."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Suggests the opening ten minutes will satisfy the viewer, expressing a favorable recommendation."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Only predicts the opening may satisfy someone; no overall film judgement."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"mixed","rationale":"Vague hedged remark implying the opening is good but the rest may disappoint; no clear overall verdict."},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"Implies the movie is only satisfying for its first 10 minutes, suggesting disappointment overall."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"Comment discusses a specific scene without a clear overall judgment of the film."}}},{"split":"heldout","movie":"Street Fighter","rank":94,"type":null,"human":-1,"excerpt":"I don't know what happened with this film. Either there was a TON of studio interference or they passed the script through AI or something. Everything looks fine from a glance. Most of the actors fit their role well, the costume design is on-point, the fights ranged from eh to pretty good, VFX strad","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.98"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.98 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.63"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.94"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The reviewer criticizes multiple aspects of the film, including editing, dialogue, characters, and overall structure, with no overall positive lean."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Criticism of multiple aspects including story, dialogue, character behavior, and ending."},"openrouter:google/gemini-4-argon":{"label":-1,"verdict":"negative","rationale":"Despite praising some surface elements, the commenter concludes the film was derailed by messy editing, dialogue, and tonal whiplash."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Praises surface elements but heavily criticizes editing, dialogue, tone, and ending; overall says the film was derailed."},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Acknowledges some good surface elements but overall harshly criticizes editing, dialogue, tone, and ending as a derailed mess."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Despite praising individual elements, they strongly criticize the film\u2019s incoherent scenes, dialogue, tonal whiplash, and ending."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Praises pieces but concludes the film was derailed and a mess."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":""},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"Despite praising some elements, they criticize the film\u2019s editing, dialogue, tone, characters, and ending as derailed and incoherent."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"Praises aesthetics and casting, but strongly criticizes editing, dialogue, tone, and execution, concluding it was derailed."}}},{"split":"heldout","movie":"Other Mommy","rank":42,"type":null,"human":-1,"excerpt":"I don't know if this is the best way to describe it, but the movie had an almost uncanny valley feel to it. The way the characters spoke to each other, the things they said, the quick transitions between locations. I felt like I wasn't watching a real movie at times. I normally dislike when a horror","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.21"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.97 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.49"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.75"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The reviewer describes the movie as having an "},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"mixed","rationale":"Criticism of the horror elements but also appreciation for the unique style"},"openrouter:google/gemini-4-argon":{"label":-1,"verdict":"negative","rationale":"Criticizes the dialogue, transitions, and lack of depth, stating it didn't feel like a real movie."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Criticizes unnatural dialogue, abrupt transitions, and too little lore; felt unlike a real movie."},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Commenter describes the film as feeling unreal and criticizes its execution, saying it went too far in the wrong direction; overall critical."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Criticizes unnatural dialogue and abrupt transitions, finding the film too sparse in lore and dialogue."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Criticizes unnatural dialogue, transitions, and unreal feel as too extreme."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":""},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"They criticize the film\u2019s dialogue, transitions, and lack of lore, saying it sometimes didn\u2019t feel like a real movie."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"The commenter felt detached and criticized the movie for going too far away from lore and dialogue."}}},{"split":"heldout","movie":"Other Mommy","rank":54,"type":null,"human":0,"excerpt":"Wow, that is a crazy ending in the book. No wonder people online were talking about how the movie would tackle it I honestly like grim endings like that and I personally would have preferred it. But I know for sure general audiences would have hated this bleak conclusion.","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.38"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.99 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.91"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.62"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The reviewer likes grim endings but disliked the bleak conclusion, leaning negative overall."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"mixed","rationale":"The comment expresses liking grim endings and preferring a bleak conclusion, but also mentions that general audiences would have hated it."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"Discusses preference for the book's grim ending over the movie's adaptation, without giving an overall verdict on the film."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses the book's ending and personal preference for grim endings; no judgement of the film itself."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses the book's ending and how the movie handled it, expressing a preference for the bleaker ending, but gives no clear verdict on the film itself."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Discusses the book\u2019s ending and a preferred adaptation choice without judging the film overall."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Discusses the book ending and adaptation preference, not an overall film verdict."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses the book's ending and what audiences would think; no clear judgement of this film's quality or recommendation."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"They express a preference for the book\u2019s grim ending, but don\u2019t judge the film itself."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"The commenter notes the film avoided the grim book ending because general audiences would have hated it."}}},{"split":"heldout","movie":"Other Mommy","rank":64,"type":null,"human":0,"excerpt":"When they set out all the weapons they had in the house on a table?? Like did you not see frank unsuccessfully try to shoot the entity? You\u2019re just giving it sharp things to kill you with!","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.40"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.97 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.78"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.89"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment criticizes the film's plot and characters, suggesting it is poorly executed."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Criticism of character's actions, suggesting ineffective use of weapons"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment mocks a specific character decision in the plot without stating an overall opinion on the film."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Jokes about characters' poor decision-making without judging the film's quality overall."},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Criticizes characters' illogical decision as foolish, implying a plot-logic complaint, though it's mostly in-scene commentary without an explicit verdict on the film."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Questions the characters\u2019 weapon choices against the entity without judging the film itself."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Mocks an illogical character decision as stupid writing."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comments on a plot moment and characters' choices in the film without judging whether the film is good or worth watching."},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"They criticize the scene\u2019s characters\u2019 decision as illogical, implying the film\u2019s writing frustrated them."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"Comment discusses plot logic and character actions without judging the film itself."}}},{"split":"heldout","movie":"Other Mommy","rank":67,"type":null,"human":0,"excerpt":"A lot of the decisions you just described can likely be explained with them trying to keep some semblance of normalcy for Bela Especially with their little \u201cparty\u201d before the final fight when Bela calls them out and says they could stop pretending As for the strangers at the restaurant, I mean what ","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.56"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.02 P(pos)=0.01"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.83"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.83"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":0,"verdict":"mixed","rationale":"The comment discusses the characters' actions and reactions but does not express a clear positive or negative opinion of the film itself."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment discusses plot and character motivations without expressing a clear like/dislike for the film."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment discusses character motivations and plot points without expressing an overall opinion on the film's quality."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Explains character decisions and plot logic without giving any judgement of the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Defends character decisions and discusses plot points without giving an overall judgement of the film's quality."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Explains characters\u2019 decisions and reactions without expressing an overall judgment of the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Explains plot choices; no judgement of the film itself."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Explains characters' motivations and plot decisions in the film without expressing any overall like or dislike of it."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Discusses and defends the characters\u2019 choices without judging the film itself."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"Discusses plot decisions and characters without expressing a judgement on the film."}}},{"split":"heldout","movie":"Other Mommy","rank":73,"type":null,"human":1,"excerpt":"Maybe I'm a dunce or something but I found it to be good and actually recommended it to people. I think there's a lack of empathy for the story being told here. There's a marriage that started off on the wrong foot. Unwanted motherhood. An affair. This is all happening while the parents think all of","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.96"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.99 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.78"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.69"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The reviewer liked the film and gave it a positive rating."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The commenter recommends the film and praises the story, despite some minor criticisms."},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly states they found it good, recommended it, and rated it 8/10 despite minor pacing criticisms."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Found it good, recommended it, rated 8/10; minor criticism of a rushed character turn."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Commenter liked the film, recommended it to others, and gave 8/10; the only criticism is a minor note about rushed character development."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Explicitly calls it good, recommended it, and rates it 8/10 despite criticizing rushed character development."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Liked and recommended it; 8/10 despite rushed character beat."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Says they found it good, recommended it to others, and rated it 8/10, despite minor criticism of one part."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They recommend the film and rate it 8/10, despite finding one character-development turn contrived and rushed."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The user explicitly states they found it good, recommended it, and gave it an 8/10."}}},{"split":"heldout","movie":"Other Mommy","rank":85,"type":null,"human":0,"excerpt":"This movie is absolutely insane, sometimes in good ways, sometimes in bad ways. It was doing too much and too little at the same time. Chastain gives a killer performance though, bravo. I like that everyone can just see the entity off the bat. Hate how they dealt with that after. I like that the pow","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.20"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=0.01"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.98"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.97"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":0,"verdict":"mixed","rationale":"The reviewer praises the performance and the lore, but criticizes the pacing and the powers, giving a mixed verdict."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise for Chastain's performance and some aspects of the plot, overall lean is positive despite criticisms."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"mixed","rationale":"The commenter lists equal pros and cons, calling it fun overall with a lukewarm 6/10 rating."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"mixed","rationale":"Balanced likes and dislikes throughout; 'fun overall' but 6/10 with major criticisms, no clear lean."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"mixed","rationale":"Praises Chastain's performance and calls it fun overall, but lists many criticisms; 6/10 with no strong lean either way."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Calls it fun overall and rates it 6/10, despite substantial criticism of its execution."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"mixed","rationale":"Balances likes and hates; fun overall but flawed, 6/10."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":""},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Despite numerous criticisms, they call it fun overall and rate it 6/10."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"mixed","rationale":"The reviewer highlights both good and bad elements, concluding it was fun overall but messy with a 6/10 score."}}},{"split":"heldout","movie":"Other Mommy","rank":88,"type":null,"human":0,"excerpt":"The Coraline comparisons are driving me crazy. I\u2019ve read and loved the book source material, and this is wholly different. People say that it\u2019s copying Coraline, but that\u2019s only because of the name \u201cOther Mommy.\u201d However, \u201cOther Mommy\u201d is the best name for a woman-like monster trying to manipulate a","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.82"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.77 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.72"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.42"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":""},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise for the name and concept of the monster"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment defends the title and praises the book source material, without giving a verdict on the film itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Defends the film against Coraline comparisons and praises the book, but gives no judgement of the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Defends the film against Coraline comparisons and praises the title, but gives no clear overall judgment of the film itself."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Defends the title and disputes Coraline comparisons, but gives no overall judgment of the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Defends the title against Coraline comparisons; no clear like or dislike of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":""},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They defend the film as distinct and praise \u201cOther Mommy\u201d as an especially fitting name for its monster."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The comment discusses comparisons to another movie and the source material without judging the film itself."}}},{"split":"heldout","movie":"Other Mommy","rank":93,"type":null,"human":0,"excerpt":"I haven\u2019t seen it and by all accounts it\u2019s not good but I am tired of horror movies that spend a lot of runtime leading on if the monster is real and/or no one believing the main character. You can still have tension without using those tropes.","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.88"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.93 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.45"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.92"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":0,"verdict":"mixed","rationale":"The comment criticizes the film's use of horror tropes but also suggests the film can be tense without them."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Commenter expresses lack of knowledge and preference for different horror styles"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The commenter explicitly states they haven't seen the movie and is only discussing horror tropes in general."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Commenter hasn't seen the film; relays others' opinions and complains about horror tropes generally."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Commenter hasn't seen the film; they only relay hearsay and complain about general horror tropes, offering no personal verdict."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Hasn't seen the film; relays others' opinions and criticizes general horror tropes rather than judging this film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Commenter has not seen the film; only secondhand and genre complaints."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Commenter explicitly hasn't seen the film and is critiquing horror tropes generally, not giving a verdict on this film."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"They explicitly haven\u2019t seen the film; their criticism is of horror tropes generally."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The commenter has not seen the film and discusses general horror tropes instead."}}},{"split":"challenge","movie":"Wildwood","rank":7,"type":"dark_words_positive_verdict","human":1,"excerpt":"I don't think it can be overstated how absolutely gorgeous this film was. The final battle was one of the most beautiful spectacles I've seen on screen in a fair amount of time. The cast was strong across the board and the story was very dark when it had to be. If I had one complaint it would be tha","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.93"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.93"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.59"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is overwhelmingly positive, praising the film's beauty and strong cast, but also acknowledging a minor issue with pacing."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praises the film's beauty, cast, and story; expresses mild disappointment but forgiving."},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"Strong praise for visuals, cast, and story, explicitly forgiving a minor pacing complaint."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Strong praise for visuals, cast, story; minor pacing complaint explicitly forgiven."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Heavy praise for visuals, final battle, cast, and story; only a minor complaint about pacing, which they forgive."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Strong praise for visuals, cast, and dark storytelling outweighs a minor pacing complaint."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Praises visuals, cast, and story; forgives pacing drag."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Strong praise for visuals, battle, and cast; one minor complaint about pacing, overall leaning positive and forgiving."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Strong praise for the visuals, cast, and story outweighs a minor complaint that it dragged at times."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Praises the film's visuals, cast, and story, dismissing minor pacing issues because it was amazing."}}},{"split":"challenge","movie":"Wildwood","rank":21,"type":"dark_words_positive_verdict","human":1,"excerpt":"I think the \"sudden fatality\" award for the year goes to that eagle. We knew going in that the movie was going to be more intense than the average animated family movie, but I don't think most of us expected it to be quite as bloody as it ended up being (not that this was a bad thing at all).","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.88"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=0.99"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.55"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.47"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the intensity and bloodiness of the movie, which is a positive aspect."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Comment praises the intensity and violence of the movie without any negative criticism."},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter notes the movie's unexpected intensity and bloodiness, explicitly clarifying that this was not a bad thing at all."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Notes the surprising bloodiness and explicitly says it wasn't a bad thing, a mildly approving take."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Notes the surprising bloody intensity but explicitly says it wasn't a bad thing; mild approval, though no overall recommendation is stated."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Approves of the film\u2019s unexpectedly bloody intensity, explicitly noting that it was not a bad thing."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Notes unexpected bloodiness but explicitly says that was not a bad thing."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":""},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They approve of the film being bloodier and more intense than expected, saying that was not a bad thing."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter notes the film is intense and bloody, explicitly stating 'not that this was a bad thing at all'."}}},{"split":"challenge","movie":"Wildwood","rank":24,"type":"dark_words_positive_verdict","human":1,"excerpt":"There\u2019s quite a few scenes where characters are just killed so suddenly and brutally. Maybe I\u2019m just a sucker for it, but I like when violence is portrayed like that because it\u2019s so cruel. Like the Eagle being shot down so suddenly after that beautiful flight, or that brief bit where a bandit shoves","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.41"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.97 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.50"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.82"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The commenter appreciates the brutal and sudden violence in the film, describing it as 'cruel' and 'enunciates it to me.'"},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise of violent and disturbing content, describing scenes as 'beautiful' and 'enunciates it to me'."},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly states they liked the sudden, brutal portrayal of violence in the film."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Commenter praises the film's sudden, brutal portrayal of violence as effective and impactful."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Commenter explicitly likes the sudden, brutal violence and cites specific scenes approvingly, indicating appreciation of the film's choices."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Praises the film\u2019s sudden, brutal violence as effectively conveying cruelty."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Praises the film\u2019s sudden, brutal violence as effective and cruel."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":""},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They praise the film\u2019s sudden, cruel violence and say they like how it is portrayed."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter praises the sudden and brutal violence in the film, stating they like it."}}},{"split":"challenge","movie":"Wildwood","rank":61,"type":"dark_words_positive_verdict","human":1,"excerpt":"Holy shit Wildwood was one of the best Western animated movies I\u2019ve seen. I knew Wildwood was PG-13, but I was still surprised by how dark and violent the movie got. In many ways, Wildwood reminded me of Princess Mononoke. Both movies have a protagonist thrown in the middle of a forest conflict with","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.73"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.98"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.96"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The commenter uses positive language like 'Holy shit' and 'one of the best', and praises the movie's dark and violent elements."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise of the movie's dark and violent content, comparison to a well-regarded film, and personal connection with the characters"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly states it is one of the best Western animated movies they have ever seen."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Calls it one of the best Western animated movies, rates it above Princess Mononoke; darkness noted approvingly."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Calls it one of the best Western animated movies and says it's stronger than Princess Mononoke for character connection."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Calls it one of the best Western animated movies and prefers it to Princess Mononoke."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Calls it one of the best and stronger than Mononoke."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":""},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Calls it one of the best animated movies and says it is stronger than Princess Mononoke."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter calls it one of the best Western animated movies and stronger than Princess Mononoke."}}},{"split":"challenge","movie":"Wildwood","rank":65,"type":"dark_words_positive_verdict","human":1,"excerpt":"Unapologetically brought my elementary school aged kids to this movie. We had a lot of mature themes to discuss, but what a wonder to behold as a kid and this is absolutely a great way to start discussions about grief and letting go. Mufasa getting trampled is far more traumatic IMO.","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.43"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.87 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.74"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.71"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The reviewer praises the mature themes and calls it a great way to start discussions about grief and letting go."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise for the movie's ability to start discussions about mature themes and grief, and the positive impact on the viewer's children."},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter calls the film 'a wonder to behold' and praises how it handles mature themes."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Calls it a wonder to behold and a great way to discuss grief; defends bringing kids."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Commenter praises the film as a wonder for kids and a great way to start discussions about grief; recommends it despite mature themes."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Calls the film a wonder to behold and praises its value for discussing grief and letting go with children."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Calls it a wonder and a great way to discuss grief."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter brought kids, calls it a wonder and a great way to start discussions; clearly recommends it."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Calls the film a wonder to behold and a great way to start meaningful discussions."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Praises the film as a wonder for kids and a great tool for discussions."}}},{"split":"challenge","movie":"Primetime","rank":24,"type":"dark_words_positive_verdict","human":1,"excerpt":"He was terrific - Rent at the end was chilling","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.46"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.94 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.42"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.50"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment praises the actor's performance and describes a chilling ending, indicating a positive overall impression."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise for the villain and mention of a chilling scene"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter praises a performance as 'terrific' and calls the ending 'chilling' in an admiring way."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Praises the actor's performance as terrific and the ending as chilling, which reads as approval of the film."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Calls the performance terrific and the ending chilling, which is praise of the film's dark content, so the verdict is positive."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Praises the performance as terrific and the ending as chilling."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Praises the performance and finds the ending chilling in a complimentary way."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Praises the performance as terrific and calls a scene chilling, which reads as approval of the film."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Praises his performance and describes the ending as chilling, expressing admiration for the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Praising the acting ('terrific') and the ending ('chilling') indicates a positive verdict."}}},{"split":"challenge","movie":"Primetime","rank":35,"type":"dark_words_positive_verdict","human":1,"excerpt":"The preceding fallout of the affair combined with the rabbi gave this offputting vibe and it kind of worked. Like, the whole room felt disgusting.","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.23"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.75 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.74"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.46"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment criticizes the film for its offputting vibe and describes it as "},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Describes the scene as disgusting without clear praise or criticism"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter praises the film's effective, intentionally off-putting and disgusting atmosphere, stating that 'it kind of worked'."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Says the off-putting, disgusting atmosphere 'kind of worked' \u2014 praising the film's intended discomfort."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Commenter says the offputting, disgusting vibe 'kind of worked', praising the film's atmosphere rather than criticizing it, though the praise is mild."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Says the offputting atmosphere worked, praising the film\u2019s ability to make the room feel disgusting."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Offputting, disgusting atmosphere 'kind of worked'\u2014praise of the film's vibe."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter says the offputting, disgusting atmosphere 'kind of worked', praising the film's unsettling effect despite no explicit recommendation."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"The unsettling, disgusting atmosphere is praised as effective; the offputting vibe kind of worked."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter describes the offputting vibe as something that 'kind of worked', indicating a favorable reception."}}},{"split":"challenge","movie":"Primetime","rank":52,"type":"dark_words_positive_verdict","human":1,"excerpt":"A film about exploitation that wasn't afraid to show the hypocrisy inherent in doing something like that for television. The predators wanted to exploit kids. The production wanted to exploit the predators. The National Inquirer wanted to exploit Chris Hansen. Chris Hansen wanted to exploit everyone","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.86"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.89"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.66"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the film's exploration of exploitation and its ability to make the audience feel sick at the same time. The reviewer also compliments the performances of the main actors."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise for the film's approach, emotional impact, and performances"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter praises the performances, themes, and calls it a \"great feel bad time at the movies.\""},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Praises the film's handling of hypocrisy and the performances, and calls it fascinating and 'a great feel bad time.'"},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Calls it fascinating, praises performances, and 'a great feel bad time at the movies'; the sickening feeling is praise of its themes."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Calls it fascinating and a great feel-bad experience, praising the performances and exploration of exploitation."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Calls it fascinating and a great feel-bad movie; praises the cast and approach."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Calls it fascinating, praises the performances, and ends with 'A great feel bad time at the movies.'"},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Calls it fascinating, praises the performances, and describes it as a great movie experience despite its disturbing subject."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Praises the film's themes, performances, and overall impact, calling it a great film despite making them feel sick."}}},{"split":"challenge","movie":"Primetime","rank":72,"type":"dark_words_positive_verdict","human":1,"excerpt":"Same, I think that this is my favorite movie cinematography-wise in a long time. That shot where he's confronting the rabbi, and it slowly puts his face more in shadow until his eyes are going black gave me goosebumps. A lot of absolutely beautiful and haunting shots in this movie.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.84"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.99 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.83"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.97"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the cinematography and describing beautiful and haunting shots, which are positive attributes."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise of cinematography and specific scenes, no criticism"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter strongly praises the film's cinematography, describing the shots as beautiful, haunting, and giving them goosebumps."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Calls it their favorite movie cinematography-wise in a long time, praising beautiful, haunting shots."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Calls it a favorite film in a long time for its cinematography, praising beautiful, haunting shots that gave goosebumps."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Calls the cinematography a recent favorite and praises the film\u2019s beautiful, haunting shots."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Praises cinematography as a favorite, calling shots beautiful and haunting."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":""},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They praise the film\u2019s cinematography and haunting visuals, describing the shots as beautiful and evocative."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Praises the cinematography and calls the shots beautiful and haunting."}}},{"split":"challenge","movie":"Forgotten Island","rank":54,"type":"dark_words_positive_verdict","human":1,"excerpt":"\u201cI know she tried to kill us - but she is SO cool.\u201d","pred":{"vader":{"label":0,"verdict":null,"rationale":"compound +0.00"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.52 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.81"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.36"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the character's coolness, which is a positive aspect."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise of the villain's coolness"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment is just a quote or joke from the movie without any judgment on the film itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"A quote or joke about a character; no judgement of the film itself."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Quote about a character being cool; admires a character but gives no explicit judgment of the film itself."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Quotes dialogue praising a character, without judging the film itself."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Quoted line about a character, no judgment of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"The commenter praises a character as cool, but gives no clear verdict on the film itself."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Praises a character, but gives no opinion on the film itself."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"Quote from the film or a joke about a character, no direct review of the movie."}}},{"split":"challenge","movie":"Forgotten Island","rank":64,"type":"dark_words_positive_verdict","human":1,"excerpt":"Gorgeous animation and very fun, but too hyperactive for its own good and filled with unnecessary characters that pad the runtime. Really wished I loved this more, and found that when they were in the human world was actually the strongest part. The voice cast killed it.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.96"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=0.01"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.98"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.95"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is mostly positive, praising the animation and voice cast, but also criticizing the hyperactivity and unnecessary characters."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"mixed","rationale":"Balanced praise and criticism, leaning towards negative due to runtime and unnecessary characters."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"mixed","rationale":"Praises the animation, voice cast, and fun factor, but criticizes the pacing and unnecessary characters with no clear overall lean."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"mixed","rationale":"Praises animation, fun, and voice cast but criticizes hyperactivity and padding; wished they loved it more."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"mixed","rationale":"Praises animation, fun, voice cast, and human-world segments, but criticizes hyperactivity, padding characters, and says they wished they loved it more."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Despite praising animation and voice acting, they lean disappointed, criticizing hyperactivity, unnecessary characters, and padded runtime."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Praises animation and cast but wished they loved it more; soft disappointment."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":"Praises animation and fun, but says it's too hyperactive, padded, and they wished they loved it more; soft disappointment leans negative."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"mixed","rationale":"They praise the animation, fun, and voice cast, but say it felt overstuffed and they wished they had loved it more."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"mixed","rationale":"Praises animation and voice cast, but criticizes pacing and padding, leading to soft disappointment."}}},{"split":"challenge","movie":"Forgotten Island","rank":74,"type":"dark_words_positive_verdict","human":1,"excerpt":"Loved it! The animation was gorgeous and the voice acting was superb. Manang and Batiba were both terrifying in their own ways. I always enjoy learning about new cultures this way, so I feel it's a must watch.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.95"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.75 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.74"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.52"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is entirely positive, praising the animation, voice acting, and the film's educational value."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Clear praise with no criticism"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly loved the film, praised the animation and voice acting, and called it a must watch."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Loved it, praises the animation and voice acting, and calls it a must watch; 'terrifying' is praise for the villains."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Says they loved it, praises animation and voice acting, calls it a must watch; the terrifying characters are praised, not criticized."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Explicitly loved it, praised the animation and voice acting, and called it a must-watch."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Explicitly loved it and calls it a must-watch."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter loved the film, praising animation and voice acting, and explicitly calls it a must watch."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"The commenter loves the film, praises its animation and voice acting, and calls it a must-watch."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter explicitly states they 'Loved it!' and calls it a 'must watch'."}}},{"split":"challenge","movie":"Forgotten Island","rank":84,"type":"dark_words_positive_verdict","human":1,"excerpt":"What a fever dream of a movie (meant in the best way) It wasn\u2019t a perfect 10/10, but it was a fun ride! And my goodness the Manang was terrifying","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.95"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.99 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.64"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.83"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the movie and describing it as a fun ride, even though it wasn't perfect."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"clear praise with no overall criticism, mentions it was a fun ride and the villain was terrifying"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly praises the film as a 'fun ride' and effectively scary, despite noting it isn't perfect."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Calls it a fun ride and a fever dream in the best way; scary villain is praise; minor caveat."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Calls it a fever dream 'in the best way' and a fun ride; minor caveat of not perfect, and praises the terrifying Manang."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Calls it a fun ride and a fever dream in the best way, praising the terrifying Manang."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Calls it a fun fever dream despite not being perfect; terror is praise."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Calls it a fever dream meant in the best way and a fun ride, despite not being perfect."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Calls it a fun ride despite not being perfect; the terrifying character is praised."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Called it a fun ride and meant fever dream in the best way."}}},{"split":"challenge","movie":"Wicker","rank":11,"type":"dark_words_positive_verdict","human":1,"excerpt":"I loved the framing of the story. It reminded me so much of a Grimm's fairy tale, with the way everyone is just their title (Tailor's-wife, the Butcher, etc) and the title cards throughout. Peter Dinklage really got to showcase his Shakespearean acting chops with his monologues. And of course lovely","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.96"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.86 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.89"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.88"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The reviewer praises the story's framing, the acting, the British humour, and the villain's character development, with only a slight negative comment about the final character's actions."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Clear praise for multiple aspects of the film"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly praises the story's framing, humor, characters, and several actors' performances."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Loved the fairy-tale framing, praises the performances, humour and villain; enthusiastic throughout."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Commenter explicitly loved the framing and praises the acting, humour, and villain, with no criticism of the film."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Loved the framing, fairy-tale style, humour, performances, and villain characterization; consistently praises the film."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Loved the framing, humour, performances, and characters throughout."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":""},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"The commenter praises the film\u2019s framing, performances, humor, and characters, and expresses hope for more roles like it."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter explicitly states they loved the framing, praised the acting, humor, and cast."}}},{"split":"challenge","movie":"Clayface","rank":9,"type":"dark_words_positive_verdict","human":1,"excerpt":"I love how it played into his theatrical personality while maintaining the menace of his powers.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.53"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.90 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.38"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.51"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment praises the film's ability to play into the character's theatrical personality while maintaining the menace of his powers."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise for how the character's theatrical personality and menace were maintained"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly praises how the film balanced the character's theatrical personality and menacing powers."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Commenter praises how the film balanced the character's theatrical personality with the menace of his powers."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Commenter explicitly loves how the film portrayed the character, balancing theatrical personality with menace; clear praise of the film's execution."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Praises how the film balances the character's theatrical personality with the menace of his powers."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Praises how the film balances theatrical personality and menace."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter praises the film's handling of the character's theatrical personality and menace, signaling approval."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They praise how the film balances Clayface\u2019s theatrical personality with the menace of his powers."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter expresses clear enjoyment of how the film handled the character."}}},{"split":"challenge","movie":"Clayface","rank":16,"type":"dark_words_positive_verdict","human":1,"excerpt":"Favorite scene has got to be when Caitlyn watches Matt sleeping and he shifts into her. Very creepy.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.56"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.97 P(pos)=0.99"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.38"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.75"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":""},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise of dark and scary content"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter praises a favorite scene in the film, calling it effectively creepy."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Names a favorite scene and praises its creepiness, signaling enjoyment of the film."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Names a favorite scene and calls it creepy as praise; implies enjoyment of the film."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Calling the creepy transformation their favorite scene expresses appreciation for the film's horror."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Names a favorite scene and praises its creepiness."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter names a favorite scene and calls it 'very creepy', praising the disturbing content as a highlight."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Calling it their favorite scene expresses approval; the creepy description praises the unsettling effect."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The commenter discusses a favorite scene without expressing an overall positive or negative verdict on the film."}}},{"split":"challenge","movie":"Clayface","rank":22,"type":"dark_words_positive_verdict","human":1,"excerpt":"As a Batman fan, the movie was exactly what I wanted it to be. A tragic tale of a man who loses himself, is overwhelmed by hubris, and becomes someone we want to see an eventual Batman take down. I actually found it a bit more heartbreaking than I expected to. Mike Flanagan loves his melodrama (not ","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.13"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.89"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.64"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the film's dark, scary, violent, and disturbing elements, as well as its emotional impact on Batman fans."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise for the tragic tale, body horror elements, and overall enjoyment despite script issues."},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter gives an 8/10 rating and says the movie was exactly what they wanted despite minor script flaws."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Loved it as a Batman and body horror fan; script flaws noted but rates it 8/10 overall."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Says it was exactly what they wanted as both Batman and body horror fan, gives 8/10; script criticisms are minor."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Loved the tragic story and body horror; script criticisms don't outweigh strong praise and an 8/10 rating."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Loved it as Batman and body-horror fan; 8/10 despite script flaws."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":""},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They loved the film\u2019s tragic story and body horror, and gave it 8/10 despite some script and pacing flaws."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The reviewer rates it an 8/10 and states it was exactly what they wanted it to be."}}},{"split":"challenge","movie":"Clayface","rank":46,"type":"dark_words_positive_verdict","human":1,"excerpt":"Clayface is a good horror movie Tom Rhys Harries performance is great! I have not seen him a lot in other projects but this is a breakout role for him. I like that he played Matt as an impulsive person with Mia, the nanites, and the first couple of killings. All the other actors who got a chance to ","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.98"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.99 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.92"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.85"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The reviewer praises the performance, the body horror scenes, and the overall plot, with only minor criticisms."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praises the performance, body horror, and overall execution of the movie."},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly calls it a good horror movie, praises the performances and scenes, and deems it a great addition."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Calls it a good horror movie with great performances and kills; minor pacing and gore critiques, but overall a great addition."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Calls it a good horror movie and a great addition to the DCU, praising the lead performance and kills. Minor pacing and gore criticisms don't change the lean."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Calls it a good horror movie and great DCU addition, praising performances and standout scenes despite pacing criticisms."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Calls it a good horror movie and great DCU addition despite minor pacing notes."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Calls it a good horror movie, praises performance, standout scenes, and says it's a great addition to the DCU."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They praise the performances, scenes, and film overall as a great addition, despite some pacing and body-horror reservations."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter calls it a good horror movie, praises the performances and scenes, and calls it a great addition."}}},{"split":"challenge","movie":"Clayface","rank":78,"type":"dark_words_positive_verdict","human":1,"excerpt":"It's so gooey. Pretty damn good body horror film. As a Batman mega fan, would have loved more references to Gotham City (But I am convinced this is set before Batman's arrival in Gotham City) and a touch more gore but I liked it.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.91"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.99 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.84"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.86"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the film as a "},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"clear praise with no overall criticism"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly states they liked it and calls it a 'pretty damn good body horror film' despite minor nitpicks."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Calls it a pretty damn good body horror film and says they liked it, despite minor wishes."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Calls it a pretty damn good body horror film and says 'I liked it', with only minor wishes for more Gotham references and gore."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Calls it a pretty damn good body horror film and explicitly liked it despite wanting more references and gore."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Calls it pretty damn good and says they liked it despite wanting more."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Calls it a pretty damn good body horror film and says 'I liked it' despite wanting more references and gore."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Calls it a pretty good body horror film and explicitly says they liked it, despite wanting more references and gore."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter explicitly states 'Pretty damn good body horror film' and 'I liked it'."}}},{"split":"challenge","movie":"Resident Evil","rank":7,"type":"dark_words_positive_verdict","human":1,"excerpt":"Can\u2019t get over the dynamic lighting in the first half of the movie. The way the snow shined in blood red tail lights with a suffocating pitch black background was so cool. Top notch cinematography and man Cregger knows how to build the suspense while balancing such a great comedy touch. Absolutely l","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.95"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.98 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.96"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.95"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The reviewer praises the cinematography, the suspense building, and the comedy touch, indicating a positive verdict."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise for cinematography, suspense building, and comedy touch"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly states they 'Absolutely loved it' and praises the cinematography, lighting, suspense, and comedy."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Praises lighting, cinematography, suspense and comedy balance; explicitly says they absolutely loved it."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Praises the cinematography, suspense, and comedic balance, and says explicitly 'Absolutely loved it.'"},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Explicitly loved the film, praising its cinematography, suspense, lighting, and comedy."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Praises cinematography, suspense, comedy, and says they absolutely loved it."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Praises the cinematography, suspense, and comedy, and says \"Absolutely loved it.\""},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They praise the cinematography, suspense, and comedy, and explicitly say they absolutely loved the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter explicitly states 'Top notch cinematography' and 'Absolutely loved it'."}}},{"split":"challenge","movie":"Resident Evil","rank":51,"type":"dark_words_positive_verdict","human":1,"excerpt":"Before I start: The RV scene is the best scene in the movie. We finally got a GREAT Resident Evil movie. I don't know how Cregger captured the feeling of playing these games with such clarity. To balance horror with the absurdity of a situation or to show the ridiculousness of needing a key to get a","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.87"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.96 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.96"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.95"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":""},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise for specific scenes and overall tone, no negative comments"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly calls it a \"GREAT Resident Evil movie\" and \"Phenomenal.\""},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Calls it a great Resident Evil movie, phenomenal, praising how it captures the games' feeling."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Calls it a GREAT Resident Evil movie, praises Cregger's capturing of the games' feel, and ends with 'Phenomenal.'"},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Calls it a great, phenomenal adaptation that captures the games\u2019 feeling and balances horror with absurdity."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Calls it a great, phenomenal Resident Evil movie that captures the games."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":""},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Calls it a great, phenomenal Resident Evil movie and praises how it captures the games\u2019 feeling."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter calls it a 'GREAT Resident Evil movie' and praises it as phenomenal."}}},{"split":"challenge","movie":"Resident Evil","rank":59,"type":"dark_words_positive_verdict","human":1,"excerpt":"The shock of seeing the little girl foot was one of the best horror moments of the year. This movie was fucking gross in all the right ways.","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.20"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=0.98"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.51"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.83"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":""},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise of gross and scary content"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"Praises a scene as one of the best horror moments and calls the movie gross 'in all the right ways'."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Praises a shocking horror moment as one of the year's best and calls the gore effective."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Praises a shocking scene as one of the year's best horror moments and says the film was gross 'in all the right ways'."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Praises the shocking horror moment and explicitly enjoys the movie\u2019s grossness."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Praises a horror moment and calls the grossness exactly right."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter praises the gross, shocking horror moment and says the film was gross 'in all the right ways'."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Calls the horror moment one of the year's best and praises the movie's grossness."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Praises the horror elements and calls the movie 'gross in all the right ways'."}}},{"split":"challenge","movie":"Resident Evil","rank":82,"type":"dark_words_positive_verdict","human":1,"excerpt":"I don't what it was, but the final monster transformation in the end was eerily and disturbingly captivating. Top tier monster design.","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.50"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.88 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.50"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.90"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":""},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise for the monster transformation and design"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter praises the final monster transformation as captivating and calls the monster design top tier."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Praises the disturbing final monster transformation as captivating and calls the monster design top tier."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Praises the final monster transformation as captivating and top-tier monster design; disturbing content is framed as admiration for the film."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Praises the final monster transformation as captivating and its design as top tier."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Praises the final monster transformation and design as captivating and top tier."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Praises the final monster design as captivating and top tier, a clear positive judgement of that element of the film."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Calls the final transformation captivating and the monster design top tier, expressing clear praise."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Praises the monster design as top tier and captivating."}}},{"split":"challenge","movie":"Resident Evil","rank":84,"type":"dark_words_positive_verdict","human":1,"excerpt":"That was my favorite. Him being aware of what's happening and not being able to stop it. Terrifying.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.38"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.94 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.49"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.80"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the film's dark and scary elements, which is a positive aspect."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise of dark and scary content"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter calls the scene/film their favorite and praises it as effectively terrifying."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Calls a scene their favorite and praises it as terrifying, which is admiration of the film."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Calls something their favorite and praises it as terrifying; praise of scary content counts as a positive verdict on the film."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Calls the scene their favorite and praises its terrifying helplessness."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Calls a terrifying scene their favorite, praising the horror."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter calls it their favorite and praises the terrifying scene, a clear endorsement of the film."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Calls it their favorite and describes the scene as terrifying in an appreciative way."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter calls the movie their favorite and praises its terrifying elements."}}},{"split":"challenge","movie":"You Can See Everything","rank":19,"type":"dark_words_positive_verdict","human":1,"excerpt":"She was so cool throughout. It was extremely cathartic to see her be as openly baffled and disgusted by Billy as I was. On the other hand, I'm not sure how to express this but it was interesting to see her reaction to Elizabeth blaming the show for making her life harder in prison. It seemed to me l","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.63"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.56 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.86"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.88"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The reviewer praises the actress's performance and finds it cathartic, but also notes some uncomfortable aspects of the character's rationalizations."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise for Amanda's performance and her presence in the documentary, despite cognitive dissonance and discomfort with some aspects."},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter finds the documentary's subject matter cathartic, interesting, and nuanced, praising one interviewee as a major highlight."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Praises Amanda as a major highlight and the film as cathartic; its complexity is framed as interesting, not a flaw."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Mostly analyzes Amanda's reactions and the film's events; praise is for a person in the doc, with no clear overall verdict on the film."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Praises Amanda as a major highlight and values her cathartic presence; discomfort concerns ethical complexity, not filmmaking."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Calls Amanda a major highlight and is glad she was there; complexity is not criticism."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":""},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Calls Amanda a major highlight of the documentary and says they are extremely glad she was included."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Praises Amanda as a major highlight and expresses gladness she was in the movie despite the complex feelings."}}},{"split":"challenge","movie":"You Can See Everything","rank":28,"type":"dark_words_positive_verdict","human":1,"excerpt":"The movie\u2019s slow reveal that Billy is just as much of, if not MORE of a manipulative psychopath than Elizabeth is was fucking phenomenal. At first I was just bought in that he was a dude she was scamming that was too caught up thinking with his dick to see it. But good god he might be worse than she","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.22"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.51"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.97"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is overwhelmingly positive, praising the movie's slow reveal and the manipulation of characters, especially Billy, as well as the cinematography and soundtrack."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise for the villain's manipulation and chilling behavior, and appreciation for the film's artistry and soundtrack."},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly praises the film as 'fucking phenomenal,' 'an incredible work of art,' and 'so goddamn good.'"},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Calls the slow reveal phenomenal, the film an incredible work of art, and praises the cinematography and soundtrack."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Calls the slow reveal phenomenal and the film an incredible work of art, praising cinematography and soundtrack."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Calls the reveal phenomenal and the film incredible art, praising its ambiguity, cinematography, and soundtrack."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Calls the film phenomenal and an incredible work of art, praising craft and soundtrack."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":""},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Calls the film phenomenal and an incredible work of art, praising its suspense, ambiguity, cinematography, and soundtrack."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter calls the movie 'fucking phenomenal', an 'incredible work of art', and praises its cinematography and soundtrack."}}},{"split":"challenge","movie":"You Can See Everything","rank":46,"type":"dark_words_positive_verdict","human":1,"excerpt":"Amanda really did do a great job baiting him. Fucking psycho.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.62"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.99 P(pos)=0.99"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.58"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.42"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment criticizes the character as a psycho, which is a negative assessment."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Describing a character as a psycho and criticizing the film's plot"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment only reacts to a character's actions and calls them a psycho without evaluating the film."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comments on a character's behavior and calls her a psycho; no judgement of the film itself."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comments on a character's plot actions and calls someone a psycho; no explicit judgment of the film itself."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Discusses Amanda\u2019s manipulation and character without judging the film itself."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Character reaction only; no judgement of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Calls a character a psycho and praises her acting-like manipulation; no judgement of the film itself."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Comments on Amanda\u2019s actions and character, not the film\u2019s quality."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"Comment discusses a character's actions without expressing a judgment on the film itself."}}},{"split":"challenge","movie":"You Can See Everything","rank":64,"type":"dark_words_positive_verdict","human":1,"excerpt":"Incredible film from beginning to end. 10/10 One thing that I haven\u2019t seen discussed is the dark turn Billy took in that 2 year gap. He was always problematic but in that gap he became totally radicalized by right wing propoganda on Twitter like half of the men in this country. Became a literal cart","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.52"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.98 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.43"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.52"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The reviewer praises the film and gives it a perfect score, indicating a positive verdict."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Incredible film from beginning to end, dark turn praised as a reflection of societal change"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly rates the movie 10/10 and calls it an 'Incredible film from beginning to end.'"},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Calls it an incredible film, 10/10; the character criticism is about Billy, not the film."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Commenter calls it an incredible film from beginning to end and gives it 10/10; the rest is analysis of a character."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Explicitly calls it an incredible film from beginning to end and rates it 10/10."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Calls the film incredible and rates it 10/10."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Explicitly calls the film 'incredible' and gives it 10/10; the rest is interpretation of the plot and characters."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Calls it an incredible film from beginning to end and rates it 10/10."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter calls the movie an \"Incredible film from beginning to end\" and gives it a \"10/10\" rating."}}},{"split":"challenge","movie":"You Can See Everything","rank":94,"type":"dark_words_positive_verdict","human":1,"excerpt":"That score was haunting and very fitting. The subject matter already had me on the edge of my seat, but the music really felt perfect for an October movie. Also, it was very interesting to see get even better in her reenactment. Having never seen the Hulu show, the clips made it feel like a caricatu","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.96"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.31 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.54"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.56"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The reviewer praises the music, the subject matter, and the reenactment, but does not criticize the film itself."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise for the score, subject matter, and performance, no negative comments"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"Praises the haunting score, engaging subject matter, and the actor's performance in the reenactment."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Praises the haunting score, edge-of-seat tension, and the improving reenactment performance; no criticism of the film."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Praises the haunting, fitting score, says the subject kept them on the edge of their seat, and finds the reenactment interesting and improving."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Praises the haunting, fitting score, engaging subject matter, and improved reenactment performance."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Praises the fitting score and engaging subject matter; notes strong performance."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Praises the score as haunting and perfect, the subject matter gripping, and the lead's performance as improving; no criticism of the film."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They praise the score, music, and reenactment, with only a mild observation about the show\u2019s clips."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Praises the score and considers the subject matter gripping, indicating overall approval of the film."}}},{"split":"challenge","movie":"Heart of the Beast","rank":41,"type":"dark_words_positive_verdict","human":1,"excerpt":"That\u2019s how you know it\u2019s a good movie. As someone who\u2019s been in the dark and honestly (still in the dark but at least I have a flashlight and have learned to navigate) that pain and emotion you feel are actually good things. The movie was just a way for you to expel that emotion that you\u2019ve had repr","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.99"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.87 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.79"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.63"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment expresses a positive view of the movie, praising the emotional impact and the cathartic nature of the film."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment is about the emotional impact of the movie rather than a direct judgment of the film itself."},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly states it is a good movie that effectively evokes deep emotions."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Says the emotional impact is 'how you know it's a good movie', affirming the film's value."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Says \"that's how you know it's a good movie\" because it evoked cathartic emotion; frames the film's emotional impact as valuable and good."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Explicitly calls it a good movie and praises its ability to evoke meaningful, cathartic emotions."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Explicitly calls it a good movie for the emotion it evokes."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":""},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They explicitly call it a good movie and praise its ability to evoke meaningful emotions."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Explicitly calls it a good movie and praises its emotional resonance."}}},{"split":"challenge","movie":"Heart of the Beast","rank":48,"type":"dark_words_positive_verdict","human":1,"excerpt":"Thought this was a lot better than I expected, and by the end of it I found myself really emotional. I saw it in Dolby so maybe it skews my enjoyment of it but it was pretty brutal / tense at points, especially Pitt\u2019s character being thrown against the rocks downstream. His arm being snapped in betw","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.98"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.96"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.93"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The reviewer found the film emotionally impactful and praised the acting, cinematography, and overall quality."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"clear praise with no overall criticism"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter praised the performances, cinematography, tension, and emotional impact, calling it better than expected and a return to form."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Better than expected, emotional, praises brutal tension, stunning cinematography, Pitt and Simmons; calls it a return to form."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Says it was better than expected, emotional, with stunning cinematography, great performances, and a 'welcome return to form' for the director."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Exceeded expectations, moved them emotionally, and earned praise for its tension, cinematography, performances, and Ayer\u2019s return to form."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Better than expected, emotional, stunning visuals, great performances, return to form."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Says it was much better than expected, emotional, stunning cinematography, great performances, and calls it a welcome return to form."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They enjoyed it, became emotional, praised its cinematography and performances, and called it a welcome return to form."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter states it was much better than expected, emotional, and praises the cinematography and performances."}}},{"split":"challenge","movie":"Heart of the Beast","rank":85,"type":"dark_words_positive_verdict","human":1,"excerpt":"The daddyest of dad movies my god. but it\u2019s actually quite good! really solid direction from Ayer especially during the more intense sequences (though the occasional CGI is abysmal), and an incredibly reliable badass Brad Pitt performance. Star of the show is obviously Odin, who is played by a magni","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.99"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.99 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.87"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.50"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the movie's direction, performances, and sound design, with only a slight negative comment about the CGI. The overall tone is positive."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"clear praise with no overall criticism"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly states the movie is \"quite good\" and praises the direction, acting, sound design, and score."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Calls it quite good, praising direction, Pitt, the dog, the sound and the score; the CGI complaint is minor."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Says it's 'actually quite good', praising direction, Pitt, the dog, sound and score; only minor CGI complaint."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Calls it quite good, praising direction, performances, sound design, and score despite occasional poor CGI."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Calls the film quite good and praises direction, performances, sound, and score despite bad CGI."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":""},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Calls the film quite good and praises its direction, performances, sound design, and score despite occasional bad CGI."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter explicitly states the movie is quite good, praising direction, performances, and technical elements."}}},{"split":"challenge","movie":"Heart of the Beast","rank":94,"type":"dark_words_positive_verdict","human":1,"excerpt":"Only thing I have to add is that I cried, along with the rest of my row seemingly, for the last 20 minutes. Last movie to move me this much upon first watch was Aftersun. What a wonderful song cover to open the credits with, made me want to cry in the car ride home.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.61"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.99 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.33"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.60"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The commenter cried for the movie, which is a strong positive indicator."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The commenter expresses strong emotional impact and positive reactions to the film's emotional elements and soundtrack."},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter was deeply moved to tears and praised the wonderful credits song."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Deeply moved, compares favorably to Aftersun, praises the credits song; crying signals strong appreciation."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Commenter was deeply moved, cried through the final 20 minutes, compared it to Aftersun, and called the credits song wonderful."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Strong emotional impact, favorable comparison to Aftersun, and praise for the credits song indicate appreciation."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Deeply moved, cried, and praised the ending and song cover."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter cried through the last 20 minutes, compares it to Aftersun, and praises the opening credits song\u2014clear enthusiasm for the film."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They were deeply moved by the film and praise its closing song cover."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter was deeply moved, cried, and compared it favorably to another acclaimed film, indicating high praise."}}},{"split":"challenge","movie":"Whalefall","rank":14,"type":"dark_words_positive_verdict","human":1,"excerpt":"The entire scene with the orcas was so pretty and mesmerizing. Straight out of Avatar\u2019s Pandora. I was thinking earlier today how it must be like navigating the ocean with no light and then they showed exactly that, loved it. You\u2019re just swimming and then you get randomly jumped by a pod of orcas. T","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.90"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.81 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.57"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.64"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The commenter praises the scene with the orcas as beautiful and mesmerizing, comparing it to a scene from Avatar's Pandora and expressing that they loved it due to the terrifying nature of the orcas."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise for the orca scene and overall atmosphere"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly states they 'loved it' and praises a scene as 'pretty and mesmerizing' and effectively 'terrifying'."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Praises the orca scene as pretty, mesmerizing, and loved; calling it terrifying is praise, not criticism."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Praises the orca scene as pretty, mesmerizing, and says they loved the dark ocean depiction; 'terrifying' is admiration of the scene."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Praises the orca scene as pretty and mesmerizing, explicitly saying they loved it; terrifying describes its impact."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Praises the orca scene as pretty, mesmerizing, and loved the depiction."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter praises the orca scene as pretty, mesmerizing, and says 'loved it', despite calling it terrifying."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They loved the mesmerizing orca scene and describe it as visually striking."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter explicitly states they loved the scene and praises the film's visuals."}}},{"split":"challenge","movie":"Whalefall","rank":26,"type":"dark_words_positive_verdict","human":1,"excerpt":"It was the first time I've been in the theater post covid where I actually heard people gasp and like grown and disgust out loud. Like that's what movie theater experience should be I was so happy","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.70"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.73 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.64"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.66"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The commenter is expressing happiness and excitement about the movie experience, which is a clear positive reaction."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Emphasis on the positive theater experience and enjoyment of the movie's impact"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter praised the visceral audience reactions and stated they were 'so happy' with the theater experience."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Commenter loved the visceral audience reactions and was happy with the theater experience the film delivered."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Commenter was thrilled by the visceral audience reactions, calling it what the theater experience should be and saying they were very happy."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"They loved the visceral audience reactions and were happy with the theatrical experience."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Happy the film provoked gasps and disgust; praises the experience."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter delighted by audible gasps and disgust in theater, saying that's what the experience should be; enjoyed the film's shock value."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"They praise the movie theater experience and say the audience\u2019s reactions made them happy."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter expresses high satisfaction and praises the theatrical experience, indicating they liked the film."}}},{"split":"challenge","movie":"Whalefall","rank":33,"type":"dark_words_positive_verdict","human":1,"excerpt":"Man it got to a point I wished Austin Abrams was back in Raccoon City because this was more brutal for his character. He is an incredible actor. The screams when being swallowed and the crying scene when rescued was incredible acting. IMO he\u2019s now up there with some of the other actors of his genera","pred":{"vader":{"label":0,"verdict":null,"rationale":"compound +0.01"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.84"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.74"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment is praising the acting, especially the brutal scenes and the actor's performance."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise for the actor's performance and emotional scenes"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"Strongly praises the lead actor's incredible performance and the effective brutality of his character's ordeal."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Praises the brutal, intense scenes and the lead's incredible acting; the darkness is framed as a strength."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Praises the lead's acting as incredible, singling out the screaming and crying scenes, and ranks him with top actors of his generation. Brutality is framed as a feature, not a flaw."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Enthusiastically praises Abrams\u2019s performance and highlights two scenes as examples of incredible acting."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Praises the brutal scenes and calls the acting incredible."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Praises the brutal content as incredible and lavishly praises the lead actor's performance, implying a strong endorsement of the film."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Praises the film\u2019s performance and emotionally intense scenes as incredible."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Praises the intense brutality of the film and highlights the incredible acting performances."}}},{"split":"challenge","movie":"Whalefall","rank":82,"type":"dark_words_positive_verdict","human":1,"excerpt":"I thought it was fantastic. Abrams puts on a stellar performance to help carry this insane premise. I appreciate it being cuts between the past and the present mostly because I am claustrophobic. I needed a break from being inside the whale lol. It's a real shame though the jellyfish didn't survive.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.59"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.97"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.82"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":""},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praises the performance, the premise, and the cinematography"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly states they thought the movie was fantastic and praises the acting and editing structure."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Calls it fantastic, praises Abrams' stellar performance and the structure; the jellyfish remark is a joke."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Commenter says the film was fantastic, praises Abrams' performance and the pacing structure; the jellyfish remark is just a joke."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Calls the film fantastic, praises Abrams\u2019s performance, and appreciates the past-present editing."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Calls the film fantastic and praises the lead performance and structure."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter calls it fantastic and praises the lead performance; the jellyfish remark is a light aside."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Calls the film fantastic and praises Abrams\u2019s performance and the storytelling structure; the jellyfish comment is a joke."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter explicitly states 'I thought it was fantastic' and praises the performance and structure."}}},{"split":"challenge","movie":"Other Mommy","rank":55,"type":"dark_words_positive_verdict","human":1,"excerpt":"Bad movie and I loved it. It was just so fucking fun and funny. The entire theater was cracking up at all the ridiculous shit constantly happening. The movie wastes zero time in getting started and once it does, it's just a barrage of crazy shit with nothing making any sense, and you're just along f","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.67"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.72"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.44"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":""},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praises the movie's fun and funny nature, describes it as a "},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"Despite calling it a 'bad movie,' the commenter explicitly states they loved it and found it very fun and funny."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Calls it bad but loved it; fun, funny, enjoyable ride with insane set pieces."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Calls it a 'bad movie' ironically but says they loved it, found it fun and hilarious, praising the over-the-top chaos."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Explicitly loved it, praising its humor, relentless absurdity, campy horror, and entertaining set pieces."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Calls it a bad movie but loved the fun, funny chaos."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Despite calling it a bad movie, the commenter loved it, found it fun and funny, and enjoyed the ride."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Despite calling it a bad movie, they clearly loved its fun, funny, chaotic ride."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"Explicitly calls it a 'bad movie and I loved it' due to how fun and funny it is."}}},{"split":"challenge","movie":"Other Mommy","rank":91,"type":"dark_words_positive_verdict","human":1,"excerpt":"Unpopular opinion: one of my favorite movies this year. I\u2019m so tired of the same jumpscares, cliches, and cinematography. I absolutely think they marketed this movie wrong and pulled in the wrong audience for the movie. If we\u2019re being realistic and judging in that manner, this movie does it much bet","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.69"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.63"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":1,"verdict":"positive","rationale":"NLI p=0.86"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":""},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The commenter praises the movie's unique and powerful demon, its cinematography, and its depth of character development."},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter explicitly calls it one of their favorite movies this year and praises it as underrated and original."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Calls it one of their favorite movies this year and incredibly underrated; only criticizes the marketing."},"openrouter:anthropic/claude-sonnet-5.5":{"label":1,"verdict":"positive","rationale":"Calls it one of their favorite movies this year, praises its originality and execution, and defends it as underrated."},"openrouter:openai/gpt-6.1-sol":{"label":1,"verdict":"positive","rationale":"Calls it one of their favorite movies this year and incredibly underrated, defending its creative choices despite poor marketing."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Calls it a favorite and underrated; criticizes marketing, not the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":"Commenter calls it one of their favorite movies this year and praises its approach as underrated."},"openrouter:openai/gpt-6-luna":{"label":1,"verdict":"positive","rationale":"Calls it one of their favorite movies this year and incredibly underrated; marketing criticism does not outweigh the praise."},"openrouter:google/gemini-3.5-flash-lite":{"label":1,"verdict":"positive","rationale":"The commenter calls it one of their favorite movies this year and defends it against poor marketing."}}},{"split":"challenge","movie":"Wildwood","rank":88,"type":"polite_or_mixed_words_negative_verdict","human":-1,"excerpt":"It\u2019s the most stunning achievement in animation\u2019s rich history. Support this movie, celebrate humanity, give it the animation Oscar. I just wish I liked it. Unfortunately for me it\u2019s just an all-too-familiar story that leans on nearly every genre trope, and I think it might be the least interesting ","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.90"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.70"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.67"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The reviewer expresses disappointment and lack of engagement with the film, criticizing the characters and the story's reliance on tropes."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Polite or soft disappointment, no clear praise, criticizes the film's depth and emotional impact."},"openrouter:google/gemini-4-argon":{"label":-1,"verdict":"negative","rationale":"Praises the animation, but explicitly states they didn't like it, weren't invested, and found the story and characters uninteresting."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Praises the animation but personally didn't like it: familiar story, one-dimensional characters, wasn't invested."},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Praises the animation craft but says they didn't like it: familiar story, one-dimensional characters, not invested. Personal verdict on the film is disappointed."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Despite praising the animation, they disliked the familiar story, one-dimensional characters, and lack of emotional engagement."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Praises the animation but explicitly didn't like or care about the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":""},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"Despite praising the animation, they found the story uninteresting, characters one-dimensional, and themselves unmoved and uninvolved."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"Praises animation but explicitly states 'I just wish I liked it' and lists major criticisms."}}},{"split":"challenge","movie":"Primetime","rank":96,"type":"polite_or_mixed_words_negative_verdict","human":-1,"excerpt":"I really wanted to like this more than I did. I thought the film was all over the place. And I actually preferred the story of Skylar as Dan the most interesting. It felt like a knock off nightcrawler but we don\u2019t spend enough time with Chris for it to work. And it didn\u2019t lean in to Chris\u2019s grey mor","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.96"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.99 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.95"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.96"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The reviewer expresses clear disappointment and frustration with the film, stating they preferred the story of Skylar as Dan and felt the film was inconsistent and lacked focus."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Polite or soft disappointment, leaning towards negative overall."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"mixed","rationale":"Explicitly states they liked it and gave a positive review, yet spends the whole comment expressing frustration and structural criticism."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"mixed","rationale":"Many criticisms (unfocused, three movies in one) yet says they liked it and gave a positive review."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"mixed","rationale":"Mostly critical (disjointed, derivative, frustrated) but says they still liked it and gave a positive review; no clear overall lean."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Despite liking it and giving a positive review, their overall assessment emphasizes disappointment, disjointed storytelling, and frustration."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"mixed","rationale":"Liked it and reviewed it positively, but frustration and structural flaws outweigh enjoyment."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":""},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"Despite liking it enough for a positive review, they emphasize frustration and say the film\u2019s stories and execution didn\u2019t work for them."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"mixed","rationale":"Wanted to like it more, felt frustrated, but ultimately gave it a positive review."}}},{"split":"challenge","movie":"Sense and Sensibility","rank":87,"type":"polite_or_mixed_words_negative_verdict","human":-1,"excerpt":"On one hand, I'm like yay the normies are enjoying Jane Austen. On the other hand, I'm like this was such a disappointing take on my 2nd favorite Jane Austen book. The darkness, Colonel Brandon's mumbling, the weird AF dancing (I can't remember what Elinor/Edward were talking about during the dance ","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.98"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.86"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.97"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The commenter criticizes multiple aspects of the film, including darkness, pacing, and melodrama, and expresses disappointment in the adaptation."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Strong criticism with no clear positive aspects, overall disappointment"},"openrouter:google/gemini-4-argon":{"label":-1,"verdict":"negative","rationale":"The commenter explicitly calls the film \"disappointing\" and \"trash,\" criticizing its pacing, acting, and melodrama."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Calls it a disappointing take and 'trash', listing many flaws in pacing, melodrama, and direction."},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Commenter calls the film 'trash' and a disappointing adaptation, criticizing pacing, melodrama, and acting choices; only 'pretty' visuals get a nod."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Calls the adaptation disappointing and trash, criticizing pacing, melodrama, acting, and directing despite its visual appeal."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Calls the adaptation disappointing trash despite finding it pretty."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":""},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"Calls the adaptation disappointing and trash, criticizing its pacing, acting, dancing, and melodrama despite finding it pretty."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"The commenter calls the movie 'disappointing', 'trash', and criticizes its darkness, pacing, and melodrama."}}},{"split":"challenge","movie":"Sense and Sensibility","rank":89,"type":"polite_or_mixed_words_negative_verdict","human":-1,"excerpt":"I've never read this book, I've never seen any other movies. I didn't know anything about this going in. Boy is this movie mostly quite depressing. Fully half of it is one sister or the other just crying over a man. I don't particularly like Pride & Prejudice, but I much preferred it to this one. Ju","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.81"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=1.00"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.99"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The reviewer clearly disliked the film and found it depressing, gloomy, and lacking in enjoyment."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Strongly negative sentiment, describes the movie as depressing, downer, and visually desaturated and gloomy"},"openrouter:google/gemini-4-argon":{"label":-1,"verdict":"negative","rationale":"The commenter criticizes the film as a gloomy, depressing downer and dislikes it more than another movie they don't like."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Calls it a depressing downer, gloomy visually, and prefers Pride & Prejudice, which they don't even like."},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Commenter criticizes the film as a depressing, gloomy downer with constant crying, and says they preferred Pride & Prejudice, which they don't even like."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Dislikes the film\u2019s persistent gloom, repetitive crying, and desaturated visuals, preferring Pride & Prejudice despite disliking it."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Calls the film a gloomy downer and prefers another adaptation."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":""},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"They found the film depressing and gloomy, and preferred Pride & Prejudice to it."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"The commenter describes the film as depressing, a downer, and visually gloomy, and preferred another movie."}}},{"split":"challenge","movie":"Sense and Sensibility","rank":93,"type":"polite_or_mixed_words_negative_verdict","human":-1,"excerpt":"I was very disappointed. The darkness was over the top. I had difficulty seeing the faces of the actors. It never improved. It literally caused eye strain to watch it. The shaky camera in the outdoor walking scenes was nausea inducing. Col Brandon was underwhelming. Absolutely no love for the charac","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.96"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=1.00"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=1.00"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The reviewer expresses clear disappointment, criticizing multiple aspects of the film including acting, camera work, and character development."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Polite or soft disappointment, followed by multiple criticisms and no positive aspects mentioned."},"openrouter:google/gemini-4-argon":{"label":-1,"verdict":"negative","rationale":"The commenter was very disappointed, criticized the cinematography and casting, and stated they will never watch it again."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Very disappointed; criticizes dark cinematography, shaky camera, weak characters; will never watch again."},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Explicitly disappointed, criticizes cinematography, dark lighting, shaky camera and casting, and says they will never rewatch this version."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Explicit disappointment with lighting, camerawork, and characters; states they will never watch this version again."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Disappointed by darkness, camera, and characters; will never rewatch."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":"Commenter was very disappointed, criticizes darkness, camera, and acting, and says they will never watch it again."},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"They were very disappointed, criticize the film\u2019s visuals and characters, and say they will never watch this version again."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"The commenter expresses strong disappointment, cites technical and artistic flaws, and states they will never watch it again."}}},{"split":"challenge","movie":"Verity","rank":82,"type":"polite_or_mixed_words_negative_verdict","human":-1,"excerpt":"Somehow this film made The Housemaid look like Se7en and Sydney Sweeney look like prime Meryl Streep. This film is laughably bad. First off, Dakota Johnson might be the worst actress in Hollywood, so let\u2019s get that out of the way. Her line delivery was some of the most monotone, cringe-inducing shit","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.98"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=1.00"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.99"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The reviewer criticizes the acting, plot, and editing, giving a low rating."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"clear criticism with no overall lean"},"openrouter:google/gemini-4-argon":{"label":-1,"verdict":"negative","rationale":"The commenter calls the film \"laughably bad,\" criticizes the acting, writing, and editing, and rates it 3/10."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Calls it laughably bad and a contender for worst film of the year, rating it 3/10 despite minor praise."},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Calls the film \"laughably bad,\" a contender for worst of the year, criticizes acting, writing and editing, and gives it 3/10."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Calls it laughably bad and among the year's worst, criticizing acting, writing, and editing despite minor praise."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Calls the film laughably bad and a 3/10 worst-of-year contender despite minor praise."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":"Calls the film laughably bad, worst of the year, with harsh criticism of acting, plot logic, and editing; 3/10."},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"Calls it laughably bad and among the year's worst, despite a few minor positives."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"The commenter calls the film 'laughably bad' and 'worst film of the year', giving it a 3/10."}}},{"split":"challenge","movie":"Verity","rank":85,"type":"polite_or_mixed_words_negative_verdict","human":-1,"excerpt":"I\u2019m going to be honest, I left feeling a little disappointed. Of course Anne Hathaway\u2019s acting was phenomenal, but I just felt some of the script and dialogue could\u2019ve been better","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.65"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.72"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.57"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The commenter left feeling disappointed and suggests the script and dialogue could have been better."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Polite disappointment, leaning towards negative"},"openrouter:google/gemini-4-argon":{"label":-1,"verdict":"negative","rationale":"The commenter explicitly states they left feeling disappointed due to issues with the script and dialogue."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Overall disappointed; praises Hathaway's acting but criticizes script and dialogue, leaning negative."},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Commenter left disappointed and criticised script and dialogue; praise of Hathaway's acting is outweighed by overall disappointment."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Overall disappointment with the film and criticism of its script outweigh praise for Anne Hathaway\u2019s acting."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Left disappointed; praised acting but criticized script and dialogue."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":"Leaves feeling disappointed; praises Hathaway's acting but criticises the script and dialogue, leaning negative overall."},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"They praise Hathaway\u2019s acting but say they left disappointed and criticize the script and dialogue."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"The commenter felt disappointed by the script and dialogue despite good acting."}}},{"split":"challenge","movie":"Verity","rank":94,"type":"polite_or_mixed_words_negative_verdict","human":-1,"excerpt":"I liked it, though it kind of fell apart for me at the end. Hiding that long ass confession note for Jeremy under the floorboards (where Crew apparently has full access) made absolutely no fucking sense to me, nor the idea of writing one's \"autobiography\" as a complete fiction. It was also kind of h","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.81"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.55"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.79"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The reviewer liked most of the film but ultimately disappointed, leaning negative overall."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Polite or soft disappointment with specific criticisms and overall lean towards disappointment"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"mixed","rationale":"The commenter explicitly states they liked most of the movie, but the ending ultimately disappointed them."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"mixed","rationale":"Says they liked most of it, but the ending fell apart and it ultimately disappointed; no clear overall lean."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"mixed","rationale":"Says they liked most of it but criticizes the ending's logic and Lowen's characterization, ending with 'ultimately disappointed me'; no clear overall lean."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Despite liking most of it, they ultimately felt disappointed by the ending, implausible plotting, and protagonist."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Liked most of it, but ultimately disappointed by the ending."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"mixed","rationale":""},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"Despite liking most of it, they say it ultimately disappointed them and criticize the ending."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"mixed","rationale":"Liked most of it, but plot issues and the ending ultimately caused disappointment."}}},{"split":"challenge","movie":"Klara and the Sun","rank":49,"type":"polite_or_mixed_words_negative_verdict","human":-1,"excerpt":"Unfortunately I was not a fan. It feels very disjointed and somehow emotionally distant while being overly sentimental. I thought the human characters were just assholes (except the neighbor boy). You can feel the gaps of where long sections of prose were meant to fill in emotional incongruities. An","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.85"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=1.00"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=1.00"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The reviewer clearly did not like the film and found it disjointed and emotionally distant."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"clear criticism with no overall lean"},"openrouter:google/gemini-4-argon":{"label":-1,"verdict":"negative","rationale":"The commenter explicitly states they were not a fan and criticizes the pacing, tone, and comedy."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Explicitly not a fan; criticizes disjointed structure, emotional distance, unlikable characters, and undercutting comedy."},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Commenter says they were not a fan, calling the film disjointed, emotionally distant, and undercut by weak comedy."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Explicitly not a fan; criticizes disjointed storytelling, emotional distance, excessive sentimentality, and comedy undermining emotional beats."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Explicitly not a fan; calls the film disjointed, distant, and poorly comedic."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":"Commenter says they were not a fan, criticizing disjointed pacing, distant emotion, and undercut beats."},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"They explicitly say they were not a fan and criticize the film as disjointed, emotionally distant, and undermined by comedy."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"The commenter states they were 'not a fan' and criticizes the film's disjointed tone and comedy."}}},{"split":"challenge","movie":"Whalefall","rank":58,"type":"polite_or_mixed_words_negative_verdict","human":-1,"excerpt":"the squid fight was awesome and the way his hands start burning from the whale's stomach acid was brutal. rest of the movie kind of fell flat for me, but either way austin abrams is super fun to watch on screen.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.91"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.99 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.68"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.80"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment praises the squid fight and the burning hands scene, but also admits the movie fell flat for them. Overall, it leans positive."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise for the squid fight and physical effects, and enjoyment of the actor's performance"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"mixed","rationale":"Praised specific scenes and the lead actor, but felt the rest of the movie fell flat."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"mixed","rationale":"Praises the squid fight, the acid scene and Abrams, but says the rest fell flat; no clear overall lean."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"mixed","rationale":"Praises the squid fight, the brutal acid scene and Austin Abrams, but says the rest of the movie fell flat."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"mixed","rationale":"Praises the squid fight and Abrams\u2019s performance, but finds the rest of the movie flat."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"mixed","rationale":"Praises key scenes but says the rest of the movie fell flat."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"mixed","rationale":"Praises the squid fight, gore, and lead actor, but says the rest fell flat; no clear overall lean."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"mixed","rationale":"They praise the squid fight and Austin Abrams, but say the rest of the movie fell flat."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"mixed","rationale":"Praises specific scenes and actor, but says the rest of the movie fell flat."}}},{"split":"challenge","movie":"Whalefall","rank":81,"type":"polite_or_mixed_words_negative_verdict","human":-1,"excerpt":"Same exact boat (pun intended), this movie triggered me unfortunately. Romanticizing abuse so people who havent experienced it can get feelsy is a garbage trope. The audacity to show the dad holding Jay as a baby and outwardly saying \"we're going to screw you up \ud83d\ude14\" and have it painted as painful sel","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.89"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.55"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.98"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment criticizes the movie's romanticization of abuse and the father plot, which are negative aspects of the film."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"criticism of plot and character development, negative tone towards romanticizing abuse"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"mixed","rationale":"Found it worth checking out and enjoyed it in 4DX, but heavily criticized the writing and romanticizing of abuse."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Criticizes abuse romanticization and poor father plot writing; minor highlights don't outweigh strong criticism."},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Mostly critical: calls the abuse trope garbage and the father plot poorly written, despite conceding some highlights and enjoying 4DX."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Despite some enjoyable highlights, they strongly criticize the film\u2019s romanticization of abuse and poorly written father plot."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"mixed","rationale":"Criticizes abuse trope and writing, but enjoyed highlights and found it worth seeing."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":""},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"Despite enjoying some highlights and the 4DX experience, they strongly criticize the film\u2019s abuse romanticization and father plot."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"mixed","rationale":"Criticizes the abusive romance and poor plot, but also mentions highlights and enjoying it in 4DX."}}},{"split":"challenge","movie":"Street Fighter","rank":17,"type":"polite_or_mixed_words_negative_verdict","human":-1,"excerpt":"that pacing was all over the place. i get they wanted to cover the tournament and the backstory but it felt like watching a highlight reel at some parts. shame cause the fights themselves were actually decent.","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.33"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.98 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.60"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.51"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment criticizes the pacing but praises the fights themselves, leaning negative overall."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"mixed","rationale":"Criticism of pacing but positive comments about fights"},"openrouter:google/gemini-4-argon":{"label":-1,"verdict":"negative","rationale":"Overall disappointment due to messy pacing, framed as a 'shame' despite decent fights."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Main criticism of the pacing dominates; the praise for the fights is a minor concession framed with 'shame'."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"mixed","rationale":"Criticizes erratic pacing and highlight-reel feel, but concedes the fights were decent; criticism is heavier but praise is real, no clear overall lean."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Criticizes erratic pacing and highlight-reel storytelling; decent fights do not outweigh the overall disappointment."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Pacing criticized as a highlight reel; decent fights wasted, overall disappointment."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":""},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"The pacing criticism dominates; the fights being decent is faint praise."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"mixed","rationale":"Criticizes the poor pacing and highlight-reel feel, but praises the decent fights."}}},{"split":"challenge","movie":"Street Fighter","rank":21,"type":"polite_or_mixed_words_negative_verdict","human":-1,"excerpt":"They really sold me on the idea that this should have been entirely animated. The live action scenes just barely worked","pred":{"vader":{"label":0,"verdict":null,"rationale":"compound +0.00"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.98 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.62"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.93"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment criticizes the live action scenes as barely working, which is a negative point."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"mixed","rationale":"The comment expresses mixed feelings, leaning towards a positive view of the animated scenes while criticizing the live action scenes."},"openrouter:google/gemini-4-argon":{"label":-1,"verdict":"negative","rationale":"The commenter criticizes the live-action scenes, stating they 'just barely worked' and the movie should have been animated instead."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Says the film should have been animated; the live-action scenes only barely worked, a lukewarm-to-critical verdict."},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Says the film should have been entirely animated and that the live action scenes barely worked, a critical judgment of the film."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Criticizes the live-action scenes as barely working and suggests the film should have been entirely animated."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Live-action scenes barely worked; implies the film should have been animated."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":"Commenter says the film should have been fully animated and that its live-action scenes barely worked, a lukewarm-to-negative judgement."},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"They say the live-action scenes barely worked and imply the film would have been better entirely animated."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"Criticizes the live-action execution and suggests it should have been animated instead."}}},{"split":"challenge","movie":"Street Fighter","rank":84,"type":"polite_or_mixed_words_negative_verdict","human":-1,"excerpt":"My main complaint is that the fights had no consequences. It became a series of things happening without the plot progressing to the point of feeling tedious. The movie failed to make me care about anyone, which is hard to do. I\u2019m easily swayed by movies generally. So I also didn\u2019t care what happene","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.75"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.94"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.99"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The reviewer criticizes the lack of consequences in the fights, fails to care about characters, and finds the plot confusing. Despite some positive aspects, the overall impression is negative."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"multiple complaints about the plot, character development, and overall impact of the movie"},"openrouter:google/gemini-4-argon":{"label":-1,"verdict":"negative","rationale":"The commenter criticizes the tedious plot, lack of character investment, and anticlimactic ending, giving it a 4/10."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Criticises the tedious plot, inconsequential fights and lack of investment; minor praise; rated 4/10."},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Mostly criticism: tedious, uncaring characters, messy plot, anticlimactic ending; minor positives noted but gives 4/10."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Despite enjoying animation and occasional humor, they found the plot tedious, characters unengaging, and ending anticlimactic; rated 4/10."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Tedious, uncaring plot and 4/10 outweigh minor chuckles and animation praise."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":""},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"They call the film tedious, criticize its plot and characters, and rate it 4/10 despite enjoying a few scenes and jokes."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"The commenter lists multiple complaints about the plot and characters, giving it a 4/10 rating."}}},{"split":"challenge","movie":"Other Mommy","rank":11,"type":"polite_or_mixed_words_negative_verdict","human":-1,"excerpt":"Ridiculous plot\u2026 Bella to babysitter: hey, check out my creepy demonic friend lingering in the house. Babysitter: scared shitless. Probably should tell the parents. Bella (6 yo?) Don\u2019t tell my mom & dad. Babysitter: Cool, back to the party!","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.53"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.79 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.63"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.81"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment criticizes the plot and the character's actions, indicating a negative verdict."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"mixed","rationale":"Criticism of plot structure, but no clear positive or negative overall judgment."},"openrouter:google/gemini-4-argon":{"label":-1,"verdict":"negative","rationale":"The commenter criticizes the movie's plot as 'ridiculous' and mocks the illogical behavior of the characters."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Calls the plot ridiculous and mocks the babysitter's illogical choices, criticising the film's writing."},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Opens with 'Ridiculous plot' and mocks the babysitter's illogical behavior, criticizing the film's writing."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Calls the plot ridiculous and mocks the babysitter\u2019s implausible behavior."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Mocks the plot as ridiculous and illogical."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":""},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"Calls the plot ridiculous and mocks the characters\u2019 implausible behavior, expressing criticism of the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"Criticizes the plot as ridiculous by highlighting poor character decisions."}}},{"split":"challenge","movie":"Other Mommy","rank":46,"type":"polite_or_mixed_words_negative_verdict","human":-1,"excerpt":"Absolutely this. The Boogeyman, his other horror film, was not well done. It\u2019s mid at best. The minute I saw he was doing this one I lowered my expectations. Still didn\u2019t like it. Maybe he gets his shit together someday and makes a classic, but this ain\u2019t it.","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.05"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.97"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.99"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The commenter clearly did not like the film and had low expectations."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Clear criticism and no positive aspects mentioned"},"openrouter:google/gemini-4-argon":{"label":-1,"verdict":"negative","rationale":"The commenter explicitly states they lowered their expectations and still did not like the film."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Commenter lowered expectations yet still didn't like it; says 'this ain't it.'"},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Commenter states they still didn't like the film, even with lowered expectations, and says it's not a classic."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Explicitly says they still disliked this film despite lowered expectations and that it falls short."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Explicitly says they still didn\u2019t like it and this isn\u2019t it."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":"Commenter calls the director's previous film mid, says they didn't like this one, and doubts it's a classic."},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"They explicitly say they still didn\u2019t like this film and that it isn\u2019t a classic."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"The commenter states they lowered expectations and still didn't like the film, calling it not a classic."}}},{"split":"challenge","movie":"Other Mommy","rank":47,"type":"polite_or_mixed_words_negative_verdict","human":-1,"excerpt":"Once they went to grandma's castle mansion next to the lighthouse by the sea, I could no longer take any of it seriously, and I was barely holding on already. The batshit seance scene is wear it really jumped off a cliff. It's not grounded in any kind of reality, and there is no way anybody working ","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.95"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.53"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.88"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The reviewer criticizes the film's lack of grounding in reality, its over-the-top sequences, and the use of CGI that detracts from the performance."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"criticism of the film's seriousness and tone, disappointment expressed"},"openrouter:google/gemini-4-argon":{"label":-1,"verdict":"negative","rationale":"The commenter criticizes the film's baffling tone, poor CGI, and weak material, stating they couldn't take it seriously."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Calls film absurd, tonally baffling, impossible to take seriously; says Chastain deserves better material despite some creepiness."},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Commenter couldn't take the film seriously, found the tone baffling, criticized the CGI and material; only minor praise for Chastain being creepy."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Criticizes absurd plotting, inconsistent tone, weak material, and distracting CGI, despite occasional praise for Chastain\u2019s creepiness."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Couldn't take it seriously; baffling tone, bad material, CGI undermines it."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":""},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"They criticize the film\u2019s incoherent tone, implausibility, and distracting CGI; brief praise for Chastain\u2019s creepiness doesn\u2019t offset the overall disappointment."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"The commenter criticizes the film's tone, implausibility, and poor material, calling it baffling and advising the lead deserves better."}}},{"split":"challenge","movie":"Other Mommy","rank":63,"type":"polite_or_mixed_words_negative_verdict","human":-1,"excerpt":"It was... not good. I appreciate that they didn't go for the tired trope of \"nobody believes this thing is real\" and jumped straight to, \"oh it's fucking real and what do we do about it?\" but there was so much behavior that just came out bizarre because they believed this thing existed. Just an over","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.97"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.92 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.98"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.95"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":""},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Criticism of the plot and character behavior, no clear praise"},"openrouter:google/gemini-4-argon":{"label":-1,"verdict":"negative","rationale":"The commenter explicitly states the movie was \"not good\" and heavily criticizes the bizarre, illogical character behavior."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Opens with 'not good'; one small appreciation, then extensive criticism of characters' illogical, bizarre behavior."},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Opens with 'not good' and spends the comment criticizing illogical character behavior and tone; the one small appreciation doesn't change the overall negative lean."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Explicitly calls it not good and extensively criticizes implausible character behavior despite appreciating one avoided trope."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Opens with 'not good' and criticizes illogical, blas\u00e9 character reactions throughout."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":"Calls the film 'not good' and mocks its character behavior throughout; the overall verdict is clearly disappointed."},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"The commenter explicitly says the film was not good and criticizes its characters\u2019 implausible behavior."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"The commenter explicitly states the movie was 'not good' and criticizes its illogical character behavior and plot."}}},{"split":"challenge","movie":"Other Mommy","rank":92,"type":"polite_or_mixed_words_negative_verdict","human":-1,"excerpt":"Very disappointing, the trailer was much better than the movie and it was misleading. There is absolutely no suspense at all because everyone knows about Other Mommy in the first 5 minutes. The majority of the movie should have been tense not knowing if it was Other Mommy or the actual mom. Movie wa","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.94"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=1.00 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=1.00"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=1.00"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The reviewer criticizes the film's plot, scares, and characters, indicating a negative verdict."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Polite disappointment, multiple criticisms without clear praise"},"openrouter:google/gemini-4-argon":{"label":-1,"verdict":"negative","rationale":"The commenter explicitly states the movie was 'very disappointing', lacked suspense, was not scary, and had baffling plot elements."},"openrouter:anthropic/claude-opus-5.5":{"label":-1,"verdict":"negative","rationale":"Calls it very disappointing, not scary, no suspense, misleading trailer, criticizes tropes and abrupt scenes."},"openrouter:anthropic/claude-sonnet-5.5":{"label":-1,"verdict":"negative","rationale":"Calls the film very disappointing, criticizing lack of suspense, not scary, tired trope, abrupt scene, and misleading trailer."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Explicitly disappointed, criticizing the misleading trailer, lack of suspense and scares, clich\u00e9d resolution, and unexplained scene."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Disappointed; not scary, no suspense, disliked tropes and character."},"openrouter:anthropic/claude-haiku-5.5":{"label":-1,"verdict":"negative","rationale":"Calls the film disappointing, misleading, not scary, and lists multiple criticisms with no praise."},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"They found the film disappointing, suspense-free, unscary, and frustratingly written."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"The commenter expresses clear disappointment, calling the movie misleading, not scary, and lacking suspense."}}},{"split":"challenge","movie":"Primetime","rank":86,"type":"dark_description_no_verdict","human":0,"excerpt":"I didn\u2019t find that ending to be happy. Before we even saw if he got his SAG AFTRA, I was thinking that it\u2019s a loss either way after all he\u2019s been through. They kind of hint that he\u2019s reclusive after the show. He couldn\u2019t even answer when that guy asked \u201care you an actor\u201d. It felt almost like a cruel","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.31"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.96 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.80"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.93"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The commenter clearly did not like the ending and found it cruel."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Criticism of the ending and character behavior, indicating disappointment with the film."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment only discusses and interprets the plot and ending without expressing an opinion on the film's quality."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Interprets the ending's bleakness and the character's fate without judging the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Interprets the ending's meaning and tone without judging the film's quality or recommending it; it is analysis, not a verdict."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Interprets the ending as unhappy and cruel for the character without judging the film\u2019s quality."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Interprets the ending as unhappy and ironic, without judging the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses the ending's plot and character outcome without clearly judging whether they liked or would recommend the film."},"openrouter:openai/gpt-6-luna":{"label":-1,"verdict":"negative","rationale":"They describe the ending as unhappy and getting his card as a cruel joke, expressing a negative view of the film\u2019s conclusion."},"openrouter:google/gemini-3.5-flash-lite":{"label":-1,"verdict":"negative","rationale":"The commenter expresses disappointment with the bleak ending, describing it as a \"cruel joke.\""}}},{"split":"challenge","movie":"You Can See Everything","rank":9,"type":"dark_description_no_verdict","human":0,"excerpt":"And maybe it was just the added discomfort of being on camera, but the dad seemed like he wanted to keep talking but was worried about irritating Billy and obviously it\u2019s such a complex situation for the parents like the insanity of their daughters life for the past 15 years or whatever, immediately","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.93"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.18 P(pos)=0.01"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.85"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.53"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment criticizes the dad's behavior and the complexity of the situation, which is a negative verdict."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment is focused on plot and characters without a clear positive or negative judgment of the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment analyzes the situation and subjects shown, offering no opinion on the quality of the film itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses the parents' situation and scene dynamics without judging the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Commenter discusses the parents' situation and the dad's behavior on camera, with no judgement of the film itself."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Discusses the subjects\u2019 family dynamics and circumstances without judging the film itself."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Describes a tense family scene and characters without judging the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Describes scenes and characters' situations in the documentary without explicitly judging whether the film is good or recommended."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Discusses the characters\u2019 situation and the awkwardness of being filmed without judging the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"Comment discusses plot points and character situations without expressing a judgment on the film itself."}}},{"split":"challenge","movie":"You Can See Everything","rank":30,"type":"dark_description_no_verdict","human":0,"excerpt":"When her dad just confidently and casually said where it all went wrong\u2026 that was shocking. And it really made me feel like genuine damage was done to her brain and that she never recovered, and now doesn\u2019t have a single original thought in her head outside of what her partner says.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.40"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.73 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.66"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.50"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment criticizes the film's portrayal of the protagonist's mental state, suggesting it is overly dramatic and damaging to the character's mental health."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Criticism of character's mental state and development"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"Discusses a character and plot reaction without offering a judgement on the quality of the film itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses a shocking scene and character's damage; no judgement of the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses a plot moment and character impact without explicitly judging the film's quality or recommending it."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Discusses a shocking scene and its implications for a character without judging the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Describes a shocking plot moment and character damage, no film judgement."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":""},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Describes a shocking scene and its effect on a character, but gives no judgment of the film itself."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The commenter discusses plot points and character impact without expressing a clear judgment on the film itself."}}},{"split":"challenge","movie":"Clayface","rank":31,"type":"dark_description_no_verdict","human":0,"excerpt":"Caitlyn gets a very horrifying ending. Always having to look over her shoulder and not truly knowing if whoever she\u2019s talking to is actually them.","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.75"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.93 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.48"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.34"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment criticizes the ending of the film, describing it as 'very horrifying' and 'always having to look over her shoulder and not truly knowing if whoever she\u2019s talking to is actually them.'"},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise of the horrifying ending"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"Describes a character's horrifying plot ending without expressing an opinion on the film's quality."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Describes a character's horrifying fate without judging the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Describes a character's ending and its horror without judging the film's quality or recommending it."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Describes the horror of Caitlyn\u2019s ending without judging the film\u2019s quality or recommending it."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Describes a character's horrifying ending without judging the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Describes a plot point about a character's ending and paranoia; no judgement of the film itself."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Describes a character\u2019s horrifying ending without expressing an opinion about the film itself."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The comment discusses a plot point and a character's fate without expressing a judgment on the film itself."}}},{"split":"challenge","movie":"Clayface","rank":35,"type":"dark_description_no_verdict","human":0,"excerpt":"The problem with people like Clayface is that even if you give them what they want, you never know when they're gonna be unhinged enough to just snap on you and go for the kill. He's insane in every sense of the word.","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.38"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.75 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.74"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.51"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":""},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Criticism of the character's unpredictability and insanity"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment analyzes the character's psychology and behavior without expressing an opinion on the film itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Describes the character as insane and dangerous; no judgement of the film itself."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Describes the character's unpredictability and insanity; no judgment of the film's quality or recommendation."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Describes Clayface's dangerous, unstable character without judging the film itself."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Discusses the character as insane, not the film itself."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Describes the character as unhinged and dangerous, but says nothing about whether they liked the film."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Describes Clayface as dangerous and insane, but gives no opinion on the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The comment discusses the character's traits without giving a verdict on the film."}}},{"split":"challenge","movie":"Heart of the Beast","rank":22,"type":"dark_description_no_verdict","human":0,"excerpt":"It was definitely from blood loss, that bite wound on his side was too gnarly it needed immediate care. When I first saw how bad he was bleeding on the rock, that was the first moment of oh shi he's not gonna make it","pred":{"vader":{"label":0,"verdict":null,"rationale":"compound +0.03"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.43 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.45"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.63"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment criticizes the film for its graphic violence and bloodiness, which is a negative aspect."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment is discussing a scene from the movie without providing a verdict"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment only discusses a plot point and character death without expressing an opinion on the movie itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses a character's injury and death cause; describes a reaction but gives no judgement of the film."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses plot details (cause of character's death, bite wound) without judging the film's quality or recommending it."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Discusses a character's wound and likely cause of death without judging the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Plot speculation about a character's injury, no film judgement."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Discusses a plot detail (cause of death from a bite wound) without judging the film's quality or recommending it."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Describes the character\u2019s injury and reaction to the scene without judging the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"Comment discusses plot details and character fate without expressing a judgment on the film itself."}}},{"split":"challenge","movie":"Verity","rank":49,"type":"dark_description_no_verdict","human":0,"excerpt":"That whole situation pissed me off, it\u2019s not like you overslept, you just saw someone get killed and crushed in front of you spraying you with blood??","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.86"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.79 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.56"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.60"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The commenter is expressing strong dislike for the film's plot and situation, which is a clear negative verdict."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Direct criticism and frustration with the situation described"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The commenter reacts to a character's situation in the plot without giving a verdict on the film itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Reacts to a character's behavior in a scene without judging the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Commenter is angry at a character's situation or reaction to a violent scene, not judging the film itself."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Expresses frustration about a plot situation, not a judgment of the film itself."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Frustrated by a plot situation, not judging the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Commenter is reacting to a plot situation (character witnessing a death) with no judgement of the film itself."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"They\u2019re frustrated with a situation in the story, but don\u2019t judge the film itself."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The commenter is discussing a plot point and expressing frustration with a character's situation, not the film itself."}}},{"split":"challenge","movie":"Resident Evil","rank":14,"type":"dark_description_no_verdict","human":0,"excerpt":"I think it\u2019s more like how Brian was at the end; pleading with the scientists to cure him while his body is acting on the monster instincts","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.42"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.70 P(pos)=0.99"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.50"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.37"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"The comment praises the character's actions, which is a positive aspect of the film."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"Comment is discussing character motivations without a clear positive or negative judgment on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The comment merely discusses a character's situation and plot details without expressing an opinion on the film's quality."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Plot interpretation comparing a character's ending; no judgement of the film's quality."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment draws a plot/character comparison about Brian's ending, offering no judgment of the film's quality or recommendation."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Discusses a character\u2019s behavior and monster instincts without judging the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Plot interpretation of a character's ending, no film judgement."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Comment offers a plot interpretation of a character's ending, with no judgement of the film's quality or recommendation."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Speculates about a character's behavior without expressing an opinion on the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The comment discusses plot and character details without expressing any opinion or verdict on the film itself."}}},{"split":"challenge","movie":"Wildwood","rank":27,"type":"dark_description_no_verdict","human":0,"excerpt":"Yeah it really doesn\u2019t sugarcoat violence. Even in the scenes where the violence is justified, it\u2019s still ugly and brutal.","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.83"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.98 P(pos)=0.01"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.59"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.63"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The comment criticizes the film for its violent content, describing it as ugly and brutal, which is a negative verdict."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":"Praise of dark, scary, violent content"},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"Describes the portrayal of violence without expressing an opinion on the quality of the film itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Describes the film's brutal violence without clearly praising or criticizing the film overall."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Describes the film's depiction of violence as ugly and brutal without stating whether they liked it or would recommend it."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Describes the film\u2019s unflinching depiction of violence without expressing approval or criticism of the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Describes unsugarcoated violence without judging the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Describes the film's violence as ugly and brutal without stating whether the commenter liked or would recommend it."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Observes that the film depicts violence as ugly and brutal without judging the film itself."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"Discusses the film's violence without expressing a clear like, dislike, or recommendation."}}},{"split":"challenge","movie":"Wildwood","rank":44,"type":"dark_description_no_verdict","human":0,"excerpt":"What surprised me most honestly is that when they get to the governess\u2019s backstory, it turns out she really went to the dark side and decided to put the boy\u2019s soul in a robot. And he actually couldn\u2019t handle that fact so he killed himself. Like, this movie held next to nothing back here.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.79"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.99 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.66"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.53"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":""},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"Describes a horrifying and disappointing plot twist"},"openrouter:google/gemini-4-argon":{"label":1,"verdict":"positive","rationale":"The commenter expresses impressed surprise at a dark backstory and praises the movie for holding nothing back."},"openrouter:anthropic/claude-opus-5.5":{"label":1,"verdict":"positive","rationale":"Impressed the film held nothing back with its dark governess backstory; praises its boldness with disturbing content."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"Describes plot points and expresses surprise at the film's darkness, but offers no clear judgement of quality or recommendation."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Describes surprise at the dark backstory and uncompromising content without explicitly judging the film\u2019s quality."},"openrouter:x-ai/grok-4.7":{"label":1,"verdict":"positive","rationale":"Praises the film for not holding back on dark plot turns."},"openrouter:anthropic/claude-haiku-5.5":{"label":1,"verdict":"positive","rationale":""},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"Describes a shocking plot reveal and the film\u2019s willingness to show it, without clearly judging the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The commenter is discussing the plot without expressing a clear opinion or recommendation about the film."}}},{"split":"challenge","movie":"Klara and the Sun","rank":98,"type":"dark_description_no_verdict","human":0,"excerpt":"Can anyone describe the ending of this movie to me in detail? I had a vivid dream of the ending and I can't find the actual ending anywhere, I'm so scared that what I dreamed is actually what it ended like because I was watching a video about the movie and the book, and I can't seem to find the vide","pred":{"vader":{"label":-1,"verdict":null,"rationale":"compound -0.28"},"siebert+opinion (live)":{"label":0,"verdict":null,"rationale":"P(op)=0.01 P(pos)=0.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.89"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.83"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":-1,"verdict":"negative","rationale":"The commenter is clearly scared and frustrated that they can't find the actual ending of the movie, indicating a negative verdict."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"The comment is asking about the ending and dreaming about it, but does not express a verdict on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"no_verdict","rationale":"The commenter asks for the plot of the ending and has not expressed an opinion on the film itself."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"no_verdict","rationale":"Asks about the ending after a dream; no judgement of the film itself."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"no_verdict","rationale":"The commenter asks about the movie's ending and describes a dream, without giving any judgment of the film itself."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"no_verdict","rationale":"Asks for ending details and describes a dream without judging the film."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"no_verdict","rationale":"Asks for the ending; no judgment of the film."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"no_verdict","rationale":"Asks about the ending and a lost video; expresses confusion, no judgement of the film's quality."},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"no_verdict","rationale":"They ask for an ending description and express fear about a dream, but give no judgment of the film."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"no_verdict","rationale":"The commenter is asking for the ending of the movie based on a dream and has not expressed an opinion on the film itself."}}},{"split":"challenge","movie":"Klara and the Sun","rank":92,"type":"mixed_verdict","human":0,"excerpt":"I wouldn't say 'so bad it's good' but more 'It's so mid that it is actually ok'. I don't feel the hate train others have for it to call any aspect bad, but it never rises in any aspect to be 'good' either. Perfectly fine. I don't regret my time with it, but I'll likely forget about it by tomorrow.","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.88"},"siebert+opinion (live)":{"label":-1,"verdict":null,"rationale":"P(op)=0.63 P(pos)=0.03"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.58"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":-1,"verdict":"negative","rationale":"NLI p=0.50"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":1,"verdict":"positive","rationale":""},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"no_verdict","rationale":"The comment does not provide a clear positive or negative verdict on the film itself."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"mixed","rationale":"The commenter explicitly rates the film as 'mid', 'ok', and 'perfectly fine'\u2014neither bad nor good."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"mixed","rationale":"Calls it mid and perfectly fine: not bad, never good, forgettable but not regretted. No clear lean."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"mixed","rationale":"Calls it mid and forgettable but 'perfectly fine' with no regrets; no aspect bad or good, so no clear lean."},"openrouter:openai/gpt-6.1-sol":{"label":-1,"verdict":"negative","rationale":"Calls it mediocre and forgettable, with no aspect good, despite finding it watchable and not regretting the time."},"openrouter:x-ai/grok-4.7":{"label":-1,"verdict":"negative","rationale":"Calls it mid, not good, and forgettable despite not hating it."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"mixed","rationale":""},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"mixed","rationale":"They found it acceptable and not regrettable, but also mediocre and forgettable."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"mixed","rationale":"Describes the film as 'mid' and 'ok', neither hating nor praising it, resulting in a mixed stance."}}},{"split":"challenge","movie":"Klara and the Sun","rank":78,"type":"mixed_verdict","human":0,"excerpt":"Just saw it with no expectations and it was entertaining without knowing anything about the book (always tricky to adapt a book). I wouldn\u2019t say this is great and quite forgettable. Ortega was brilliant and I liked Adams. Tharia was the least likeable character and she didn\u2019t do much. Even Rick was ","pred":{"vader":{"label":1,"verdict":null,"rationale":"compound +0.87"},"siebert+opinion (live)":{"label":1,"verdict":null,"rationale":"P(op)=0.97 P(pos)=1.00"},"local-nli:MoritzLaurer/deberta-v3-base-zeroshot-v2.0":{"label":0,"verdict":"mixed","rationale":"NLI p=0.62"},"local-nli:MoritzLaurer/deberta-v3-large-zeroshot-v2.0":{"label":0,"verdict":"no_verdict","rationale":"NLI p=0.55"},"local-llm:qwen2.5-1.5b-instruct-q4_k_m":{"label":0,"verdict":"mixed","rationale":"The reviewer liked the acting and the story, but found the characters and plot forgettable."},"local-llm:qwen2.5-3b-instruct-q4_k_m":{"label":0,"verdict":"mixed","rationale":"The comment describes the movie as entertaining but forgettable, with praise for some performances and characters but criticism of others."},"openrouter:google/gemini-4-argon":{"label":0,"verdict":"mixed","rationale":"Found it entertaining with good performances, but called it 'not great and quite forgettable' with weak character dynamics."},"openrouter:anthropic/claude-opus-5.5":{"label":0,"verdict":"mixed","rationale":"Entertaining with brilliant performances, but called not great and quite forgettable; no clear overall lean."},"openrouter:anthropic/claude-sonnet-5.5":{"label":0,"verdict":"mixed","rationale":"Says it was entertaining and praises Ortega and Adams, but calls it not great and quite forgettable, with no clear overall lean."},"openrouter:openai/gpt-6.1-sol":{"label":0,"verdict":"mixed","rationale":"Found it entertaining and praised performances, but called it forgettable and criticized the central character relationship."},"openrouter:x-ai/grok-4.7":{"label":0,"verdict":"mixed","rationale":"Found it entertaining, but not great and quite forgettable."},"openrouter:anthropic/claude-haiku-5.5":{"label":0,"verdict":"mixed","rationale":""},"openrouter:openai/gpt-6-luna":{"label":0,"verdict":"mixed","rationale":"Found it entertaining and praised the performances, but called it not great and forgettable."},"openrouter:google/gemini-3.5-flash-lite":{"label":0,"verdict":"mixed","rationale":"Entertaining and praised acting, but also called quite forgettable with flawed character bonds."}}}]}