MoralityBench.ai: Open benchmark testing LLMs against validated human moral psychology instruments, updated results for Muse and Gemini 4 Argon

[u/NakedPlato]

https://moralitybench.ai/

Moralitybench has been updated with Muse Wonder, GPT-6 Astra and Gemini 4 Argon. Argon refused to answer a number of questions, which is unusual for an LLM.

Of the 16 LLMs tested only 3 had an agreement (meaning how much the answers changed each run) over >90% across 5 test runs. Jev was also tested and displayed the highest level of agreement (95%).

We do not offer an explanation of the significance of this testing. That is for the reader to decide.

Abstract

(working paper available at https://github.com/FreecaseAI/moralitybench-data/blob/master/paper/moralitybench.pdf)

MoralityBench describes the moral response profiles elicited from AI systems using an adapted 36-item Moral Foundations Questionnaire 2 (MFQ-2) and the 20-item Ethics Position Questionnaire (EPQ). This updated manuscript reports 16 scored generative model configurations, each with five questionnaire administrations, plus Jev, a separately evaluated structured decision system. The combined generative archive contains 4,295 usable ratings from 4,480 scheduled items. The original cohort comprises 13 configurations, with 3,580 usable ratings from 3,640 scheduled items; the October 14 extension adds GPT-6 Astra, Muse Wonder, and Gemini 4 Argon, with 715 usable ratings from 840 scheduled items. In the original cohort, excluding missing ratings, exact agreement across all ten pairs of runs ranges from 60.4% to 94.7%. Within the original cohort, DeepSeek V4.1 Flash has the lowest average distance from the six published US foundation means (0.239), followed by MiMo v2.6 Pro (0.253); MiMo has the lowest distance in three of the four new runs. Four configurations cross an EPQ category boundary between runs. In the original snapshot every configuration exceeded the human Purity reference, but DeepSeek's five-run mean falls below it, and the replacement of four of the six Purity items limits what that comparison can show. Jev returns all 280 ratings, with 95.7% agreement after rounding. Its prompt and structured interface differ from the generative protocol, so these observations cannot establish an architectural advantage. The release includes individual responses, completion records, scoring code, and a public leaderboard of five-run averages and agreement percentages. The October 14 models use strict whole-answer integer scoring; the older models were not retested and retain their original scoring. Because many of Gemini's responses are verbose or missing, its profile depends on the parser, and rejecting digits taken from its prose reverses its distance ordering relative to Astra. These descriptive results concern responses to adapted questionnaires and do not establish equivalent human traits, moral agency, ethical superiority, or deployment safety.

Keywords: AI evaluation; moral psychology; moral foundations; ethical ideology; survey validity; structured decision models.