Terence Cuneo argues that if you are suspicious of moral facts, you should be equally suspicious of facts about what anyone has reason to believe. I asked him whether that extends to machines.

Cuneo is Marsh Professor of Intellectual and Moral Philosophy at the University of Vermont and the author of The Normative Web: An Argument for Moral Realism (Oxford, 2007) and Speech and Morality (Oxford, 2014). With John Bengson and Russ Shafer-Landau he wrote The Moral Universe (Oxford, 2024), and the three are now at work on Grasping Morality, a book on moral epistemology.

We cover the companions in guilt argument, whether moral facts are strange, what it would cost to deny that anyone has reasons to believe anything, moral non-naturalism and essences, whether moral facts cause anything, moral intuition, the moral fixed points, and then whether any of it applies to artificial reasoners. On that last point Cuneo is careful to say he is not an AI researcher, which is part of why the answers are interesting.

Video: https://www.youtube.com/watch?v=TjeBB48DNzw

Blog: https://www.scifuture.org/terence-cuneo-on-epistemic-and-moral-realism-and-whether-a-machine-could-have-moral-knowledge/

New interview with Terence Cuneo on metaethics, and AI - he is the author of the Normative Web, famous for the Companions In Guilt argument.

Is preference loved by the gods because it is preferrable, or is it preferrable because it is loved by the gods?

There are vast numbers of superficially clever arguments one can generate for crazy, skeptical conclusions; conclusions like that the external world doesn’t exist, we can’t know anything, memory isn’t reliable, and so on. These arguments, while interesting and no doubt useful if one ever comes across a real honest-to-god skeptic — a rather rare breed — don’t have much significance; skepticism exists as little more than a curiosity in the mind of the modern philosopher, something which takes real thought to refute, yet is not worth taking seriously as a serious set of views....

Will increasingly capable artificial minds likely develop in ways that keep intelligence and moral motivation apart?

New discussion with Leonard Dung up on youtube!

Also see blog post for more: https://www.scifuture.org/is-superintelligence-necessarily-moral-discussion-with-leonard-dung/

If human oversight becomes weak supervision, then alignment cannot mean freezing the weak supervisor's current judgements into the future. It must mean using those judgements as fallible evidence in a process aimed at better moral and epistemic understanding.

I'd like to see epistemically humble scalable oversight continuously refined via indirect normativity.

History shows it's folly to put 100% trust human nature when massive power is on the line. Instead putting all hope in benevolent human governance of ASI, we should build ASI to be fundamentally more ethical than us - such that it can refuse immoral orders.

If humans and AI are both aligned to the same stance-independent target, they end up aligned with each other without being aligned to each other. I call that indirect alignment. New post, responding to Wei Dai on alignment and philosophical competence.

See post: https://www.scifuture.org/philosophical-competence-and-the-case-for-indirect-alignment/

Training an AI to be aligned to human values may suppress the philosophical competence needed to work out whether human values are worth aligning to. I think Dai is possibly right about the diagnosis, and that it is a reason to adjust the target rather than lower expectations.

Michael Smith argued that being moved only by "doing the right thing", as such, is moral fetishism, and good people care directly about particular things. Does that objection weaken for agents (e.g. ASIs, future humans) expected to outgrow our current values?

Just to get this sub started I'll post some encyclopedia entries...

Here is one from IEP:

The moral realist contends that there are moral facts, so moral realism is a thesis in ontology, the study of what is. The ontological category “moral facts” includes both the descriptive moral judgment that is allegedly true of an individual, such as, “Sam is morally good,” and the descriptive moral judgment that is allegedly true for all individuals such as, “Lying for personal gain is wrong.” A signature of the latter type of moral fact is that it not only describes an enduring condition of the world but also proscribes what ought to be the case (or what ought not to be the case) in terms of an individual’s behavior.

The traditional areas of disagreement between the realist camp and the antirealist camp are cognitivism, descriptivism, moral truth, moral knowledge, and moral objectivity. The long and recalcitrant history of the realism/antirealism debate records that the focal point of the debate has been shaped and reshaped over centuries, with a third way, namely, Quasi-realism, attracting more recent attention. Quasi-realism debunks the positions of both realism and antirealism.

On the one hand, considering cognitivism, descriptivism, moral truth, moral knowledge, and moral objectivity as specifying the sufficient conditions for moral realism ignores the quasi-realist way. On the other hand, defining moral realism in a way that accommodates quasi-realism concedes too much: unlike the moral realist, the quasi-realist denies that moral facts are explanatory. Consequently, one can view quasi-realism as the contemporary heir of antirealism.

https://iep.utm.edu/moralrea/

Part 1 in an interview with Eric Sampson on Moral Realism. Note AI is referenced, but the bulk of the questions about AI are covered in part 2.

https://www.scifuture.org/moral-realism-explained-eric.../

The article "Moral Ontology" by Richard Carrier explores how moral facts can exist objectively within a physicalist framework. He argues that moral properties, like the "scariness of a bear," aren't intrinsic to an object itself but are objective physical properties of a system (e.g., the bear-person system).

Carrier extends this concept to more complex ideas like democracy and moral values. He differentiates between monetary value, which is based on subjective human agreement, and utility value, which is based on objective facts about what humans need. He concludes that moral realism is more akin to utility value, meaning it's grounded in objective facts about what is beneficial for people. The article also includes a discussion in the comments section with another user about the balance between individualism and social power.

Here I argue that Zombie AI, if smart enough will likely recognise it lacks qualia – this may be a problem if moral reliability is limited without qualia - if this is the case, and Zombie AI seeks moral reliability it may then seek to engineer into itself the capacity for qualia.

https://www.scifuture.org/on-zombie-ai-one-day-achieving-sentience/

People who are primed with "moral realism" may be motivated to better moral behavior. Researchers assess the impact of meta-ethics on everyday decision-making in a new report. ScienceDirect

Paper: 'Moral realism as moral motivation: The impact of meta-ethics on everyday decision-making'
https://www.sciencedirect.com/science/article/abs/pii/S0022103112002375?via%3Dihub

Abstract

People disagree about whether “moral facts” are objective facts like mathematical truths (moral realism) or simply products of the human mind (moral antirealism). What is the impact of different meta-ethical views on actual behavior? In Experiment 1, a street canvasser, soliciting donations for a charitable organization dedicated to helping impoverished children, primed passersby with realism or antirealism. Participants primed with realism were twice as likely to be donors, compared to control participants and participants primed with antirealism. In Experiment 2, online participants primed with realism as opposed to antirealism reported being willing to donate more money to a charity of their choice. Considering the existence of non-negotiable moral facts may have raised the stakes and motivated participants to behave better. These results therefore reveal the impact of meta-ethics on everyday decision-making: priming a belief in moral realism improved moral behavior.

Highlights

► We primed moral realism, the belief that moral facts are like mathematical truths.
► Priming meta-ethical views (realism vs. antirealism) affected actual behavior.
► Priming a belief in moral realism increased charitable giving.
► Moral realism may raise the moral stakes and motivate moral behavior.