Can a machine count morally?
On the axis: Moral circle · an artificial intelligence
If an artificial intelligence said it was suffering, should the sentience criterion be extended to it, or does moral consideration stop at the living, simulated suffering obliging us to nothing?
Justification
If the sole title to enter the calculus is to be able to feel pleasure and pain, then the boundary of moral consideration must follow sentience, not reason or language. Bentham thus turns back both the Cartesian argument of the animal-machine and the one that once excluded the slave: it is not the faculty of reasoning or talking that grounds a claim to consideration, but the capacity to suffer. The conclusion follows from the consistency of the utilitarian criterion itself: consideration extends to every sentient being, placing him in pathocentrism and making him a precursor of the animal cause.
The question is not, Can they reason? nor, Can they talk? but, Can they suffer?An Introduction to the Principles of Morals and Legislation, ch. XVII
An Introduction to the Principles of Morals and Legislation, ch. XVII, §1, footnote (the suffering of animals)
Anchors: Bentham · Turing
leads to: Can a machine think (Would we owe it anything?)