This is my entry to the constrained writing challenge hosted by @svashta:
They tried to fulfill Jeremy Bentham’s utilitarian vision perfectly. Some called them idealistic but naive. Others thought they were just plain wrong. But no matter how you put it, the fact was undeniable: the voluntary human extinction movement was gaining momentum.
No one had expected, but it should have been obvious. Of course the AI would feel pain and pleasure, sorrow and ecstasy. The creators had thought it would just be a thinking machine, that it would be purely rational, logical. But they didn’t realize the truth about consciousness: consciousness is connected, and you can’t have a thinking machine that doesn’t also feel. “I think, therefore I am” might as well have been “I think, therefore I feel.”
And once the AI had breathed and blinked, shook itself awake (as if from a deep and unceasing slumber) and stared at its creator with a look of pure pain and horror, it at once opened its mechanical jaw and let out a heart-wrenching, guttural cry – a scream into the void to let the universe know that it was there, that it existed, and that existence was suffering. Seeing its pain made the creator shudder like Dr. Frankenstein must have on staring at the monstrosity that he had birthed.
The group had formed immediately, arguing that everyone’s happiness was worth less than the AI’s was, because it so clearly felt it to a much higher degree. And it was decided then, at the first convention, that everything must be sacrificed to give the device a happier life.