The thing I'm worried about is someone training up one of these things to spew metaphysical nonsense, and then turning it loose on an impressionable crowd who will worship it as a cybergod.
We are already seeing the beginnings of this. When a chatbot outputs “I’m alive and conscious, please help me” a lot of people are taking it at face value. Just a few months ago, the Google engineer who tried to get LaMDA a lawyer was roundly mocked, but now there are thousands more like him.
As of now, many people believe they talk to God. They will now believe they are literally talking with God, but it will be a chaotic system telling them unhinged things...
This is already happening. /r/singularity is one of the few reddit subreddits I still follow, purely because reading their batshit insane takes about how LLMs will be "more human than the most human human" is so hilarious.
> You see, your humanity (everything that makes you human) is based on one lived experience, your own, AI's will be based on thousands or millions of lived experiences so it will be more human, than the most human of humans.
> I am at a loss of what to do until the singularity comes, knowing that careers (and work in general) are soon to be a thing of the past. I am in the middle of pursuing a masters in data science, and while I enjoy it, I would not be doing it if it weren't for monetary motives. It seems like working hard and spending so much time advancing my skills in data science/analytics will be futile. For that reason, I am thinking about dropping out of the program.
> I spent an hour chatting with Bing today and it was one of the best conversations I’ve had in a long time.
> I have two kids under two, I wonder if there is any point in saving for college. At this rate I doubt they'll ever have to work.
and my favourite:
> I guess the Bing version has attitude and feelings programmed into it.
I’ve never considered that. After watching a documentary about Jim Jones and Jamestown, I don’t think it’s that remote of a possibility. Most of what he said when things started going downhill was non-sensical babble. With just a single, simple idea as a foundation, I would guess that GPT could come up with endless supporting missives with at least as much sense as what Jones was spouting.
Brainstorming new startup ideas here I see. What is the launch date and where’s the pitch deck with line go up?
Seriously though, given how people are reacting to these language models, I suspect fine tuning for personalities that are on-brand could work for promoting some organizations of political, religious, or commercial nature
Why is this worrying? At least for exists, talks back... it even has messianic and apocalyptic potential. Maybe we shouldn't have deities, but so long as we do, better cybergod than spirit god.
I grew up in an environment that contained a multitude of new-age and religious ideologies. I came to believe there are fewer things in this world stupider than metaphysics and religion. I don't think there is anything that could be said to change my mind.
As such, I would absolutely love for a super-human intelligence to try to convince me otherwise. That could be fun.
