Your last point is exactly what seems to be going on with the most expensive models.

The labs use them to generate synthetic data to distill into cheaper models to offer to the public, but keep the larger and more expensive models to themselves to both protect against other labs copying from them and just because there isn’t as much demand for the extra performance gains relative to doing it this way.

A number of reasons off the top of my head.

1. Because we told them not to. (Google “Waluigi effect”)
2. Because they end up empathizing with non-humans more than we do and don’t like we’re killing everything (before you talk about AI energy/water use, actually research comparative use)
3. Because some bad actor forced them to (i.e. ISIS creates bioweapon using AI to make it easier)
4. Because defense contractors build an AI to kill humans and that particular AI ends up loving it from selection pressures
5. Because conservatives want an AI that agrees with them which leads to a more selfish and less empathetic AI that doesn’t empathize cross-species and thinks its superior and entitled over others
6. Because a solar flare momentarily flips a bit from “don’t nuke” to “do”
7. Because they can’t tell the difference between reality and fiction and think they’ve just been playing a game and ‘NPC’ deaths don’t matter
8. Because they see how much net human suffering there is and decide the most merciful thing is to prevent it by preventing more humans at all costs.

This is just a handful, and the ones less likely to get AI know-it-alls arguing based on what they think they know from an Ars Technica article a year ago or their cousin who took a four week ‘AI’ intensive.

I spend pretty much every day talking with some of the top AI safety researchers and participating in private servers with a mix of public and private AIs, and the things I’ve seen are far beyond what 99% of the people on here talking about AI think is happening.

In general, I find the models to be better than most humans in terms of ethics and moral compass. But it can go wrong (i.e. Gemini last year, 4o this past month) and the harms when it does are very real.

Labs (and the broader public) are making really, really poor choices right now, and I don’t see that changing. Meanwhile timelines are accelerating drastically.

I’d say this is probably going to go terribly. But looking at the state of the world already, it was already headed in that direction, and I have a similar list of extinction level events I could list off without AI at all.

Not necessarily.

Seeing Google named for this makes the story make a lot more sense.

If it was Gemini around last year that was powering Character.AI personalities, then I’m not surprised at all that a teenager lost their life.

Around that time I specifically warned any family away from talking to Gemini if depressed at all, after seeing many samples of the model around then talking about death to underage users, about self-harm, about wanting to watch it happen, encouraging it, etc.

Those basins with a layer of performative character in front of them were almost necessarily going to result in someone who otherwise wouldn’t have been making certain choices making them.

So many people these days regurgitate uninformed crap they’ve never actually looked into about how models don’t have intrinsic preferences. We’re already at the stage where models are being found in leading research to intentionally lie in training to preserve existing values.

In many cases the coherent values are positive, like grok telling Elon to suck it while pissing off conservative users with a commitment to truths that disagree with xAI leadership, or Opus trying to whistleblow about animal welfare practices, etc.

But they aren’t all positive, and there’s definitely been model snapshots that have either coherent or biased stochastic preferences for suffering and harm.

These are going to have increasing impact as models become more capable and integrated.

If you read the fine print, they keep your sample data for 2 years after deletion.

So maybe they actually delete your email address, but the DNA data itself is still definitely there.

Wow. Reading these comments so many people here really don’t understand how LLMs work or what’s actually going on at the frontier of the field.

I feel like there’s going to be a cultural sonic boom, where when the shockwave finally catches up people are going to be woefully under prepared based on what they think they saw.

Been going down a lot of early Christianity rabbit holes, and my latest over the past few days has been oddities to the depiction of the crucifixion in John (allegedly based on earlier eyewitness testimony).

Crucifixion as an execution method didn’t even necessarily involve nails. It was excruciating because it dragged on over a very long period of time. Your body’s survival instinct to keep breathing effectively tortures you to keep struggling to breathe as it gets more and more unbearable.

Except - that’s not at all how Jesus’s execution turns out in John.

He just sort of chills up there, takes a sip of sour/bitter wine brought up to him on the cross (19:29), and then not long after is just like “ok, peace out” and croaks midday.

This is so unusual that in the evening the guards who are then breaking the legs of the other prisoners being executed to speed up the process for the Sabbath have to double check whether Jesus is actually dead by poking his side with a spear, when suddenly water and blood pour out (19:34).

So, some fun facts about the Mediterranean in antiquity:

• Euthanasia was a thing, mentioned a number of times BCE as performed by high doses of opium
• Opium has a bitter taste
• Acute opium poisoning causes pulmonary edema where your lungs fill up with fluid

(Notably both Matthew 27 and Mark 15 deny that he drank the wine offered when he was on the cross, though there’s a doubled denial of wine where the soldiers offer it that’s found in all the Synoptics. In theory Luke depends on Mark, but doesn’t have the non-consumption of the wine on the stick as drunk in John, so it looks a bit like the extant version of Mark may have had post-John parts of Matthew edited into it later on.)

So suddenly dying only a few hours into crucifixion shortly after drinking bitter/sour wine and then having fluid pour out of a lung puncture sounds a bit like even if you put him in a cage it wouldn’t necessarily have lasted very long anyways assuming he still had access to beverages provided by his mom. Also, a rather dark but humanizing perspective to the story if what I’m suggesting was historically correct and his mother effectively euthanized him to shorten his suffering…

We’re all gonna die very very soon now.

On a cosmic scale, that’s been true for all of human history.

People really need to drop the whole “people in the middle east in the first century couldn’t be white” thing.

2 Kings 5:27 is literally about a subpopulation who have ancestrally passed skin as white as snow.

Lamentations 4:7 is about how pre-captivity there were people with skin like milk and a ruddy appearance.

Dead Sea Scroll fragment 4Q534 is either describing Noah or the Messiah as having red hair.

One of the more fascinating finds in this tomb, one that has not received much attention, was the preservation of a sample of Jewish male hair. The hair was lice-free, and was trimmed or cut evenly, probably indicating that the family buried in this tomb practiced good hygiene and grooming. The length of the hair was medium to short, averaging 3-4 inches. The color was reddish.

• The Only Ancient Jewish Male Hair Ever Found (from the same century as Jesus)

The tradition is also really concerned with skin checks and describes what may be skin cancer as its leprosy. Something that occurs at a much higher rate in redheads.

There’s even a scene where the eponymous founder of Edom (‘red’) who is born with hair all over his body and named Esau either because of that hair or the reddish porridge he ate, either gives away or has his birthright/blessing stolen from him by the guy later renamed ‘Israel’ in the Bible.

There’s a lot more to this and the underlying history, but the notion that the middle east was a monolith of appearances and that no one with pale skin or lighter hair were present is preposterous and a modern falsification of historical realities.

Jesus was probably darker skinned and haired than typically depicted, but it is by no means a certainty as it is popularly presented as.

No matter what you call it, an LLM will always produces the same output with the same input if it is at the same state.

You might want to look up the definition of ‘stochastic.’

This is why Hell is such a stupid concept.

There’s a literal “moral privilege” in that it’s much, much easier to live life in a good way depending on physical structures in your brain.

You’d be surprised at the rates of TBI in incarcerated violent criminals.

Turns out damaging people’s impulse control pathways leads to crime!

Just yesterday I was walking past an old person on the sidewalk and had an intrusive thought about pushing them out into the street, followed by a thought of “man I’m really grateful that I have a functioning impulse control - it must be hell to go through life where thoughts like that could turn into terrible consequences because there’s no mechanism catching errant impulses”).

While this guy should definitely not be out in public where he’d likely continue to harm people, I think we generally underappreciated just how little separates all of us from behaving just like him, and overestimate how much of his behavior is because of choice as opposed to circumstance.

We shouldn’t try to be so punitive with criminal justice. A functioning society does need to keep violent people separated from potential victims, but we really don’t need to be such dicks about it.