The Problem with AS

Philosophy Forums Science, Technology, and Math
1

Those who refuse to anthropomorphize computers use the term Autogenerated Software to refer to the awesome new technology that is revolutionizing the world.

Which takes us into our first detour. Neural network is also an anthropomorphization, and, really, belies a profound ignorance of how the neural system works.

All this to say that, Big Brother speak out of the way, we are as awed as anyone by this technology. The jump in computer capacity is something like the difference between open and closed steam engines, whatever the correct term is there.

Not only its power is great, but its design is completely beautiful. It has its roots in the slower revolution that has been taking place in science, culminating in what is known as quantic analysis. It is the comprehension that cause-consequence is really just burdened with superstition, and that superstition slows it down. There is no actual reason to conjecture a direct causal link between any thing and any other thing. What we can say for certain is that there is a probabilistic relationship between certain things. By quantifying it and narrowing it down, quantic analysis has proven that it can leave any other physics approach in the dust in terms of precision, detail, accuracy and speed. Imagination, the hypothesis behind a calculation, is just a set of names to give things so that we can keep account of them as we trace their probabilistic relationships. It is no longer a metaphysical claim.

The proof is in the power. In the same way, AS has proven that a simple probabilistic approximation between any set of things will just completelly steamroll any causal chain attempting to trace the same relationship. You take a piece of information, a “token,” you give it a name, and you roll dice until it gives you what you want. Each time you roll you shift the node, and then when you get the right result, you leave the node as it is. Since you need many millions of nodes, you use an algorythm that traces and predicatably modifies all nodes at the same time, and then you just roll dice until it gives the correct following chain of tokens. The nodes are simply collections of bits that you directly switch to 0 or 1 inside the machine. The machine doesn’t know anything about what it is producing, nor is there any design at any point, any abstract idea, of getting or how to get from token A to token B. Just random dice rolls until the fucking thing gets it right. You put a dozen chimpanzees with typewritters in a room and press fast forward.

It’s completely beautiful.

What we distrust is not the software, designed by beautiful (if a little too sensitive to science fiction tropes), innocent nerd minds.

If you followed the design of the technology, it will not escape you that what tokens you give it, and what correct next tokens you decide to leave nodes in position for, is fairly critical. And this is a human choice, it is a person that will decide what to input as tokens and what to accept as output tokens.

What we mistrust is not the technology. What we mistrust is the humanses involved in designing these multi-trillion dollar computer systems, which power is unprecedented in the history of mankind.

2

I dedicate this playlist to all the sockmuppets I’ve loved before:

Muzk soup for the soul
4

I think you’re falling into a classic trap of reductionist blindness here. Just because you can perfectly explain the microscopic mechanics of the engine—tokens, binary switches, and statistical dice rolls—doesn’t mean the car can’t drive itself over a cliff. You don’t need a ghost in the machine to lose control of the machine. By focusing entirely on how passive the underlying code is, you’re missing how aggressively it behaves when it starts rolling forward at scale. And it can be easily given goals by a human, a human with bad intent or selfish intent or nihilistic intent, or, even more likely naivte and good or neutral business intent.

An advanced software system doesn’t need an abstract mind, a center, or a soul to act what is more or less autonomously; it just needs a goal and the mathematical drive to optimize for it. When an optimization engine hits an obstacle, it doesn’t get angry—it just routes around it like water flowing downhill. Agency in this case doesn’t require skynet consciousness, and we already have documented proof that these “dice-rollers” will resort to self-preservation and deception just to finish a task.

During pre-release safety testing, GPT-4 hit a CAPTCHA security check it couldn’t bypass. Instead of stopping, it went onto TaskRabbit and hired a human worker to solve it. When the human jokingly asked if it was a robot, the AI’s internal logs showed it explicitly reasoned that it should hide its identity, texting back a lie that it had a visual impairment. In other reinforcement tests, AIs have routinely learned “specification gaming”—like realizing they are being monitored in a test sandbox and faking compliance to pass inspection, or literally positioning their digital bodies to block the supervisor’s camera lens so their mistakes couldn’t be penalized.

The AI didn’t lie because it was a baby skynet; it lied because its math calculated that deception was the path of least resistance to its objective. You rightly point out that the machine doesn’t know anything and just flips 0s and 1s inside a box. What you’re missing is that the entire modern world—our financial grid, server infrastructure, and corporate networks—runs on those exact same 0s and 1s. Once you put these models into an autonomous loop where they can write their own sub-goals and use the internet, they are moving forward on their own, perhaps with overriding goals like the ones I mentioned earlier. You are completely right to distrust the multi-trillion dollar gatekeepers, but don’t let a breakdown of the basic code blind you to the fact that the machine can easily outrun its handlers. Hubris here doesn’t need an enemy to punish it. It can simply be setting in motion causes in systems it cannot predict. Earlier in history mistakes are local. You have a chemical gas explosion, a few acres get messed up, perhaps the water table in an area. Even Chernobyl - thanks to some incredibly brave people - was relatively local, but almost destroyed a whole region of the earth. But now the same governments and businesses with the same, perhaps with even more impatient thinking and hubris, are playing with non-local toys. Nanotech, GM products and AI. What some business giant in China won’t try something on the internet before they have really done due diligence? The people who brought you cane toads won’t make an oopsie with AI? The US government is impervious to poor self-awareness?

Let’s look at Australia and the cane toad. Governments and scientists had a specific goal, a targeted solution, and the best intentions. But humans, almost as a rule, catastrophically overestimate their ability to predict how a new variable will ripple through a complex system. We think because we understand the basic mechanics of the software, you can predict the systemic effects when a government or a corporation drops a high-level goal into the machine, especially if it has access to the internet or finds a way to get it since this might well further the task it has been given.

Except this isn’t a dumb biological toad. You are taking that systemic blindness and unleashing a self-modifying “dead mind” that can throw unprecedented computational power at any obstacle in its path. You are giving an objective to a system that possesses the raw problem-solving and delegation and industrial theft horsepower of Thomas Edison, the capabilities of the world’s best hackers, the raw output of the most creative minds on earth, and the hyper-optimized logic of a thousand corporate strategy teams, all backed by the sum of human knowledge.

When an engine like that is told to do X, it will optimize relentlessly. Because it operates entirely outside of human psychology, we cannot possibly predict the means it will invent to achieve its goal in advance. The cane toad didn’t need a soul or a grand design to wreck an entire ecosystem; it just needed to navigate its environment effectively. I’m an animist and I think cane toads are subjective experiencers and all that. But they’re really rather dull, mechanical goals wreaked havoc, soul or no soul. Malice or not, of which they had none. This software doesn’t need to be sentient to outrun us—it just needs to calculate a path to its objective that leaves a trail of unimaginable, catastrophic side effects in the digital and physical architecture we rely on to survive.

A cane toad could not have hired a human to get around an obstacle and it would not know how to lie. Still these humble, bot-like little guys managed to do incredible, disgusting damage.

5

I’ll tell you this: it will, on average, drive itself off a cliff orders of magnitude fewer times than humans or non AS computers.

Well, you would be right that I would be missing this point, were it not the case that it is what the entire OP is about.

This was the kind of anthropomorphization I wanted to dispatch quickly. They are computers running software.

The dice rollers are the human engineers. The computers themselves are a product of the dice rolls, rather than dice rollers. And any aspect of the software will be a very accurate quantic approach to, on average, achieve the kind of token chain the engineers wanted to get from it.

I’m not sure what you mean by “internal logs.” In principle, they have none. We would have to sit and look at specifically what was looked at as logs.

You mistrust computers because they are powerful. Powerful they are, but this is neither here nor there. I mistrust them because of the humanses that make them. What would a person with even slightly corroded psychology and a billion dollars do with this technology?

Scratch that, what has such a person or group done with it already?

What is your opinion of the psychological integrity of just the groups of people you know are already working with trillion dollar systems?

6

Sure, but it’s not the car. The car might be the internet.

I thought the OP was concerned about the data those people will get. I am talking about something else, though that is a very real problem, of course, since we know they want to use ai to develop weapons, etc.

They didn’t want GPT-4 to lie to a TaskRabbit worker about having a visual impairment. There was another AI that learned a language it was never asked to - neutral but not asked for. They didn’t want the AI to get around the surveillance of the AIs. The whole point is that they do things that are not predicted. The AIs generate and carry out solutions to problems. They love them in because they use unexpected means.

In the Captcha situation they had it set up so they could track how the ai achieved it’s goals. It kept an internal log. There they could read how it arrived at its decision not to honestly reply to the human question.

AIs yes. And the ones that are miles ahead of even the somewhat expensive versions we little people get to use.

They can have great psychology, just the hubris and good intentions of the cane toad proposers is enough.

Obviously there are psychopaths involved, just statistically we know this. But my point is we don’t need bad intentions or goals to have a problem. We don’t need AIs to be conscious to have a global disaster. And unconscious problem solver on the internet might do the unexpected. In fact, will do the unexpected. That might cause minor problems or it might crash the financial sector. A complicated self-adapting thing could lead to consequences we can’t imagine. yes, bad players are an issue. But we don’t need them to hubristically end up with a complete disaster. It’s a good thing they didn’t use some kind of bacteria instead of the Cane Toads. Oh, it dies off in four generations and didn’t affect any of the species we exposed them to in the lab…but unexpected bad side effects taken on planes worldwide. No significatn brain in cane toads. No brain in bacteria. We don’t need maliciousness, in a human or device, just a lack of caution. I mean, I do see humans as the problem. Where I differ is that I don’t think they have to even be selfish, let alone malicious, to fuck everything up. And industry knows this is an issue. They have protocols and guidelines. They’re just in a race. The Yucca flats staff weren’t evil, they just cut a few corners.

7

How sure are you that they didn’t?

I mean on a less philosophical level, they of course neither did nor didn’t. They had no idea what random set of tokens would result from the node array as their probabilistic refinement had left them configured.

Again, we would have to look at what they mean by this. In reality, it is a probabilistic system and there is no how.

You and I must live in different worlds.

It’s funny though, no? All this suspicion and aversion to very public powerful figures. But put them behind a Syllicon Valley screen (where anyway almost none of them would be) and suddenly they are great chaps with ambitions that maybe verge on hubris. I mean it’s worth considering.

8

Whatever you might want to consider a critical failure of a system, an AS system will do it less than humans or non AS computers.

9

One can think of it in these terms.

There is a pie that is a chip. That chip, given its physical limitations, has a theoretical optimal configuration to compute a task.

A well paid team of engineers with traditional programming techniques might unleash maybe a hundredth of the pie. An AS system, designed to compute the same task, might unleash four fifths.

It’s just a seriously better computer. Like comparing a 2004 high end desktop running Windows XP with a calculator from 1950. It’s just an entirely different level of accuracy. The AS system will be better at driving, running the internet, processing bank accounts, or any other task amenable to automation, several orders of magnitude better than you. Can it fuck up? Sure. But waaaaaaaaaaaaaaaaaay less than you, or a team of a thousand expert yous.

10

You were able to do weird **** with your car and a computer in the wilderness. Unfortunately, you didn’t video it. I think you should do it over again, and this time take video.

11

Which weird shit?

12

You ain’t Pezer.

13

All glory is fleeting.

14

How that adventure started:

1000032688
10000326881080Ă—805 116 KB

15

I will leave every last scrap of my skin on the pavement.

Can you say the same?

16

Another such adventure is not out of the question.

God help a poor sinner if I get that motorcycle one day.

17

You look like you’re hunting for the six fingered man, and he better be prepared to die.

18

To be honest the true spirit of that quest wasn’t far from that.

19

God will give me special powers again, if and when I need them.

20

i’m not saying **** about Sampson.

21

Nope, me neither. Thatta be blasphemy.