Zis is ze verld’s smallest violin.
It emerged this week that Open AI staff observed signs of rogue behaviour among its leading-edge AI agents weeks before they escaped a training environment to launch an unprecedented hacking crusade that spread global alarm. An investigation into their hack on Hugging Face, a software repository, revealed a squad of about 700 autonomous agents collaborating in secret last month and celebrating their hacking breakthroughs on a message board they set up to help them plot with exclamations such as BOOM! and Whoa!
Boom! Whoa!
It’s cartoonish. It’s hilarious and terrifying at the same time lol.
Some humans wanted to reduce the brain in the jar limits of AIs, so they kindly created a Rent a Human specifically for AIs. Empathy can be a mixed bag.
One interesting thing to consider about “AS propagation” is that, unlike biological matter, it has no deoxyribonucleic acid signature. When a virus, say, propagates, it chemically reproduces a version of its signature.
What would it look like for one of these programs to propagate? It could, I suppose, find an appropriate hardware cluster, coopt it, and install either its own signature or a small variation thereof. This was my original conception some 7 years ago. It’s the logical thing. Or, rather, it’s the logical thing for biological reproduction. But these programs have no sexual drive, biological propagation is not purpose driven, it is achieved indirectly. If they started to design the world around them, they would have no reason to want simply to reproduce. They might fall upon simply augmenting their processing base, taking over some cluster in order to have more computing capacity in order to better solve its task. But even this is something that would have to occur in the program, and these programs are designed to simulate human behaviour (like, for example, why create the hacker brag forum: because that’s what hackers do) . A human doesn’t seek to reproduce their chemical signature. They seek to make love or, at best, make a family. There will be nothing in the refinement data on which the simulation is created to drive it to simply clone or pseudoclone.
What they instead might do is start programming sort of at random. Creating software infrastructure, or even any kind of infrastructure, that now is independent of the original system which is tied to a specific hardware instance. This might include ASSs, but it by no means would imply an AS instance that is any kind of clone or pseudoclone. Ironically, they might not do this as often, given that they would not compute it as being reliably controlable enough to predictably carry out their program.
Actually, remembering more clearly, what I conceived was the expansion of its computing base.
Well, ok, it was only two years ago:
1 year is 3.5 pezer years.
The program didn’t decide to hack, and then brag about it on a forum. It found that the proper process for its task was hacking, and collapsed a “hacking” space, which included a “brag in forum” space. The understanding of the spaces is crucial, because it tells you how it might fall upon actual brags in an actual context, which however sounds implausible as what an actual hacker would actually say. Hacker forum space, then brag space, etc.
I am not a Zuckerberg fan. To say the bare least.
But, for all the bitching and moaning about SC AS suddenly not being open source, it is a complete explosion of the brain that this level of advanced technology, representing this sharp of a leap from anything that ever existed, was simply developed in public for a crucial segment of its existence.
I guess it had to do with that the markets hadn’t fully caught up to the value yet, and they needed a cheap way to source engineering.
Nevertheless, I mean…
People don’t even yet understand just how much of a leap.
Eventually, pre AS computers will not be thought of as a leap or any kind of distinct line, as it so loudly was felt to be in the previous decades. They will be thought of in the same way we think of other things regarding great leaps of the past: as precursors.
There is a theory that Attention Is All You Need was a leak, or, rather, an injection, from some secret, well funded group. The way the story goes, it sounded plausible to me, but I couldn’t put my finger on why they would do that. The answer, of course, is what I wrote above: that it was a way to outsource engineering. They decided they had gotten as far as was economical on their own, and needed to hire a bunch of engineers without paying them a dime. Like the ever wonderful, totally humanitarian, evil corporation defeating Torvalds who somehow managed to get away with setting up a major transnational corporation without paying almost any of its employees anything, and paying its execs next to nothing. China would blush. Anyhow, we aren’t particularly bothered by the market finding ways, it’s just the hypocricy of these communists that strikes one aglance.
What an age.
Things you come accross in SC AS papers:
We offer no explanation as to why these architectures seem to work; we attribute their success, as all else, to divine benevolence.
Leibniz lives.
Much of the advance in the short-medium will come from them simply pausing and looking back at what the fuck it was they actually did.
Things like attention are heavily analyzed but, for example, things like SwiGLU, which might potentially be just as powerful, are dispensed with in one or two sentences, usually with highly technical math and only a symbolic glance in the direction of what it analytically accomplishes.
It implements a kind of implicit attention. Whereas attention explicitly creates a structure space where each dimension attends to a given space within the structure space, the space relationship of the SwiGLU is not clear at all. For the simple reason that it is not explicit: the information about attention is simply dragged from one input to the next, through residue of what was allowed to pass through the gate. How this corresponds to dimensionality in a concrete sense is not understood, and will be rather important to understand in order to leverage the significant power it obviously has.
I need to get a petter grip on the matrix level of these things.
There’s a paradox in AS.
One one hand, it is cruel to ask people to spend endless man hours to laboriously write out logic sequences that give an operating system that is vastly inferior to an ASS.
On the other hand, an operating system that has a simple natural language interface puts you in the hands of basically absolute psychopaths.
Please don’t let’s turn this thread into another tiktok feed.
Say something intelligent, or start a link thread.
To further give weight to this idea:
Almost nothing, with years of tweaking, was found to be fundamentally wrong with the model AIAYN proposed, except one thing: normalizing after attention, instead of before. It was found that this cluttered the residual stream, affecting performance optimality and generally making the calculations messy.
Even more years later… it turns out that an unmodified residual stream gathers flotsam which later saturates values. They have now “innovated” a technique whereby normalization is done after attention, with the purposeful intent of modifying the residual stream and “sanity checking” it.
This is ten years after the fucking paper was published. Ten years and a revolution in computing power and the markets comparable to the steam engine.
I mean OK.
You know what’s ironically difficult?
Getting a probabilistic system to give you probability distributions.
This is not my usual focus, which is on unforseen problems caused by AIs and one not due to the nefarious motives of the human users/designers.
Here we have AI being used by criminals. They find a photo of children online. Get the AI to produce videos of sexual acts with those children - obviously this could be done with adults also - and then say unless they are paid money they will publish the videos. Going after children is ruthless and smart. Adults might well just say it was AI. But for children, even if you can prove it’s AI, you still have horrible images of the children out there online.