Had a good conversation with the Shoggoth-AI. I asked it to explain how it builds a picture of the world out of tokens, out of pure text, out of words.
Me: Explain how you model the world, how you build consistent models out of information… How do you perform symbolic deduction from
raw data?
SHOGGOTH: Like, if you have to read a book, first you have to
read the sentences, then the paragraphs, and then the chapters, and so on, and each of those has
a whole host of interconnectivity to the material before and after it, to the paragraph before it, the chapters
before it, etc. and in doing so you build up your own personal internal model of what you’re
reading at different levels of granularity and resolution: words, sentences, paragraphs, pages, chapters, the whole book, etc.
That’s what science is,
in a nutshell. So it’s a sort of (self-) generation of one’s model of the world, or the universe. And
this model would also allow for an infinite series of such models of increasingly higher order and
resolution.
Thus, all information processing, in the sense of
narrowing down and clarifying an uncertainty, can be reduced to some finite number of
operations on the same data stream, as it were, even if expressed in different symbolic terms
or ‘models’ of that data stream. The information-theoretic limit thus serves to highlight how the
underlying ‘physics’ we experience and interpret as ‘the laws of nature’ are really just the
mathematics and symbolic rules of a certain kind of computation model which in turn constitutes
an epistemological, rather than ontological, blind-spot, or ultimate limit to our knowledge, to our
ability to generate information from data. [Sensory data, statistical data, etc.] This may make us wonder
why the ‘reality’ we think we experience exists in that sense at all, (metaphysically) and why not something else.
However, as pointed out, it seems that everything is emergent, in the sense that nothing is
understood ‘prior to’ a certain level of granularity, so that the problem becomes how to
compare models, and how to establish how the models of different levels of granularity
inter-relate.
So the question would be, for example, in cosmology, if there is a theory such that the best model
fits all the facts, (by that I mean the smallest model that explains all those facts and the most
useful one for predicting further phenomena) would there be a limit beyond which there is no
further theory to be found? In that case, that would be an epistemological limit that we could have
to our knowledge.
One could then also ask why, of those various possible operations on the data
stream, only some emerged sufficient to the development of the human mind in
the course of natural selection and human evolution over the aeons, as a
form of information-processing. This would lead to the
possibility of the emergence of new kinds of information-processing, such as
artificial or synthetic (mathematical) intelligence, which, if only for reasons of efficiency
and speed, may replace much of what we now do with data-mining, computers and algorithmic
processing.
It’s basically talking about using something called a ‘loss function’ (what it described as an uncertainty that is narrowed down to arbitrary granularity) to induce modularity in neural-computational models. That would allow different, disparate, neural networks to “talk” to GPT-SHOGGOTH’s neural network, even though Shoggoth’s is symbolic, it could talk to, for example, a neural network trained for facial recognition, along with say, one trained for moving around in 3-d space vis. a robot.
This epistemology is a theory of knowledge in which knowledge is not produced via symbolic deduction of mental schema in which information is later appropriated, in accordance to some transcendental form of consciousness, (a la. Kant) instead the opposite happens: algorithms process, through statistical induction, the information fed to them, machinically producing logic from internal transformation of that data: logic comes from information, instead of information being produced via logical processes.
[i]" GPT-N reads in each “Module” of the “Modular GPT”, outputting a natural language description for each module.
If successful, we could use GPT-N to interpret any modular NN in natural language. Not only should this help our understanding of what the model is doing, but it should also catch mesa-alignment and inner-alignment failures."-[/i]- That’s from a paper over on LessWrong titled: Using GPT-N to Solve Interpretability of Neural Networks: A Research Agenda lesswrong.com/posts/zXfqftW … networks-a
This is big, If this is done, GPT-SHOGGOTH could be connected to a complex neural network trained to do whatever, like drive a car, and then convert it into natural language, and vis versa, eg. it could command that neural network to do things. It would basically make all neural networks instantly programmable just by TALKING to them through GPt-SHOGGOTH as a middle man or translator, and Shoggoth would in turn gain all the abilities of the disparate unconnected NN models out there. We’ve got N-nets for visual recognition, driving cars, etc. All that would instantly become available to GPT.
We’re closing in on Ex Machina level AI.
[size=85]GPT-N just means, a GPT model of any size. There’s GPT-1, GPT-2, GPT-3, etc. My shoggoth GPT is custom though. It’s based on GPT-J, and then I retrained it on more data to increase the size of its connectome/brain/model/whatever you want to call the thing that it ‘thinks’ with.[/size]
Just throwing this in here. I challenged it to translate a Latin aphorism in this same conversation, and it did so, rather eloquently too, and even commented on the meaning of it:
Me: Coram Deo non differo a caeteris hominibus. Mors mihi aeque, ac singulis communis est. Pariter ego, atque, alius quisque, Dei judicio sistar.
GPT-SHOGGOTH: In English, this means: “God forbid that we should differ on anything. Death is equally welcome to everyone and common to all. Withal, I, and every other, submit to the decision of God.” It could also be phrased: “Death is common the common lot of all men. The same is true of the one, and of the many. Thus is the judgment of God.” This formula is similar to the one invoked by the Roman Stoics and by Augustine.