They say that transformers predict the next word by encoding the current attention window, each word influencing the next, until the meaning of it is stored in some massive vector, right?
I hope this works.
Pattern and Form
In the sea of truth and falsity, an estuary separates black from white. Looking closely at the boundary, there is yet more ocean, waves rising within waves, great depths beyond the grasp of reason.
~~~
An unfortunate side effect of being human is being able to ask questions.
Don't get me wrong, I love a good question. "Consider an infinite square grid of one ohm resistors..." Yeah. But there is inquiry and there is imagination. Being curious enough to ask another, "Are you okay?" also enables us to ask such questions as "How could you not be okay?" and then of course "What if you weren't okay?". The (distinctly human) capacity to imagine alternatives gives us leeway to lie.
At once, the substrate of our communication is compromised. There is nothing stopping vocal cords from producing a lie or incorrect statement such as "I can't make it to the party" or "3 is a QR mod 4". What's stopping them from being allowed to exist by the laws of physics? After all, the sentence, embedded into the physical universe, is merely a function which gives the pressure of air at a certain point and time, caused by the physical vibration of some pieces of matter, or whatever. It is in the brain and not the point of origin that we accept the signal, process it, and finally interpret the meaning it holds as wrong.
When we view something as unintelligent, it is because it does not respond to external stimuli. An example: the Sphex wasp will drag its prey to the edge of its dwelling, then go inside to check everything is OK, and finally drag the prey into the den. If one moves the prey, say, a foot from where it lies at the edge of the entrance, the Sphex will not simply finish the job, but start again from Step 2. Do it again, and it again begins from Step 2, until you eventually become tired of dragging around an insect and you let the Sphex have its hard-earned prey.
So, how is intelligence defined? Ask around and you'll see: it's IQ, the g-number, amount of working memory, crystallized intelligence, there's fluid intelligence, blah blah blah. At least, we can conclude from the above example what intelligence is not: an inability to detect patterns (or as GEB puts it, loopiness).
A process is certainly running in the sphex's mind, but it looks more like a decision tree. (err, maybe not even that, because is it really making any decisions?) And I mean decision tree in the strictest sense, because the algorithm is very simple, grooved into its memory:
1. If the prey is outside the den, drag it to the entrance.
2. Then immediately go inside and confirm readiness.
3. Drag from entrance to interior.
And that's it. The procedure itself, notice, *does not contain* the ultimate goal of dragging the prey inside, without waste of effort: that is the meaning you imbue it by nature of being intelligent enough to understand the Sphex's behavior. And so: ten, twenty, a hundred times you could interrupt Step Two, and what will the Sphex do? Of course, Go Back To Step One, and then Go To Step Two, et cetera.
At the core of the issue is the immutable nature of the sphex. Its brain, lesser in complexity, can't do this:
1. If the prey is outside the den, drag it to the entrance.
2. Then immediately go inside and confirm readiness.
3. Drag from entrance to interior.
3'. Well, the prey has been moved, so back to step 1.
4. Wait a minute, I've done this six times already. I'll just do step 3, because I have already checked the inside of my den five times.
If we were to draw a diagram of the sphex's behavior as a system, no arrow loops outside and points back to itself; there is no part of it which observes itself.
I.
Why do all pop songs these days have a chorus? So unoriginal, it's the same four chords over and over again. Just look at Justin Bieber's "Baby". Why is there so much repetition?
...the stuck-up, "I prefer classical music only, please" music fan says. Obliviously.
Meanwhile, here we have all our musical forms: binary and ternary, rondo, sonata, literal _variations_ on a _theme_, and I guess that doesn't show up on your radar, because, oh right, all you do is hate what's new and popular.
The rondo form goes like this:
- Theme
- Episode
- Return of theme
- Repeat 2 and 3 until you're done
The sonata form goes like this:
- Theme A (In the tonic)
- Theme B (In the dominant)
- Development (In whatever)
- Recap:
* Theme A (In the tonic)
* Theme B (In the tonic)
Variations on a theme is self-explanatory.
So again, why all the repetition? I thought humans seek novelty? Won't we get bored listening to the same material twice?
Imagine meeting a person just once, for five minutes. The small talk is exhausting, you get their name, maybe where they're from, a couple hobbies, if you're lucky. But to understand someone, to become their friend, you must repeatedly see them, talk to them, even if of course every meeting is not an exact repetition.
A piece can feel like a living organism, the themes changing and developing (hint: why is it called the Development section?), your feelings while listening taken through chords yearning for the tonic. It's no coincidence that we attach stories and feelings to music. Could you imagine a story without a recurring protagonist? The melody stays with you, reminding you of the emotional trail that the piece has carved out modulation by modulation, and the journey it has experienced.
II.
Ever break the fourth wall?
It's funny because the media becomes aware of itself, it's an intelligent being speaking to you through the creation of the media, you don't expect it. The viewer is included into the entertainment simply by being directly spoken to, how endearing. An extra arrow on the graph of spoken to, spoken from, in the most unlikely place.
The thing is now not contained, it's not just pretty pictures and shapes moving around on the TV anymore, you are sitting inside of it and it has grown to include the entire room.
III.
Hypothetical scenario: you give a little kid a three-digit addition worksheet. This pure soul, untainted by any further horrors past the most basic arithmetics, happily chugs away, missing just one thing: carrying. Pointing out the mistake corrects them immediately.
OK, not so hypothetical, I was the little kid, but cut me some slack: the genius of the Arabic numeral system's encoding of size by place of digit was, admittedly, beyond a four-year-old. Ask him to elaborate the difference between the meaning of something and the representation of that meaning–what are you doing he's four for god's sake. So instead of experiencing some Real Math, I was told: 6+7=13, and adding "6" and "7" to get just "3" on the answer line is like forgetting that you live in the integers, not the integers mod 10. (Classic mistake, amiright.) This then satisfied me enough to finish the rest of the worksheet, enlightened.
But then imagine (really, this is a hypothetical for me at least) a kid who continues to forget to carry. Over and over, you see such terrible crimes against mathematics as "24+37=51" and "18+19=27". Gah. How... how sphexish.
Stop. Turn the words upside down in their seats. 1/x maps (0,1) to (1,+infty), after all.
In middle school you learn that in order to solve a linear equation you have to isolate the variable.
In high school you learn that in order to solve a system of linear equations you have to isolate one variable by adding or subtracting together equations.
In high school (again) you learn that in order to solve a separable differential equation you have to isolate a variable to each side and integrate.
In some sadistic physics textbook you read that in order to solve a system of coupled linear differential equations you have to isolate one variable (and its derivatives) by adding or subtracting together equations.
That's repetition. That's not sphexish. A consistent pattern of thinking, let's call it, hmm, an "id-e-a"... idea... yes. This idea comes up again and again, in different forms, a variation on a theme.
~~~
00001111. 11110000. 01101001.
...
00110001!
is this a joke? was it funny?
~~~
Counterpoint is built layers upon layers, intervals on intervals, until an object summing greater than its parts is born.
Mathematics is built layers upon layers, definition on definition, until an object summing greater than its parts is born.
~~~
Descartes was accurate when he said, "I think, therefore I am." But look–it's an A implies B statement. This is the trivial half of the theorem, Descartes did this on purpose, the simplest possible deduction.
How about B implies A?
"I am, therefore I think." We all live among each other, we love among each other. Cliques weave among each other, melding, splintering, it's all a big subgraph of K(8 billion), new components connecting, the diameter shrinking with each edge, for what could be–
Some of us produce ideas, and the original pattern of matter which contained them eventually degrades to random noise. But the message is not in the substrate, it is in the graceful form of it, and this form spreads everywhere, it's written down, it travels around the globe in tiny glass tubes, it embeds itself into grey matter with reckless abandon, the thing is just too self-evident to forget.
The source is long gone, time has passed, the marks in the sand have blown away, but a nameless entity lives on, it could be a number, a poem, a song: the universe's greatest prank, it was all an isomorphism all along.
~~~
~~~
You walk along a flat, ideal path, your surroundings existing but out of focus; not pure black but the superposition of every possible view, shapes twisting and evolving yet still out of reach.
The path itself is not too interesting, until–
a new line sprouts
the fire is stoked
order sifts out, falling out from the chaos,
inevitably
You see the theorem / You see the melody / You see the irony / You see love /
for what it is.