Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- I wish I could get a model to state its assumptions.by gdiamos
- Models do not have assumptions. They have probabilities for what the next token should be. With enough context, in the context window and built into the model, that next token isn't completely random, it's correlated with something someone might choose to write.
But people write all kinds of crap manually. So far, the data people have been writing has tended to be denser around what people could agree on (there are many lies, but only one truth), so the model is more likely to go there.
If we start putting AI generated text into the training data, it's not clear what that means for the resulting model. It's already clear that some actors are trying to influence models by putting large amounts of content out there that agree with them.
Figuring out which content is safe to train from is the real problem for future model trainers.
by reichstein - > This resolved the most basic liar paradox, but not every diagonal attack, since not all functions on [0, 1] have fixed points. To make this work in general, we can for example allow only continuous functions on [0, 1] (which always have a fixed point by Brouwer’s fixed-point theorem). But that restriction comes at the cost of expressivity: "This sentence has truth score less than 0.5" is not a continuous function of the truth score of the sentence.
Ok, I get it. Either expressiveness or completeness, but the question arises: did mathematicians explore systems with limits on expressiveness? In a field of computer programming there is Rust with limited expressiveness that doesn't solve all the problems, but still makes things much simpler. How about a mathematics with limited expressiveness and some unsafe blocks here and there?
by ordu - Of course this has been explored: https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_... The bar is rather low, however (like Robinson arithmetic). Basically, you’d have to forgo integer arithmetics with multiplication.by layer8
- H: All right, all right... and the limitation of human intelligence?
M: Mr Gödel is telling you that the theorem limits formal systems. So you see your intelligence as a formal system, as a machine?
H (pompously): Indeed. I have the impression that everything I do ought to be done by a machine, which could moreover speak just as well in my place.
G: From where I am, it is difficult to tell whether you exist or whether you are the virtual creation of a GAT - a Generator of Automatic Truisms. Intelligence does not exist without error, perhaps even without obstinacy in error; but who would take the risk of giving a computer that kind of psychology? As for the incompleteness theorem, it certainly did not foresee bad-tempered theories...
---
Gödel's Theorem, or an Evening with Mr Homais Jean-Yves Girard
Translation: https://files.catbox.moe/kac0wu.pdf
Original: https://perso.ens-lyon.fr/pierre.lescanne/ENSEIGNEMENT/LOGIQ...
by Xmd5a - Someone at MIRI must know how to solve this. Good luck getting them to tell you how!by Vecr
- For context: Logical Induction https://arxiv.org/abs/1609.03543
Sadly the line of research seems to have been abandoned. Kind of understandable with the ascent of LLMs: there is no time left for a multi-decade research program.
- > this sentence has no proof
As a programmer I was never impressed in such paradoxes. For me it was kinda obvious that in any sufficiently complex language you can create eqivalent of buggy infinite loop/recursion.
by scotty79 - The issue is that you can’t generally determine whether a statement is “buggy” in that way, because under the assumption that you could, you can construct another paradox.
Regarding programming languages, it means that for any programming language with a non-trivial type system, either there will be correctly typed programs that however fails the type checker, or there will be ill-typed programs that cause the type checker to run into an infinite loop.
by layer8 - A direction that is 99.99% accurate survives this argument completely. For all practical purposes one does not need totality.by ziofill
- Ironically, some users took the “all” in my first post too literally. It should have read “for most practical purposes”by ziofill
- A basic course in statistics will inform you of why a 99.99% accurate test should be looked at with skepticism when diagnosing a rare disease. Yet we see the fancy 9s and think somehow this many 9s is enough.by voxl
- For a one time coin flip, sure. For a lot of them, depending on the stakes, that can be very, very wrong.
https://www.google.com/search?q=why+99.99+accuracy+is+not+en...
It doesn't take a lot of thought experiments to realize this. Imagine if every bite of food we eat had a 0.01% chance to turn into something instantly lethal in our mouth. Average lifespans would be reduced measurably, and apart from anxiety, we'd develop all sorts of strategies and laws around that. E.g. absolutely NO eating for airplane pilots. You wouldn't go on a date to have dinner, dancing and sex, you'd go dancing and have sex, and then have breakfast. People would modify their jaws and stomachs so they could eat less, but bigger chunks of food. It would be a whole thing!
And that's not even talking about water changing on us, or a tiny chance of getting sucked into the toilet whenever we use it, and a lot of other things where going from damn near 100% to 99.99% would change everything for the worse, by so much.
by customguy - > Second, some people do really believe in a kind of platonic representation space that all models converge on, and that represents the “true” state of the world. If truth is indeed an objective part of the world, then you might expect such a universal truth direction to emerge as models get better. This
This really sounds like all those scifi stories with people trying to find god in the computer. How could it be there, in the jumbled mirror of internet scraped texts.
by tomaskafka - Easy! It is... emergent!by abc123abc123
- I've heard some weird definitions of God. I always thought God was supposed to be a giant man sitting in the clouds, but some people apparently equate God with the meaning of life, or human civilization, or morality, or the sum of all human knowledge or experience.by inigyou
- Aren't there definitions of Truth that are not the negation of Falsehood? Can't there be a function True(x) that is not equal to !False(x)? Can't there be a third function Paradox(x) such that these counterexamples can be considered paradoxes and therefore outside of the truth?
I'm admittedly not a logician and don't formally study paradoxes, but I never quite understood the whole category of "this sentence is false" paradoxes, I feel that the sentence is paradox and therefore it's neither false nor true.
I do agree that it's a truth vector sounds like a silly panacea fantasy, though. But more logical formality is not the counter argument that would convince me of it, rather I believe that there's less formal and rigorous ways to get closer to truth.
by TZubiri - Fuzzy logic?by cgio
- There are a LOT of different logical system.
Paraconsistent Logic
Intuitionistic Logic
Dialetheism
by UltraSane - You run into the same problems when you have True(x) = NOT (False(x) OR Paradox(x)). You can define NotTrue(x) := False(x) OR Paradox(x), and then you have True(x) = NOT NotTrue(x), which one can build the usual diagonalizations on.by layer8
- I’m not a logician either but believe this is what Tarski’s definition of truth solves for. In order to make a statement about the statement itself, you have to introduce a new meta language. Then a statement in the meta language is only true if the underlying statement is true.
Much more rigorous explanation: https://plato.stanford.edu/entries/tarski-truth/
- There is no lack of more elaborate logics:by roywiggins
- Interesting point about "self-referential sentences". I tend to agree. In my view a sentence saying something like "This sentence ..." does not have valid semantic meaning. It says nothing because, what "This" in "This sentence" means is ill-defined.
If terms we use are not well-defined, then sentences using such terms can not have meaning.
But for the sake of argument let's explore, what could the "this" in (so called) "self-referential" sentences refer to?
Do they refer to a specific encoding of the sentence you are reading, as some bits in computer memory perhaps?
That would require that those bits somehow have a unique "identity" and the "this" in a self-referential sentence would have to refer to those bits in specific addresses of a specific memory-chip.
But of course the "this" does not specify which memory chip, which specific (concrete) encoding of its (purported) meaning it is referring to. And if it did, then it would be talking about that specific set of bits in that specific memory-chip, not of "itself".
The fallacy is that what we perceive as a "sentence we read" is somehow "speaking" of something. But no, the sentence is not a subject, a sentence can not speak, and THEREFORE it specifically can not speak of itself.
A sentence can not speak, only actors, only subjects, like humans and AI, can "speak". And their speech must be encoded in some physical medium. A written sentence like "This sentence is ..." gives us the false impressions that somehow the SENTENCE IS SPEAKING of itself!
But speech can not speak, speech is the product of speaking.
Hence, in my view, "self-referential sentences" do not have any meaning and whatever paradoxes they might seem to create are results of confusion between ontological levels of "Subject" vs. "Speech".
by galaxyLogic - Title is a bit clickbaitish, but the content is well worth reading - came in with my pitchfork ready and left agreeing with basically all of it, with questions like ‘what if the probe could return 3 dimensions: truthfulness, knowledge confidence and decidability?’
Also the observation that people treat LLMs like oracles when they’re everything but is spot on, something I’ve also been thinking about and it’s quite a bit scary.
by baq - > what if the probe could return 3 dimensions: truthfulness, knowledge confidence and decidability?
can't be done because the other two dimensions depend on the first. or in other words it's all the same dimension only with a different name.
by lstodd - Fun, though as hinted at the end, the point of LLM "truth" probes is to measure the model's internal judgment of truthfulness. There's no reason this judgment, even if measured with 100% accuracy, couldn't be mistaken or logically inconsistent.by aesthesia
- I think this article pushes the premise farther than is reasonable.
The best anyone expects from an LLM "truth vector" is that it would encode the model's belief about whether the statement is true. Of course a perfect truth oracle is impossible.
by Legend2440 - > The best anyone expects from an LLM "truth vector" is that it would encode the model's belief about whether the statement is true.
I think even that's too-optimistic: The LLM is a document-extender, so its "belief" is whether a token seems like it would statistically fit-next in a partial document, based on prior documents. This is usually not the kind of analytic truth we're interested in, and we've already figured out how to constantly extract it.
If we peek at vectors and weights, we'll we'll probably end up measuring the moods and styles for whatever tokens are about to get emitted next, whether that's dialogue for a fictional character (of various kinds), a narrator, or an impersonal memo conclusion paragraph. We'll be measuring "earnestness and conviction", on the same level as "loquaciousness" or "pleading" or "talking like a pirate."
So is Truthiness [0] what we really want? Probably not. If our document described the character as Yoda, then The Force connecting all existence ends up truthy. Using "a really gullible person" can repeat anything you supply as truthy. Even if we set things up as "a respected encyclopedia article" or "a relentlessly logical super-genius", we're really changing the influence mix of styles and biases, rather than creating a logical mind independent of text inside the LLM.
by Terr_ - If anything, the whole vector space is the LLM's truth.by TZubiri
- Not only that, but nobody really cares about whether truth vectors can work correctly under carefully constructed paradox edge cases. Well, maybe mathematicians do, but nobody else.by mike_hearn