I did this as two separated conversations exactly to avoid the “context” window. It shows that the LLM in question (ChatGPT 3.5, as provided by DDG) has the information necessary to correctly output the second answer, but lacks the reasoning to do so.
If I did this as a single conversation, it would only prove that it has a “context” window.
So if I asked you something at two different times in your life, the first time you knew the answer, and the second time you had forgotten our first conversation, that proves you are not a reasoning intelligence?
Seems kind of disingenuous to say “the key to reasoning is memory”, then set up a scenario where an AI has no memory to prove it can’t reason.
So if I asked you something at two different times in your life, the first time you knew the answer, and the second time you had forgotten our first conversation, that proves you are not a reasoning intelligence?
You’re anthropomorphising it as if it was something able to “forget” information, like humans do. It isn’t - the info is either present or absent in the model, period.
But let us pretend that it is able to “forget” info. Even then, those two prompts were not sent on meaningfully “different times” of the LLM’s “life” [SIC]; one was sent a few seconds after another, in the same conversation.
And this test can be repeated over and over and over if you want, in different prompt orders, to show that your implicit claim is bollocks. The failure to answer the second question is not a matter of the model “forgetting” things, but of being unable to handle the information to reach a logic conclusion.
I’ll link again this paper because it shows that this was already studied.
Seems kind of disingenuous to say “the key to reasoning is memory”
The one being at least disingenuous here is you, not me. More specifically:
In no moment I said or even implied that the key to reasoning is memory; don’t be a liar claiming otherwise.
Your whole comment boils down a fallacy called false equivalence.
I was referring to you and your memory in that statement comparing you to an it. Are you not something to be anthropomorphed?
|But let us pretend that it is able to “forget” info.
That is literally all computers do all day. Read info. Write info. Override info. Don’t need to pretend a computer can do something they has been doing for the last 50 years.
|Those two prompts were not sent on meaningfully “different times”
If you started up two minecraft games with different seeds, but “at the exact same time”, you would get two different world generations. Meaningfully “different times” is based on the random seed, not chronological distance. I dare say that is close to anthropomorphing AI to think it would remember something a few seconds ago because that is how humans work.
|And this test can be repeated over and over and over if you want
|I’ll link again this paper because it shows that this was already studied.
I was referring to you and your memory in that statement comparing you to an it. Are you not something to be anthropomorphed?
I’m clearly saying that you’re anthropomorphising the model with the comparison. This is blatantly obvious for anyone with at least basic reading comprehension. Unlike you, apparently.
That is literally all computers do all day. Read info. Write info. Override info. Don’t need to pretend a computer can do something they has been doing for the last 50 years.
Yeah, the data in my SSD “magically” disappears. The SSD forgets it! Hallelujah, my SSD is sentient! Praise Jesus. Same deal with my RAM, that’s why this comment was never sent - Tsukuyomi got rid of the contents of my RAM! (Do I need a /s tag here?)
…on a more serious take, no, the relevant piece of info is not being overwritten, as you can still retrieve it through further prompts in newer chats. Your “argument” is a sorry excuse of a Chewbacca defence and odds are that even you know it.
If you started up two minecraft games with different seeds, but “at the exact same time”, you would get two different world generations. Meaningfully “different times” is based on the random seed, not chronological distance.
This is not a matter of seed, period. Stop being disingenuous.
I dare say that is close to anthropomorphing AI to think it would remember something a few seconds ago because that is how humans work.
So you actually got what “anthropomorphisation” referred to, even if pretending otherwise. You could at least try to not be so obviously disingenuous, you know. That said, the bullshit here was already addressed above.
|And this test can be repeated over and over and over if you want
[insert picture of the muppet “testing” the AI, through multiple prompts within the same conversation]
Congratulations. You just “proved” that there’s a “context” window. And nothing else. 🤦
Think a bit on why I inserted the two prompts in two different chats with the same bot. The point here is not to show that the bloody LLM has a “context” window dammit. The ability to use a “context” window does not show reasoning, it shows the ability to feed tokens from the earlier prompts+outputs as “context” back into the newer output.
You linked to a year old paper [SIC] showing that it already is getting the A->B, B->A thing right 30% of the time.
Wow, we’re in September 2024 already? Perhaps May 2025? (The first version of the paper is eight months old, not “a yurrr old lol lmao”. And the current revision is three days old. Don’t be a liar.)
Also showing this shit “30% of the time” shows inability to operate logically on those sentences. “Perhaps” not surprisingly, it’s simply doing what LLMs do: it does not reason dammit, it matches token patterns.
You clearly couldn’t be arsed to read the source that yourself shared, right? Do it. Here is what the source that you linked says:
The Reversal Curse has several implications: // Logical Reasoning Failure: It highlights a fundamental limitation in LLMs’ ability to perform basic logical deduction.
Logical Reasoning Weaknesses: LLMs appear to struggle with basic logical deduction.
You just shot your own foot dammit. It is not contradicting what I am saying. It confirms what I said over and over, that you’re trying to brush off through stupidity, lack of basic reading comprehension, a diarrhoea of fallacies, and the likes:
LLMs show awful logical reasoning.
At this rate, the only thing that you’re proving is that Brandolini’s Law is real.
While I’m still happy to discuss with other people across this thread, regardless of agreement or disagreement, I’m not further wasting my time with you. Please go be a dead weight elsewhere.
Yes, it is getting late and I don’t have time to discuss someone who reads three sentences into a paper proposing a fix and shouts “ah ha, the author agrees with me that there is a problem! You are an idiot, good day sir!”
You successfully attack me without refuting anything I said, so congratulations and good night.
This is blatantly obvious for anyone with at least basic reading comprehension. Unlike you, apparently
So if I’m understanding you correctly:
“Philosophical mental masturbation” = bad
“Personal attacks because someone disagreed with you” = perfectly fine
I did this as two separated conversations exactly to avoid the “context” window. It shows that the LLM in question (ChatGPT 3.5, as provided by DDG) has the information necessary to correctly output the second answer, but lacks the reasoning to do so.
If I did this as a single conversation, it would only prove that it has a “context” window.
So if I asked you something at two different times in your life, the first time you knew the answer, and the second time you had forgotten our first conversation, that proves you are not a reasoning intelligence?
Seems kind of disingenuous to say “the key to reasoning is memory”, then set up a scenario where an AI has no memory to prove it can’t reason.
You’re anthropomorphising it as if it was something able to “forget” information, like humans do. It isn’t - the info is either present or absent in the model, period.
But let us pretend that it is able to “forget” info. Even then, those two prompts were not sent on meaningfully “different times” of the LLM’s “life” [SIC]; one was sent a few seconds after another, in the same conversation.
And this test can be repeated over and over and over if you want, in different prompt orders, to show that your implicit claim is bollocks. The failure to answer the second question is not a matter of the model “forgetting” things, but of being unable to handle the information to reach a logic conclusion.
I’ll link again this paper because it shows that this was already studied.
The one being at least disingenuous here is you, not me. More specifically:
You just said they were different conversations to avoid the context window.
|You’re anthropomorphising it
I was referring to you and your memory in that statement comparing you to an it. Are you not something to be anthropomorphed?
|But let us pretend that it is able to “forget” info.
That is literally all computers do all day. Read info. Write info. Override info. Don’t need to pretend a computer can do something they has been doing for the last 50 years.
|Those two prompts were not sent on meaningfully “different times”
If you started up two minecraft games with different seeds, but “at the exact same time”, you would get two different world generations. Meaningfully “different times” is based on the random seed, not chronological distance. I dare say that is close to anthropomorphing AI to think it would remember something a few seconds ago because that is how humans work.
|And this test can be repeated over and over and over if you want
|I’ll link again this paper because it shows that this was already studied.
You linked to a year old paper showing that it already is getting the A->B, B->A thing right 30% of the time. Technology marches on, this was just what I was able to find with a simple google search
|In no moment I said or even implied that the key to reasoning is memory
|LLMs show awful logical reasoning … Thus they’re unable to hold knowledge.
Oh, my bad. Got A->B B->A backwards. You said since they can’t reason, they have no memory.
I’m clearly saying that you’re anthropomorphising the model with the comparison. This is blatantly obvious for anyone with at least basic reading comprehension. Unlike you, apparently.
Yeah, the data in my SSD “magically” disappears. The SSD forgets it! Hallelujah, my SSD is sentient! Praise Jesus. Same deal with my RAM, that’s why this comment was never sent - Tsukuyomi got rid of the contents of my RAM! (Do I need a /s tag here?)
…on a more serious take, no, the relevant piece of info is not being overwritten, as you can still retrieve it through further prompts in newer chats. Your “argument” is a sorry excuse of a Chewbacca defence and odds are that even you know it.
This is not a matter of seed, period. Stop being disingenuous.
So you actually got what “anthropomorphisation” referred to, even if pretending otherwise. You could at least try to not be so obviously disingenuous, you know. That said, the bullshit here was already addressed above.
Congratulations. You just “proved” that there’s a “context” window. And nothing else. 🤦
Think a bit on why I inserted the two prompts in two different chats with the same bot. The point here is not to show that the bloody LLM has a “context” window dammit. The ability to use a “context” window does not show reasoning, it shows the ability to feed tokens from the earlier prompts+outputs as “context” back into the newer output.
Wow, we’re in September 2024 already? Perhaps May 2025? (The first version of the paper is eight months old, not “a yurrr old lol lmao”. And the current revision is three days old. Don’t be a liar.)
Also showing this shit “30% of the time” shows inability to operate logically on those sentences. “Perhaps” not surprisingly, it’s simply doing what LLMs do: it does not reason dammit, it matches token patterns.
You clearly couldn’t be arsed to read the source that yourself shared, right? Do it. Here is what the source that you linked says:
You just shot your own foot dammit. It is not contradicting what I am saying. It confirms what I said over and over, that you’re trying to brush off through stupidity, lack of basic reading comprehension, a diarrhoea of fallacies, and the likes:
LLMs show awful logical reasoning.
At this rate, the only thing that you’re proving is that Brandolini’s Law is real.
While I’m still happy to discuss with other people across this thread, regardless of agreement or disagreement, I’m not further wasting my time with you. Please go be a dead weight elsewhere.
Yes, it is getting late and I don’t have time to discuss someone who reads three sentences into a paper proposing a fix and shouts “ah ha, the author agrees with me that there is a problem! You are an idiot, good day sir!”
You successfully attack me without refuting anything I said, so congratulations and good night.
So if I’m understanding you correctly:
“Philosophical mental masturbation” = bad
“Personal attacks because someone disagreed with you” = perfectly fine