In a landmark study, OpenAI researchers reveal that large language models will always produce plausible but false outputs, even with perfect data, due to fundamental statistical and computational limits.
I don’t know where I read it but sort of said that it to have that much information inside the models it was basically similar to a compression algorithm.
From logic, if we have a lossy compression then its mostly luck if the output is equal to the original. Sometimes it will tip one way and sometimes the other.
I don’t know where I read it but sort of said that it to have that much information inside the models it was basically similar to a compression algorithm.
From logic, if we have a lossy compression then its mostly luck if the output is equal to the original. Sometimes it will tip one way and sometimes the other.
With the caveat that there is no LLM where the “compression” is lossless on this analogy.