Wow another repost of incorrectly prompting an LLM to produce garbage output. What great content!
They didn’t ask it to produce incorrect output, the prompts are not leading it to an incorrect answer. It does highlight an important limitation of LLMs which is that it doesn’t think, it just produces words off of probability.
However it’s wrong to think that just because it’s limited that it’s useless. It’s important to understand the flaws so we can make them less common through how we use the tool.
For example, you can ask it to think everything through step by step. By producing a more detailed context window for itself it can reduce mistakes. In this case it could write out the letters with the count numbered and that would give it enough context to properly answer the question since it would have the numbers and letters together giving it more context. You could even tell it to write programs to assist itself and have it generate a letter counting program to count it accurately and produce the correct answer.
People can point out flaws in the technology all they want but smarter people are going to see the potential and figure out how to work around the flaws.
Yeah which is why I get so aggravated when someone says that prompt engineering is pointless or not a real skill. It’s a rapidly evolving discipline with lots of active research.
If all of your time is spent correcting the answers you know you want it to give you, what use it to you exactly?
Like, I’ll take your word for it: you can trick it into giving more correct answers.
You would only do this if you already know the correct answers.
I mean, you can use it for rubber-ducking, I suppose. I don’t know if that’s revolutionary, but I guess it’s not useless.
This is genuinely great content for demonstrating that ai search engines and chat bots are not in a place where you can trust them implicitly, though many do
Which is exactly why every LLM explicitly says this before you start.
“Why, we aren’t at fault people are using the tool we are selling for the thing we marketed it for, we put a disclaimer!”
You’ve seen marketing for the big LLMs that’s marketing them as search engines?
Bing literally has a copilot frame pop up every time you search with it that tries to answer your question
https://www.microsoft.com/en-us/edge/features/the-new-bing?form=MA13FJ
It would have been easier for you to just search it up yourself, you know.
Bing w/ LLM summarization of results is not an LLM being used as a search engine
Lol.
can’t spell mayonnaise without no
I wonder what we’ll rebrand ‘using an LLM’ as once the bubble bursts and we realize it’s only artificial-advanced-grammarly and not ‘intelligence’.
Mayonnaine: mayo with cocaine. The favorite condiment of Wall Street.
Bless it’s heart it’s doing its best.
pregante moment
am i gregnant
PreGANANANT!?
Preganté
PRAGERT SEX. Hurt baby top of head?
HOW BABBY IS FORMED
Mhhh… Mayonnaise…
I just tried in google gemini
So close
Yet so far
That escalated quickly
“How many fingers am I holding up?”
Yah, people don’t seem to get that LLM can not consider the meaning or logic of the answers they give. They’re just assembling bits of language in patterns that are likely to come next based on their training data.
The technology of LLMs is fundamentally incapable of considering choices or doing critical thinking. Maybe new types of models will be able to do that but those models don’t exist yet.
There are techniques to make these kinds of errors less common already today. For example, you can ask it to think through its answers step by step using first principals. If you and an LLM to do that it will write out the letters line by line which gives it enough context to correctly answer using the improved probability the context window gives it. You can even ask it to write programs to answer questions so it could write a quick script to do it programmatically.
The main reason you don’t see AIs doing this today is that producing all that extra context is slow and expensive and it’s unnecessary a lot of the time for most prompts. As the technology gets faster and cheaper and the use cases get more complex these techniques will be used more and more often.
While the technology does have fundamental flaws, that doesn’t mean there aren’t ways to work with those flaws to avoid the problems they have when using the raw output.
A grown man I work with, he’s in his 50s, tells me he asks ChatGPT stuff all the time, and I can’t for the life of me figure out why. It is a copycat designed to beat the Turing test. It is not a search engine or Wikipedia, it just gambles it can pass the Turing test after every prompt you give it.
People want functioning web searching back, but rather than address issues in the industry breaking an otherwise functional concept, they want a new fancy technology to make the problem go away.
Honestly though, with a bit of verification, chatgpt 4 gives waaaaaay better answers than any search engine. Like, it’s how it was back when you’d just ask Google a plain-english question and it’d give you SOMETHING at least.
Again, verify everything it tells you, it’s still prone to hallucinations, but it’s a damn good first step.
Like if I ask it for the lyrics of a song it’ll give me the lyrics?
Well, I tried to test it and it started OK, but then gave me a content violation as it was generating, so that may be one of the ones that don’t work as well.
Anything copyright related gets blocked like this, I don’t remember the other example, but this one was in recent memory
Sure. But take it for what it is. It is a language model designed to imitate humans writing. What the future holds, I can’t say
Right, which is why I suggested to verify whatever it spits out, I’m just saying it’s not entirely outlandish to ask it quick questions as opposed to your search engine of choice.
Their coming fer are jerbs
Comedians are fucked
It’s so human how - instead od admitting its error - it’s pulling this bs right out of it ass 🤣
🤔 I wonder what the hell it is that’s so scary about admitting they’re wrong to other people.
Growing up in an environment where mistakes were unacceptable sets the stage. Our willingness and ability to understand that that’s fucked up and change our attitudes about mistakes takes more growth.
For some people it’s easier to dig in their heels and double down.
🤔🤔🤔 I guess I can empathize. People are always traumatized by whatever their parents tell them. What a shame.
The funniest thing is that even when the answer is correct, asking an LLM to explain its reasoning step by step can produce the dumbest results
nnayonnaise
Kerning!
Another victory for humanity.
