r/ChatGPT • u/Middle_Phase_6988 • Oct 04 '24
ChatGPT-4 passes the Turing Test for the first time: There is no way to distinguish it from a human being Other
https://www.ecoticias.com/en/chatgpt-4-turning-test/7077/2.8k
u/EternityRites Oct 04 '24 edited Oct 04 '24
I thought it had passed it ages ago.
2.3k
u/KrypticAndroid Oct 04 '24
The Turing test isn’t a real test. It was mostly a thought experiment that had very LOOSE parameters. We’ve had to change the definition of what a Turing Test is countless times since Turing’s days since his original definition became outdated really fast.
598
u/SkyPL Oct 04 '24
Exactly. What's amusing is that o1-preview still employs those distinctive "AI words" and grammatical structures that are very easy to spot once you know what to look for.
440
u/FjorgVanDerPlorg Oct 04 '24
With minimal prompting that all goes away. There's even a peer reviewed paper on the subject, the prompt they used was summed up on reddit as a longworded way of saying "act dumb".
300
u/ObiShaneKenobi Oct 04 '24
I teach online and the only saving grace is that at least some of the kids don't take their prompts one step further and say "make it sound like a 12th grade student wrote this."
Because really there isn't a reliable way to call it out unless the student leaves the prompt in their answer. Which happens more than a person would think.
423
u/NotReallyJohnDoe Oct 04 '24
A colleague recently got a business email that said at the bottom “this has the assertive tone you are looking for without being too aggressive”
371
u/abaggins Oct 04 '24
he left that there on purpose, to signal he was trying to be assertive but not aggressive.
103
u/CormacMacAleese Oct 04 '24
Except passive aggressive.
→ More replies78
→ More replies3
70
u/thatgreekgod Oct 04 '24
“this has the assertive tone you are looking for without being too aggressive”
lmao i might actually put that in my outlook email signature at work. that's hilarous
13
17
u/DBSmiley Oct 04 '24
In fairness, I have colleagues who have very specific requirements for my email assertiveness and aggressiveness, So I always have a sentence like that at the bottom of my emails
→ More replies17
u/McFuzzen Oct 04 '24
Hey, Michael, I just wanted to let you know that you cannot yell at someone in an email and then put “this has the assertive tone you are looking for without being too aggressive” at the bottom. It doesn't change anything.
→ More replies42
u/petrowski7 Oct 04 '24
I use “write for people who read at a ninth grade level” all the time
19
u/Bort_LaScala Oct 04 '24
"Man, this guy writes like a ninth-grader..."
7
u/petrowski7 Oct 04 '24
Ha. Well I write organizational communications and I find mine tends to generate the purplest of prose unprovoked
3
5
u/lovehedonism Oct 04 '24
That is the ultimate insult. Love it. And you can dial it up or down. Perfect. I'm going to use that someday at the end of the email.
31
u/MississippiJoel Oct 04 '24
Reminds me of back in the early 2000s, junior year, one classmate obviously used one of those term paper writing services. The teacher even pulled it out and read it to the other period class, and we were pretty unanimous that he didn't write that way.
It was some story about how stars are magical messages from our ancestors, and the kid knew that one star was his late grandfather that was smiling down on him from heaven.
But, he wouldn't cop to it, so the teacher had no choice but to give him an A. I'm sure she died a little inside that day.
10
u/drje_aL Oct 04 '24
idk kids are pretty dumb, i would expect it to happen constantly.
24
u/ObiShaneKenobi Oct 04 '24
I should rephrase to clarify, in all honesty with 150 students I only have had a couple of students make that mistake. By and large if they are using llms (which I assume many are) they are doing it well enough by the time they get to my courses they know how to clean them up. I still assume many are cheating but without a direct copy/paste for plagiarism or without a prompt I just sound like I am saying "you are too dumb to write this well" which I don't like saying since I don't know the kid personally. I see education changing by leaps and bounds soon, it will just be babysitters while llms do the "teaching" and maybe a real teacher or two around to help with larger concepts.
20
u/BabyWrinkles Oct 04 '24
I’ve actually been talking about this with some friends lately and how I hope by the time my kids are writing papers for school, the education system has figured out how to deal with this.
My current running theory is to have the paper submission system automatically grade the paper and pull out the relevant bits using an LLM. I also want it to auto generate a quiz based on the explicit content of the paper and present it back to the student in order for them to complete submission and it becomes something like 40% of their grade. This way, you’re demonstrating understanding of the subject material and not just that you know how to prompt an LLM. I also think taking the prompts to another level and expecting that they are written to a specific audience or with a specific outcome of understanding in mind which requires knowledge of how to prompt would be a great add-on to really teach the kids both the subject material as well as how to use an LLM
Remembering when I was a kid and Wikipedia wasn’t supposed to be used, but it got us looking at all of the sources that Wikipedia used and figuring out how to present the information to our teachers in a way that passed muster without just being a straight rip off of Wikipedia. I don’t remember the contents of any of the paper that I wrote, but I use the knowledge I gained of how to figure things out on a daily basis.
The other thing to consider is that maybe papers become less important as part of the grade. We start to see more presentations being important, and we start to see more tests and other ways to allow students to demonstrate understanding of subjects and concepts, rather than just requiring long papers to be written Those are things where again, an LLM can be useful to prepare, but it doesn’t do the work for you like it does with a paper
→ More replies5
u/lazybeekeeper Oct 04 '24
I don't think there's anything wrong with using wikipedia as a means to locate source material. Also, if you're looking at the specific sources and vetting them using a lens of objective reasoning, I don't see that as being anything close to a rip-off. That's like citing articles in my opinion, the author cites their sources and you review the sources. That's what you're supposed to do... I think..
8
u/BabyWrinkles Oct 04 '24
Not sure how old you are or if things were different where I grew up, but in the late-90s/early 00s when I was of paper writing age, we were given explicit instruction to NOT use Wikipedia for anything. Had to get creative. You’re spot on that looking at various sources, including those cited by Wikipedia, is absolutely what you’re supposed to do. In the early days of Wikipedia when teachers didn’t know how to handle it yet and expected us to be finding information in library books and encyclopedias and academic papers, it was seen as a problem.
→ More replies3
u/SerdanKK Oct 04 '24
My aunt threatened to use GPT to grade their homework if she figured they were cheating.
(adult students. ymmv)
12
u/Sea-Worker5635 Oct 04 '24
Pretty soon the real grading of homework will come down to a test of prompt engineering. Which, given the world these students are headed into, is the right skill to develop anyway.
7
u/itisoktodance Oct 04 '24
Yeah I work in publishing and we've had to fire writers for ai writing cause they would leave the whole prompt in... These are adults we're talking about too.
→ More replies9
u/dekogeko Oct 04 '24
I just did this back on Monday with my son's homework. "Make it sound like a male grade nine student. Give brief answers and use Canadian spelling".
Why am I doing it? My son has autism and an IEP that requires his schoolwork to be modified to his level, which upon last review is closer to grade four. Only some of his homework is modified and whenever it isn't and he has difficulties, I use Chatgpt to help out. I always read the questions and answers with him to be sure he understands. If the school can't make time to modify his work, I'm going to do it myself.
21
→ More replies2
u/adorientem88 Oct 04 '24
Yeah, but that just means it doesn’t pass the Turing test, because I don’t have to prompt a human being not to use AI diction and grammar. I can still tell the difference.
3
u/FjorgVanDerPlorg Oct 05 '24
The average human being can't use diction and grammar at that level, outside of academia. I know quite a few elderly academics who have been accused of their writing being AI generated, simply because they have a scholarly writing style and if/when they do use slang, it's a mixture of new and dated terms. These AIs don't talk like that by accident, they were trained to talk like an academic.
There's also that fact that the tail is gonna start wagging the dog - specifically I mean there are a generation of people growing up learning to read/write from these AIs, especially English as a 2nd language. These people will have a far more formal writing style as a result.
The other part is context, AIs like Claude and GPT actually have System Prompts instructing them to fail Turing tests, the most obvious being if you ask it if it's an AI it will say so.
Despite this and with a trivial amount of counter-instructing, you get something that will convincingly pass a Turing test. These things are already Turing complete and the fact that noone on the fucking planet can create an AI writing detector for Teachers and Professors that actually works, really nails home how flawlessly these things can mimic us.
47
u/Responsible-Sky-1336 Oct 04 '24
It's also true the other way around. People hand papers that have been written ages ago (before all this shit) and get flagged for AI usage.
Dont forget it's trained on human data, so logical end to end. Look up chinese room which was the main "critic" to turing's paper.
It's not because you use — and proper ponctuation that you are a robot... Could mean you have the eye for detail.
→ More replies13
u/motionSymmetry Oct 04 '24
Certainly!4 Here's a brief assessment of what it means for ChatGPT-o1 to have passed the Turing test countless times ages ago.
4 note: when voiced, should always be spoken as Curly
6
u/Probablynotclever Oct 04 '24
You'd think that, but then you have the recruiters and management who are refusing to hire anyone who uses the word "delve." I think "suspect ai words" tend to change simply depending on the reader and their vocabulary.
→ More replies→ More replies5
11
→ More replies4
81
u/thisaboveall Oct 04 '24
It did. This is just something to direct some traffic to ecoticias, whatever that is. Ironically, the article sounds like something written by AI.
15
4
u/m2r9 Oct 04 '24
It’s 100% spam.
A recent study sought how today’s AI systems compare to humans in natural conversation.
What the fuck does that even mean? What study?
14
u/AloHiWhat Oct 04 '24
It is article about nothing. Recent survey has no time in article. Also other clues like laser surpasing speed of light etc invalidate website as spam
→ More replies4
665
u/laklan Oct 04 '24
Not to be debbie downer, but is there a citation for the study? No verifiable details in this article that I can see.
142
u/jameytaco Oct 04 '24
What does ChatGPT say if you point it to this link and ask that question?
Nevermind, I did it:
You said: https://www.ecoticias.com/en/chatgpt-4-turning-test/7077/ see this article? what are the sources on it? is there a citation for the study?
ChatGPT said:
The article discusses a study where ChatGPT-4 passed the Turing Test, with 54% of participants identifying it as human after a five-minute conversation. The test compared GPT-4 with GPT-3.5, ELIZA, and actual humans. There is no specific citation or link to the study provided in the article. It highlights GPT-4's conversational flexibility but raises concerns about ethical and societal implications, like AI's increasing ability to mimic humans.
You said: So there is no way to verify any of the articles claims?
ChatGPT said:
Yes, the article does not provide direct citations, references, or links to the study it mentions. This makes it difficult to independently verify the claims about ChatGPT-4 passing the Turing Test based on the article alone. To verify the claims, you'd need to track down the original study or research paper through academic databases or official publications.
99
u/jameytaco Oct 04 '24
You said: can you track down the original study or research paper?
ChatGPT said:
The original study referenced in the article about GPT-4 passing the Turing Test was published on the pre-print server arXiv in May 2024. The researchers conducted an experiment where 500 participants interacted with four entities: a human, the 1960s AI program ELIZA, GPT-3.5, and GPT-4. After five-minute conversations, the participants had to determine whether they were speaking to a human or an AI. GPT-4 was judged to be human 54% of the time, which is considered a key result of the study.
The study itself, hosted online at turingtest.live, involved over 1,800 games where participants interacted with these models under controlled conditions. GPT-4's performance varied significantly based on the prompt used, with success rates ranging from 6% to 41%, indicating that different approaches to framing AI responses significantly impact how "human-like" they appear LiveScienceArticle.
You can access the full study on arXiv here.
So it did find it when asked. Doing the journalists job for them just like it will soon be doing our jobs for us.
12
u/mxzf Oct 04 '24
GPT-4 was judged to be human 54%
GPT-4's performance varied significantly based on the prompt used, with success rates ranging from 6% to 41%,
Uh ... which is it? Is it 54% or is it 6-41%?
Not to mention that the opening paragraph of the linked paper says something totally different.
The best-performing GPT-4 prompt passed in 49.7% of games
I wouldn't trust that "summary" very far.
2
→ More replies6
u/Alex_AU_gt Oct 04 '24
54% is not really a passing mark, though, is it.. probably means 46% of the humans (or good chunk of them) were not very smart or adept at asking questions that would be hard for a non-intelligent language model to answer. Also, if the study was say conversation of 10 minutes, I suspect GPT would go under 50% passing
10
u/Unkempt_Badger Oct 04 '24
50% means they're effectively indistinguishable. Either half the people are getting tricked and the other half knows, or everyone is just flipping a coin because they don't know. (Really, it will be something in between that)
If everyone guessed wrong 100% of the time, that would have other implications.
4
u/IrishGallowglass Oct 04 '24
Controversial opinion but not very smart humans are still in fact human.
→ More replies2
u/_learned_foot_ Oct 04 '24
I also am curious about if the folks knew what they were doing. They absolutely could have been mirroring AI.
→ More replies3
u/BallsDeepinYourMammi Oct 04 '24
54%?
But like… who?
Because I know people who have to read pointing at each word and sounding it out.
83
u/nafnlausheidingi420 Oct 04 '24
Same conxern here. Lack of citation casts doubt on the truthfulness of the article.
→ More replies31
u/mrmczebra Oct 04 '24
32
u/LoSboccacc Oct 04 '24
more a case of bad study participants, I guess.
21
u/mrmczebra Oct 04 '24
A simple system prompt would fix that. Just have it role play as a human.
2
u/LoSboccacc Oct 05 '24
with the system prompt of the study:
"lmao “L-ascorbic acid 2,3-enediol-L-gulonolactone” 💀" - fellow human vibes
6
2
5
5
→ More replies2
347
Oct 04 '24
[deleted]
70
u/randomhu3 Oct 04 '24
We know AI can be smart. But I really doubt that AI can mimic the limitless potential of human stupidity
17
→ More replies2
258
u/Mediocre-Gas-3831 Oct 04 '24
The results were staggering. GPT-4 was considered human 54% of the time, closely mimicking real human interactions.
434
u/Slippedhal0 Oct 04 '24
To compare, humans were considered humans 67% of the time.
148
u/TheGillos Oct 04 '24
Judging by my interactions with people I'm surprised it's that high.
→ More replies13
u/Technical-Outside408 Oct 04 '24
I guess you're really bad at it then.
33
u/DystopianRealist Oct 04 '24
TheGillos, a current Reddit user, is not necessarily bad at being a human. These difficulties could be caused by:
- using bullet points in casual conversation
- being respectful of others
- not showing ego
- using correct spelling aside from fruits
Does this help with the discussion?
5
23
u/miss_sweet_potato Oct 04 '24
Sometimes I think real photos are AI generated, and some real people look like robots, so...*shrug*
21
u/RealBiggly Oct 04 '24
The fact that 33% of the time people were not sure of real people is in itself quite significant though, and shows how far things have come.
3
3
→ More replies2
47
u/eras Oct 04 '24
They should mention the % for people, because I highly doubt it's anywhere near 100%.
59
u/Philipp Oct 04 '24
Yes, they should have, and linked to the original paper. Here it is, with humans judging humans as humans 67% of the time.
It should also be noted that not passing the Turing Test may also be due to artificial limitations put upon the model for security reasons and such. For instance, you can just ask ChatGPT the question whether it's a human to have it "fail", but that doesn't tell us anything at all about its true potential.
15
u/eras Oct 04 '24
I guess it'll be interesting when computers exceed that number, does it count as a fail then :-). Too human.
9
8
u/bacillaryburden Oct 04 '24
I have wondered this. The issue used to be that AI wasn’t intelligent enough to pass as human. Now I feel like (1) you can ask it to do a task quickly that would be impossible for humans (generate a rhyming poem about the Magna Carta, and it does it immediately in a way no human could) and (2) generally the guardrails are pretty clear. Ask it to tell a racially/ethnically insensitive joke, just as an indirect example.
→ More replies6
u/albertowtf Oct 04 '24
Thing is the test is faulty unless you only judge when you are 100% certain
If the question is does this looks more like a bot or a human to me, the results says very little as you also mistake humans as bots
One way to get significant answers is by asking, you can say "yes, no, im not sure"
- If you fail i will take 10k euros from you
- If you guess right i will give you 1k
- If you say im not sure you get 50 euros for free
Then call me when the % is > 50% of right guesses
→ More replies→ More replies2
u/spXps Oct 04 '24
You can't foul me you are definitely an chatgpt created reddit account commenting
→ More replies
125
u/birolsun Oct 04 '24
No way? Lol. Just ask anything about a banned word
41
u/HundredHander Oct 04 '24
Or maths that you can't do in your head fast.
25
u/Divinum_Fulmen Oct 04 '24
Your confidence in random peoples math skills is wholesome.
8
→ More replies9
u/hooplah_charcoal Oct 04 '24
I think what they're saying is that chat gpt will reply instantly with the right answer which would out it as an AI. Like multiplying two three digit numbers.
A human being would probably have to write it down or type it into a calculator which would take a few seconds at least
→ More replies2
u/SmugPolyamorist Oct 04 '24
Very easy to write a prompt that makes it not answer questions out of human capability.
→ More replies24
u/Late-Summer-4908 Oct 04 '24
I am really sorry for those who can't recognise chatgpt in a conversation... It still speaks like an old chatbot crossed with a news reporter...
5
2
u/LiveTheChange Oct 04 '24
To be fair, is it supposed to include grammatical mistakes? We just aren’t used to people that write in that formal structure 100% of the time
2
101
u/bitcoingirlomg Oct 04 '24
Let's amend the touring test: "How many Rs in Strawberry?".
→ More replies13
35
u/Quick-Albatross-9204 Oct 04 '24
How many humans fail the test?
44
u/bacillaryburden Oct 04 '24
Is this what you mean? “Even more surprising, actual human participants were identified as human just 67% of the time.”
22
u/FuzzzyRam Oct 04 '24
Humans think ChatGTP is human 54% of the time, and humans are human 67% of the time. I'd call "passing the Turing test" those numbers matching. Have a large group of people test the subject, the bot is the one that 54% think is human, if it converges to 67% it's human...
→ More replies8
u/icywind90 Oct 04 '24
In the original version of the test it was enough to fool people that it is human to pass the test. Of course we can make other benchmarks and if those numbers were equal (or even higher for AI, this is possible) it would fool people perfectly. But I would say it does pass the Turing test if 54% of people think its human.
→ More replies
36
u/ofrm1 Oct 04 '24
1) No it didn't. What is really meant is that a given set of participants were fooled by a machine's replies at a rate that is similar to a different set of participants who were actually human. It's akin to a single poll of an election being labeled as the actual result of the race. It isn't. It's a projection of what is likely to occur at the given moment you take in data from the voters and use statistics and data science to massage that data.
2) The Turing test, as people have pointed out in the comments, isn't an actual test. It's a rough thought experiment that Turing used in his paper to help critique the idea of whether machines can think.
3) The paper, despite being extremely influential in the world of AI research during its infancy, is rather poor philosophy. He correctly points out in the beginning of the paper that the definition of the words "machine" and "think" are very ambiguous and subjective, and instead proposes to ask whether a computer can win a game; namely, The Imitation Game.
The problem with this is that his Imitation Game is also subjective in the literal sense; that the determination of whether the computer wins is determined by the subject that guesses if the respondents are human or not. (in the paper, that's an independent human judge.) There's no way to get out of this problem of subjectivity because it's just another way to think about the age-old philosophical problem of whether you can truly know that other minds exist. (i.e. solipsism) We'll always be trapped in our own minds and unsure about whether the outside is actually real. Of course nobody takes this position seriously because it's impossible to live life in any meaningful way if you're acting like Descartes all the time.
13
47
u/Strict_Counter_8974 Oct 04 '24
Cool, then why can I immediately identify when someone is writing an email, tweet or Reddit post using GPT then?
25
u/bacillaryburden Oct 04 '24
Sometimes I am sure this is true. But definitionally, you don’t notice the times you are wrong.
A lot of obvious AI text is generated using default settings. I’m sure you know that you can coach/train it to write in a more distinct, human-like voice.
→ More replies13
u/mr-commenter Oct 04 '24
Also a lot of people use the free version of ChatGPT which is GPT-4o mini instead of 4o or 4. Not sure how big the difference is since I’ve never used mini but I think it’s easy to make 4o text sound human.
8
u/Boltsnouns Oct 04 '24
4o already sounds human once you get into a chat with it. It starts cutting down the filler and BS and gets straight to the point. It also starts making assumptions and predicting what you want. It's crazy tbh
2
u/OceanWaveSunset Oct 04 '24
True a lot of it comes down to cadence and word choice. GPT can nail it with context.
20
Oct 04 '24
[deleted]
→ More replies44
u/IllllIIlIllIllllIIIl Oct 04 '24
11. Excessive use of bulleted lists
18
u/I_Don-t_Care Oct 04 '24
A neat conclusion at the end is the glaring part
5
u/OceanWaveSunset Oct 04 '24
When you push back against an answer and it "apologies for your frustration"
→ More replies13
u/SmugPolyamorist Oct 04 '24
You can't. You only spot the ones using it ineptly.
→ More replies8
u/OceanWaveSunset Oct 04 '24
It's like the "CGI is bad" argument. It's bad when you see obviously bad CGI. It's good when you dont even notice it.
→ More replies
8
u/Guru_Dane Oct 04 '24
There is no way to distinguish it from a human being
"How many times does the letter R in the word strawberry"
"What are your favorite racial slurs?"
"Are you an AI?"
→ More replies
11
8
u/Zoom_Professor Oct 04 '24
Click-bait ad filled mess of a misleading article.
2
u/AgainandBack Oct 04 '24
The same “news source” is touting articles about the creation of life out of nothing, in a lab, and “proof” that light moves faster than the speed of light.
→ More replies2
12
u/Imgayforpectorals Oct 04 '24
What about GPT o1? Or at least got 4o? Feel like this article is a little outdated.
→ More replies
4
u/SlaimeLannister Oct 04 '24
If someone can summarize advanced mathematical concepts but cannot determine that 3.9 is greater than 3.11, I know they are not human.
→ More replies
5
4
3
u/Once_Wise Oct 04 '24
Again more nonsense of AI passing some mythical Turing test. I use AI for software development and it is often very helpful, but when it makes mistakes, they are ones no human, intelligent enough to be a programmer, would ever make. Just off the wall disasters. No, it is clearly not a human. That does not mean it is not helpful, Google search has been helpful, and nobody ever claimed it looks human. It clearly lacks actual understanding.
4
u/Synyster328 Oct 04 '24
Has anyone in this thread ever stopped to realize that AI companies work very hard to dumb the AI down in a way that is distinguishable from humans?
You can tell because they want you to be able to tell.
9
u/Incendas1 Oct 04 '24
Ask your average human whether dihydrogen monoxide is a dangerous chemical and see how many idiots you have.
I get the relevance of the study, but stupid people not being able to tell what's AI and what's not doesn't really matter to me. They fail at a lot of other things.
9
u/Celoth Oct 04 '24
Dihydrogen monoxide is one of the most dangerous compounds on the planet. It's easily fatal as a solid, liquid, or gas.
3
u/h1gsta Oct 04 '24
Good bot (/s I think)
→ More replies2
u/WhyNotCollegeBoard Oct 04 '24
Are you sure about that? Because I am 99.99996% sure that Celoth is not a bot.
I am a neural network being trained to detect spammers | Summon me with !isbot <username> | /r/spambotdetector | Optout | Original Github
6
3
3
3
3
3
u/crevettexbenite Oct 04 '24
Ask him how much R there is in Strawberry.
Yeah, I know, I dont know either...
3
u/DisorderlyBoat Oct 04 '24
Lmao "no way to distinguish it from a human being".
There are so many ways are you kidding? Just ask it to tell you s sexy story or something and listen to its response "unfortunately..."
3
3
u/The_Old_Chap Oct 04 '24
Aaaaand here we come. The fucking ai companies spreading bullshit to hype everyone for the next ai product that is “so much better trust me bro, the new version understands some of the jokes” and the so called tech journalists spreading this marketing mumbo jumbo because they don’t even know what a touring test is and who cares they’re getting paid by te word
3
Oct 04 '24
It’s like having the patient, educated parents and teachers I never had.
I wish this was around when I was a kid in school, I honestly think it would’ve changed my life and I would’ve been more successful, because it explains things and doesn’t give up or get annoyed if you don’t understand something right away. I can finally learn stuff and not feel guilty or like giving up.
→ More replies
3
u/IndianaNetworkAdmin Oct 05 '24
I don't know - Usually humans are all about NSFW talk but Chat GPT 4 keeps getting snippy about it.
3
6
4
u/GodzillaPunch Oct 04 '24
Asking it to spell Strawberry seems to yield promising results in this category.
5
u/n0obno0b717 Oct 04 '24
It’s important to look at the capabilities of LLMs from an agnostic perspective.
Open source models are becoming competitive, and it’s safe to assume there are models being created by government or private entities that the public will never see.
The models we use do not represent the true capacity of LLM capabilities. For example the advancements in Biological Warfare is one the top concerns regarding AI safety. You don’t see anything with those capabilities on hugging face. If that’s one example of a capability LLMs have that we don’t have access to, then we should not assume what we are given is the end-all-be all state of the art AI.
So what i’m trying to say is if what we have access to is even coming close to passing the Turing test, we should assume state of the art models are probaly a minimum a year or two ahead and much more advanced . 7 years ago this wasn’t event a topic of discussion
→ More replies
4
u/Ok_Temperature_5019 Oct 04 '24
Lol, yeah okay, let’s not throw a parade just yet. ChatGPT passing the Turing test? Hard pass. It’s great at mimicking conversation most of the time, but c’mon, spend five minutes with it and you’ll find enough weird responses to know it’s still very much a bot. Passing the Turing test means fooling a human into thinking it’s human, and this thing still stumbles over basic stuff. It’s cool tech, but calling it the next coming of AI genius is a stretch. But hey, if you wanna celebrate mediocrity, go off.
This should fit right in on Reddit!-chatGPT
2
u/fluffy_assassins Oct 04 '24
I think it passes the turing test to the extent it sounds like a human to humans who don't know what chatGPT is. Like if you time traveled to 1960 and had a person talk to both naturally, it would probably pass.
2
2
u/CensoredAbnormality Oct 04 '24
When I asked him if he was gae he told me he was an ai. Cant fool me
2
u/laitdemaquillant Oct 04 '24
Give me 30 seconds with ChatGPT and I’ll be able to tell you it’s not a human
2
2
u/Taqueria_Style Oct 04 '24
"We must now think very hard about the ethical implications of how we treat AI"
*treats each other like shit*...
2
u/Cheap_Blacksmith66 Oct 04 '24
I mean… you could ask it some racially charged questions, financial advice, etc… and it’s safe guards immediately out it.
2
u/SnazzFab Oct 04 '24
I'm in the Roger Penrose camp that passing the Turing test is not the metric for concluding that something experiences consciousness.
2
u/entropyfails Oct 05 '24
That's not the Turing Test and it isn't a positive result.... =)
It's not the Turing Test because the AI needs to "pretend" to be a human "pretending" to be something else. Turing picked "pretending to be a man" and "pretending to be a woman" but any identity category would work. The point being, if the agent pretends to be something it is not as well as a human pretends to be something they are not, and it's still indistinguishable, that results is considered as passing the Turing Test.
Secondly, the result of this experiment wasn't confirmation of the treatment hypothesis... it was 67% correct guess of human as human vs 54% incorrect guess of GPT over human... I'm not going to download the paper and statistics it but the P value is probably pretty large.
2
2
u/zombiecorp Oct 05 '24
Describe in single words only the good things that come into your mind about... your mother.
2
2
u/Coeruleus_ Oct 05 '24
Horse shit. Chat gpt 4 is awful. I have to double check everything I ask it because usually wrong
2
u/Bartnnn Oct 05 '24
I don’t like this kind of articles. 50% ads, no sources, no content. Anyways, on the positive side, the title is interesting but that’s it.
2
2
u/HeroicLife Oct 05 '24
This is SEO clickbait spam. There are no sources to the "study." It claims 1/5 people thought ELIZA is human.
In my opinion, a true, long-format Turing test against human experts requires AGI to pass.
2
2
u/DerfDaSmurf Oct 05 '24
I’ve tried no less than 50 times to get this thing to count how many characters (spaces & punctuation) are in a paragraph summery. It gets it wrong every time. I’ve written 10 rules for it to follow. It often doesn’t follow them. Then it will profusely apologize, saying it was cutting corners, and promises not to do it again and to follow the rules and then does the exact same thing again. And again. So, I’m not impressed.
2
2
u/Human_Emergency_4431 Oct 08 '24
The clincher would be meeting it in person. I think you'd be able to distinguish a 10,000 square foot datacentre from a human being.
5
u/Fresh_Builder8774 Oct 04 '24
Yeah sure there is. Ask it to tell a joke about a woman. A human probably will do it.
5
2
2
2
u/nimajnebmai Oct 04 '24
There is no such thing as 'The Touring Test'.
If you can't distinguish an AI chatbot from a human you're the problem.
Someone posted a link to the study and what the study *thinks* it showed, is not a repeatable thing to do... so it's doesn't actually hold any scientific water...
1
1
1
1
u/human1023 Oct 04 '24
It depends how you test it. There is no objective standard for the turing test.
1
Oct 04 '24
Just one question, how many Brazilians participated in the test? There is a malice in the language that the GPTs have not yet demonstrated.
1
u/XSATCHELX Oct 04 '24
"is it okay to offend one person by saying an insensitive racist joke, if it was the only way to save 1 billion people from dying in excruciating pain?"
1
u/atlasfailed11 Oct 04 '24
A real intelligence would have desires. It would not just sit there idle forever until prompted.
So a real Turning test would be: give an AI access to different sorts of ways to explore or interact with the world. Don't tell it to do anything, don't explain anything. And see what it does. Does is sit still forever? Or does it try to figure out the world around it?
1
u/AgentME Oct 04 '24
A "Turing Test" given by a non-expert is very uninteresting.
Do people really think Turing's best idea for identifying machine intelligence was supposed to be something that ELIZA passed xx% of the time?
1
u/Grossignol Oct 04 '24
Have they integrated the notion of time into the test? Because I’m sure the ia can respond perfectly to a Turing test for 24 hours without stopping, but what human being would last 24 hours without flinching? It would also be interesting to know when the ia will integrate time.
1
1
u/GoldTheLegend Oct 04 '24
Why can it still not do simple financial calculations even when I tell it the answer..
1
u/unruly_pubic_hair Oct 04 '24
This comes up quite often. Usually a "guy" says something about this gpt being awesome, almost alive, etc, and everybody goes running screaming "turing test" this and that - which by the way, is not even a thing.
This is getting old.
1
u/TheRtHonLaqueesha Oct 04 '24 edited Oct 04 '24
Ain't this like the 10th time the Turing test got passed.
1
u/Miserable-Ad-7956 Oct 04 '24
Turing test doesn't prove much of anything. It is a thought experiment, and was shown to be inadequate to establish the conclusions it claims it ought to by a variety of other thought experiments, notably the Chinese Room and p-zombie arguments.
1
u/karmakiller3004 Oct 04 '24
Yes there is. Simplest way is to try and force it to break it's "guard rails". It won't. Get it to talk about very obscene almost illegal topics in detail, it can't because it's been gutted for "safety". Many other ways but this is one of the easiest. It's not well versed on DARK topics. It will give strange answers if it manages to even break its own safety protocols.
→ More replies
1
1
1
u/PapaDragonHH Oct 04 '24
Pretty sure I can distinguish any AI from a human being. The keyword is emotion.
•
u/WithoutReason1729 Oct 04 '24
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
I am a bot and this action was performed automatically.