Home → News → AIs Are Being Tested to See How Well They ­nderstand... → Full Text

AIs Are Being Tested to See How Well They ­nderstand Our Thoughts

By New Scientist

September 6, 2018

[article image]

University of California, Berkeley (UC Berkeley) researchers have created a set of tests to explore the progress of artificial intelligence (AI) in understanding the world.

Each test is comprised of a short paragraph describing a scenario involving people and a few questions for the AI to answer about it; these queries revolve around identifying first and second order beliefs.

UC Berkeley's Aida Nematzadeh produced 10,000 scenarios and associated questions that tested theory of mind via first and second order beliefs, and her team fed them to four AIs, none of which successfully passed (the pass level was 95%). The highest-scoring AI achieved a score of 94.3%.

The team found inserting a sentence with no relation to the tested scenarios could fool the AIs, whose scores declined by 5% to 20%. The implication was that the AIs were failing to properly grasp the meaning of the text.

From New Scientist
View Full Article


Abstracts Copyright © 2018 Information Inc., Bethesda, Maryland, USA


No entries found