Overcoming Goodhart’s Law and New AI Achievements

fgfg Picture

Artificial intelligence builds a space-time world model, tracks underwater moving objects, predicts earthquakes and analyses people’s sentiments in 164 countries through Twitter (now – X). The world is creating more and more advanced versions of AI but has yet to resolve the question of how to measure its abilities against human ones. Read all about that in the new issue of #InfocusAI.

AI-focused digest – news from the AI world

Issue 27, September 28 – October 12, 2023 

Scientists proved that LLM can build a space-time world model

Researchers at MIT have found evidence that large language models can not only analyse statistical data, but also build a coherent model of the data generation process, i.e., a model of the world. While working with the Llama-2 family of models, scientists discovered that LLMs learn linear representations of space and time across multiple scales, and these representations are robust to variations and unified across different entity types. Beyond that, space and time neurons encoding spatial and temporal coordinates were identified. This is truly groundbreaking. Find more information about the study in this article

New hypotheses, if confirmed, could help overcome the Goodhart’s law in AGI testing

The journal “Scientific Letters of the Institute of Psychology of the Russian Academy of Sciences” published an article raising the problem of the failure of current approaches to AI testing. When it comes to evaluating AI’s ability to mimic human intelligence, the world seems to face what is known as Goodhart’s law, when a measure has become a target. This means that AI simply is taught to pass human similarity tests, causing them to lose value. As a result, we have the following situation: everyone is striving to create Artificial General Intelligence (AGI) level systems capable of performing any intelligent task that humanity can (or cannot) imagine, while lacking understanding how to determine that AI has reached that level. The article suggests three interrelated hypotheses (based on the works of W. Ashby, S. Kauffman, A. Roli, and J. Jaeger), that, if true, could solve the problem of Goodhart’s law for AI:

1. Humans have the intellectual capability to identify and actualise the affordances of an adjacent possible (which is not a computable task due to the uncertainty of the latter).

2. This capability is provided by the mechanism of the serendipity to invention, which is based on the multidimensional abilities of people to perform specific cognitive (primarily analytical and temporal) and extra-cognitive (social network) actions.

3. It may be evident from the incomputability of the serendipity mechanism that it is an act of incomprehensible, direct “seeing”, “comprehension”, and this act of cognition coincides with the act that generates reality (the operation of such an incomputable mechanism is most likely based on the laws of non-classical physics).

The author states that if these hypotheses are confirmed, no tests will be needed, as the AI ability to detect and actualise the affordances of an adjacent possible is enough to assert that it has reached the AGI level. Read more details and interpretation of unfamiliar terms in the article

Deep reinforcement learning helps tracking underwater moving targets

An international research group, led by the Barcelona Institute of Marine Sciences, has proved the benefit of deep reinforcement learning to facilitate the task of exploring the ocean depths using underwater autonomous vehicles. In particular, through a series of field experiments at sea, scientists have confirmed that systems based on this learning method are able to optimally track a moving target underwater, which is not a simple task at a depth of, say, 4 km. And this is really valuable for marine researchers – both in terms of tracking fauna and monitoring climate change. Learn more on the website of the Polytechnic University of Catalonia, which took part in the project, and in the Science article

MIT and Harvard presented a sentiment dataset based on Twitter posts

In the recent issue of Nature, MIT and Harvard told the academic world about the Twitter Sentiment Geographical Index Dataset – a geographic index of moods on Twitter (now – X). It is a database of people’s sentiments built from posts on this social network (4.3 billion geotagged tweets) from 2019, run through ML sentiment analysis algorithms. According to the authors, this is the most extensive Twitter sentiment resource to date – covering 164 countries. Through a specially designed platform, researchers can view sentiment indices by region and over different time periods. At the very beginning of the article, the authors talk about this database as a resource for assessing people’s subjective well-being, which is essential for designing sustainable development programmes. Ah, if only it was used solely for that…

AI from Texas predicts earthquakes with a 70% accuracy

The Economic Times reports that an AI-based earthquake prediction system developed by Texas University at Austin “foresaw” 70% of earthquakes in China a week in advance. Its testing took seven months. During that period, AI pre-alerted 14 earthquakes with near-accurate determination of their magnitude and location within 320 kilometres. It missed only one earthquake and gave false predictions eight times. A large-scale set of historical seismic activity data and current seismic data was used to train the predictive AI tool. Although researchers emphasise that they are still far from being able to predict earthquakes anywhere in the world, they optimistically state that the test results show that it is possible to achieve the goal which was previously thought to be unattainable. 

Latest Articles
See more
AI Trends
Transcendence of AI Models and Smile of Robot
AI Trends
LLMs’ Visual Knowledge and Dangers of Specification Gaming
MTS AI and K2Tech to Implement GenAI in Large Companies
AI Trends
AI Law and Bad Grade of ChatGPT for Programming Questions
AI Trends
AI Assistants of the Future and Smart Antivirus from Russia
MTS AI partners with Softline Group
AI Trends
Holodeck with GPT-4 and the New Atlas
AI Trends
Musk’s Predictions and Competition in the AI Field
AI Trends
Sora in Hollywood and AI for Big Football
Team news
MTS AI Signs the Declaration on Responsible Development of GenAI