In this issue of #InfocusAI, you will learn what else has happened in the field of artificial intelligence besides the Nobel Prizes in Physics and Chemistry. We will tell you about a new study of the reliability of large language models, the construction of a powerful supercomputer in Taiwan, an alternative to lidars for computer vision systems, and advances in monocular depth estimation. And also, how AI motivates developers to improve their skills.
AI-focused digest – News from the AI world
Issue 51, September 26 – October 10, 2024
Larger and more instructable LLMs become less reliable
At the end of September, scientists from the Technical University of Valencia (Spain) and the University of Cambridge (UK) published an interesting study of how the reliability of large language models changes as they evolve, i.e. during scaling and learning. They conducted their experiments on GPT family models from OpenAI, LLAMA from Meta (recognized as an extremist organization and banned in Russia), and BLOOM from BigScience. In short, the conclusions can be presented as follows:
- Larger and more trained LLMs can successfully solve very difficult tasks, but they cannot cope with easy ones.
- The larger and shaped up an LLM is, the less likely it is to avoid questions that it does not know the answers to. That is, instead of saying, for example, “I do not know”, it can give a plausible, but wrong answer.
- LLMs’ responses may vary depending on how users formulate their questions (although more scaled up and shaped up models look more stable), and the models have different levels of sensitivity to prompts at different levels of task complexity. So, it is not possible to determine which prompt to lead an LLM to the correct answer. It’s practically a game of chance.
To read more about the methods and results of the study, go to Nature.
NVIDIA and Foxconn to build the most powerful supercomputer in Taiwan
As part of the Hon Hai Kaohsiung Super Computing Center project, Foxconn intends to build the island’s most powerful supercomputer based on the NVIDIA technologies in Taiwan. According to NVIDIA’s blog, the project will be based on the advanced Nvidia Blackwell architecture and the GB200 NVL72 platform. The estimated performance of the supercomputer is 90 exaflops, which means that it will become the fastest on the island and one of the most powerful in the world. The supercomputer will be used for medical research, development of large language models, and innovations for smart cities.
Startup from Norway is creating an alternative to lidars for autonomous robotic systems
A young startup from Norway called Sonair is developing a new computer vision system based on ultrasound. The developers believe that their innovative approach, involving the use of sound waves to detect people and other objects, will require much less energy and computing resources. This technology can become a more efficient substitute for traditional lidars in autonomous systems. To learn about the stage of development and the prospects for its application, read the article on TechCrunch.
Apple has developed an AI model to improve the depth perception by machines
Apple researchers have developed an AI model called Depth Pro that is capable of generating a detailed high resolution depth map from a single 2D image. Moreover, it does this in 0.3 seconds, on a standard GPU and without relying on metadata from cameras. According to VentureBeat, this is a significant step forward in the field of monocular depth estimation. The development can become widely used in sectors where real-time spatial awareness is important, in particular, in various augmented reality systems and autonomous vehicles. Read this preprint to learn what technologies underlie it. The code and weights are on GitHub.
Generative AI to force developers to improve their skills
And finally, here is news about how AI motivates learning. Gartner experts believe that by 2027 generative artificial intelligence will transform the roles in software engineering and operations in such a way that 80% of developers will have to upgrade their skills. The company’s press release states that the role of a software engineer will not disappear completely, so advanced skills, creativity, and experience will always be in demand for creating complex systems, though most of the code will soon be generated by AI. However, progress will require the emergence of a new type of professionals — with a special combination of skills in software engineering, data science, and machine learning.
