In this issue of #InfocusAI, we will talk about the transcendence of generative models, the improvement of small LLMs, China’s AI leadership, and the invention of Japanese scientists that allowed robots to smile. And finally, we will dwell on the plans of Russian healthcare professionals to integrate neural networks into the fight against cancer.
AI-focused digest – News from the AI world
Issue 45, June 27 – July 11, 2024
Scientists have found out how to achieve transcendence of generative models
The scientists from Harvard, with the assistance of their colleagues from the University of California, Santa Barbara, Princeton, Apple, and Google DeepMind, have found a way to train generative models so that they surpass the human experts who generated data for their training. The scientists call this superiority transcendence. In their study, using the example of a model trained in playing chess, they experimentally confirmed that transcendence is enabled by an approach to selection of training data called low-temperature sampling. In fact, low-temperature sampling leads to noise reduction in data (i.e. reduced impact of erroneous chess moves), which increases the likelihood of choosing the most effective solutions (in the case of chess, these are moves). Due to this, the model can outperform those people on whose data it was trained, and theoretically this can be applied not only to chess. However, according to experts, it is probably too early to declare superhuman AGI. The transcendence in these scientific experiments arises by reducing the influence of human errors in training the model, which does not prove that the model can develop new abstract thinking and derive solutions that humans would be incapable of producing.
Meta AI* has found a way to improve performance of small LLMs
The researchers from Meta AI* (owned by Meta Platforms, a company recognized as extremist and banned in the Russian Federation) are working on large language models that can be deployed on smartphones and other mobile devices with limited computing resources. It deals with models with less than one billion parameters, whereas in LLMs like GPT-4, according to some estimates, there are about a trillion of them. In an article on the MobileLLM family of models being developed by Meta AI*, the scientists described some design features that positively influence the accuracy and performance of models with a small number of parameters. In short, it is about the fact that deeper neural networks, specifically transformers, demonstrate better quality than wide ones with the same number of weights. Additionally, MobileLLM uses a modification of the attention mechanism called Grouped-Query Attention, as well as the embedding sharing, and for specifically sequential layers. According to the scientists, this approach makes it possible to improve the accuracy of small models in a number of control tasks. To learn more, click here.
*Meta AI is owned by Meta Platforms, a company recognized as extremist and banned in the Russian Federation.
China takes the lead in terms of generative AI adoption
A recent survey by SAS and Coleman Parkes Research showed that China is ahead of the entire planet in terms of adoption of generative artificial intelligence technologies. This is according to Reuters. More than 80% of respondents from China said that their companies already use generative AI, whereas in the United States only 65% turned out to be such. The global average level of AI adoption is 54%. In total, more than 1,600 people participated in the survey. They are those who make decisions in industries such as banking, insurance, healthcare, telecommunications, manufacturing, retail, and energy. By the way, according to the UN, China also turned out to be the leader in the number of patents in the field of generative AI. The Chinese have 6 times more patent applications than the United States, the agency reports.
Japan scientists figured out how to attach skin to robot’s surface and let it to smile
Just a few words about the achievements in biohybrid robotics. Scientists from Japan have created a robot face covered with a skin equivalent and capable of smiling almost like a human. This was made possible by the way they found to attach the equivalent of skin to the robotic surfaces. The secret is to use perforation-type anchors that mimic the operation of human skin ligaments. A smile is just the beginning, scientists are going to “teach” robots other facial expressions. Details of the project and plans for development thereof can be found in this article in Cell Reports Physical Science.
Neural network to be involved in creation of personalized cancer drugs In Russia
Russian scientists are working on a neural network that will help create patient-specific drugs against cancer, RIA Novosti reported with reference to Alexander Ginzburg, the Director of the Gamaleya National Center of Epidemiology and Microbiology. AI will analyze the patient’s tumor and issue a kind of “blueprint” of a vaccine for its treatment, on the basis of which scientists will be able to develop a patient-specific drug. It is assumed that the creation of drugs applying this method will take about a week.
