This time in #InfocusAI, we will tell you how to reduce the cost of training for large ML models and how GPT-4 is changing the US labor market, the work of startups and the plans of Chinese technology giants. Also in this issue, you will learn about another project to search for alien life using AI.
AI focused digest – News from the AI world
Issue 14, 9-23 March 2023
Scientists find a way to reduce training costs for large ML models
Scientists from several leading American technological universities have teamed up to find a way to speed up, reduce the cost and “green up” training of large ML models. The fact is that large models are built on a neural network architecture called a transformer, and the larger the transformer, the better the results of the model. For this reason, companies are competing with each other, trying to train constantly growing transformers based on expanding data sets. And everyone is interested in reducing the time, monetary and environmental costs of this. One way to do this is to use previously trained smaller models. The scientists propose a method they call a learned Linear Growth Operator (LiGO or Linear Growth Operator), which increases the width and depth of a larger network from the parameters of a smaller network. ML is used to transform the set of input values, specifically the parameters of the smaller model, to the set of output values, the parameters of the larger model. This approach reduces computational costs of training by up to 50%. The results of its testing are set out in detail in this scientific article.
Chinese tech giants enter the race with GPT-4
The Chinese tech giants have stirred after OpenAI and Microsoft last week announced GPT-4, a large multimodal LLM capable of working with texts and images and vastly superior in ability to its previous version. Just a few days later, Baidu introduced its analogue of ChatGPT – Ernie Bot, showing its capabilities in such areas as literary creation, business writing, mathematics, Chinese language understanding and multimodal generation. So far, as Baidu Chairman and CEO Robin Lee admits, the Ernie model is not perfect, but the company is hoping for its rapid improvement as feedback from users arrives. Now Ernie Bot is available to a limited audience by invitation codes, but soon it is promised to be brought to the masses. Other Chinese tech heavyweights, such as Alibaba Group and JD, announced their intentions to enter the race with ChatGPT too. Details about how the Chinese market perceived the challenge of GPT-4 are exposed in the China Daily article.
LLMs will affect 80% of workers in the US
Just a week after OpenAI introduced GPT-4 and once again blew up the market, the company’s experts, in collaboration with scientists from the University of Pennsylvania, released an article on how this technology will change the labor market. They took the O*NET database of professions in the United States and analyzed the professions represented in it for compliance with the capabilities of GPT. Their findings are as follows: GPT can affect at least 10% of the tasks of approximately 80% of workers in the United States, and 19% of workers may face the fact that at least 50% of their tasks will be transformed. The influence of GPT in one way or another will spread to all industries and types of professions, especially this will affect specialists whose work is related to programming and text creation. And at greater risk, as reported by the researchers, are jobs with high wages. While the copywriter who prepared this issue of the digest is looking for a new job, you can read the OpenAI report on the impact of large ML models on the labor market at this link.
IT startups begin to use GPT-4 to reduce programming costs
Speaking of the impact of GPT-4 on programming… Motherboard reports that having noticed the advanced coding capabilities of this model, some startups have already begun to use it in order to spend less on programmers. In particular, the article cited a viral post of Joe Perkins, the founder of a startup that creates tools to help venture capitalists. He stated that he used GPT-4 to write code for five microservices of a new product, this took three hours and cost him $0.11. A professional developer estimated the cost of this work as £5K, and that it would take two weeks to complete it. The publication writes that the abilities of GPT-4 cause strong concern among computer science students and software developers, but a number of optimistic experts assure that generative models will not replace all programmers, but will be very useful for their training, increase productivity and contribute to even greater democratization of programming. We’ll have to wait and see…
AI to help find out whether there is life on Mars
An international group of scientists has created a neural network capable of identifying patterns of biosignatures and thus help in the search for life on other planets. The results of their experiment are published on the pages of the journal Nature Astronomy. To train the model, the scientists used data from an environmental survey of an area of 3 km² in the basin of the dried-up lake Salar de Pajonales in South America. Natural conditions there are almost Martian. The researchers collected more than 7700 images and 1150 samples and mapped photosynthetic microbes living in the salt domes, rocks and alabaster crystals that make up the surface of the area. Next, the ground-based sampling data and three-dimensional topographic mapping were combined with drone images of the study area. On this set, the AI model was trained to determine with what probability certain geological materials can contain biosignatures. Tests showed the effectiveness of the approach: with the help of AI, scientists were able to find biosignatures in 87.5% of cases, which is impressive, since with a random search, the success rate is less than 10%. Plus, it was possible to reduce the search area by 97%. It is expected that their development will contribute to the study of the presence of life on Mars, as well as on the satellites of Jupiter and Saturn.
