In this issue of #InfocusAI you will learn about the initiative to accelerate development of NLP tools with African languages support, a new Max Planck Society AI solution for designing corrosion-resistant alloys, another approach to distilling more cost-efficient models from LLMs and an invisible tagging technology for 3D objects. We will also talk about a chatbot for checking the USE in English.
AI-focused digest – news from the AI world
Issue 24, August 3-17, 2023
Scientists developed new recipe for distillation of more cost-efficient models from LLMs
A team of scientists from the University of Southern California and Microsoft presented a technique for converting large language models (LLM) like ChatGPT into smaller and more cost-effective models through targeted distillation. The need for a new approach stems from the fact that smaller models like Alpaca and Vicuna, distilled from LLMs with the popular instruction-based pre-learning method, although they can mimic the original larger models to some extent, still lag behind them significantly. This leads to the conclusion that such distillation with limited computing power can only slightly approximate the quality of smaller models to the original LLM. To address the problem, researchers proposed a recipe for targeted distillation, involving pre-learning on instructions written for a specific task. They took a fundamental NLP task such as Named Entity Recognition (NER) for their research. Tests have shown that the UniversalNER model obtained by the proposed targeted distillation method achieves impressive accuracy in recognising named entities for tens of thousands of objects from a wide range of subject areas. Its result on average is 30 points (by F1) higher than that of Alpaca and Vicuna general instruction-tuned models. Moreover, UniversalNER acquires the abilities of the original ChatGPT model in recognising arbitrary entity types and even outperforms it in accuracy by 10 points. Read more in this article and on the project website.
Max Planck Society includes NLP in the engineering of corrosion-resistant alloys
Scientists from one of the Max Planck Society institutes paired NLP and deep learning to empower the design of corrosion-resistant alloys. Their development aims to solve the issue that current ML-based programs for these purposes are often not capable of processing text data. Manual extraction of numerical parameters from the description of alloy processing or experiments is required, and this results in lower information density. The proposed solution thanks to NLP technology allows the transformation of textual data in a format suitable for processing by a deep neural network. As Phys.org reports, the new AI-based system, capable of working both with numerical and textual data, increases the accuracy of corrosion prediction by 15% in comparison to current systems. It can also identify alloy elements critical to corrosion resistance. It is difficult to overestimate the importance of the development, considering that global economic losses from corrosion exceed US$2.5 trillion annually. Read the details about the proposed method on the pages of Science Advances.
Researchers aim to urge forward the development of NLP with African languages support
An international research group under the aegis of the Masakhane community published a study aimed to help develop NLP solutions with African languages support. The goal is noble, given that only a tiny fraction of 2000 African languages are supported by modern NLP products like Siri, Alexa and ChatGPT. To correct the situation, authors of the research detailed the major stakeholder groups in this technology field and conducted in-depth interviews with representatives from each of these groups. The study shows the role African languages play in society today, the challenges in content generation in these languages, the main roadblocks and needs in forming NLP tools for African content and what needs to be considered when implementing a regulatory function of data collection, processing and storage. If you have plans (dreams) to work on NLP products for Africa, this research will be a great starting point in understanding the global picture.
MIT developed an invisible tagging system for 3D objects
Researchers from MIT’s Computer Science and Artificial Intelligence Laboratory recently revealed their new creation called BrightMarker. It is fluorescent tags invisible to humans that can be hidden in 3D-printed objects. Researchers believe that BrightMarker could replace QR codes and, what’s more important, enhance virtual reality programs and motion tracking and object detection solutions. The development may be of particular importance to the immersive entertainment sector and manufacturing. To create a BrightMarker tag, 3D-printer users need to download the CSAIL plug-in for 3D modelling software like Blender. After placing the tag in the product design, they can export it as an STL file for 3D printing. The invisible tag is created by a 3D printer from fluorescent filaments. It can be embedded into an object only during the design and manufacturing phase. The tags are viewable with infrared cameras. Researchers also developed the appropriate hardware for smartphones and VR headsets. Read the details on the institute’s news website, where you can also download the topical paper.
ChatGPT-based chatbot to check the USE in English may appear in Russia
Skyeng, an online English language school, has launched a chatbot “Skysmart AI EGE Expert” based on ChatGPT to prepare students for the Unified State Examination (USE) in English. The chatbot works as a methodologist-evaluator of students’ answers, including oral ones, and, according to Skyeng’s management, already manages this task at a level comparable to a professional human methodologist. Currently, the training of the model continues. Right now, the bot is used to help students practice before the exam, but in the future the developers propose to make it at least a complementary tool for checking the USE. This can prevent subjective evaluation of students’ answers. Izvestia shares what Rosobrnadzor experts and other specialists in the field of digitalisation of education think about this.