This issue of #InfocusAI will tell you about the nature of the theory of mind in large language models, the ability of artificial intelligence to moderate online forums discussions and current achievements in disease diagnostics technologies. You’ll also learn about ChatGPT performance in the exam for doctor. Let’s go!
AI-focused digest – News from the AI world
Issue 12, 9-22 February 2023
Theory of mind can spontaneously evolve in large language models
Stanford University professor Michal Kosinski has tested large language models on their ability to “impute” unobservable mental states, such as thoughts and desires, to others. Better known as “theory of mind” or ToM, this ability plays an important role in human social interactions, communication, self-consciousness, and morality. Kosinski applied classic ToM tasks to several language models without any examples or prior training. Models published before 2022 show virtually no ability to solve ToM tasks. The January 2022 version of GPT-3 showed a performance comparable with that of seven-year-old children, and its November 2022 version reached a performance level comparable with nine-year-olds. The development of ToM-like abilities of artificial intelligence will have a positive impact on safety and efficiency. For example, if self-driving cars better “understand” the intentions of pedestrians and drivers, they will be able to avoid traffic incidents. However, the scientist believes that the theory of mind as an ability does not necessarily need to be purposefully built into AI systems while they are being developed. It can emerge spontaneously as a “byproduct” when training models to other tasks. Particularly in LLMs, this may be a consequence of improving language skills. Details are available in the preprint.
AI helps to conduct healthy online discussions
Scientists from Cornell have developed an AI tool that will help participants to conduct more productive online discussions even on the most acute topics without offending opponents. The tool called ConvoWizard is a browser extension which is based on a deep neural network. The model is trained on an array of data from the Change My View forum, where conscientious discussions are held on heated political, economic and cultural issues. Testing of the tool also took place in this forum. ConvoWizard essentially determines how a given comment will affect a conversation. It notifies the discussion participants if tensions escalate. It can also analyze the upcoming answers in real time and inform the users how their responses may affect the dialogue. More than half of convoWizard’s test participants said that thanks to these tips, they did not publish comments that they would later regret. Details of the project can be found on the university’s news website.
Artificial intelligence fights myopia in Singapore
Researchers from the Singapore National Eye Centre (SNEC) and the Singapore Eye Research Institute (SERI) have decided to adopt AI to combat severe forms of myopia (near-sightedness). This disease is a real problem for Singapore, where 65% of children become myopic by the sixth grade of school. Usually, as the scientists report, myopia progresses in children between the age of 5 and 15, and the earlier the disease is detected, the more chances to prevent its development to a severe form. They designed an AI-tool that, using deep learning algorithms, processes retinal images and clinical data of young patients (6-12 years old), and evaluates the risk of developing high myopia in their older age. The researchers hope that early diagnosis of the disease will reduce the incidence of severe forms of myopia among Singaporeans and reduce the risks of complications such as glaucoma, cataracts, myopic maculopathy and retinal detachment. Currently, the tool is being tested in a real world setting. More information about the tool and the problem of myopia in Singapore you can find here.
Intelligent system to help in Parkinson’s diagnosis in China
Artificial intelligence will now assist Chinese neurologists in diagnosing Parkinson’s disease, according to Parkinson’s News Today. The China National Medical Products Administration has approved for use the MoDAS system, which combines machine learning and computer vision to assess motor symptoms of this disorder and aid in making clinical decisions. MoDAS does not require the use of any wearable motion sensors and the participation of a doctor for medical examination. It needs only video that can be recorded on a smartphone by the patients themselves, guided by the instructions of the system. Next, the tool will analyze the video for motor dysfunction and the severity of symptoms, and then make a conclusion about the presence / absence of the disease or its development. According to the test results, the system copes with the diagnosis no worse than experienced doctors.
ChatGPT almost passed exam for doctor
OpenAI’s ChatGPT came close to passing the medical licensing exam in the US. An international team of researchers tested the chatbot’s medical knowledge using the standard USMLE exam, which all medical students pass to obtain permission to practice medicine. The exam consists of three parts. Usually, it takes about 300-400 hours of preparation to successfully pass it, but ChatGPT did not undergo any training. Nevertheless, its results are impressive. The chatbot scored from 52.4% to 75% in various parts of the exam, while the passing score was 60%. In 88.9% of the responses, the AI made meaningful observations, which the researchers rated as “non-obvious” and “clinically sound.” The numerical data of the experiment, the methodology and the results are described in detail in the PLOS Digital Health journal.