AI failed miserably! GPT-4 only scored 15 points on a question that a human scored 92 points on... | Technology Weekly

Compiled by Zhou Shuyi

Does the "miracle anti-cancer drug" cause cancer? FDA has launched an investigation

On November 28, the U.S. Food and Drug Administration (FDA) issued an announcement stating that it had received reports showing that a small number of patients developed new T-cell lymphoma after receiving chimeric antigen receptor T cell (CAR-T) therapy, and was investigating the "serious risk" that CAR-T therapy may induce malignant tumors.

CAR-T therapy is known as the "magic anti-cancer drug." It genetically modifies T cells, introduces genes encoding chimeric antigen receptors (CARs), and infuses them back into the patient's body. The modified T cells can specifically target and kill tumor cells. As of the 28th, the FDA's FEARS database lists 12 cases of adverse reactions to the above-mentioned new malignant tumors, and some patients have been hospitalized or died as a result. According to the announcement, the six CAR-T therapies targeting BCMA or CD19 currently approved for marketing in the United States all contain potential risks. Among them, Carvykti (targeting BCMA), Tecartus and Yescarta (targeting CD19) have been approved for marketing in my country. The FDA stated that in terms of approved uses, these therapies still have more benefits than disadvantages overall, and the FDA is evaluating the need for regulatory action. The announcement recommends that patients receiving these therapies and clinical trial subjects should be monitored for new malignant tumors throughout their lives. (FDA)

Planet formation theory faces challenge: small stars can drive large planets

According to the classical theory of planet formation, it is difficult for high-mass planets to form around low-mass stars, but the latest research challenges this understanding. Researchers published a paper in Science on November 30, saying that they discovered an exoplanet LHS-3154b, which is 13 times more massive than the Earth, while the mass of the star LHS-3154 it orbits is only 11% of the Sun's. The planet-star mass ratio is more than 100 times that of the Earth-Sun, far exceeding the limit predicted by current theories.

Planets are born in protoplanetary disks that orbit young stars and are filled with gas and dust. The size of the protoplanetary disk determines the mass of the born planet. Researchers used the Habitable Zone Planet Finder (HPF) to detect the Doppler shift caused by LHS-3154b through the radial velocity method. Observations show that the planet is very close to its host star, with an orbital period of only 3.7 days. Current planet formation theories (core accretion theory and gravitational instability mechanism) cannot explain the high planet-star mass ratio observed this time (>3.5×10^(-3)). Simulation calculations of nuclear accretion show that the existence of LHS-3154b can only be explained if the mass of the protoplanetary disk is more than 10 times that expected. (Science)

Materials science ushered in a revolutionary breakthrough! AI predicted the structures of 2.2 million inorganic materials

From electronic products to photovoltaic power generation, materials can be seen everywhere in modern life. However, synthesizing new materials is not easy, and it requires years of trial and error and a large number of hard-working graduate students. On November 30, the Google DeepMind team published two papers in Nature, which are expected to change this situation: they used AI tools to predict the structure and properties of 2.2 million inorganic materials, of which nearly 400,000 can exist stably; the research team also used AI and robotics technology to achieve fully automatic synthesis of inorganic materials.

The Materials Project is the most comprehensive online open source database in the field of materials. Researchers used 48,000 data from the Materials Project to train a deep learning model called Graph Networks for Materials Exploration (GNoME). Through active learning and multiple iterations of training, GNoME successfully predicted 2.2 million possible inorganic compounds. Formation energy calculations showed that 381,000 of these materials can exist stably. 736 predicted materials have been verified for stability through independent experiments. Tests have shown that GNoME's success rate in predicting stable structures has increased from 50% in previous studies to 80%; the accuracy of every 100 experiments in predicting composition has increased from 1% to 33%.

Researchers from Lawrence Berkeley National Laboratory also built a laboratory platform called A-Lab, which can realize the fully automatic synthesis of inorganic solid powder materials. The target product data comes from Materials Project and GNoME, and the synthesis formula is proposed by a natural language model based on literature training and optimized through active learning. After 17 days of closed-loop operation, A-Lab conducted 355 experiments and successfully synthesized 41 of the 58 target products. The researchers said that with only a slight modification of the decision-making algorithm, the success rate can be increased to 74%, and if the computing technology is improved simultaneously, it can be further increased to 78%. (Nature News)

AI failed miserably! GPT-4 scored only 15 points on a question that humans scored 92 points on.

Scores and answering times of various methods in GAIA. GPT-4 Plugins requires manual selection of different plugins according to the problems encountered, while AutoGPT-4 plugins are selected automatically. The Human item refers to the score of the annotator when verifying the problem. | Image source: Grégoire Mialon ET AL

AI is becoming increasingly powerful, and its performance in many professional fields (such as law and chemistry) has even surpassed that of humans. However, the latest tests show that AI is still far from being as "smart" as we thought. Researchers recently published a paper on the preprint website arXiv, proposing a new AI benchmark test GAIA. In this test, humans can answer 92% of the questions correctly, while the average score of GPT-4, which uses a human-selected plug-in, is only 15%.

An example of a GAIA question. The answer is clear and does not appear in the training set text. | Image credit: Grégoire Mialon ET AL

Previous AI benchmarks tended to design tasks that were increasingly difficult for humans, but GAIA took a different approach, focusing on examining basic abilities needed in the real world, such as reasoning, multimodal processing, web browsing, and general tool usage. These problems are relatively simple for humans, but they often stump the most advanced AI. GAIA contains 466 questions designed and labeled by humans, divided into three levels according to difficulty, covering daily personal tasks, scientific problems, and general knowledge. The answers to the questions are short and unique, so they are easy to verify. Researchers believe that AGI (artificial general intelligence) should show robustness similar to that of ordinary people on such problems. If it can "pass" GAIA, it will become a milestone in artificial intelligence research. Researchers also acknowledge that GAIA has limitations, including the inability to evaluate multiple solutions to the same problem, ambiguous problems in reality, and that all questions are only asked in English, lacking language and cultural diversity. (arXiv)

Remote collaboration is not conducive to scientific research innovation

Division of labor and cooperation has become an important trend in modern scientific research, and remote technology has fueled it, helping researchers overcome geographical barriers and greatly improving collaboration efficiency. But researchers also revealed its negative side: an analysis of tens of millions of papers showed that remote collaboration may be detrimental to innovation and more difficult to achieve breakthroughs than offline scientific research teams. The study pointed out that remote collaboration teams find it difficult to fully integrate ideas, which makes them focus more on technical tasks involving explicit knowledge, and perform poorly in conceptual tasks such as conceiving ideas and designing research; "distributed" teams are good at incremental progress but poor at disruptive research.

The researchers analyzed 20 million papers from 22.5 million authors from 1960 to 2020, and 4 million patent applications from 2.7 million authors from 1976 to 2020, and analyzed the proximity between collaborators based on the geographical coordinates of cities. They also introduced the "D score" to evaluate the degree of subversion of papers, ranging from -1 to 1, with 1 being the most subversive. The D score is based on the following considerations: if a paper is often cited with its own references, then the paper is considered "incremental" work; if a paper is independent when cited (its own references are not cited), then the paper is considered a pioneering work. The study found that the average distance between collaborators has increased significantly in the past 50 years, from 100 kilometers to nearly 1,000 kilometers in papers and from 250 kilometers to 750 kilometers in patents. But correspondingly, the subversiveness of research is decreasing. When the collaboration distance increases from the same city to 600 kilometers, the rate of disruptive research (D score greater than 0) drops from 28% to 22%; the rate of disruptive patents drops from 67% to 55%. The relevant paper was published in Nature on November 29. (Nature News)

New method for studying large-scale entangled systems

Entanglement is a prominent feature of quantum many-body systems. When several elementary particles interact with each other, the properties of each particle are related in a way that it is impossible to describe the properties of each particle individually, only the overall properties of the system. Researchers have pioneered a new method to experimentally confirm the predictions of quantum field theory for the first time, deepening the understanding of quantum entanglement.

In quantum materials, measurements of strongly entangled particles produce random results with large fluctuations, which scientists call "hot" particles, and vice versa, "cold" particles. Only by measuring all entangled objects can the exact state of the system be revealed. In quantum many-body problems, the complexity brought by entanglement grows exponentially with the scale of the system, which makes measurement difficult. Researchers have developed a new more efficient description method that can extract entangled information from the system with only a very small number of measurements. They prepared the ground state and excited state of a one-dimensional XXZ Heisenberg chain on a programmable quantum simulator with 51 ions, and effectively "learned" samples of the entangled Hamiltonian (EH) of subsystems with up to 20 grid points. The researchers determined its temperature distribution through a feedback loop between the computer and the quantum system. The computer continuously generated new distributions and then compared them with the measurement results. The obtained temperature distribution curve shows that particles that interact strongly with the environment are "hot", while particles that interact weakly are "cold", verifying theoretical predictions. The relevant paper was published in Nature on November 29. (University of Innsbruck)

African penguins use unique 'QR codes' to identify mates

African penguins (Spheniscus demersus) have distinctive black spots on their white breast feathers. In a study published today in Animal Behaviour, researchers found that these spots act like name tags, helping penguins recognize their mates among similar-looking species.

African penguins | Photo credit: BACIADONNA ET AL

The researchers showed penguins two photos of other penguins at the same time and measured the time they looked at the photos and stayed near the photos. They speculated that long stares indicated that the penguins were more familiar with the individuals in the photos. The results showed that penguins spent longer looking at photos of their mates than photos of unfamiliar individuals, an average of 23 seconds; and stayed near photos of their mates twice as long as other photos. Next, the research team showed the penguins two photos of their mates, but the black spot on the chest of one of them was erased. As a result, the penguins spent more time looking at the photo that retained the black spot. Finally, the penguins were shown photos of their mates and unfamiliar individuals again. This time, the black spots on the chests of both photos were erased. As a result, there was no significant difference in the time the penguins looked at and stayed near the photos - they seemed to no longer recognize their mates. The researchers said that identifying individuals based on appearance is "a trait that we used to think was very special and cognitively difficult." The new study shows that penguins can do this, too. (Science News)

Switching to a healthier diet could extend your life by 10 years

Adhering to healthy eating habits can prevent non-communicable diseases and extend life expectancy. It is estimated that unhealthy diet causes more than 75,000 premature deaths in the UK each year, of which 17,000 are in the age group of 15-70 years old. Researchers analyzed data from a prospective population cohort of 467,354 people from the UK Biobank and found that if an unhealthy dietary pattern is switched to a dietary pattern recommended by the Eatwell Guide and maintained, the life expectancy of 40-year-old men and women will be extended by 8.9 years and 8.6 years respectively. For the same population, if a dietary pattern associated with longevity is switched to and maintained, life expectancy will be extended by 10.8 years and 10.4 years respectively. For 40-year-old British people who are accustomed to a moderate dietary pattern, changing their eating habits can extend life expectancy by about 3 years.

Image source: Pixabay

Studies have shown that dietary patterns associated with longevity include moderate intake of whole grains, fruits, fish and white meat, large intake of milk and dairy products, vegetables, nuts and beans; small intake of eggs, red meat and sugary drinks; and small intake of refined grains and processed meat. Among them, eating more whole grains, nuts and fruits, drinking less sugary drinks and eating less processed meat brings the greatest benefits. The relevant paper was published in Nature Food on November 20. (Nature News)

This article is supported by the Science Popularization China Starry Sky Project

Produced by: China Association for Science and Technology Department of Science Popularization

Producer: China Science and Technology Press Co., Ltd., Beijing Zhongke Xinghe Culture Media Co., Ltd.

Special Tips

1. Go to the "Featured Column" at the bottom of the menu of the "Fanpu" WeChat public account to read a series of popular science articles on different topics.

2. Fanpu provides a function to search articles by month. Follow the official account and reply with the four-digit year + month, such as "1903", to get the article index for March 2019, and so on.

Copyright statement: Personal forwarding is welcome. Any form of media or organization is not allowed to reprint or excerpt without authorization. For reprint authorization, please contact the backstage of the "Fanpu" WeChat public account.