Comparing the two years of Stanford AI reports: 32 charts to help you fully understand the development of the industry

The AI ​​Index project of Stanford University's AI 100 aims to track the activities and progress of artificial intelligence and study the impact of artificial intelligence on people's lives. The AI ​​Index promotes the public's understanding of AI by focusing on tracking and observing the activities and progress of AI and based on reliable and verifiable data.

AI Index published its first report since its establishment at the end of 2017, and recently released its second report. By comparing the data changes in the two reports, we will explore the development trends of the AI ​​industry in 2018, as well as the similarities and differences in the development directions of AI in China compared with other countries in the world.

PDF reference: https://t.zsxq.com/BaYRB6q

Number of AI academic research papers

2017: The number of academic papers and studies has increased more than 10-fold since 1996

Since 1996, the number of academic papers and research published each year in computer science has increased more than 9 times. Academic papers and research often lead to new intellectual property and patents. In the entire Scopus database, there are more than 200,000 (200237) papers in the field of computer science containing the keyword "Artificial Intelligence". There are nearly 5 million (4868421) papers in the field of "Computer Science" in the Scopus database.

2018: The total number of papers in the field of AI has increased sharply

The above chart shows the annual publication rate growth of academic papers compared to 1996. This chart compares the growth of paper publication rates in the field of computer science (CS) and AI. From 1996 to 2017, the number of papers in the field of AI increased by 7 times (8x), and the number of papers in the field of CS increased by 5 times (6x).

Comparative summary: Compared with the 2017 report, the 2018 report further focuses on sub-sectors. The focus is on the number of papers in the computer and AI fields. In addition, the publication of papers in various regions and related fields is also supplemented. Among them, the growth of the number of papers in the AI ​​field in China and the United States is particularly eye-catching.

AI Venture Capital

2017: In the United States, venture capitalist (VC) investment in AI startups has increased sixfold annually since 2000.

The above chart shows the total annual investment of VCs in AI startups across all financing stages in the U.S. The amount of money VCs invest in startups each year plays an important role in certain key areas.

2018: In terms of venture capital, from 2013 to 2017, venture capital in the field of artificial intelligence increased by 4.5 times, while all venture capital increased by only 2.08 times. These data are annual data, not cumulative year by year.

Comparative summary: Overall, the investment enthusiasm in the AI ​​field is still very high, and the 2018 report pays more attention to the recent investment situation. From the report, it can be seen that the proportion of investment in the AI ​​field has further increased in recent years. Compared with other industries, the development speed of the AI ​​field will be further accelerated with the help of capital.

AI Startups

2017: The number of AI startups increased 14-fold.

In the U.S., the number of VC-backed AI startups has increased 14-fold since 2000. This number includes any VC-backed company listed in Crunchbase’s VentureSource database.

2018: From January 2015 to January 2018, the number of AI startups grew 2.1x, while all active startups grew 1.3x.

The chart shows that the number of startups is cumulative year on year. For the most part, the growth of startups overall has remained relatively stable, while the number of AI startups has grown exponentially.

Comparative summary: Compared with the past two years, the number of AI companies has grown faster and faster, and the number of active companies has also increased. This further reflects the hot AI market and the enthusiasm for entrepreneurship is also further increasing.

AI Talent Demand

2017: Jobs requiring AI skills have grown 4.5 times since 2013.

The share of jobs requiring AI skills on the Indeed.com platform is calculated by determining whether they are related to artificial intelligence through the titles and keywords in the job descriptions. The AI ​​Index study also calculated the growth of the share of jobs requiring artificial intelligence technology on the Indeed.com platform in different countries. The report shows that Canada and the United Kingdom account for 5% and 27% of the absolute size of the US AI recruitment market, respectively.

2018: The report statistics ML is the largest skill requirement. The increase from 2016 to 2017 is particularly significant. The global demand for AI talents increased sharply in 2016. Deep learning (DL) has the fastest growth rate - from 2015 to 2017, job openings requiring DL skills increased 35 times.

Comparative summary: 2016 was a year when the demand for AI talents soared. Since then, the demand for talents in the AI ​​industry has been growing rapidly, and the talent gap may further intensify. At the same time, the field of deep learning has developed the fastest, and talents with this skill are the most sought-after.

Core skills and hot fields in AI

2017: Machine learning, deep learning, and natural language processing (NLP) are the top three most important skills. In addition to creating AI applications, the most popular skills include machine learning techniques, Python, Java, C++, experience with open source development environments, Spark, MATLAB, and Hadoop.

2018: The number of GitHub stars for open source frameworks, TensorFlow is far ahead in popularity among developers and is growing steadily. Scikit-learn and BVLC/caffe are in second and third place respectively. Keras, which is officially promoted by TensorFlow, ranks fourth, and the other two popular frameworks PyTorch and MXNet rank seventh and sixth respectively. In particular, PyTorch, as a young framework, has seen its number of GitHub stars increase by at least 4 times since its release in early 2017.

Comparative summary: From the past two years, we can roughly see the popularity of various AI programming frameworks and programming skills. The two major trends that have emerged recently are that frameworks supported by large companies are becoming more and more popular, such as Google's TensorFlow, Facebook's Pytorch, Amazon's mxnet, etc., and TensorFlow has shown an increasingly obvious advantage in popularity over other languages.

Achievements in various tasks in the field of AI

2017: Since 2010, the error rate for image annotation has dropped from 28.5% to less than 2.5%.

In addition, the report also elaborated on many fields such as visual question answering, word parsing, speech recognition, theorem proving, etc. The report shows that in some fields, AI is still far behind human performance. Taking visual question answering as an example, as of August 2017, the accuracy of the best AI system was less than 70%, while the human level was around 85%.

2018: 2017 was the last ImageNet competition, and the competition was no longer held in 2018. It can be seen that by 2015, the ability of machines in image classification tasks had clearly surpassed that of the human eye.

After the ImageNet Challenge "retired", friends in the CV field focused on Microsoft's COCO, challenging semantic segmentation and instance segmentation. Over the past four years, the accuracy of the image segmentation challenge on the COCO dataset has increased by 0.2, and the results in 2018 have increased by 72% compared to 2015. In the task of determining sentence structure, such as grammatical analysis, AI performance (F1 Score) has improved by nearly 10% in the 15 years from 2003 to 2018.

Comparative summary: AI's performance in various fields is improving rapidly, and in some fields it has even surpassed humans. It can be seen that as AI capabilities continue to improve, in the near future, on a larger scale, AI's general capabilities will surpass humans.

Number of robots installed

2017: Internationally, robot imports have increased from around 100,000 units in 2000 to around 250,000 units in 2015.

International Data Corporation (IDC) predicts that spending on robots will accelerate over the next five years, reaching $230.7 billion by 2021, with a compound annual growth rate (CAGR) of 22.8%.

2018: From 2012 to 2017, annual robot installations in China grew by 500%, while other regions, such as South Korea and Europe, grew by 105% and 122%, respectively. Among regions with smaller installations, Taiwan stood out, with the fastest growth from 2012 to 2017.

Comparative summary: Compared with the 2017 report, the 2018 report further subdivides the international market. Compared with the previous North American market, the growth rate of the robot market in the Chinese market is particularly remarkable, and it is currently far ahead of other countries and regions.

The latest supplementary content in 2018

The 2018 report further breaks down the publication of papers in different fields: Papers are published at a faster pace

In 2017, 56% of papers were in the field of machine learning and probabilistic reasoning, compared to 28% in 2010. For most sub-fields, papers were published faster in 2014-2017 than in 2010-2014. For example, the compound annual growth rate (CAGR) of papers in the field of neural networks (red line in the figure above) was only 3% from 2010 to 2014, while the CAGR of this field from 2014 to 2017 was 37%.

AI papers on ArXiv: Rapid growth

Since 2010, the number of AI papers on arXiv has grown rapidly, from 1,073 in 2010 to 13,325 in 2017. Computer vision (CV) has been the fastest growing field since 2014 (blue line in the figure above). This trend shows that AI researchers tend to disseminate their research, whether it is recognized by peers or tested by top conferences, which also shows that the field is highly competitive.

Gender of AI professors in universities: Mostly male

In the process of collecting relevant data, improving faculty diversity became the focus of the team. In the schools studied, they found that an average of 80% of AI professors were men, and this was the case in major universities around the world.

Gender gap in employment: AI is still a man’s game

In the US, on average, male job seekers make up 71% of all AI job seekers, as machine learning has the highest number of job seekers, on average, driven primarily by machine learning job seekers. Beyond machine learning, deep learning, and robotics have even larger gender disparities relative to other categories.

Government attention to AI: The number of references to “artificial intelligence” and “machine learning” in parliamentary records in the United States, the United Kingdom, and Canada has increased rapidly.

In the parliaments of these three countries, the frequency of mentions of "machine learning" and "artificial intelligence" has increased rapidly since 2016. Relatively speaking, "machine learning" was rarely mentioned before, and it has only started to increase rapidly in the past two years.

Summary of the 2018 report on China

Papers published by region: The number of papers published in China increased by 150% from 2007 to 2017. Europe is the largest producer of AI papers. In 2017, 28% of AI papers on Scopus came from Europe, 25% from China, and 17% from the United States.

In terms of field emphasis (RAI): Chinese AI papers focus more on engineering technology and agricultural science

RAI can show the professional tendencies of a region by comparing it with the global research activities in AI. When the RAI value is 1, it means that the country's research activities in AI are consistent with the world; when it is higher than 1, it means that the country places more emphasis on this field; when it is lower than 1, it means less attention. As can be seen from the figure, China's AI papers focus more on engineering technology and agricultural sciences, while AI papers in the United States and Europe tend to focus on humanities and medicine and health sciences.

Comparison of research led by the government, enterprises and the medical community: From 2007 to 2017, China grew by 400%, while corporate papers grew by 73% during this period. In the United States, a relatively large proportion of AI papers come from enterprises, and the proportion of AI papers by American enterprises is much higher than that of China and Europe.

FWCI is the field-weighted citation impact coefficient, which can be used to measure the influence of a paper. In 2016, the citation rate of Chinese AI papers was 44% higher than in 2000, but it was still below the average level overall. The United States performed outstandingly in this regard, with the citation rate of AI papers by American authors being 83% higher than the world average. Europe has been hovering around the average level.

In terms of the mobility of AI researchers, among the three countries and regions of the United States, China and Europe, China has the largest proportion of low-mobility (Sedentary) AI authors (76%), followed by Europe (52%) and the United States (37%). The survey shows that AI papers published by scholars with high mobility have higher citation rates and influence, and are more likely to publish papers more frequently.

About 70% of the papers submitted to the 2018 AAAI Summit came from the United States or China. China submitted the most papers, with 1,242, followed by the United States, with 934. However, papers submitted by American institutions received a higher acceptance rate of 29%, with 268 accepted, while the acceptance rate of Chinese papers was slightly lower at 21%, with 265 accepted. Papers in German and Italian received the highest acceptance rate (41%), but the number of submissions was smaller.

Among the non-US universities, Tsinghua University has the highest growth rate in the number of students enrolled in AI and ML courses, which is about twice that of the second-ranked University of Toronto. In addition, the number of students enrolled in AI and ML courses in 2017 was 16 times that of 2010.

Among the top five regions in the world that have visited ROS.org the most since 2012, China has the fastest growth rate. In 2017, the number of visits from China was 18 times that of 2012.

ROS.org said the growth in visits from China was structural, not the result of increased marketing and resources in China. The United States and Europe ranked first and second in terms of visits.

AI Index concluded its 2018 report by stating that artificial intelligence will undoubtedly continue to become more sophisticated and be used more and more, but there are still many obstacles, both technical and bias and security. At the same time, with the popularization of automation, although large-scale unemployment will not come soon, as a society, we need to be prepared for a shift in the nature of work, toward more unstable, lower-paid, and less secure jobs such as health insurance.

It may be too early to reliably measure the impact of AI on society—the industry is only just getting started—but preparing for it, understanding what it means, and how AI will impact daily life, work, and public institutions like healthcare, education, and law enforcement is as important as investing in AI research and product development. Only by investing in both can we make the world a better place.