A Data-Driven Look At AI Development

By datatrekresearch in Blog A Data-Driven Look At AI Development

The latest AI Index Report was out last week, a comprehensive look at global progress in developing artificial intelligence. Produced in collaboration between the likes of Stanford, MIT, Harvard, OpenAI, McKinsey, SRI and others, this 3rd annual report is a useful tool to assess where we are in worldwide AI development.

There are links to the report as well as an excellent summary in The Verge at the end of this section, but here are the 4 takeaways that we found most important:

The US is still in the lead on both AI research and funding. China actually publishes more academic papers on the topic than either the US or all of Europe, but the US papers’ “citation impact” (how often they are cited by other work) is 50% higher than Chinese papers. AI investment in the US was just under $12 billion in 2018 as compared to $7 billion in China.
“Autonomous vehicles (AVs) received the largest share of global investment over the last year with $7.7 billion (9.9% of the total), followed by Drug, Cancer and Therapy ($4.7B, 6.1%), Facial Recognition ($4.7B, 6.0%), Video Content ($3.6B, 4.5%), and Fraud Detection and Finance ($3.1B, 3.9%).”
58% of large companies adopted AI in at least one function/business unit in 2019, up from 47% last year but very few (19%) are taking steps to mitigate the risk of explaining why the algos do what they do (“explainability”) and even less (13%) are evaluating how they might drive bias and discrimination in corporate decision-making.
For basic use cases like image recognition, AI is getter cheaper (less than $100) and faster to train (3 hours in October 2017, 88 seconds in July 2019) according to the article in The Verge citing the study’s work. (We would credit Amazon’s Rekognition software for much of that improvement.)

What we take away from all this:

That the US still leads in AI research and development is the study’s most important finding in terms of investing. American researchers – public and private – may not pump out as many papers as their Chinese counterparts, but what they do produce is more widely used by other researchers.
The Verge article had a great line worth quoting: “No matter how fast AI improves, it’s never going to match the achievements accorded to it by pop culture and hyped headlines.” The AI Index Report is replete with information about how quickly AI is developing, but there are still many challenges to bring it into real world applications.

As the article mentions, quoting machine learning pioneer Andrew Ng, “If a typical person can do a mental task with less than one second of thought, we can probably automate using AI either now or in the near future.” The trick to future AI development will be when a task takes 5, 20, or even 60 seconds. Or, as is the case with driving, 35 or so minutes on a morning commute.
That so few companies are addressing the risks to AI applications (that 13% - 19% statistic above) says we’re likely in for many public hiccups in 2020 as this technology sees more wide scale usage. Google and Microsoft even have disclosures in their annual reports essentially saying, “things might go wrong as we use more AI”. We suspect many more companies will be adding boilerplate language in 2020 to the same effect…

Summing up: we see future AI development as more linear than exponential. Yes, the technology will continue to progress, but Moore’s famous law about a double every 2 years does not seem to apply here. And very visible mistakes will happen along the way.

Sources:

Article in The Verge: https://www.theverge.com/2019/12/12/21010671/ai-index-report-2019-machine-learning-artificial-intelligence-data-progress

Full 2019 AI Index Report: https://hai.stanford.edu/sites/g/files/sbiybj10986/f/ai_index_2019_report.pdf