What incident accidentally showed your intelligence

manage it

A look back at the biggest AI stories of 2020.

 

Last year there were technical advances in model size (175 billion parameters!) And the possibilities of giving AI attention - the ability to learn which part of the data to focus on. Attention-based AI has now generated texts comparable to human writers and solved complex physical problems such as protein folding. We have only just begun to scratch the surface of what such AI can do.

display

For Vectra AI, provider of cybersecurity based on artificial intelligence, three experts take a close look at the development of artificial intelligence in the past year: Christopher Thissen (Senior Data Scientist, Vectra AI), Ben Wiener (Data Scientist, Vectra AI) and Sohrob Kazerounian ( Senior Data Scientist, Vectra AI).

The year 2020 also gave a glimpse of how much AI is starting to invade everyday life. It seems likely that we will see regular (and unknowingly) AI-generated text on our social media feeds, in advertisements, and on the news for the next several years. The effects of using AI in the real world raise important questions about the ethical use of AI. There is every reason to believe that AI could add to bias in training data. With AI now being used in loan applications, law enforcement, and more, the need to understand and eliminate these prejudices has never been greater. Again this year we saw some great stories about the ethics of AI.

display

 

GPT-3: AI-generated text

Perhaps most of the attention in 2020 went to OpenAI's GPT-3 model. GPT-3 (Generative Pretrained Transformer 3) is an AI that can understand and generate text. The capabilities of this AI are impressive. Early adopters made the AI ​​answer trivia questions, create fiction and poetry, and generate simple web pages from written instructions. Perhaps most impressive is that people cannot distinguish between articles written by GPT-3 and those written by humans. For example, an AI-generated article recently made it to the top of HackerNews, a popular technology news site, without most users realizing that the content was artificially generated.

GPT-3 uses a technology called attention that allows AI to learn which parts of the text to focus on, such as nouns and verbs. Although this attention technology has been around for a number of years, the new capabilities of GPT-3 seem primarily to result from a mere expansion of the network. The new model has an astonishing 175 billion parameters, more than ten times as many as previous language models. The capabilities of the model suggest that the limits of the transformer architecture on which the AI ​​is based have not yet been reached. In fact, recent work has shown initial success in image generation too.

display

Images generated by a modified version of GPT-3 that predicts pixels instead of words. (Photo: OpenAI)

 

GPT-3's uncanny ability to write and respond in human-like ways has sparked some discussion about the intelligence of this AI. Despite its ability to generate compelling text, GPT-3 doesn't understand the world the way humans do. Instead, the AI ​​simply predicts the next word. This works well on topics that humans have already written extensively about, but the knowledge of AI does not translate into new information like human knowledge does.

For example, the AI ​​cannot answer simple common sense questions, such as whether a pencil is heavier than a toaster. Since there is (obviously) not enough text comparing the properties of pencils and toasters, the AI ​​cannot predict a meaningful answer. In fact, the AI ​​doesn't know what a pencil is, it just knows what text tends to surround the word pencil. AI lacks such common sense perhaps precisely because these are the kinds of things not worth writing about.

Presumably these topics are also not worth reading, so that from a practical point of view it should be less important that GPT-3 has problems with these questions. Rather, it is about the fact that GPT-3, despite the first appearances, has no general intelligence. Imagine being able to read all of the texts ever created but not being able to physically interact with the world. You would have huge knowledge gaps. Indeed, the question of whether an AI can learn the meaning of texts on its own is an important area of ​​research. If a picture is worth a thousand words, reality could be worth more than the 500 billion words in GPT-3's training set.

While GPT-3 is not yet approaching the technological singularity, this model and others like this one will prove incredibly useful in the years to come. Companies and individuals can request access to the model results via an API (currently in a private beta test). Microsoft now owns the license for GPT-3 and other groups are working to produce similar results. I expect we will soon see a variety of new features related to AI that understand language.

 

AlphaFold: protein folding

Outside of natural language processing, there were also important advances in biotechnology in 2020. Right at the beginning of the year, there was rapid and timely progress in mRNA vaccines. During the year, these were shown to be highly effective in clinical studies. At the end of the year there was another bang. DeepMind's AlphaFold seems like a giant step forward, this time in the area of ​​protein folding.

Proteins are absolutely essential for life. They are the machinery that replicates DNA, enable cellular signal transmission, facilitate chemical reactions, and even provide the physical structure of cells. A protein is a polymer, a chain of units called amino acids. It almost sounds like you can understand a protein simply by looking at the sequence of amino acids that make up it. However, it turns out that the sequence is only a small piece of the puzzle. When a protein is in its natural environment, different parts of the chain stick together, causing the molecule to fold into a complex shape. This three-dimensional structure is of crucial importance for the behavior of the protein. Unfortunately, in many cases this folded structure is both difficult to find experimentally and difficult to predict from the amino acid sequence.

This fall, the latest version of AlphaFold competed against other modern methods in a biennial competition to predict protein folding, the CASP Assessment. In this competition, algorithms were entrusted with the task of converting amino acid sequences into protein structures and were judged on the basis of the proportion of amino acid positions that the model correctly predicts within a certain range. In the most demanding category, »Free-Modeling«, AlphaFold was able to predict the structure of unseen proteins with a median of 88.1. The next best predictor in this year's competition achieved 32.4 points. This is an amazing leap forward.

 

From a computational point of view, predicting protein folding is notoriously difficult. A famous guess is that a typical protein can break down into an unimaginable 10300 plausible conformations could fold. For comparison: there are only about 1080 Atoms in the known universe. AlphaFold addressed this frightening problem by training a deep neural network on publicly available datasets of known protein structures. The DeepMind team has not yet released technical details of their approach, but has provided some basic facts. AlphaFold uses a deep learning approach to predict which amino acids will be close together in a chain and then uses a separate model to solve the structure based on those approximate predictions. Like OpenAIs GPT-3, we know that AlphaFold uses what is known as the attention mechanism, which allows the model to learn which parts of the input are relevant to each other.

In the future, scientists can use models like AlphaFold to accelerate their research on diseases and genetics. Perhaps in late 2021 we will celebrate the technology that made such work possible.

 

Democratize deep learning

As mentioned earlier, deep learning (the primary methodology underlying many modern AIs) is proving useful in areas as diverse as biology and natural language. Efforts to make deep learning more accessible to professionals and practitioners are accelerating the adoption of AI in many areas.

FastAI is a great example of this effort. FastAI consists of an online course, a Python library, and a textbook, and is aimed at people with about a year of Python programming experience and high school math. The teaching philosophy of the course is example-oriented and motivated by early successes to continue learning. In the first lesson, the students build a modern AI. Past FastAI course graduates have won international machine learning competitions, received offers from top companies, and founded machine learning nonprofits. The FastAI library uses best practices by default and is currently focused on visual and natural language related tasks. A new version of the library was released along with the book this year.

Democratization efforts are not limited to programmers. Anyone with an Internet connection can now create a realistic but completely fake photo of a human face. Similar technology has been used to create more realistic - and harder to spot - fake social media accounts in disinformation campaigns, including some in the run-up to the 2020 US election. And OpenAI plans to make GPT-3's capabilities a comparatively easy one to make usable API available for verified users. There is a legitimate concern that the easier it is to use deep learning technology as a weapon as it becomes more accessible.

But the coupling of AI with human experts can also be used for good. Domain experts can direct the AI ​​to effective, solvable problems and diagnose when the AI ​​is biased or has drawn incorrect conclusions. AI is able to process enormous amounts of data quickly (sometimes with greater accuracy than humans), making analysis cheaper and faster and uncovering insights that might otherwise be inaccessible. User-friendly tools, APIs, and libraries make it easier to adopt AI, especially in areas where established techniques such as image classification can be used. In biology, for example, the number of publications mentioning deep learning has risen from a few hundred to several thousand in recent years. In the short term, we are likely to continue to see rapid adoption of AI techniques in a number of areas.

 

AI ethics

One of the interesting consequences of the increasingly easy accessibility of AI and ML systems is the resulting shift in priorities in the field of AI ethics. In the not-too-distant past, when black box machine learning algorithms did not work so well on tasks such as computer vision, speech recognition, and language comprehension, the moral puzzles and ethical dilemmas raised by AI ethicists tended to be disconnected from practical considerations. Examples include early discourses such as Isaac Asimov's Three Laws of Robotics, which combined the possibility of robots as moral agents with a kind of Kantian deontological ethic, to Roko's Basilisk, a thought experiment in which a future AI punishes people who didn't help bring it to life. In order to save Asimov's honor, it should be said that the Three Laws of Robotics were not only written in the middle of the last century, but were also intended as science fiction.

Nonetheless, until the recent commercial feasibility and widespread use of AI and ML systems, such topics received an overwhelming amount of attention and reverence within the field. Issues that would otherwise have been treated as simple legal matters related to new technology were treated as profound questions about the philosophy of ethics simply because they had something to do with AI. For example, discussions about liability in accidents involving self-driving cars have led to a wealth of comparisons with the philosophical trolley problem. Though unspoken, these comparisons often fluctuated between suggesting that either sentient AI drivers would one day make value judgments about who to kill or, conversely, that engineers would soon be writing code that weighed the relative worth of one life against five . It has never been clear why any of this should be treated as anything other than the failure of an automaker's complex braking system. It's also unclear what the unique ethical dimensions are when it comes to AI weapon systems killing the wrong people or AI algorithms crashing the stock market. These do not equally apply to situations where (for example) a human drone operator kills the wrong person or a human stock trader accidentally crashes the market.

What sets the field of AI ethics apart in 2020 is not a single achievement or breakthrough, but rather the sheer amount of work that has been done to refocus and focus attention on issues of immediate interest. This includes questions ranging from dealing with racial and gender bias in datasets to injustices resulting from poorly paid gig work that characterizes the very data that is used to train algorithms. Some of these problems are now being faced as the interaction with AI systems increases. Distance learning, for example, has led to the use of exam software that relies on facial recognition to prevent fraud, but has led to horrific scenarios in which colored students are forced to take exams with bright lights on their faces all the time seem just so as not to be accused of fraud. These are the same types of facial recognition algorithms that were used in China to track the Uighur population. Even medical devices, which could be assumed to be subject to rigorous testing, have been shown to have a higher error rate in black patients than in white patients.

Some of these problems have been uncovered by the proliferation of commercial applications based on AI. The other driving factor was a small but dedicated group of researchers, often from groups that are underrepresented in the broader AI community. Not only did they sound the alarm about these ethical concerns, but also pushed for greater diversity and representation in the field itself. Progress has been made on this front. The increasing presence of affinity groups at leading AI conferences and efforts to evaluate the criteria for sponsorship and connections to various companies have raised awareness of the ethical shortcomings of AI (for example by examining anti-Muslim prejudices in language models). At this point, mainstream personalities outside the realm of AI ethics also feel compelled to interfere, such as players from the Boston Celtics who have called for facial recognition to be regulated.

Despite all the advances made so far, a great struggle remains. In early December, Google fired its co-head of ethical AI, Timnit Gebru. While you can read the full story in Gebrus's own words here, the news has been very disturbing to the broader Ethical AI community.This is not only true because Gebru attempted to publish a research study on the environmental consequences of large-scale training of language models (which are central to Google's business) and the lack of diversity issues that were uncovered as a result of the review process. The incident also raises questions about how the academic research community should relate to industry.

Nonetheless, the achievements in this burgeoning field lay the foundation for who and what AI should be used for. At least hopefully we no longer need to explain why you don't have to worry about Roko's Basilisk.

 

Looking towards 2021

Early in 2020, some researchers voiced concerns that AI research could soon enter yet another winter where progress comes to a standstill and both interest and funding dry up. While the novelty and excitement surrounding deep learning might indeed wear off, it is certainly interesting to note that two of the more exciting breakthroughs in 2020 were GPT-3 and AlphaFold, both of which took advantage of existing theoretical approaches but the practical applications of AI Significantly advanced algorithms in their respective areas. In the case of GPT-3, the improvements were achieved by significantly increasing the size of the model (with a corresponding increase in data requirements and training costs), while AlphaFold was combined with other computation models that together reached the state of the art. We suspect that in the future the focus will shift to enabling learning from smaller amounts of data and at the same time improving generalizability and interpretability in order to make AI models more practicable.

Human domain experts will continue to play an important, albeit different, role as democratization efforts push AI capabilities into new areas. As these changes continue to transform the landscape in which AI is used and the ways in which we interact with such systems, we will likely continue to focus on pragmatic problems with real societal implications and ongoing discussions about the role AI plays in see society.

In any case, practical applications seem to have considerable leeway before they exhaust the theoretical advances available. And unlike in previous decades, AI's penetration into society and the promise of achievable pragmatic solutions seems to be sustaining AI advancement for the foreseeable future.

 

NEWS | BUSINESS | BUSINESS PROCESSES | ARTIFICIAL INTELLIGENCE

In the English language there is the catchy expression of "no-brainer". Attempts at translation such as "self-evident" or "child's play" do not quite get to the point. A no-brainer describes, among other things, a situation that is clear. A decision does not have to be considered very carefully. Everything speaks for - or against. Despite all the complexity of the implementation, the ...

Continue reading →

NEWS | ARTIFICIAL INTELLIGENCE | WHITEPAPER

Everyone knows them, nobody wants them - complaints are a burden for companies, but unfortunately they are part of the business. But if they cannot be completely avoided, they must at least be processed in an uncomplicated manner and handled efficiently. This is in the interests of the company, but also of the customers, who want their ...

Continue reading →

TRENDS 2021 | NEWS | TRENDS 2030 | ARTIFICIAL INTELLIGENCE

The use of artificial intelligence (AIs) is already a reality in many industries, but at the same time has significantly more potential. The analysts from Accenture and Frontier Economics predict that labor productivity in developed countries can increase by up to 40 percent by 2035 due to the influence of artificial intelligence. As the Statista infographic shows, ...

Continue reading →

NEWS | BUSINESS | ARTIFICIAL INTELLIGENCE | STRATEGIES | ISSUE 9-10-2020

With the support of AI, experts have to spend less time looking for the cause and have more time to fix problems, for innovative and creative developments. In order to optimize processes and make them efficient, the strengths and knowledge of employees must be correctly combined with AI applications.

Continue reading →

NEWS | ARTIFICIAL INTELLIGENCE

Artificial intelligence (AI) is permeating more and more areas. In addition to devices and machines, the new technologies now also support IT operations: They enable networks to operate autonomously, for example. For companies that want to take this path, the IT system house Circular Information Systems recommends an approach in 3 stages. Chatbots answer customer questions, intelligent robotics control system maintenance and speech recognition simplifies ...

Continue reading →

NEWS | TRENDS 2020 | ARTIFICIAL INTELLIGENCE | TIPS

+++ Large majority demand clarification of questions of liability and clear labeling. +++ Only every tenth company is using AI so far. +++ Economy expects strong effects on the labor market. +++ TÜV Association presents a study on artificial intelligence. German companies have a very positive view of artificial intelligence and hope to gain decisive competitive advantages through the use of the technology. ...

Continue reading →

NEWS | TRENDS SERVICES | ARTIFICIAL INTELLIGENCE | SERVICES

The market analysis and consulting company PAC has published the PAC INNOVATION RADAR "AI-related Services in Germany 2020". As part of the evaluation process, the services relating to artificial intelligence (AI) from almost 30 service providers were analyzed. 21 of them were included in the final evaluation. The overall result has five providers with a rating of "Best in Class" ...

Continue reading →

NEWS | TRENDS 2020 | TRENDS ECONOMY | ARTIFICIAL INTELLIGENCE

Artificial intelligence is one of the most important technologies of the future. A new study by the Institut der Deutschen Wirtschaft (IW) in cooperation with the Bundesverband Digitale Wirtschaft (BVDW) now shows how AI has developed in Germany over the past year, who drives it - and what inhibits it. Artificial intelligence (AI) can save resources ...

Continue reading →

NEWS | ARTIFICIAL INTELLIGENCE | WHITEPAPER

+++ Regulate AI-supported applications depending on the risk. +++ Ensure the security of particularly critical AI systems through independent tests. +++ VdTÜV publishes position paper. The TÜV association has called for legal regulation of products and applications with artificial intelligence (AI) in order to be able to guarantee the safety of people and the protection of the environment in the digital world. ...

Continue reading →

NEWS | ARTIFICIAL INTELLIGENCE

Even if many companies are still skeptical about the topic of artificial intelligence, AI technology is now very relevant in the casino industry and supports important processes in the gaming sector. An AI trains itself by consuming billions of data from a wide variety of sources and thereby improves its knowledge itself. Through so-called »deep learning« ...

Continue reading →

NEWS | IT SECURITY | ARTIFICIAL INTELLIGENCE

320,000 new malware programs daily: The Federal Office for Information Security (BSI) published this number in its 2019 management report. In addition to the increasing number of attacks, these are becoming more and more professional. Cyber ​​defense solutions that use artificial intelligence (AI) to detect and evaluate cyber attacks provide valuable information for cyber defense teams and are of fundamental importance for ...

Continue reading →

NEWS | TRENDS 2020 | TRENDS SECURITY | IT SECURITY

"Insight the Mind of a Hacker 2020" offers an overview of the status quo of the international ethical hacker community. Bugcrowd presented the results of the study "Inside the Mind of a Hacker 2020" [1]. Accordingly, creativity and ingenuity play an important role in staying that decisive step ahead of criminally motivated hackers. That's enough for 78 ...

Continue reading →

NEWS | ARTIFICIAL INTELLIGENCE

AI will change the professional world and that also affects the fields of activity of the IT specialists themselves. Routine activities, sometimes even programming, can be taken over by algorithms. Comprehensive knowledge and advisory skills, on the other hand, will be in demand in the future, as will specific industry knowledge. Future-oriented training and further education for young professionals should keep these development trends in mind. Artificial intelligence, AI for short, ...

Continue reading →

NEWS | HEALTHCARE IT | ARTIFICIAL INTELLIGENCE | ISSUE 3-4-2020

The challenges in healthcare are enormous. Costs are rising, money is tight and margins are falling. At the same time, there is a lack of nursing staff and the existing specialists have been revised. Hospitals have to operate efficiently in order to guarantee patients the best possible care - and at the same time ensure data security and compliance requirements when processing the resulting data.

Continue reading →

NEWS | HEALTHCARE IT | ARTIFICIAL INTELLIGENCE | ISSUE 3-4-2020

The importance of technology in healthcare is currently more ubiquitous than ever. Regardless of whether it is telemedicine, the quick evaluation of findings or simply data management - modern information and data technology plays a central role in healthcare.

Continue reading →

NEWS | DIGITIZATION | ARTIFICIAL INTELLIGENCE

Intel is partnering with Accenture and the Sulubaï Environmental Foundation to help conserve coral reefs through the use of artificial intelligence. "Project: CORaiL" is based on intelligent underwater cameras that enable researchers to make data-based decisions about the protection of reefs. What's new? On the occasion of Earth Day 2020, Intel, together with Accenture and Sulubaï…

Continue reading →

NEWS | DIGITIZATION | ARTIFICIAL INTELLIGENCE

The importance of technology in healthcare is currently more pervasive than ever. Regardless of whether it is telemedicine, the quick evaluation of findings or simply data management - modern information and data technology plays a central role in healthcare. The use of artificial intelligence in healthcare is also growing rapidly, as reported by Pure Storage. AI is already being used to improve ...

Continue reading →

NEWS | DIGITIZATION | EDITOR'S FAVORITES | ARTIFICIAL INTELLIGENCE | SERVICES

Artificial intelligence: TÜViT is working with Fraunhofer AISEC to develop an approach for the certification of AI algorithms. Innovative test procedures are intended to make the trustworthiness of AI solutions measurable. Machine learning processes and artificial intelligence (AI) systems are already being used in numerous IT infrastructures and networked end devices as a central component for analysis, forecasting and control, and at the same time, in Germany, ...

Continue reading →

NEWS | TRENDS 2020 | BUSINESS | INFRASTRUCTURE TRENDS | TRENDS SERVICES | ARTIFICIAL INTELLIGENCE

DW providers implement AI functions either themselves or through partners. Artificial intelligence (AI) functions are now standard in digital workplace solutions. AI-supported help, for example through service bots or automated error analyzes, is no longer only offered by the large providers on the German market. This is the finding of the new "ISG Provider Lens Digital Workplace of the Future Germany 2019", a study ...

Continue reading →

NEWS | TRENDS 2020 | TRENDS ECONOMY | BUSINESS | TRENDS BUSINESS PROCESSES

Global survey shows how companies can leverage new technology to improve finance and operations. Companies that use AI and other new technologies in finance and operations are increasing their annual profits 80 percent faster than companies that do not use such solutions. This is the result of a new study by the Enterprise ...

Continue reading →

NEWS | EDITOR'S FAVORITES | ARTIFICIAL INTELLIGENCE | ONLINE ARTICLE

Machine learning, artificial intelligence, data science, data mining, ... These terms have changed over the years, but how much is really new and what has been around for decades? Will machine learning still exist in 20 years or will the boom just end one day? Where are we currently in the ...

Continue reading →

NEWS | ARTIFICIAL INTELLIGENCE | SOLUTIONS

Possible uses of AI in interaction with corporate applications. There is no question that artificial intelligence (AI) is one of the trending topics. A current PwC study, however, comes to the conclusion that only six percent of companies in Germany are using or implementing AI - so there is a lot of catching up to do. Oracle has now put together application areas in the ERP and business software environment in which AI ...

Continue reading →

NEWS | TRENDS 2020 | BUSINESS | BUSINESS PROCESS MANAGEMENT | DIGITIZATION | TRENDS BUSINESS PROCESSES | BUSINESS PROCESSES | ARTIFICIAL INTELLIGENCE

Study: Automation and AI in accounting and controlling - what do medium-sized companies expect? Automation and artificial intelligence (AI) are revolutionizing the commercial sector. Processes, tasks and role models - everything is changing. What effects does this have for medium-sized companies and organizations? The study »Who do we want to be - ... provides answers and food for thought to this question.

Continue reading →

NEWS | ARTIFICIAL INTELLIGENCE | PRODUCT NOTIFICATION

We live in a time in which technical progress is spreading every minute. Our scientists work, among other things, in the field of artificial intelligence in order to book great achievements there. This form of technology can work modern miracles in the field of medicine, for example. Robots have been aids there for several years, which are always ...

Continue reading →

NEWS