AI¶
Artificial intelligence (AI) is the field concerned with building systems that perform tasks normally associated with human intelligence, such as understanding language, recognizing images, planning and reasoning. The term first appeared in a proposal dated 31 August 1955 by John McCarthy, Marvin Minsky, Nathaniel Rochester and Claude Shannon for a summer workshop held at Dartmouth College in 1956.
Early AI research was dominated by symbolic approaches: hand-written rules, logic and expert systems. The progress of the last decade, in contrast, rests almost entirely on machine learning, in particular on deep neural networks trained on large data sets. This part follows that line of development.
Contents¶
- Neural Networks & Deep Learning: from the perceptron to deep networks, backpropagation, and logistic regression as a single neuron.
- Transformers: the attention mechanism and the architecture behind modern language models.
- Language Models: BERT and GPT, pretraining, fine-tuning and RLHF.
- Recent Developments: multimodal models, retrieval-augmented generation, agents, reasoning models and diffusion models.
A short timeline¶
| Year | Milestone |
|---|---|
| 1956 | Dartmouth Summer Research Project on Artificial Intelligence |
| 1958 | Frank Rosenblatt publishes the perceptron |
| 1986 | Rumelhart, Hinton and Williams show that backpropagation learns useful internal representations in multilayer networks |
| 2012 | AlexNet wins the ImageNet image classification challenge by a large margin, starting the deep learning boom |
| 2017 | The transformer architecture is introduced in "Attention Is All You Need" |
| 2018 | GPT and BERT demonstrate large-scale pretraining of transformers on text |
| 2022 | InstructGPT and ChatGPT: language models fine-tuned with reinforcement learning from human feedback (RLHF) to follow instructions |
| 2024 | Reasoning models trained with reinforcement learning to generate long chains of intermediate steps |
AI, machine learning, deep learning
Deep learning is a subfield of machine learning, which in turn is a subfield of AI.
What comes next¶
This part concludes the learning path. The glossary collects the key terms from all three parts.