Recent Developments¶
AI research moves quickly, and specific models and products change from month to month. This page focuses on the ideas behind important recent trends, which tend to be more durable than any particular system.
About this page
This page describes concepts, not products or rankings. It will be updated as the field develops.
Multimodal models¶
Multimodal models process, and sometimes generate, more than one kind of data, such as text, images, audio and video. A common technique, introduced with the vision transformer (Dosovitskiy et al., 2020), is to cut an image into small patches and treat each patch like a token. CLIP (Radford et al., 2021) learned a shared embedding space for images and text by training on image-caption pairs, so that matching images and captions end up close together.
Applications include describing images, answering questions about charts and documents, and voice assistants that respond to speech directly.
Retrieval-augmented generation¶
Retrieval-augmented generation (RAG; Lewis et al., 2020) combines a language model with an external knowledge source. A typical system works as follows:
- Documents are split into passages, and each passage is converted into an embedding vector and stored in an index.
- For a user question, the passages whose embeddings are most similar to the question's embedding are retrieved.
- The retrieved passages are given to the language model as context together with the question.
Similarity is often measured with the cosine similarity
RAG gives a model access to current or private information without retraining, makes it possible to cite sources, and can reduce hallucinations. Its quality depends heavily on the retrieval step: if the relevant passage is not found, the model cannot use it.
Agents¶
An AI agent is a system in which a language model does more than answer a single prompt. It plans multi-step tasks and uses tools such as web search, code execution, databases or other software interfaces, observing the results and deciding what to do next. The basic pattern is a loop of reasoning, acting and observing.
Risks of autonomy
The more actions an agent can take on its own, such as sending messages or making payments, the more important it becomes to limit its permissions and to require human approval for consequential steps. A particular threat is prompt injection: text from an untrusted source, such as a web page or an email, may contain instructions designed to hijack the agent.
Reasoning models¶
Language models often perform better on multi-step problems when they write out intermediate steps before giving an answer. Wei et al. (2022) showed this with chain-of-thought prompting, in which the prompt contains worked examples. Reasoning models, which appeared from 2024 onwards, are trained, largely with reinforcement learning on problems with checkable answers such as mathematics and programming, to produce long chains of reasoning on their own.
This has made test-time compute a second axis of scaling: performance can be improved not only by training larger models but also by letting a model spend more computation on each question, for example by reasoning longer or by sampling several solutions and choosing among them. The price is higher cost and latency per answer.
Diffusion models¶
Diffusion models are the leading approach to generating images, and increasingly video and audio. They are based on two processes:
- a fixed forward process that gradually adds noise to an image until nothing but noise remains, and
- a learned reverse process in which a neural network removes the noise step by step.
To generate a new image, the model starts from pure random noise and applies the reverse process, usually guided by a text prompt. The approach goes back to Sohl-Dickstein et al. (2015) and became practical with Denoising Diffusion Probabilistic Models (Ho et al., 2020). Latent diffusion (Rombach et al., arXiv 2021, CVPR 2022) runs the process in a compressed latent space rather than on pixels, which makes it far more efficient.
Further trends¶
- Smaller, efficient models that run on laptops and phones, made possible by techniques such as quantization and knowledge distillation.
- Mixture-of-experts architectures, in which only a subset of the parameters is active for each token.
- Open-weight models, whose trained parameters are publicly released, in addition to models available only through an API.
- Longer context windows, allowing models to process entire books or code bases at once.
What comes next¶
This page concludes the learning path. The glossary collects the key terms from all three parts.