<aside> <img src="/icons/help-alternate_gray.svg" alt="/icons/help-alternate_gray.svg" width="40px" /> Started as a long Tweet:
A brief(ish) proposal for the periodicization of AI history based on periods of human history. Suggestions and criticisms welcome.
Note: as with history, the closer to the present we get, the shorter the periods become.)
</aside>
timeline
title AI History Timeline
section AI Prehistory (Until 1930s)
Early archetypes and philosophical foundations : Golem (Jewish legend), Mechanical Turk (1770), Frankenstein (1818)
: René Descartes (1596-1650) - Mechanism, mind-body dualism
: Gottfried Leibniz (1646-1716) - Symbolic logic, calculus ratiocinator
: Thomas Bayes (1701-1761) - Bayesian probability
: George Boole (1815-1864) - Boolean algebra
: Charles Babbage (1791-1871) - Difference Engine, Analytical Engine
: Ada Lovelace (1815-1852) - First computer program
: Bertrand Russell & Alfred North Whitehead (1910-1913) - "Principia Mathematica"
section AI Ancient History - Axial Age (1936 - 1973)
Foundational ideas emerge : "On Computable Numbers" by Alan Turing (1936)
: "A Mathematical Theory of Communication" by Claude Shannon (1948) - Foundations of information theory
: "A Logical Calculus of Ideas Immanent in Nervous Activity" by McCulloch & Pitts (1943)
: "The Organization of Behavior" by Donald Hebb (1949)
: "Computing Machinery and Intelligence" by Alan Turing (1950)
: "Modes of Meaning" by J.R. Firth (1951) - Contextual theory of meaning
: "Syntactic Structures" by Noam Chomsky (1957) - Transformational generative grammar
: Dartmouth Conference coins "Artificial Intelligence" (1956)
: "The Perceptron - A Probabilistic Model for Information Storage and Organization in the Brain" by Frank Rosenblatt (1958) - Introduces first trainable neural network
: "Perceptrons" by Minsky & Papert (1969)
: Various demo implementations - Checkers (Arthur Samuel, 1950s), ELIZA (Joseph Weizenbaum, 1964-1966), Mark I Perceptron (Frank Rosenblatt, 1957-1958)
section AI Middle Ages (Mid-1970s - Early 1990s)
Patchy progress, less funding : "The Lighthill Report" (1973)
: Minsky, McCarthy, Simon - Symbolic AI and knowledge representation
: Expert systems (MYCIN, DENDRAL)
: Early machine translation attempts (SYSTRAN)
: Cyc project launched by Douglas Lenat (1984)
: "Learning representations by back-propagating errors" by Rumelhart, Hinton & Williams (1986)
: "Parallel Distributed Processing" by Rumelhart & McClelland (1986) - Foundations of connectionism
section First AI Industrial Revolution (Mid-1990s - 2000s)
Stochastic approaches bear fruit : "Long Short-Term Memory" by Hochreiter & Schmidhuber (1997)
: Statistical machine translation
: Hidden Markov Models for speech recognition
: NLP advancements - Named Entity Recognition (NER), Latent Semantic Analysis (LSA)
: Recommendation engines (e.g., Amazon, Netflix)
: Sentiment analysis and opinion mining
: Information retrieval and search engine improvements
section Deep Learning Revolution (2012 - 2017)
Neural networks prove viable : "ImageNet Classification with Deep CNNs" (AlexNet) by Krizhevsky et al. (2012)
: "Efficient Estimation of Word Representations in Vector Space" by Mikolov et al. (2013)
: "Sequence to Sequence Learning with Neural Networks" by Sutskever et al. (2014)
: "Generative Adversarial Networks" by Goodfellow et al. (2014)
: "The Unreasonable Effectiveness of Recurrent Neural Networks" by Karpathy (2015)
section Transformer Revolution (2017 - 2022)
Transformers dominate AI : "Attention Is All You Need" by Vaswani et al. (2017)
: "Improving Language Understanding by Generative Pre-Training" (GPT) by Radford et al. (2018)
: "BERT - Pre-training of Deep Bidirectional Transformers for Language Understanding" by Devlin et al. (2018)
: "Language Models are Unsupervised Multitask Learners" (GPT-2) by Radford et al. (2019)
: "Language Models are Few-Shot Learners" (GPT-3) by Brown et al. (2020)
: "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models" by Wei et al. (2022)
: "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks" by Lewis et al. (2020)
: "InstructGPT - Training language models to follow instructions with human feedback" by Ouyang et al. (2022)
section PC (Post ChatGPT) Age (2023 - Present)
Rapid advancements in AI capabilities : ChatGPT launch (2022)
: "Constitutional AI - Harmlessness from AI Feedback" by Anthropic (2022)
: GPT-4 release by OpenAI (2023)
: Claude 2 release by Anthropic (2023)
: "Direct Preference Optimization - Your Language Model is Secretly a Reward Model" by Stanford University (2023)
: "Towards Monosemanticity - Decomposing Language Models With Dictionary Learning" by Elhage et al. (2023)
gantt
title AI History Timeline
dateFormat YYYY
axisFormat %Y
section Post ChatGPT Age
Rapid advancements post ChatGPT big bang : 2022, 2024
section Transformer Revolution
Transformers dominate : 2017, 2022
section Deep Learning Revolution
Neural networks prove viable : 2012, 2017
section First AI Industrial Revolution
Stochastic approaches : 1995, 2012
section AI Middle Ages
Patchy progress : 1973, 1995
section Axial Age
Foundational ideas : 1936, 1973
section AI Prehistory
Formation of archetypes : 1600, 1936
We can see the development of some archetypes (Frankestein, Mechanical Turk, adding machines) and early versions of tools (calculus, probability) we will see later. But cannot find a direct developmental links.
Many of the foundational ideas and conceptual structures we recognise today and to which we can trace direct ancestry. Many ideas from many areas (mathematics, logic, neurology, linguistics) coming together or emerging at the same time. Happening in the context of great social and technological upheavals (wars, invention of computers, new societal structures and sources of funding for science).
We can pick arbitrary dates as start (1936 Turing "On Computable Numbers" or 1943 Mcculloch and Pitts "On logical calculus immanent in in the ideas of nervous activity" to 1969 Perceptrons or 1973 The Lighthill Report). It also surrounds the turning point namely the coining of the term AI (1956) and the first implementations of various algorithms.
Most of what was written then is still something we can learn from but it's not a place to start. Also, despite the direct links when we look closely, we see as many dissimilarities and discontinuities as similarities.
The end of the great churn of the Axial age. Often given as an example of a period of 'darkness' which is unjust because many things were happening that lay the foundations for later but were much more patchy and subdued (with less funding) than earlier. Saw periods of rises and falls in various areas - expert systems, connectionism, generativist NLP. Most of those ideas developed directly into what we have today but in ways that are mostly of historical interest. Writings from this time are widely cited but not widely read.
This time gave the shape to the type of AI we have today. The symbolic AI failed to produce meaningful gains while stochastic approaches started to bear fruit with first usable image recognition and speech recognition. But each system had to be created separately, we had machines that could learn but lacked computing power, data sets and techniques. The idea of a general AI seemed almost beyond reach. Yet, it revitalised the AI field (called data science or machine learning) and many of the technologies still power the world around us. But despite the direct links, many of the things we take for granted now were not present.
During this time, AI was a collection of ML algorithms that had little in common powering things like spam filters, recommendation engines, speech recognition, basic NLP (NER, LSA...). We also saw the launch of Roomba but overall robotics progress was slow.