AI / ML terms

History

AI and machine learning emerged from theoretical computer science in the 1950s, with early symbolic AI systems attempting to encode human reasoning explicitly. The field shifted substantially after the 1980s AI winter, when rule-based approaches proved limited, leading researchers to focus on data-driven methods and neural networks. The deep learning revolution of the 2010s, powered by GPUs and large datasets, transformed computer vision and natural language processing, culminating in transformer-based models like BERT and GPT. Today, AI/ML development encompasses specialized practices—prompt engineering, vector databases, retrieval-augmented generation, and LLM observability—that treat large language models and foundation models as primary development primitives rather than experimental tools. Modern AI engineering balances capability gains against costs, alignment risks, and inference latency, reflecting a maturation from research-focused exploration to production-grade software systems.

Key concepts

• Large Language Models (LLMs)
• Neural Networks — Conceptual Overview
• Tokenization in LLMs
• Embeddings
• Prompt Engineering
• RAG — Retrieval-Augmented Generation
• AI Hallucination
• Fine-Tuning LLMs

Best references

• 
  Attention Is All You Need ↗ The foundational 2017 paper introducing the Transformer architecture, which underpins all modern LLMs. Essential reading for understanding the core mechanism behind contemporary AI models.
• 
  OpenAI API Documentation ↗ Canonical reference for working with production LLM APIs, covering models, prompt engineering, fine-tuning, and cost management patterns used across 48 terms in this category.
• 
  Anthropic's Constitutional AI Paper ↗ Seminal work on AI alignment and safety through constitutional methods. Directly relevant to AI Governance, AI Guardrails, AI Hallucination, and Constitutional AI (CAI) terms.
• 
  Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks ↗ The original 2020 RAG paper by Lewis et al., establishing the framework for combining retrieval with generation—critical context for RAG and vector database terms.
• 
  Deep Learning (Goodfellow, Bengio, Courville) ↗ Authoritative textbook covering neural networks, optimization, and foundational concepts that underpin embeddings, diffusion models, and ML types in this category.
• 
  Hugging Face Model Hub & Documentation ↗ Practical reference for model selection, fine-tuning, quantization, and tokenization with real implementations. Covers hands-on aspects of AI Model Selection, Knowledge Distillation, and model deployment.