From chatbots to personal AI: How foundational models made private AI possible

Artificial intelligence often appears mysterious. A system like ChatGPT can answer questions, write essays, analyse documents, and even assist with research. To many people, it feels as if a new kind of intelligence has suddenly appeared.
 

Yet the reality is both simpler and more fascinating. Modern AI systems are the result of several technological evolutions that unfolded over the past decade. What began as a method for predicting the next word in a sentence has evolved into a new architecture of digital assistants that help individuals and organisations navigate large bodies of knowledge.
 

Understanding this evolution requires looking at three major developments: the rise of foundational models, the separation of knowledge from reasoning, and the emergence of agentic systems capable of performing tasks rather than merely answering questions.
 

Teaching Machines to Predict Language
 

Modern language AI began with a deceptively simple task:

Predict the next word in a sentence.

Consider the phrase:

“Tea grows in _____”

If a computer predicts “Assam,” it has captured a pattern found in real language. Training a system to perform this task millions or billions of times gradually teaches it the structure of language itself.

Mathematically, the model learns to estimate: P(next word ∣ previous words)

This probability function becomes the basis of the system’s linguistic ability.
 

The mechanism that learns this function is a neural network, a large mathematical structure composed of layers of interconnected parameters called weights. Each word is converted into a vector of numbers, and these vectors are repeatedly transformed through matrix multiplications and nonlinear operations.

When you type a sentence, the system first converts words into tokens—numerical representations of language. Each token is then transformed into a vector:

word→[0.13,-0.72,0.55,...]

These vectors pass through multiple layers of matrix multiplications and nonlinear transformations.

In simplified form: Input vector×Weight matrix=New vector

Stack enough layers of these operations together and train them on vast datasets, and the system begins to capture grammar, semantics, and reasoning patterns embedded in language.
 

The Transformer Revolution
 

For decades, language models struggled to understand long sentences or complex relationships between words. A breakthrough came in 2017 with the Transformer architecture introduced in the paper: Attention Is All You Need

The key idea was self-attention.

Self-attention allows each word in a sentence to examine every other word and determine which ones matter most for understanding the meaning.

For example, in the sentence: “The Brahmaputra flows through Assam.”

The system learns relationships such as:

• Brahmaputra ↔ river
• flows ↔ Brahmaputra
• Assam ↔ flows

This mechanism allows models to capture context across entire sentences or paragraphs, dramatically improving language understanding.
 

The Rise of Foundational Models
 

Once the Transformer architecture proved effective, researchers began training very large neural networks using enormous text datasets drawn from books, articles, websites, and code repositories.

These systems became known as foundation models because they provide a general-purpose linguistic capability upon which many applications can be built.

Examples include:

• GPT-4
• LLaMA
• Mistral
• Gemma

Training such models requires extraordinary computational resources, thousands of specialised processors running for weeks or months.

The result is a system capable of writing, summarising, translating, coding, and answering questions across a wide range of subjects.

However, these models are still generalists. They contain broad knowledge of language and concepts but may lack detailed information about specific domains.
 

Separating Knowledge from Intelligence
 

A major conceptual shift in AI engineering occurred when researchers realised that models do not need to memorise all knowledge internally. Instead, knowledge can be stored externally in documents or databases, and the AI can consult those sources when answering a question.

This architecture is known as retrieval-augmented generation (RAG). The workflow looks like this:

1. A user asks a question.
2. The system searches a document collection for relevant passages.
3. Those passages are given to the language model.
4. The model reads them and generates an explanation.

In effect, the AI becomes a scholar with access to a library, rather than a system trying to memorise every book ever written. This separation dramatically reduces the size of the model required for many tasks.
 

Why Small Private AI Is Now Possible
 

Because knowledge can reside in documents rather than inside the model itself, relatively small models can perform sophisticated tasks when supplied with the right context.

Models with only a few billion parameters, tiny compared with the largest models, can now:

• analyse specialised documents
• answer questions about archives
• summarise research papers
• assist with legal or medical guidelines

Such models can run on:

• a workstation
• a small server
• even a modern laptop

This development has opened the door for organisations to build private AI systems tailored to their own knowledge. Hospitals, universities, law firms, research institutes, and archives can now deploy assistants trained on their own documents while keeping sensitive data within their own infrastructure.
 

From Chat Assistants to AI Agents
 

The earliest public AI tools were primarily chat assistants. Systems like ChatGPT respond to questions and produce text in a conversational style. But the architecture is evolving further.

AI systems are increasingly organised into agentic workflows, systems that can plan and execute multiple steps toward a goal.

A useful way to see this evolution is through four stages:

Chat assistants → Copilots → AI agents → Multi-agent systems

Chat assistants

Respond to questions in conversation.

Copilots

Assist with tasks inside software environments—editing documents, analysing data, or writing code.

AI agents

Plan and perform multi-step tasks such as research, document analysis, or report preparation.

Multi-agent systems

Multiple specialised agents collaborate, much like a team of experts.

For example, a research request might involve:

• a search agent gathering information
• an analysis agent interpreting the material
• a writing agent preparing a report
• a review agent checking consistency

This structure mirrors human organisations, in which complex work is distributed among specialists.
 

Private AI for Institutions and Individuals
 

These technological developments are creating a new paradigm: institutional and personal AI systems. Instead of relying entirely on large centralised AI platforms, organisations can build assistants tailored to their own needs. Examples might include:

• a clinical assistant trained on hospital protocols
• a legal assistant analysing case law databases
• a historical archive assistant interpreting manuscripts
• a research assistant for academic literature

Such systems combine private documents, retrieval systems, small language models, and agentic workflows.

The result is an AI that understands a specific body of knowledge and can help users explore it effectively.
 

A New Interface to Knowledge
 

At its core, modern AI remains a mathematical system trained to predict language patterns. Yet when combined with large collections of documents and sophisticated retrieval systems, it becomes something profoundly useful: a natural language interface to knowledge itself.

Instead of searching through documents manually, users can ask questions and receive explanations grounded in sources. This shift may ultimately transform how information is accessed and understood.

The story of AI, therefore, is not simply about building smarter machines. It is about building systems that help humans navigate the vast landscape of human knowledge, and doing so in increasingly personal, specialised, and accessible ways.

 

(Author’s Note: Dr Jayanta Biswa Sarma writes on politics, institutions, and society through the lenses of history, philosophy, and systems thinking, drawing on both Indian and Western intellectual traditions. Artificial intelligence tools may be used in preparing this article as research and editorial aids. All arguments, interpretations, and final editorial judgement remain the author’s responsibility)