3.0 Concepts
(was Agents + AI / Intro)
CONTENT FROM SEVERAL FILES MIXED TOGETHER ON THIS PAGE…. will clean up later….
from “qs” 26.0615
The center of the Agentic AI universe is the AI agent.
The agent and LLM together can doing amazing things. But they also have severe limitations. “Tuning” then to work together is the core focus.
Just like the LLM internal agent and TF must work together, so must the extAgent and LLM.
- 1 Agents existed long before AI
- 2 Agentic AI (LLMs designed for agents)
- 3 AI agents (agents that use AI assistants to make decisions)
- 4 Model interface layer (MIL)
External agent (left) and the LLM (right)
1 Agents existed long before AI
GPT: “Your entire architecture is basically (1) Deterministic agent systems existed before LLMs (but were expensive and rigid) ….”
It took a decade for AI-assist to be added to Palantir agents (SW). Before AI Palantir agents used ontology (their own mini-dictionary) to limit the human language that could be used in their systems. And it worked. It required “sanitzing” all inputs, but it could process vast amounts of data, looking for that needle in the haystack that might signal another secret 9/11 style terrorist attack.
2 Agentic AI (LLMs designed for agents)
GPT: “…and (2) TF/UFA semantic capabilities made those systems vastly more practical and flexible. That is actually a very strong and coherent thesis.”
TF/UFA semantic capabilities made such systems vastly more practical and flexible by enabling:
- semantic interpretation
- structured outputs
- ontology alignment
- planning
- contextual reasoning
- generalization
- explanation generation
This is one of the core reasons modern agentic AI became practical.
The following are just a few examples of how AI is a very helpful assistant for agentic apps:
- Interpret/sanitize messy human language text
- Create human language explanations
- Planning (breaking up complex tasks into granular tasks that the agent can process reliably)
- Input and output (of “messy” human language representations)
- Rule injection (by AI into the main loop)
These capabilities make agentic AI apps
- vastly more flexible and capable
- at the cost of unpredictabilty
For many apps this is an excellent tradeoff.
About planning and other “intelligent” AI functionality
For background on VL, storylines, etc, discussed below, see The gist of LLMs.
AI is more about pattern matching than language. The TF generates from “messy” human language a vector language (VL) representation of the input “storyline” (I don’t prefer term “vector” for the 12288 FP numbers for each GPT-3 token; to me they are just scalar (numeric) indicators of detected semantic (meaning) features). The VL for the final token is pattern matched against all tokens in the vocabulary. The closest matching token for that pattern is selected as the next token.
Although based solely on patterns detected in massive amounts of training data, this functionality can do much more. It can be used for planning planning (taking complex planning input prompts and splitting up into smaller atomic parts that can be more easily be verified and executed by the agent), JSON specification of LLM output, etc.
3 AI agents (agents that use AI assistants to make decisions)
AI’s arrival vastly enhanced Palantir system performance because their machines could now “communicate” freely with messy chaotic human language input. But Palantir’s systems, like all AI-ic agents systems, are assisted by AI (not controlled by AI). The LLM is simply a “helpful assistant”. LLMs control nothing, push no buttons. But AI hype usually talks about “agentic AI”, not “AI agents”. Agentic AI that might one day take over the world (Skynet style; see my substack post #73 Understanding Palantir Maven / Why AI will never become Skynet). That hype generated a lot of publicity (and investment).
The following diagram is a good example. “What Is Agentic AI? Agentic AI is an innovative advancement in artificial intelligence (AI), characterized by its ability to independently make decisions and implement goal-oriented actions on behalf of users or systems.” (link)
An example of the LLM-centric agentic AI universe
4 Model interface layer (MIL)
The LLM internal Agent and TF are customized to work together to maximize the reliability and accuracy of agentic AI functionality.
But this functionality will invariably vary to some degree between all LLMs. Therefore the AI agentic may need to “tune” its prompts and interpretations of responses. To fix this problem, Palantir created a “Model Interface Layer” (MIL) between the LLM and AI agent.
2 Agent concepts
- 2.1 The agentic mission
- 2.2 Ontology
2.1 The agentic mission
Palantir agentic systems (no AI) started around 2003 in response to the 9/11 attacks. The US government wanted to make sure such an event did not happen again.
The mission goals were quite clear: Detect any strange new activity that might indicate a security threat (such as a bunch of foreign nationals wanting all of a sudden to learn on flight simulators how to fly airliners). Palantir was to be the (automated) system for this critical mission. A system that watches the world (like a sorcerer looking into his crystal ball) 24/7, 365 days a year.
A palantír is one of several indestructible crystal balls from the novel”The Lord of the Rings” used to see events in other parts of the world (Wiki)
2.2 Ontology
Palantir ontology is something between free-form text and a rigid database. You simply (1) restrict the allowed vocabulary and (2) specify relationships between DB entries. This ensures that the deterministic agent loop can process the DB content without errors (you cant simply automatically ingest “messy” human language without AI).
3.2 Agentic + AI
The core control loop is traditional non-AI deterministic programming that performs (with the help of an LLM)
- Input and output (of “messy” human language representations)
- Task planning (breaking up complex human language tasks into smaller deterministic tasks)
- Rule injection (by AI into the main loop)
Conceptual summary: Traditional deterministic agents existed long before AI, but were rigid and difficult to scale because human language and semantic relationships required hardcoded logic.
Modern TF/UFA systems made such agents vastly more practical by providing semantic capabilities such as:
- Semantic interpretation of human language
- Structured constrained outputs (JSON, schemas, workflows)
- Contextual reasoning over long token sequences
- Task decomposition / workflow synthesis
- Ontology alignment / semantic normalization
- Generalization beyond explicit hardcoded rules
The core control loop itself remains ordinary deterministic software (“extAgent”), while the LLM provides semantic functionality.
3.2.1 Intro)
Just like iAgent and TF work together internally, the extAgent and LLM must work together externally.
- 1 Deterministic agents existed before AI
- 2 TF/UFA semantic capabilities made modern agentic systems practical
- 3 extAgent orchestrates deterministic execution
- 4 LLM provides semantic functionality
- 5 Model Interface Layer (MIL)
“3.2.2 Agents + AI / Diagrams” 3.2.2-ai-agent-diagrams/
- 1 Ecosystem
- 2 Agent <> LLM interface
- 3 Workflow diagram / simple
- 4 Workflow diagram / detailed
1 Ecosystem
The LLM as a helpful assitant for the agent
2 Agent <> LLM interface
Details of the agent <> LLM interface
3 Workflow diagram / simple
In the diagram below:
- Agent (external) collects human language (HL) data (blue square) from the UI, DB, APIs, etc.
- Agent creates a HL prompt that contains specifications (JSON) for the LLM response.
- LLM (red square) responds with HL that conforms to the JSON specs (I sometimes refer to this as HL “machine” language because its content and structure is fairly predictable).
- Agent can then process this “machine” language” response realiably.
What AI does for an agent
The following diagram shows a demo of this.
- (1) 08b The agent uses LLM AI to generate a granular multi-step atomic plan that matches the agent’s deterministic logic.
- (2) 08 The agent validates (deterministicly) the plan.
- (3) 08 The agent executes (deterministicly) the plan.
Diagram from Substack post #75 The Real Job of AI in Enterprise Apps shows how AI is only used to generate a plan (“08b AI”)
4 Workflow diagram / detailed
In the diagram below:
- 1 UI/DB/API (human language sources and destinations) interface with 2 Agent (the main loop).
- 2 Agent adds extra info/requirements (JSON) to LLM input. This defines the desires LLM response content/structure.
- 3 Model (LLM) interface layer. Every LLM may require slightly different prompts and output slightly different responses. Palantir uses MIL to support “hot” swapping of LLMs.
- 4 Internal agent (iAgent) controls 5 TF. The iAgent is custom designed to interface with the TF.
- 5 TF creates output text for the input text (based on patterns detected during training).
- 4 LLM iAgent sends response to 2 (external) agent.
- 2 Agent processes the response.
Workflow diagram
title: “3.2.3 Agents + AI / AI infrastructure” permalink: /3.2.3-ai-agent-ai-infrastructure/ —
1 Before / after TF/UFA
Before TF/UFA systems
- tools/APIs/retrieval already existed
- BUT required rigid symbolic programming.
After TF/UFA systems
- semantic interpretation/generalization
- systems flexible enough for real human language (dynamic orchestration practical).
RAG and MCP fundamentally depend on these TF/UFA semantic capabilities.
2 RAG
Without LLM semantics retrieval systems were mostly:
- keyword matching
- rigid search
- hardcoded queries
RAG becomes powerful ONLY because of TF/UFA semantic capabilities.
- semantic interpretation “What affected Site 1?”
- semantic generalization understanding related concepts without exact keyword matches
- contextual tracking maintaining retrieval relevance across conversation flow
- explanation synthesis explaining retrieved content naturally
So modern RAG is deeply dependent on:
- semantic embeddings
- latent similarity
- TF contextual understanding
- semantic normalization.
Agent retrieves text/context and injects it into the prompt
3 MCP
Without TF/UFA capabilities: MCP degenerates into ordinary API plumbing.
The entire value of MCP is dynamic semantic interaction with tools/context.
- understand available tools
- infer which tools matter
- synthesize proper usage
- maintain contextual workflows
- integrate returned information semantically.
Agent uses AI to select the required tool based on the human language tool description from the MCP server
26.0520