← 2.2 PAL demos


TOC:

  • CHAT 2
  • CHAT 1

Alex Karp sounding the alarm sovereignty in the enterprise.
xxx


CHAT 2 ======================================================================


ME 2.1: so i have not seen a demo that specifically focuses on ai sovereignty…

maybe there is one…. but if not , what kind of demo would you recommend? can you give me the basic outline?

FDE:

I would recommend a comparative sovereignty demo, not merely another model-training demo.

Use the same business task and the same synthetic confidential data in three architectures:

A. External AI application
B. Foundry-controlled application using an external model
C. Foundry-controlled application using an internal model

That would directly test Karp’s argument about retaining the application, Ontology, and proprietary operational knowledge.

Proposed title

Sovereign AI: External Model vs Governed External Model vs Internal Model

Basic outline

1. Define AI sovereignty

Separate sovereignty into several dimensions:

  • Data sovereignty: What information leaves the organization?
  • Model sovereignty: Who owns and operates the model?
  • Application sovereignty: Who controls prompts, workflows, memory, and tools?
  • Operational sovereignty: Who controls deployment, monitoring, replacement, and shutdown?
  • Decision sovereignty: Who validates and executes the resulting decisions?

This prevents “sovereignty” from being reduced to only data privacy.

2. Create a controlled business scenario

Use synthetic but realistically sensitive data—for example:

  • Patient and Clinical Trial data;
  • proprietary product documents;
  • confidential operating procedures;
  • a unique synthetic trade secret.

Build one clearly defined task:

Read proprietary context
→ make a recommendation
→ optionally update operational state

Do not use actual confidential information.

3. Architecture A: external AI application

Implement the task primarily through an external provider:

Company data
→ external model/agent platform
→ provider-hosted prompt, memory or RAG
→ answer

Record:

  • what data is transmitted;
  • where prompts and documents are stored;
  • who controls memory and retrieval;
  • who controls the user interface;
  • whether the provider owns the workflow and tools;
  • what happens if the provider becomes unavailable.

This represents the maximum sovereignty concern.

4. Architecture B: Foundry application with an external model

Keep the operational system in Foundry:

Foundry Ontology
→ select/minimize context
→ approved external LLM
→ validate response
→ governed Ontology Action

Demonstrate that Foundry retains:

  • the complete domain model;
  • source data and relationships;
  • application logic;
  • permissions;
  • Action definitions;
  • operational history;
  • the user interface.

The external model receives only the context selected for a particular inference.

This architecture demonstrates Karp’s phrase “selectively use their models.”

5. Architecture C: Foundry application with an internal model

Replace the external model while retaining the same Ontology, application, and Action:

Foundry Ontology
→ internal model and/or internal RAG
→ validate response
→ governed Action

Compare:

  • data egress;
  • privacy;
  • model quality;
  • latency;
  • cost;
  • deployment complexity;
  • maintenance;
  • hardware requirements.

This shows what additional sovereignty is gained by controlling the model endpoint.

6. Demonstrate model replaceability

This may be the most important part of the demo.

Keep these components unchanged:

  • Ontology;
  • UI;
  • business workflow;
  • Action;
  • Automation.

Then switch among:

External model A
→ external model B
→ internal model

If the application continues operating with limited changes, Foundry—not the model provider—owns the operational application.

That is a concrete demonstration of sovereignty rather than a marketing claim.

7. Compare RAG placement

Demonstrate three possibilities:

External model + provider-hosted RAG
External model + Foundry-controlled RAG
Internal model + internally controlled RAG

For each one, identify:

  • where documents are stored;
  • where embeddings are stored;
  • where retrieval occurs;
  • what retrieved text is sent externally;
  • who can access the index;
  • whether prompt and retrieval logs are retained.

This makes clear that RAG does not automatically imply sovereignty.

8. Add agent-action controls

For the action-taking versions, compare:

  • unrestricted model-generated action;
  • structured output validation;
  • constrained Ontology Action;
  • staged human approval;
  • automatically committed edits;
  • human override and rollback.

This extends sovereignty from control of data and models to control of operational decisions.

9. Simulate loss of the external provider

Disable or replace the external model connection and ask:

  • Does the company retain its data?
  • Does the Ontology still function?
  • Does the UI remain usable?
  • Can the model be replaced?
  • Is historical decision context retained?
  • Can critical workflows continue with an internal model or manual review?

A sovereign architecture should degrade gracefully rather than losing the complete application.

10. Produce a sovereignty scorecard

Conclude with a comparison table:

Dimension External platform Foundry + external model Foundry + internal model
Company controls source data Partial Yes Yes
Data leaves controlled environment High Selected context Minimal/none
Company controls Ontology No/limited Yes Yes
Company controls workflow and Actions Provider-dependent Yes Yes
Model replaceability Low/medium High High
Company controls model deployment No No/partial Yes
Operational complexity Low Medium High
Overall sovereignty Low Medium/high Highest, with tradeoffs

Relationship to your existing demos

You already have much of the proposed progression:

D2
→ external models used in a data workflow

D3
→ external model used in an operational agent

D3b
→ protect data and Actions around the external model

D8 series
→ create, deploy and operate internal models

D3c
→ replace D3’s external model with an internal model

Sovereignty comparison
→ evaluate all architectures together

Therefore, the sovereignty demo may work best as a capstone comparison after D3b, D8, and D3c rather than as an isolated tutorial.

Its central test would be:

Can the company replace the model provider while retaining its proprietary data, Ontology, application, workflows, Actions, history, and customer relationship?

That question captures Karp’s sovereignty argument much more directly than simply asking whether a model is local.


ME 2.2: maybe in 2.2 PAL demos

TOC:
1 Basic demos with external AI models. This is the first set of demos because ZAI focuses primarily on 
    AI and external models are simpler than internal models.
2 Basic demos with internal AI models. Demos that show how to create and host internal models.
2b Soveignty demos
3 Basic demos without AI. Palantir started out with no AI. And AI now is still just a helpful assistant. 
   This section focuses on the core non-AI Foundry basics.
4 Other demos. Duplicate or unimportant demos (or demos I could not get to work).

NOTE i guess 2b would perhaps reuse previous demos, but alter them and focus specifically on demo-ing sovereignty

FDE:

Yes, this structure makes sense. Placing sovereignty after both external- and internal-model demos is especially logical because sovereignty can only be demonstrated meaningfully by comparing those alternatives.

1 External-model demos
        ↓
2 Internal-model demos
        ↓
2b Sovereignty comparison and experiments

Reusing previous demos is an advantage, not duplication. The sovereignty demos would hold the business task constant while changing:

  • where the model runs;
  • what data leaves Foundry;
  • where RAG and memory reside;
  • who controls the application and Ontology;
  • whether the model provider can be replaced;
  • who controls decisions and Actions.

Suggested TOC description:

2b AI sovereignty demos. Comparative extensions of the preceding external- and internal-model demos. These demos keep the same data, Ontology, applications, and business tasks while changing the model provider, deployment location, RAG architecture, and control boundaries. Their purpose is to demonstrate which proprietary assets remain under company control and what risks or dependencies remain.

A possible sequence is:

2b.1 Define and measure sovereignty
2b.2 External model with provider-controlled application
2b.3 Foundry-controlled application with external model
2b.4 Replace the external model with an internal model
2b.5 Compare external, internal and provider-hosted RAG
2b.6 Model-provider outage and model replacement
2b.7 Agent Action, approval and decision sovereignty
2b.8 Final sovereignty scorecard

One minor conceptual caution: in section 3, saying AI is “still just a helpful assistant” conflicts somewhat with D3’s automatically executing agent. A safer description is:

This section isolates the non-AI capabilities that remain the foundation of Foundry: data integration, Ontology, applications, Actions, governance, and operational workflows. AI extends these capabilities but does not replace them.

Overall, the four-part organization is strong:

External AI
→ internal AI
→ sovereignty comparisons
→ Foundry without AI
→ miscellaneous/archived demos


ME 2.3: i dont consider the LLM to be part of the agent

“One minor conceptual caution: in section 3, saying AI is “still just a helpful assistant” conflicts somewhat with D3’s automatically executing agent. A safer description is:”

you may be right … but i dont consider the LLM to be part of the agent. Yes, its specified in the agent dialogs, but not part of the agent.

FDE:

That is a valid and internally consistent boundary:

Agent
├─ control loop
├─ state/context access
├─ tool and Action execution
└─ call to an external AI service

External LLM
└─ supplies inference when called

Under your definition, the LLM is used by the agent but is not part of the agent, just as a database or external API used by a Python agent is not necessarily part of the agent program.

In D3:

  • Automate supplies persistent triggering.
  • AIP Logic coordinates one execution.
  • Ontology supplies state.
  • Actions alter state.
  • The LLM provides an advisory eligibility judgment when called.

Although the current Logic maps that judgment directly into an automatically committed Action, the LLM itself does not monitor Patients, invoke the workflow, maintain the Ontology, or execute the write.

Therefore, your original point can be retained, but perhaps made more explicit:

Palantir’s core platform does not require AI. The Ontology, pipelines, applications, Actions, and Automate provide the operational system. An LLM is an optional external inference service that the system can call as a specialized assistant.

Or more concisely:

The agent owns the loop and the Actions; the LLM supplies advice.

Common agent diagrams often place the model inside the broader “agent system” box, but that is a choice of system boundary—not an objective requirement. Your diagram can place the external LLM outside the agent boundary as long as you state that definition clearly.


CHAT 1 ======================================================================


My interpretation of the Palantir post below:

  • Frontier labs (OpenAI, Grok, etc; companies creating LLMs on the frontier, on the edge, of what is possible) are using prompts they get about sensitive topics to train their future models on (for example, GPT knows everything about ZiptieAI, although there’s not much super-secret in all that knowledge). I think this is what he means by giving up sovereignty.
  • These labs will take that knowledge and start to offer competing services based on their LLMs. Thats just the logical next step.
  • The solution: You use your own local models. No data goes outside the company. Your proprietary data stays in-house. You use RAG (you feed your data into LLMs that have not been trained on that data; basically the super-sensitive knowledge is not “baked-in” to the LLM).
  • How do you do that? First of all, you start off with a trial version of Palantir Foundry, you learn the basics doing the ZAI demos D1, D2, and D3 (based very much on original Foundry demos), and then you do the following D8 demo “Deep Dive: Model integration”. After that you truly understand the problem and the solution.

Alex Karp sounding the alarm about the risks of AI in the enterprise.
xxx


ME1: (ref the above text) this is what i wrote about the attached pic. you agree?

No—not completely as written. Your central sovereignty argument is reasonable, but the text attributes several claims to Karp that the post itself does not establish.

The important distinctions are:

  1. “Alpha being migrated out” does not necessarily mean prompts are being used for training.
    Karp may mean that companies surrender their proprietary advantage, workflows, decisions, and customer relationships to frontier-model vendors. Prompt training is one possible mechanism, but the quoted text does not explicitly say that.

  2. You cannot assume every frontier lab trains on enterprise API prompts.
    Data retention and training policies vary by provider, product, and contract. Sending sensitive data still creates external disclosure and dependency risk, even when the provider promises not to train on it.

  3. Karp does not say companies must exclusively use local models.
    He explicitly says companies may “selectively use their models,” provided the company retains the application/Ontology layer containing its proprietary context and operational logic.

  4. RAG does not guarantee that data stays inside the company.
    If RAG retrieves proprietary passages and sends them to an external LLM, those passages still leave the company at inference time. RAG means the information is supplied as context rather than trained into model weights.

  5. Grok is a model; xAI is the frontier lab.

Suggested revision

2.0 Interpreting Alex Karp’s statements about sovereignty

My interpretation of the Palantir post below:

Frontier labs such as OpenAI, xAI, Anthropic, and Google build the most capable general-purpose models. However, a company’s competitive advantage—or “alpha”—often resides outside those models: in its proprietary data, institutional knowledge, business processes, decisions, and understanding of its operating environment.

Karp appears to argue that companies give up sovereignty when they allow a frontier-model provider to become the owner or controlling layer of those operational capabilities. There is also a separate confidentiality risk when proprietary data is sent to an external model service. Whether that data is retained or used for training depends on the provider, product, deployment, and contractual controls.

The proposed solution is not necessarily to reject frontier models. Karp explicitly says that companies can use them selectively while retaining control of the application and Ontology layers. Those layers hold the company’s proprietary context, business logic, permissions, workflows, and Actions.

Internally controlled models can reduce dependence on frontier labs and prevent inference data from leaving the company’s controlled environment. RAG can provide proprietary knowledge to a model without training that knowledge into its weights. However, if RAG uses an external LLM, the retrieved context may still be transmitted externally. Internal models and RAG solve different parts of the sovereignty problem.

The following D8 demos explore how models can be integrated, trained, deployed, and used for inference in Foundry. Later demos can compare external and internally controlled models and examine what Foundry’s Ontology and application layers contribute to operational sovereignty.

Concise interpretation

The strongest reading of Karp’s statement is:

Frontier lab owns the general model.

The company should retain control of:
- proprietary data;
- operational context;
- business logic;
- permissions;
- workflows;
- Actions;
- resulting decisions and outcomes.

That is a broader and more defensible concept of sovereignty than simply:

“Frontier labs train on every prompt, so use RAG and a local model.”

Prompt confidentiality is important, but Karp’s quoted argument appears to concern control of the company’s entire operational advantage, not only model training.


ME2: so you are saying that the ontology is the key secret, not necessarily the prompt content.

Yes—in Karp’s apparent framing, the Ontology/application layer is the strategic asset, not merely the content of an individual prompt.

The Ontology can embody:

  • proprietary data and relationships;
  • the company’s operational model;
  • business rules and workflows;
  • permissions and governance;
  • Actions and decision rights;
  • accumulated human and organizational knowledge.

An external LLM may receive a selected fragment of that context for one inference, while the company retains the complete operational system:

Company-controlled Ontology
→ selectively provide context to LLM
→ receive model output
→ validate and apply it through company-controlled Actions

But prompt content may still be confidential. These are two related risks:

  1. Confidentiality risk: sensitive prompt/context data is exposed externally.
  2. Sovereignty risk: the frontier lab comes to own or control the company’s operational knowledge, application layer, customer relationship, or decision process.

So the concise answer is:

The Ontology is the larger source of proprietary “alpha”; prompt content may contain pieces of that secret and must still be protected.


ME3: ok, fine, but i think you dont explain how ontology would be sent to the external world. do you mean that without foundry, you would end up with ontology-type info being embedded into prompts?

Yes—partly. Karp probably does not mean that a company literally uploads an “Ontology file.” He means that, without a company-controlled application layer, the information and capabilities that would constitute an Ontology gradually migrate into the external provider’s system.

That can happen through:

  • system prompts containing business rules;
  • user prompts containing proprietary facts;
  • documents uploaded to the provider;
  • RAG content stored in provider-hosted vector databases;
  • fine-tuning datasets;
  • tool definitions and business workflows;
  • conversation histories and memory;
  • application logic built on the provider’s agent platform;
  • provider-hosted user interfaces and customer interactions.

For example, without Foundry, a clinical agent might send an external platform:

Patient records
+ trial criteria
+ eligibility rules
+ organizational policies
+ workflow instructions
+ available actions

That collection begins to resemble an externalized Ontology and application layer—even if nobody calls it an Ontology.

With Foundry, the intended architecture is:

Internal Ontology retains:
- complete Patient and Trial model
- relationships
- permissions
- workflow state
- business Actions

External LLM receives:
- only the selected context needed for one decision

Foundry receives the answer:
- validates it
- governs the Action
- retains operational control

So there are two forms of “migration”:

Literal data migration

Pieces of proprietary knowledge leave through prompts, uploaded documents, RAG context, logs, or training data.

Functional migration

The external provider increasingly owns the representation of the business, its workflows, memory, tools, application interface, and decision process.

Karp’s sovereignty argument appears to focus especially on the second, broader form. Foundry’s Ontology keeps that operational representation under the company’s control.

However, Foundry does not automatically prevent the first form. D2 and D3 still send selected data to an external LLM. The implementation must minimize and govern what is included in each call.

So the concise answer is:

Without a company-controlled Ontology/application layer, Ontology-like knowledge tends to accumulate in prompts, documents, RAG stores, agent memory, tool definitions, and workflows hosted by the external AI provider. Foundry aims to keep the complete operational model internal while exposing only selected fragments to external models.


26.0905 (v1 26.0905)