See the working text for this QS in

#619_.docx.
Demo URL there are actually many … this was a chaotic demo.

1 demo goal: AI “haystack needle” detection

so i used pipeline for training… but not for batch inference.

so batch inference is teh next logical thing for me to build is a pipelline that uses model for inference?

Exactly right. The next logical step would be a batch inference pipeline —

a Transform that:

Reads a dataset of housing records

Calls your trained model on each row

Writes the predictions back to a Foundry dataset

This is a natural extension of what you already built, and it reuses the same model artifact — no retraining needed.

There are actually two tools you could use for this: 1

Python Transforms (Code Repository) (BEST FIT)

Write a Transform that uses ModelInput to load your model and run .transform() on a dataset. Supports pinning a specific model version and running the model as a sidecar (recommended to avoid dependency conflicts). 2

from palantir_models.transforms import ModelInput

model_input = ModelInput(“", use_sidecar=True)

Pipeline Builder

A no-code option — drag a trained model node onto the canvas and connect it to your dataset. Simpler, but only supports models with a single tabular input/output. 1

Since your demo is already built in a Code Repository, Python Transforms is the more natural fit and gives you more control. It would also make a great next section for your tutorial — showing the same model used for both live inference (Workshop) and batch inference (Transforms).

with a batch inference pipeline (or also a stream input) , we could detect any strange situations

in very large datasets. any abnormaliities. …. this gets to the core of what palantir was orginally designed to do… look for a needle in a haystack , some kind of activity that might indicate some hidden threat ?

That is exactly the right framing. Palantir was built around the idea of finding meaningful signals in massive, noisy datasets — and what you are describing maps directly onto that core mission.

Here is how your demo could evolve into that pattern using Foundry’s tools:

Batch Inference — Find Anomalies at Scale

Run your model across an entire dataset on a schedule. Records where the model’s prediction diverges significantly from the actual value become your flagged anomalies — the needles in the haystack.

Tools: Python Transforms or Pipeline Builder for the inference pipeline, writing flagged records back to a Foundry dataset. 1

Streaming — Real-Time Detection

For truly real-time detection, Foundry supports streaming pipelines that process records as they arrive with very low latency — on average under 15 seconds from ingestion to availability in the Ontology. 2 This means your model could score incoming records in near real-time as events happen.

The key trade-off is cost and complexity — streaming compute runs continuously, whereas batch runs on a schedule. For use cases that can tolerate more than 10 minutes of latency, batch is simpler and cheaper. 3

Surfacing the Results

Once anomalies are flagged, the Foundry stack lets you operationalize them:

Ontology —

flagged records become Objects with properties like anomaly_score or risk_level

Workshop —

analysts see a live operational UI showing flagged cases

Foundry Rules —

trigger automated alerts when thresholds are breached 2

Actions —

analysts can act directly on flagged objects (escalate, dismiss, investigate)

So the progression from your demo is clean: same model, same tools, just pointed at a larger dataset with a pipeline that runs continuously or on schedule — and suddenly you have a genuine threat-detection or anomaly-detection system, which is exactly what Palantir was originally designed to power.

THE CORE GIST OF THE DEMOS !!!!!!!!!!!!!!!!!!!!!!!

(1) so in our specific example, we could detect what kinds of abnormalities?

(2) but in real examples, what might the dataset and modell be about? for example, for 9/11, if there had been a palantir app running looking for unusual numbers of foreign residents wanting to train to fly large airliners. or

(3) in current wars, if tracked leaders of a rogue state all start doing soemthing unusall at one time… can you give some examples?

Great framing — let me take each part in turn.

(1) In Your Housing Demo — What Anomalies Could You Detect?

Your model predicts house prices based on features like square footage, location, etc. Anomalies would be records where the actual value diverges sharply from the predicted value, which could indicate:

A property sold far below market value (distressed sale, fraud, money laundering)

A property sold far above market value (inflated appraisal, market manipulation)

Unusual feature combinations (e.g. tiny square footage but extremely high price — could indicate a data error or something more interesting)

(2) & (3) Real-World Examples

This is where Palantir’s original mission becomes clear. The pattern is always the same: model what “normal” looks like, then flag deviations at scale.

Counter-terrorism (your 9/11 example) Exactly right.

A model trained on normal flight school enrollment patterns could flag clusters of foreign nationals enrolling simultaneously in heavy aircraft training with no commercial aviation career profile. The signal existed — the challenge was connecting it across siloed datasets.

Military / Geospatial Intelligence Tracking movement patterns

of known military or government figures. If leaders of a rogue state all simultaneously move to hardened bunkers, go radio-silent, or alter convoy patterns — that behavioral deviation from baseline is the signal. Palantir’s Gotham product was built precisely for this kind of pattern-of-life analysis.

Financial Crime A model trained on normal transaction behavior flags

accounts with sudden spikes in cross-border transfers, structuring behavior (many transactions just below reporting thresholds), or connections to flagged counterparties.

Pandemic / Biosurveillance Unusual spikes in hospital admissions,

pharmacy purchases of specific drugs, or search query patterns in a geographic cluster — all deviating from seasonal baseline models — can indicate an emerging outbreak before it is officially reported.

Supply Chain / Critical Infrastructure Unusual procurement patterns

— a state actor quietly buying large quantities of specific industrial components that have dual military use — flagged against a baseline of normal trade flows.

The common thread across all of these is exactly what you described:

a very large, noisy dataset, a model of what normal looks like, and a pipeline that continuously scores incoming data and surfaces the deviations for human analysts to act on. That is the Palantir loop — ingest, model, detect, operationalize, act.

26.0903 (v1 26.0903)

D8.1 Demo goal (0)