We labeled the data. We engineered the features. Then we found out that knowing what users did isn’t the same as knowing why.
In previous posts, we walked through building a data pipeline and the feature engineering that sits on top of it. Since then, we’ve been hard at work using our behavioral data platform to predict user behavior.
Recently, we tried to ask the hard question: given this user’s experience so far, will they buy? Spoiler alert: it’s hard. It took us some time, but we found a way through. Here’s what we learned along the way.
Recap
Fullstory captures the live stream of what a person actually does: every movement, click, and scroll, including the ones nobody thought to tag. That stream lands in a warehouse as nested JSON, one object per event. Rich, deeply structured, but hard to query. So we unnest it, decide what counts as a qualifying session, attach a label, and load the result into a flat table an analyst can use.
That table is one row per event, which is great for an analyst but not for a model. We have to do the work to tidy up this data so that it is one row per session and one column per signal.
We split into train, validation, and test, and use the training set for model training. Three kinds of transformation (numeric aggregations, binary occurrence flags, categorical attributes) are performed, and what comes out the other side is a feature matrix.
The model we built.
We trained a classifier offline using the has_not_purchased label described earlier. When we were happy with the results, we made real-time inference possible: an endpoint a live session can hit, and a data layer wired up to carry the prediction back to the browser. It’s pretty cool.
But there was an interesting problem we didn’t anticipate. When should the prediction be calculated?
Most of our customers had a fuzzy idea of the stretch of journey they wanted to operate in, and a fuzzier one about when to act inside it. The boundaries we called the flow start and the flow goal: the first item into the cart, through to the order confirmed. The moment to act we called the intervention point. The second time a visitor opened their cart, say, or the point where cart value crossed some threshold.
The intervention point is a tripwire. Crossing it trips exactly one prediction: one payload, one score, one answer for the whole session. The session gets read at that instant and never again. How long the session currently is, what the visitor has done, where the cart stands. That snapshot goes to an endpoint hosting the model, which returns a propensity score for exit intent, the odds this visitor leaves without buying. The score comes back into the page through the data layer, ready for the page to act on.
All of this happens while the visitor is still on the page, in real time. Whatever was captured by the time the tripwire fired is everything the model will ever get to see.
One moment isn’t the whole story.
The original framing was a single intervention point. Pick the moment, score the session there, and act on the answer.
It worked in theory. Then we started building with co-creation customers, and the more we talked the more rigid it looked. The whole design rested on one question: when, exactly, should we fire? And we were answering it once, universally, for every visitor. Static. Not dynamic. A type three error, the right answer to the wrong question. The right moment isn’t a property of the site or the journey. It’s a property of the session, and it has to account for how each person moves through that journey.
It is easy to see how this breaks down once you picture the permutations the same events can take across sessions. Person A adds an item, takes it back out, adds two more, reads the reviews, and crosses the cart threshold somewhere around their hundredth event. Person B treats the cart like a wishlist, drops six things in, and leaves. One rule (cart > $150) fires for both, and hands the model two very different things. Person A arrives with a rich bundle. Person B trips the same wire seconds into the session with almost nothing in it.
Or take someone who is lost on a site and drifting toward abandonment. Wait for a static moment and you can miss them by being late. What you want is a way to trigger dynamically, on the momentum of the prediction toward abandonment.
So we changed the question. The pipeline was fast, fast enough that scoring a session once was a choice rather than a limit. Score it as the session unfolds and you get a prediction heartbeat, an abandonment EKG running the whole length of the visit. The trigger stops being a moment in the flow and becomes a measure of how fast this person is heading for the exit. Tailored to the session rather than to the site. Easier to catch early. The question is no longer when, exactly, should we fire? It is when does this visitor start showing clear signs of exit intent?
Right moment. Right message.
Momentum shipped. We worked with early customers on in-session marketing and on AI chat integrations. The one that worked best paired a momentum trigger with an agent that wakes up live in the session. Momentum decides when to wake it. Our session summary API then summarizes every event in that session into plain text and hands it over, so the agent arrives already knowing what happened. That is a lot cheaper than keeping an agent listening the whole time.
What we watched in replay was the thing we had been aiming at. The agent showing up at the moment a person would have walked away, with worthwhile advice. Timely, specific, and different every time. It never had to ask how can I help? It already knew. Looks like you are stuck between two sizes, here is the guide for this cut.
Intent is hard. Surprise.
Momentum had bought us real ground. No intervention point to agree on up front, and because the model was now scoring continuously, we could stop it at any moment in a session and ask which features were carrying the predictive power. Feature importances across the whole set, SHAP values tick by tick, so we could watch the weights move as the session went on.
The results were good. Not great. The features were not generalizing as well as we hoped, and when we dug into why, we found the next problem. Intent, actual intent, is hard. Someone clicks Add to Cart. Are they buying, or filling a wishlist to come back to? Someone sits on a review page for half a minute. Are they reading, or have they stalled out?
A click is a fact. What it meant is a guess.
Every feature we had was predictive. None of them carried intent. Counts, flags, attributes. A model trained on those learns that clicking Remove pulls toward abandonment, because in the training data it usually did. Then the wishlist shopper arrives, strips the cart down to the two things they actually came for, and checks out. The intent was to empty it and buy. The model read it as a pull toward the door.
Order matters.
The features were not wrong. They were flat. Each one answered did this happen. None of them answered when, or after what. We were saying, “This session contained A, B and C.” But nobody moves through a site as a bag of events. They move through it as a journey, with a direction and a shape.
Instead, think about the order of A, B and C. Not presence. Position. A then B then C is not C then A then B.
Take two sessions with the same four events in them. Session A goes product, size guide, add to cart, checkout. Session B goes checkout, size guide, product, add to cart.
A is a straight buy. Somebody picking out an outfit, and the right move is to put the matching shoes in front of them. B came in from an order already placed, checked the sizing, went back to the product and added it again. That is a wrong size, and the right move is to make the exchange painless.
Same four events. Different intent. Different next move. Our feature vector printed a similar row for both.
Enter embeddings.
The unlock is position, not presence. And because we capture the whole stream rather than a tagged subset, we have that position down to the finest grain we could ask for. Which means we had been collapsing away the one advantage we started with. The fine-grained temporal data was sitting right there, and our features threw it out.
So we stopped collapsing. We encoded the session’s event order as an embedding vector instead. Take the JSON event blob, turn it into a proper timeline of events, and hand that to an embedding model. Get back a vector that encodes the timeline itself, position and all, rather than a tally of what showed up.
That changes the shape of the problem. Instead of reading a bag of events and predicting whether this person buys, we can ask what kind of session this is. Event order inside the session starts carrying semantic meaning. Journeys become comparable across sessions, and searchable. Sessions that mean similar things land near each other. The distances in embedding space carry the relationships: what is similar, what is different, and how far apart.
Here is what that looks like.
The company an event keeps.
Linguists have a line about this: you shall know a word by the company it keeps. It is the idea every word embedding is built on.
Subtract man from king and what is left is royalty. Add woman to that and you land next to queen. Nobody taught the model what royalty is, or what gender is. It read enough text that the relationship became a direction you can travel in.
Sessions behave the same way. Three of the five events below are identical. The intent is not.
Once we started comparing sessions by distance instead of by counts, a semantic layer began to surface. A burrito order and a shirt purchase share no button, no URL, not one string. Both are the same four moves. Pick the category, choose the item, configure it, commit. Count events and they have nothing in common. Embed them and they land in the same neighborhood.
How we embed a session.
Four steps, run against a customer’s own warehouse. Nothing is mapped by hand and nothing is predicted. No model is trained at run time. It is a data pipeline that carries raw events through to named flows.
Tokenize. Every captured event becomes a token. A click on Continue becomes btnclick:Continue. A page becomes nav:/host-signup. When the same token fires twice in a row it collapses into one, so a burst of typing in a single field counts once. On the warehouse we ran this against, 298 million raw events came down to about 100 million tokens.
Sequence. Roll the tokens up per session, in order. One ordered array each. Of 2.3 million sessions, 2.2 million carried at least three tokens, which is the floor for anything worth mining.
Mine. Walk every contiguous run of three to eight tokens and score how tightly that run holds together, using normalized pointwise mutual information. A high score means these tokens travel as a unit. A low one means they landed in the same window by accident. The bar for calling a run a real flow is an nPMI of 0.8.
Name. A run that clears 0.8 is an ordered flow, and gets a name that carries the meaning. * Rental Booking. The tokens underneath stay as the evidence.
Above is one session, eight tokens in the order they happened. A five-token window sweeps across it, and every position it stops in becomes a flow candidate. Three stops, three candidates, each scored. Two come back under the bar at 0.61 and 0.42, which means those tokens tend not to sit next to each other in the rest of the sessions. The middle one comes back at 3.70. That one is a real flow, so it gets saved and named.
Not everything is a flow.
We also considered the things that do not fit the flow definition. Plenty of surfaces on a site are places people mill around on rather than march through. A car detail page, a date picker, a comparison view. Run the process above over one of those and we would never turn up a meaningful signal, because there is no one order everybody follows.
So we surface those a different way, off an anchor event. The anchor is the one token that says somebody is here, and on a car detail page it is click:car-result. From that anchor we run forward to whichever comes first: the next anchor, a page that means they moved on, ninety seconds of silence, or the end of the session. What that carves out is an episode rather than an explicit flow. Anything under five seconds gets dropped.
Each episode becomes a fixed-width vector of binary behavior flags, one per token on a committed list. Same idea as the feature engineering from the last post, so we get to reuse both the code and what we already understood about it. The twelve centroids it gets measured against came out of a k-means run over historical episodes, done once and offline, then lifted out of the warehouse and written into config. They are never recomputed while the pipeline runs. Whichever one an episode lands nearest decides the name it carries.
So one path answers did this person complete a flow? and the other answers how far into this surface did they get? Both come out the same shape. A named event with a business name on it, and the detection method invisible to whoever reads it later.
Before any of this ships, a human verifies. Somebody pulls up the real sessions behind a candidate and watches them, then confirms the name or throws it out. That is how cluster 7 became * Deep Car Consideration, and how the semantic layer ends up on the events instead of in somebody’s head. Token names are not ground truth. The replay is.
Intent becomes an event.
This is the part that closes the loop. Point the whole thing at a customer’s own captured stream, mine what their users actually do, and write the answer back into their Fullstory as named behavior. They instrument nothing. They decide nothing up front. The intent lands on the sessions it came out of.
Every named flow and every named surface arrives as a first-class event, carrying when it started, how long it took, and the evidence behind it. A segment. A funnel step. Something to hand an agent. Ask one a question against raw events and it reasons from ingredients, then hands back something plausible. Give it the same session with named behavior on it and it reads what happened instead of inferring it.
Both halves of the work land here together. * Car Search Sequence came out of the miner. * Deep Car Consideration came out of the episodes and the centroids. On the session itself they sit on the stretch of events they describe.
We set out to predict intent. We ended up teaching the product to name it.
Conclusion.
You can’t embed what you didn’t capture.
A gap in the sequence never announces itself. The model reads it as part of the session, learns the hole along with everything else, and hands back an answer with nothing in it to say a piece was missing. Embeddings cannot repair what was never recorded.
One way to get this data is to tag it. Pick the interactions that matter, wire them up, fire an event each time one goes off. We think that is the wrong approach. Tools built that way say plainly what they drop. Clicks on blank areas. Text highlighting. Anything judged low value. The stated aim is to avoid a stream of noisy user interactions.
Hiding in that noise is intent. The hesitation. The hover that went nowhere. The drift back up the page. Drop those and the flow they belong to cannot be found. You don’t get a weaker answer, you get no answer.
So we take all of it. Every click, every navigation, every form change, every element seen, every form abandon, every highlight, every paste, every pinch, every mouse thrash, every console error, every layout shift, every input delay.
Only full capture leaves you holding the whole sequence. And only the whole sequence gives you an embedding space built on the session that actually happened. The difference is what you keep.
Tag the click and you keep the fact.
Capture the session and you keep the story.
The dead ends were the route.
Nothing here worked the first time. That’s fine. We learn. We assess. We adapt. Every wall we hit looked like a dead end, and it turns out none of them were. Each was pointing at the way through.
One intervention point was the right answer to the wrong question. That roadblock sent us to scoring the whole session. Aggregate features were predictive but lacked intent. That sent us to event order. Position over presence only holds up if you have a complete view of the session. That sent us to what we capture.
Every dead end taught us something, and they all ran the same direction. They were the route.
This route was built on one huge foundation: Fullstory Fullcapture. Full resolution, every event, nothing dropped. That is what an accurate embedding needs, and we have been collecting it at this fidelity for more than a decade.
An accurate embedding space opens a lot of doors. Intent. Segmentation built on what people do instead of who they are. Similarity search across sessions that share nothing on the surface. Archetypes nobody had a name for. All of it running on data that was already sitting there.
Fullstory stopped being a capture company a while ago. Now we can claim to be a behavioral data platform, and this proves it.
Welcome to the future.
Fullstory Gold
The pipeline in this post has a name. It runs against a customer’s own Anywhere Warehouse data, mines the flows and the surfaces their users actually move through, and writes them back as named events on the session timelines they came from. No tagging, no deciding up front what matters.
We are building it with a small number of customers right now. If your warehouse is already connected and you want to find out what is in it, we would like to hear from you.