Personalization with AI

45 min intermediate Lesson 6

Learning Outcomes

Distinguish the three layers of a user model and explain what data feeds each one
Select the right adaptation strategy — tone, complexity, recommendations, or defaults — for a given feature and user signal
Design a progressive personalization arc that starts generic and earns trust over time
Identify where personalization helps users and where it creates filter bubbles, over-reliance, or discomfort
Apply privacy-respecting techniques — user-controlled profiles, transparent explanations, and on-device processing — to a real product scenario

Lesson Plan

Segment	Duration	Topic
Intro	3 min	The personalization promise — and where it goes wrong
User models	7 min	Explicit preferences, observed behaviour, inferred attributes
Adaptation strategies	8 min	Tone, complexity, recommendations, and defaults
Progressive personalization	7 min	Starting generic, adapting over time
The boundaries	7 min	Filter bubbles, creepiness, and user autonomy
Privacy-respecting approaches	8 min	Transparent profiles, on-device signals, "why this?"
Design exercise	3 min	Applying the strategy to a learning platform
Wrap-up	2 min	Key takeaways and next lesson

Before You Begin

Pre-work:

Complete Lesson 5: Feedback Loops — personalization is the outward result of the feedback systems you design, so the two lessons are tightly linked
Skim Lesson 3: User Trust & Transparency to refresh the trust vocabulary (confidence indicators, explainability, user control) — all three concepts resurface here
Think of a product you use every day that feels uncannily well-matched to you, and one that feels intrusive or odd. Hold both examples in mind throughout the lesson.

Shopping List:

A browser and the course landing page open at AI Product Design
A whiteboard, sticky notes, or a notes document for the design exercise in Step 6
Optional: Google's People + AI Guidebook (pair.withgoogle.com) — the "personalization" and "mental models" sections complement this lesson

1 The Personalization Promise — and the Creepiness Line

Personalization is the difference between a product that says "here is everything" and one that says "here is what you, specifically, need right now." Done well, it is one of the most powerful tools in product design: the right suggestion at the right moment, content phrased in a way that clicks for this particular person, defaults already set to what they prefer. Done badly, it feels like surveillance dressed up as helpfulness.

The clearest way to understand the gap is a short comparison. Consider two versions of the same writing assistant:

Version A — a suggestion panel appears with: "You tend to open with a long preamble before getting to the point. Would you like me to move your key ask to the first sentence?" This is useful. It acts on a pattern the user can recognise in their own writing. It offers control (would you like me to…). It is transparent about the signal it used.

Version B — a suggestion appears with: "We noticed you wrote three emails after 10pm this week. Here is a template for late-night correspondence." This is unsettling. The observation is about personal behaviour outside the task, not the task itself. The connection to helpfulness is thin. The user did not ask to be watched at that level.

Both versions used exactly the same kind of behavioural data. The difference is whether the inference is about the work or about the person, whether control is offered, and whether the explanation feels proportionate.

A useful shorthand before building any personalization feature: would the user be surprised that you knew this, and would they feel the inference was used in their interest? If either answer is uncomfortable, you have work to do.

NOTE

Key Concept

Personalization that feels helpful is usually tied to the task the user is trying to accomplish. Personalization that feels creepy is usually tied to the person's identity, habits, or circumstances beyond the task. Keep your models task-scoped first.

TIP

From the PAIR Guidebook

Google's People + AI Guidebook recommends making it easy for users to understand what the system knows about them and why — and providing a clear, low-friction path to correct or delete that information. This is not just a privacy consideration; it is a core trust design principle.

2 Building a User Model: Three Layers

A user model is the internal representation your product builds of who a user is and what they need. In an AI product, the model feeds the system's decisions about what to surface, how to phrase things, and what to do by default. There are three distinct layers, and they carry different tradeoffs:

Layer	What it is	How it is collected	Tradeoff
Explicit preferences	Things the user directly tells you — preferred language, notification settings, topics of interest, communication style	Onboarding surveys, settings panels, profile pages	High accuracy, but users don't fill things in, and stated preferences often diverge from actual behaviour
Observed behaviour	Things you infer from what the user does — which suggestions they accept, what they edit away, which paths they take, how long they spend	Interaction logs, acceptance rates, session patterns	Abundant and honest, but noisy: a user might accept a suggestion because it was good enough, not because it was right
Inferred attributes	Things you deduce about the user that they never told you — expertise level, role, context, emotional state	Pattern-matching across behaviour over time	High leverage when accurate, but brittle: a wrong inference (treating an expert as a novice) degrades trust fast and can be hard to surface and correct

Most mature AI products use all three layers, but the weight given to each should vary by feature type and stakes.

> Use as a design prompt: "For [feature], list what we would know about a user from each of the three model layers after one week of use. Then identify which layer should drive the personalization decision and why."

WARNING

Inferred Attributes Are High-Risk

Inferring an attribute — especially one tied to identity, ability, or circumstance — can entrench bias or create self-fulfilling loops. If you infer a user is a beginner and serve them beginner content forever, you may never let them grow. Build a mechanism to surface and update inferred attributes, and consider letting users see and correct them.

The key design question for each feature is not "what do we know about this user?" but "what is the minimum we need to know to be genuinely useful here, and is what we collect proportionate to the value it creates?" This is the product-design version of data minimisation — and it applies whether you are dealing with personal data regulations or simply trying to avoid building something invasive.

NOTE

Layer Priority Rule

Start with explicit preferences and observed behaviour. Add inferred attributes only when the first two layers cannot give you the signal you need, and always provide a correction path when you do.

3 Adaptation Strategies: Four Things You Can Personalise

Knowing what a user model contains is one thing; knowing what to do with it is another. There are four levers available to product designers, and they are not equally appropriate for every feature:

1. Tone and register. Adjusting how the product communicates. A support tool that detects a user is frustrated and shifts to a warmer, more direct style. A writing assistant that mirrors the user's own vocabulary level. A dashboard that presents findings in the level of formality the user's past actions suggest they prefer.

2. Complexity and depth. Adjusting how much detail or sophistication the product offers. A legal review tool that gives a junior associate a plain-language summary and a senior partner the clause-level analysis. A data product that surfaces the headline metric for an executive and the drill-down for an analyst. This is one of the highest-leverage personalization levers because the right depth of information feels like the product understands your role.

3. Recommendations and ranking. Adjusting which options surface first, which are hidden, and what is suggested. A project management tool that surfaces the three tasks most likely to be worked on today, based on patterns in past behaviour. A reading app that orders an inbox by relevance rather than recency. This is the most familiar form of personalization, and the one most prone to filter-bubble effects (covered in Step 5).

4. Defaults and pre-fills. Adjusting what the product assumes before the user has done anything. A form that pre-selects the options a user almost always picks. A scheduling tool that defaults to the meeting duration and format this user prefers. A draft email that opens with the greeting style the user consistently uses. Defaults are powerful because most users never change them — meaning a good personalised default saves friction without any explicit interaction.

Signal available	Best lever to pull
User's vocabulary and phrasing from past interactions	Tone and register
User's role or stated expertise level	Complexity and depth
Past acceptance or engagement with similar content	Recommendations and ranking
Frequently repeated choices or form completions	Defaults and pre-fills

> Use as a design prompt: "For [feature], which of the four adaptation levers — tone, complexity, recommendations, defaults — would create the most value for users in the first month? Which lever has the highest risk of going wrong? Design a test for each."

TIP

Defaults Are the Most Underused Lever

Changing defaults based on user patterns rarely requires sophisticated ML. It can be as simple as 'last used' or 'most used.' But it compounds significantly: a user who never has to re-select the same option feels understood, and that feeling builds trust across the whole product.

4 Progressive Personalization: Starting Generic, Earning It Over Time

One of the most common personalization mistakes is demanding everything upfront. A new user encounters a fifteen-field profile form on day one, fills in three fields, and skips the rest. The product is now operating on a half-complete model it will never fully fix. A worse variant: the product makes strong personalization decisions immediately, on the basis of almost no signal, and gets them conspicuously wrong — teaching the user that personalization here is unreliable.

Progressive personalization is the alternative pattern. You start with a useful but generic experience, observe real behaviour over time, and let the experience evolve toward the individual as evidence accumulates. The result is a personalization arc that feels earned rather than assumed.

A useful arc has four phases:

Phase 1 — Generic with good defaults. The new-user experience is well-designed for the median user of this product. It does not try to personalise; it tries to be broadly useful. Duration: first session to first few sessions.

Phase 2 — Observe and infer quietly. The product collects behavioural signal without surfacing it. It builds an initial picture of the user's patterns. No personalization decisions are made from it yet. Duration: first few sessions to a few weeks.

Phase 3 — Make small, visible adaptations. The product starts making low-stakes personalization decisions and signals them: "Based on what you work on most, I've moved these to the top." The user can see the adaptation, approve it, and correct it. Duration: a few weeks onward.

Phase 4 — Deeper adaptation with user control. Once trust is established, the product offers richer personalization — and may prompt the user to fill in explicit preferences to make it sharper. The user now has reason to engage with profile settings because they have seen the benefit.

> Use as a design prompt: "Map our [feature] to the four phases of progressive personalization. What does the product look like at the end of Phase 1? What is the first visible adaptation in Phase 3, and how do we surface why it happened?"

NOTE

The Phase 3 Signal Moment

The transition from passive observation to visible adaptation is a critical trust moment. Microsoft's Human-AI Interaction guidelines (guideline 4: 'make clear why the system did what it did') highlight this specifically. If users see an adaptation happen without understanding why, they feel surveilled. If they see it and understand it, they feel served.

TIP

Don't Hide the Cold-Start State

A product that is not yet personalised for a user should not pretend otherwise. A subtle UI cue — 'This gets smarter as you use it' — sets the right expectation and stops users from interpreting the generic state as a failure.

5 The Boundaries: Filter Bubbles, Autonomy, and When Personalization Harms

Personalization is not always better than a neutral experience. There are three distinct failure modes that product designers need to design against explicitly — not just think about as edge cases.

Filter bubbles. When a recommendation system shows users more of what they have already engaged with, it can narrow their view of what exists. In a content product, this means a reader who clicked three articles about one approach will see fewer articles challenging it. In a professional tool, it means a user whose workflow already skews toward one method may never discover a better one. Filter bubbles are especially pernicious because they look like success on engagement metrics — the user keeps engaging — while the experience is quietly degrading.

Design responses to filter bubbles include intentional diversity injection (surfacing content from categories the user has not seen recently), "explore" modes alongside "for you" modes, and periodic prompts that invite the user to recalibrate their interests.

Autonomy erosion. Personalization that makes too many decisions on the user's behalf can gradually reduce their ability to operate without it. A user who always accepts the AI's recommended response eventually stops knowing what they would have said without it. A scheduling tool that always fills in preferences stops the user from noticing when those preferences have changed. Good personalization supports the user's judgment; it does not substitute for it.

The creepiness threshold. Even benign personalization can feel invasive when the user perceives a gap between what they shared and what the product seems to know. This is not purely about data privacy — it is a perception gap. A product can be fully privacy-compliant and still feel creepy if users do not understand the inference chain.

Failure mode	Warning sign	Design response
Filter bubble	Engagement goes up; variety of user behaviour goes down	Inject diversity; add an "explore" mode
Autonomy erosion	Users cannot complete the task when personalization is off	Keep the unassisted path accessible; celebrate manual choices
Creepiness threshold	Users report "how did it know that?" negatively	Add "why am I seeing this?" and a correction path

WARNING

High Engagement Is Not the Same as High Value

A recommendation engine that shows users only what they have liked before will produce high click-through rates and a worsening experience. When setting success metrics for personalization features, always pair an engagement metric with a diversity or breadth metric. If breadth is declining, your personalization may be optimising against user interests.

NOTE

The Autonomy Test

Before shipping a personalization feature, ask: if this feature were turned off tomorrow, would the user still be able to accomplish their goal without us? If the answer is no, you have not built a personalization feature — you have built a dependency. That requires a different design conversation.

6 Privacy-Respecting Personalization: Transparent Profiles and "Why This?"

Respecting privacy in personalization is not just about data handling — it is a design discipline. There are three approaches that materially change the user's experience of being personalised for:

User-controlled profiles. Rather than building a silent model the user can never inspect, surface the model as a visible profile the user can read, edit, and reset. "Your writing assistant profile: prefers concise phrasing, works in legal and finance contexts, tends toward formal register." A user who can see and edit this profile has agency. They can correct errors, update outdated signals, and decide what to share. The profile becomes a collaborative tool rather than a surveillance artefact.

Transparent "why this?" explanations. Every personalised output should have a reachable explanation of why it was surfaced. This does not need to be a technical explanation of the model — it needs to be a human-readable reason. "Suggested because you often work on Q3 planning documents this time of year." "Matched to your reading level based on your past article choices." The explanation should be available on demand, not in the user's face, but clearly reachable (a small info icon that expands is the standard pattern).

On-device and minimal-share approaches. Where technically feasible, personalizing from signals that never leave the user's device eliminates a category of privacy concern and a category of user worry. Browser history used to personalise suggestions locally, keyboard patterns used to improve text prediction on-device, document-specific preferences stored in the document rather than a central server — these approaches limit data exposure while preserving the personalization benefit.

> Use as a design prompt: "Design the user-controlled profile screen for [feature]. What would it show? Which signals would be visible, editable, and deletable? What would happen to personalization if a user cleared everything?"

Approach	Privacy benefit	Design cost	Best fit
User-controlled profile	High transparency; user agency	Settings UI complexity	High-trust, professional tools
"Why this?" explanations	Reduces creepiness threshold; builds trust	Per-surface copywriting	Recommendation surfaces
On-device processing	Minimal data exposure	Engineering investment	Consumer products with sensitive contexts
Minimal-signal defaults	Reduces inference surface	Lower personalization ceiling	Early-stage features, regulated industries

TIP

The Erasure Test

Before shipping any personalization feature, design the 'erase everything' path first. What does the user see after they clear their profile? Is the product still useful? If not, you have built a product that holds data hostage to function — a design pattern that will cost you trust the first time a user tries to use it.

NOTE

Transparency as a Competitive Advantage

Many users are suspicious of personalization because they have been burned by products that used their data in ways they did not understand or consent to. A product that shows its workings — 'here is what we know, here is why, here is the off switch' — can turn that suspicion into a differentiating trust signal. Transparency is not just an ethics checkbox; it is a product decision.

7 Putting It Together: A Personalization Strategy for a Learning Platform

The concepts in this lesson connect most clearly when applied to a specific design challenge. The outline from Lesson 6 of the original course proposes a learning platform — an ideal context because personalization genuinely helps (matching content to learner level, pacing to engagement patterns), but the failure modes are real (a learner stuck forever in "beginner" content because the system never updated its model, or a platform that knows their struggle patterns in a way that feels uncomfortable).

Here is how a coherent personalization strategy comes together for this scenario:

User model for a learning platform:

Explicit: Topics of interest, stated expertise level, preferred session length, notification preferences
Observed: Lesson completion rates, rewatch behaviour, quiz correction patterns, which sections users skip
Inferred: Current expertise level (updated from quiz performance), preferred learning pace, likely professional context (from content choices)

Adaptation strategy:

Complexity and depth: surface beginner or advanced versions of lessons based on inferred expertise, and update that inference when quiz results diverge from the model
Recommendations: surface "next lesson" and "related topic" based on completion patterns, but include an "explore something different" option on every recommendation
Defaults: default session length and notification frequency based on observed behaviour, never the platform's preference

Progressive arc:

Weeks 1–2: generic recommended learning path with a single onboarding question ("What brings you here?")
Weeks 3–4: first visible adaptation ("We noticed you skip theory sections — your path now goes straight to examples")
Month 2+: full adaptive path with visible profile, editable inferred expertise, and a monthly "how is this working for you?" prompt

Privacy choices:

A learner profile page shows the three inferred attributes (current level, pace, context) with an edit button on each
Every "recommended for you" lesson shows a one-line reason
A full reset option returns the learner to the generic path without deleting their completion history

> Use as a design prompt for this exercise: "Apply this personalization strategy to [your own product or a product you know well]. Where does the model break down? What signals are you missing in the first two weeks? What is the biggest filter-bubble risk for this specific context?"

TIP

The Personalization Audit

Once a quarter, review your personalization feature against three questions: Are users' experiences diverging in ways that reflect genuine differences, or in ways that limit them? Are inferred attributes still accurate for long-term users? Is the correction path still discoverable? These questions rarely get asked after launch — putting them on a calendar is how you catch drift before it costs you trust.

Questions & Answers

Q: We have very little data on new users. How do we personalise anything in the first session?

You mostly do not — and that is fine if you set the expectation correctly. A well-designed generic experience with a clear signal that "this gets smarter as you use it" is better than personalisation based on near-zero signal that gets things conspicuously wrong. Use the first session to collect one or two explicit preferences (the highest-leverage ones, not a full survey) and start building the observational layer. Phase 1 of progressive personalization is intentionally generic. The payoff comes later.

Q: Our product team is under pressure to increase engagement. Won't a recommendation engine that shows users what they already like just maximise that?

Short-term, possibly yes — and that is the trap. A filter-bubble-heavy recommendation engine will inflate engagement numbers while slowly degrading the quality of the user's experience. Users who feel they are being shown the same things repeatedly begin to disengage, often without leaving an obvious signal. Pair every engagement metric with a diversity or breadth metric: if users are engaging more but exploring less, your personalization is working against them. That tension is worth bringing to the conversation explicitly when engagement pressure is high.

Q: A user complains that our personalisation "feels creepy" but we are fully privacy-compliant. What are we getting wrong?

Privacy compliance addresses data handling, not user perception. Creepiness is a perception gap: the user has noticed something the product seems to know, and they cannot see the inference chain that produced it. The fix is almost always transparency, not less data. Add a "why this?" explanation, surface the signals that drove the decision, and make the correction path visible. Users who understand how an inference was reached rarely object to it; users who cannot explain it tend to assume the worst.

Q: How do we handle users who want zero personalization? Some of our users actively distrust it.

Design the fully-off path first, not as an afterthought. A product that functions well without personalization active is a product that has earned the right to offer it. For users who opt out, deliver the best possible generic experience, and do not quietly degrade it to pressure them back toward personalization. Some users will turn it on once they trust the product; others will not — and their use is still valid. Treating the opt-out path as a second-class experience is a design signal that damages trust for everyone.

Q: We inferred a user attribute — say, expertise level — and it is wrong. The user has been getting the wrong content for weeks. How do we recover?

The most important thing is to make the inference visible and correctable before this happens, not after. But when it has already gone wrong, surface the inferred attribute explicitly and invite correction: "We have been treating you as a beginner in this area — does that match how you see yourself?" Then rebuild the experience quickly from the corrected signal. The recovery moment, handled well, can actually increase trust — it shows the product is listening and responsive. Handled badly (buried correction path, slow rebuild), it confirms the user's worst suspicion about the model.

Key Takeaways

Personalization is task-scoped first — inference that connects to the work a user is doing feels helpful; inference about who the user is beyond the task tends to feel intrusive. Start with the former.
User models have three layers — explicit preferences, observed behaviour, and inferred attributes each carry different accuracy and risk tradeoffs; use the minimum layer necessary for each feature.
There are four adaptation levers — tone, complexity, recommendations, and defaults. Defaults are the most underused and compound most quickly with trust.
Progressive personalization earns trust — starting generic, adapting visibly in Phase 3, and offering user-controlled profiles prevents the cold-start failure and the wrong-inference failure at the same time.
Design for the failure modes explicitly — filter bubbles, autonomy erosion, and the creepiness threshold each require specific design responses, not just good intentions. Pair every engagement metric with a breadth metric.
Transparency is a product decision, not just an ethics checkbox — user-controlled profiles, "why this?" explanations, and clear erasure paths turn privacy-respecting personalization into a competitive trust signal.

Next Steps: Lesson 7: Prototyping AI Features