AI Product Ethics

Product teams often treat ethics as something the legal team handles — a checklist that arrives late and gets filed. That framing is expensive. Ethics in AI product design is about product decisions your team makes every sprint: what data to collect, how to present uncertain output, whether the AI acts or just suggests, who gets access on what terms. Most happen before legal is in the room.

The stakes are higher than traditional software because AI harms at scale. A bug affects one user. A biased recommendation engine affects everyone the product touches. The blast radius of a single bad design decision is qualitatively different.

AI amplifies dark patterns because they can be personalised to each individual at scale. Three types carry the highest risk.

Manipulation. An AI that learns what emotional triggers drive purchases and consistently exploits them is a persuasion machine. A travel app that surfaces pricier options when it detects time pressure — because your data shows you stop comparing when anxious — is manipulation dressed as personalisation.

Addiction mechanics. Optimising purely for engagement time structurally incentivises designing for dependency. The PAIR guidebook distinguishes engagement that advances user goals from engagement that is its own end — make that distinction explicit in your metrics (see Lesson 8: Measuring AI Product Success).

Deceptive AI identity. If a user sincerely asks whether they are talking to an AI, the answer must be honest. One discovery destroys trust in the feature and the company.

The most consequential design decision in many AI features is whether the AI augments human judgment or replaces it.

Augmentation means AI suggests and a human decides. A doctor whose AI diagnostic tool surfaces patterns before they choose a treatment plan is augmented — judgment is enhanced, not bypassed.

Replacement means the AI decides and the human either ratifies without real review or is not in the loop. An automated moderation system that bans users without human review is replacement. The ethical problem is accountability: when an automated system makes a consequential decision about you with no appeal path, the power relationship has shifted in a way most users find deeply uncomfortable.

Informed consent means users understand what they are agreeing to before they agree — in the interface at the moment it matters, not buried in a privacy policy. Three disclosures are load-bearing for AI features.

What is AI-generated. Disclosure prominence should match consequence. An AI-generated legal summary with no label is a transparency failure. An AI writing assistant where the user knows they are getting suggestions is not.

What data is used and for what. Did the AI read my full email history? Is my usage training the model? These are reasonable questions that deserve honest, findable answers — reachable in under 30 seconds.

What the AI cannot do. A medical information AI that answers every question with the same confidence is failing users who cannot distinguish reliable from unreliable answers.

A feature that works brilliantly for some users and poorly for others has an equity problem, even if the disparity was unintentional. Three structural properties of AI create this risk.

Training data gaps. A hiring tool trained on historical promotion decisions encodes whatever biases those decisions contained. The gap is invisible in aggregate if your primary user group is the one the model performs well for — segmenting by demographic proxy reveals the reality.

Access gaps. AI features are often locked to paid tiers, newer devices, or high-bandwidth connections. Users who most need certain capabilities may be systematically excluded, widening rather than narrowing existing disparities.

Literacy gaps. AI interfaces reward users who know how to prompt and interpret uncertainty. A feature that works well only for technically confident users is not equitable by design.

Run this before a feature enters engineering so findings shape the design rather than being retrofitted. A team can work through it in 90 minutes.

Step 1 — Define the feature and population. One paragraph: what does it do, who uses it, at what scale? Ten internal analysts have a different risk profile from ten million consumers.

Step 2 — Dark pattern audit. For each dimension, identify the risk and how the design mitigates it.

> Describe the feature. For each item, identify the risk and mitigation:
> 1. Does it exploit emotional vulnerabilities for commercial gain?
> 2. Does it optimise for engagement that conflicts with user wellbeing?
> 3. Does it present AI as human, or obscure the AI's role?
> 4. Does it use consent flows designed to maximise data extraction?

Step 3 — Autonomy assessment. Classify each AI decision using the table from Step 3. Flag any high-consequence decision lacking human review.

Step 4 — Consent review. Apply the three-question rubric from Step 4. For any red answer, name the design change and an owner.

Step 5 — Equity audit. Who is in the training data — and who is not? Who is excluded by price, device, or language? Does the interface require AI literacy your target users uniformly have?

Step 6 — Risk register. Document: Risk, Likelihood, Severity, Mitigation, Owner, Status. Issues without a clear resolution path should block launch, not trail as backlog.