Context in Consumer AI

Two years into the AI platform shift, consumer applications are leading global adoption. ChatGPT, Claude, Perplexity, and others have introduced AI to over a billion users and are quickly becoming the default interface layer for digital life. As these platforms take on more tasks, raw intelligence is no longer enough. The difference between a helpful assistant and a generic one is context: an understanding of the user’s routines, preferences, relationships, and history.

This includes both real-time context (what the user is doing, feeling, or intending in the moment) and long-term personal memory (what the agent has learned about the user over time). Intelligence becomes commoditized; context becomes the currency. Personal memory becomes the moat.

As foundational model performance converges, AI products will compete on this layer of user understanding i.e., products that deliver utility and delight, along with a clever mousetrap for collecting new unstructured data sets like notes, texts, search. This is as much a technical challenge as a behavioral one, requiring user trust and product design that makes giving access feel natural and worthwhile.

Existing channels already building context:

• Agentic browsers: Create a meta layer over search, workflows, and connected apps to build a unified view of user behavior

• Voice agents: Capture intent and surface-level knowledge through conversational interactions and query history

• AI meeting notetakers: Convert discussions into structured knowledge, adding context to professional memory graphs

• AI/AR eyewear: Gather visual, spatial, and social signals by observing what users see, where they go, and who they interact with

• Ambient recorders (pins, wearables): Collect audio, dictated input, and contextual signals throughout the day

The behavioral hurdle (user interaction + trust) is the key gating factor for effective AI agents and one that’s largely been overlooked in the technical conversations on agentic systems. To better understand how consumers think about digital context, we surveyed 250 U.S. adults on a simple question: what parts of your digital life best reflect who you are?

We used four lifestyle domains — health, finances, commerce, and social — as examples to ground the question in everyday behavior (although our results are likely equally applicable to other categories like knowledge, education, media or travel). The responses serve as a useful proxy for a broader question: which digital signals consumers would trust to stand in for them when it comes to AI agents.

TLDR: the richest context signals aren’t the most structured or obvious. They live in screenshots, message threads, spending patterns, saved media, photos as much as they do in trackers and spreadsheets.

I. Finance

For AI agents to manage personal finances, whether simple tasks like budgeting to complex tax filings or debt refinancing, they need a cohesive picture of the user’s financial life. But in practice, that context is deeply fragmented across sources, formats, and degrees of visibility.

First is where financial data lives. For some, it’s stored in formal tools like budgeting apps, credit card dashboards, shared spreadsheets. For others, it’s scattered across notes apps, text threads, shared bank access, TikTok confessionals, Instagram infographics about Roth IRAs, or even saved screenshots of job offers and bank balances. Some users follow creators for financial education; others post openly about salary expectations or career frustrations on LinkedIn. The financial graph doesn’t live in one system but rather is braided across a digital footprint.

Second is how financial behavior is expressed. Some people speak openly about money with friends, partners, or audiences. Others treat it as entirely private. That sociability often tracks with age: in our survey, respondents under 30 were significantly more likely to say text messages best reflected their financial mindset, while those over 40 ranked texts near the bottom.

Finally, even when the numbers are legible, the meaning can be misleading. A healthy savings balance may look like financial control, but a text thread shows it’s set aside for a looming student loan restart. Consistent on-time payments may indicate stability, but a budgeting note shows they’re being made by a parent or partner. A dip in discretionary spend could look disciplined; or could reflect unpaid leave, a breakup, or burnout.

Traditional financial systems are good at describing the what, but not always the why.

This is reflected in our data. When asked which parts of their digital life is the best reflection of their financial habits and mindset, there’s no obvious winner:

• 46% said credit or debit transactions

• 42% said shopping or order history

• 32% said notes or journaling

• 31% said text conversations, strongly inversely correlated with age

• 28% said budgeting apps

Meaning, even financial intelligence, which is, on the surface quite quantitative, can’t rely on structured inputs alone. Building agents that can reason across spending patterns, behavioral cues, and soft signals like text or even loose screenshots is what will separate tools that report financial behavior from those that can act on it.

II. Health

In consumer health, many health-related behaviors and symptoms are underrepresented not because they’re invisible, but because they’re captured out of context.

Take mental health as an example. Intakes like the PHQ-9 or GAD-7 which are the most common screening instruments for depression and anxiety are rarely administered at the point of emotional acuity. They’re static diagnostic checklists developed in the 1990s, often filled out in a waiting room or portal, reflecting how someone remembers feeling, not how they feel in the moment of pain. The result is a backward-looking snapshot that can be more useful for clinical billing and payer rationalization than timely or personalized care.

The alternative is to capture signals in the flow of experience like when stress first hits, when plans are canceled, when the fourth night of poor sleep sets in. That data may not live in a traditional health app. It may surface through screen time patterns, skipped routines, journal entries, changes in language in personal messages.

Our survey reflects this. While health records (39%) and fitness apps (35%) were still seen as top signals, a significant share of respondents pointed to more ambient sources: text conversations (32%), notes or journals (24%), even changes in calendar behavior. These may not meet the standards of structured clinical data, but they often reflect legitimate shifts in mental and behavioral health before a formal system detects them.

This makes health platforms uniquely complex because they need to bridge two axes at once, data types (formal, regulated inputs like EHRs and lab results vs. informal, unstructured signals like notes, routines, even social media behavior) and stakeholders (consumers, clinicians, and often payers). The most effective platforms will know how to toggle between these layers i.e., to translate clinical protocols, intake flows, care plans into the personalized, lightweight interfaces that belong on a consumer’s home screen, and vice versa. Put differently, delivering real, holistic value in the next generation requires platforms to be both consumer-ready and clinician-grade.

III. Commerce

Much of today’s digital commerce infrastructure is optimized to capture inferred intent and push users toward checkout. From the first programmatic impression to post-purchase flows, the stack, including targeting, attribution, retargeting, affiliate, and personalization tooling, is built to interpret signals like search, clicks, and cart behavior as buying intent, and to act on them quickly.

This system supports massive businesses: it underpins the attention economies behind Alphabet and Meta, and the conversion engines powering platforms like Amazon, Shopify, and Klaviyo. But it’s narrow by design. A user browsing a high-end household item, a niche beauty product, or a gift for someone else may trigger ads, emails, and recommendations, all of which are based on a signal that wasn’t representative of their ongoing taste, need, or purchasing power.

The engine knows what happened, but not why.

A critical evolution in the commerce stack is shifting more of the intelligence upstream from the transaction i.e., building systems that can interpret aesthetic preferences, financial constraints, evolving intent before anything is purchased. That means looking beyond structured product data and toward signals like what gets saved, screenshotted, sent to a friend, revisited, or worn. 

Our data backs this up. When asked which parts of their digital life best reflect their style and preferences:

• Photos (49%) were the strongest signal, a proxy for what’s already in use or admired

• Shopping/order history (39%) and text-based sharing (38%) followed closely

Wishlists, bookmarks, and social likes trailed slightly, but still held weight

IV. Social

From meeting assistants to messaging agents, every AI product operating in the social layer needs a working map of the consumer’s relationships. More specifically, if an agent is going to help manage consumers’ calendar, triage messages, or suggest who to check in with, it needs to understand the real-world social dynamics like who they care about and how they actually engage.

The default assumption is that tools like email or calendar offer reliable proxies for this. But they miss the texture like tone, frequency, and social prioritization. Like which meetings someone will actually cancel or skip. Or how to address two connections in a contact book with wildly different context.

Accordingly, when asked what best reflects their social world, consumers overwhelmingly pointed to text messages (54% said “a great reflection”) and shared photo albums (41%). Email and calendar, by contrast, scored much lower: fewer than 21% saw them as strong representations, and over 30% said they weren’t reflective at all.

In closing…

Takeaways for founders building consumer AI apps:

• Differentiate on context, not just intelligence. Foundational model performance is broadly available; the edge lies in how well a product understands the individual.

  • Leverage the data sources that matter most. Shopping history, text conversations, notes, and photos often reflect more than trackers or dashboards.

• Create new knowledge graphs around user behavior. Capture context incumbents can’t see i.e., the ambient, behavioral, and emotional signals that live outside legacy data sets.

• Design delightful context-capture mechanisms. Products should gather meaningful data through use, not through friction-heavy setup or repeated prompts.

• Design for earned trust. In memory-driven systems, platforms earn access to personal data through repeated (and early) proof of value. 

A few products already building on this new frontier:

• Blockit: a personal coordination layer that lives inside email, functioning like a high-power EA

• Text AI: social planning and support built around group texts, one of the most underutilized but persistent fabrics of modern social life

• Doctronix: medical Q&A and care navigation through a clinician-linked AI interface, while passively assembling a longitudinal health record

• Granola: a workplace context engine that synthesizes meetings, notes, and project inputs into persistent memory

• Perplexity: a personal knowledge assistant that turns search and query behavior into a structured, recallable knowledge graph

• Final Round AI: career support and coaching trained on past interviews and role-specific context

• Limitless: an ambient productivity layer built from wearable-captured moments, notes, and routines

• Doji: captures user likeness to personalize fit and style recommendations, bridging physical data and aesthetic preference in commerce

Keeper: matchmaking built on richer intake data and continuous learning from real relationship dynamics,  moving beyond static profiles to adapt with each interaction

If you’re building in this space, we’d love to hear from you. Email abordetsky@nea.com and hworland@nea.com to continue the conversation.