compare new vs power users

Power users rate setup 4.2 on ease; new users rate it 2.9. The gap opens at the permissions screen and stays open through the first task.

quotes from tune-outs

New users who tuned out pointed to the same moment: “I didn’t know why it needed my contacts.” “Too many steps before I saw anything useful.”

which step tested best?

Connecting the first account. Sentiment peaks there with both segments, and users who reach it are far more likely to say they’d keep using the app.

Welcome to your plan.

“People sign up, then drop off before they get value. We need to see where we lose them.”

see how we test your product

step 01

choose your tested audience

first-week sign-ups

power users

churned subscribers

We recruit the users your product is made for.

Multiple panels field at once without duplicate respondents.

Don’t worry about bad respondents. We catch that in real time, so every reaction is a real one.

26M+

vetted respondents

30+

countries fielded

duplicate respondents

24/7

quality detection

streaming app

onboarding flow

40 sessions

40 participants

step 02

kickoff

Get from kickoff to fielding in as quick as 24 hours.

Understand why new users drop off in week one.

goal 01

Find the moments onboarding loses people.

goal 02

Know which features to lead with.

goal 03

A request form quicker than email auto-prices your study and books the kickoff call.

step 03

fielding

Fielding that hits the right quotas, securely.

We field until every quota is hit, so you capture the right data from the right users.

onboarding walkthrough · react while you watchdrm · watermarked

respondent 4471 · mobile

14:22 / 42:10

step 04

results

Your users live react using our Sentiment Dial. We’ll tell you why.

You see every moment the audience leaned in, and every moment you lost them, while there is still time to change it before it ships.

max norm

min norm

0011230

😳😍🥱🤨

new users

power users

Sentiment climbs during setup, then drops at the permissions screen. Most tune-outs cite “didn’t know why it needed this,” so add one line of context before the ask.

dianne kamin

director, APAC consumer insights

Attention drops when the walkthrough explains a feature users have already found. Rec. cutting the line and showing the tap instead.

nina goetzen

consumer insights manager

step 05

reporting

Let’s get deliverables to your stakeholders.

We don’t dump data. We tell stories.

Pilotly’s researchers turn your data into a story your product, design, and growth teams can act on.

With clips and verbatims in users’ own words, in your custom format, delivered in days.

Talk to your study in plain language.

and ask it anything

where do new users get stuck?

The permissions screen. Sentiment drops sharply and most tune-outs say they didn’t know why it was needed.

don’t leave just yet

come to the Pilotly side

We support product teams in several ways.

Feature concept testing

use case

Design A/B testing

methodology

In-depth interviews

methodology

Virtual focus groups

methodology

App store asset testing

use case

Messaging & claims

use case

Ready when your release is.

Book a demo and we’ll walk this exact study through with your product, design and growth teams.

book a demo tell us about a new project