As we focus on building a great user experience for the tens of millions of existing Pinners, it’s equally important to engage and retain new Pinners through the new user experience (NUX).
We recently rebuilt our new user experience and created a new framework to power it. Through the process, we determined the best content to show that would educate without overwhelming. Here you’ll learn how we arrived at a NUX that performs significantly better than the previous experience across all of our core engagement metrics.
Rethinking NUX from the ground up
We started by conducting qualitative and quantitative research to better understand new Pinners. The user experience research team interviewed a group of inactive Pinners to understand major pain points, while the data team analyzed a large sample of existing Pinners and determined the core set of actions that would increase the likelihood of a retaining a new person joining the site.
After looking at the insights and iterating on dozens of versions, we gathered new learnings about retaining new Pinners:
Demonstrate a simple value proposition that clearly shows off utility. A Pin is our primary value proposition so we immediately educate the person about how Pins work, and their value.
Image may be NSFW.Clik here to view.
Actualize the value proposition immediately. Searching and discovering Pins is a core feature, so immediately following the Pin step, we give education on how to find and save interesting Pins.
Image may be NSFW.Clik here to view.
Educate new Pinners at their own pace. The previous Pin and Search steps are mandatory for new Pinners because we’ve found they lead to increased long term engagement. However, if the Pinner doesn’t seem to get it the first time we’ll gently re-educate them on subsequent visits. For example, if he or she still hasn’t saved a Pin on their second visit, we’ll provide reeducation, and conduct the same process for board creation, following, and other features.
Image may be NSFW.Clik here to view.
Encourage immediate action. Understanding what it means to Pin early substantially increases the likelihood of retaining the new Pinner. He or she will get a simplified experience where Pinning is highlighted and other advanced features are hidden, until they save their first Pin. We call this the First Pin Experience (more on that below).
Image may be NSFW.Clik here to view.
After becoming active for the first time and saving a Pin, the Pinner will graduate to a richer experience.
Image may be NSFW.Clik here to view.
The need for a framework
The updated NUX is a multi-session experience that differs based on Pinner state such as what they’ve done, how long they’ve seen an experience, etc. Therefore we needed a system that could control what the Pinner experiences based on those variables. We also needed a way to easily run experiments to test different NUX steps, messaging, and educational units.
Similar experiences had already existed, such as new feature tutorials and education. We realized the logic powering these existing experiences were standalone and shared similar logic, and created an Experience Framework to build NUX and power new and existing experiences.
You can think of an experience as any feature on Pinterest, each of which require logic to determine when they need to be shown, persistence logic for when they’re dismissed/completed, and logging (i.e impressions vs. completions).
Image may be NSFW.Clik here to view.
Here’s how the logic was laid out in our client and backend:
Image may be NSFW.Clik here to view.
Here’s how the logic looks like with the Experience Framework:
Image may be NSFW.Clik here to view.
Each experience is configured in one place, the client delegates display logic to the backend, and persistence, logging and experimentation is all powered by the framework.
Boiling down to solutions
The Experience Framework answers one simple question: what experiences should a Pinner see on a given view within the app?
Every time the client renders a view, the Experience Framework will tell the client what experience the Pinner should see. For example, when rendering the home page, the Experience Framework tells the client whether to show a specific step in NUX, a tutorial, or a feed of Pins. How the decision is made is opaque to the client.
Image may be NSFW.Clik here to view.
The decision engine
The core of the framework is the decision engine, responsible for determining the experience a Pinner should see by considering configured and eligible instances for all potential views. The best experience is then decided upon based on static configuration (start date, seconds_to_expire, max_display_count, etc), the Pinner’s state (such as number of Pins created, level of engagement, and features experienced), the experience state (enabled, expired, view count, etc), the client type, experiment group, and many other properties. These decision parameters allow us to build complicated experiences like our First Pin Experience.
To recap, the First Pin Experience is shown to new Pinners who’ve never saved a Pin, and it lasts for no longer than 24 hours. The configuration is simple: set the seconds_to_expire to 24 hours and write a handler that will ensure it’s only enabled for new users with no Pins. In this experience we’ve also configured an experiment to further test whether 24 hours is really the best duration for this experience.
Experience.WEB_FIRST_PIN_EXP: {
'description': 'First Pin Experience.',
'start_date': '2013-11-01',
'seconds_to_expire': 60*60*24,
'handler': autobahn.FirstPin,
'experiment': {
'name': 'first_pin_duration',
'groups': {
'1_day': {'seconds_to_expire': 60*60*24,
},
'2_days': {'seconds_to_expire': 60*60*24*2, },
'7_days': {'seconds_to_expire': 60*60*24*7, }
}
}
}
We then associate the experience with a unique placement in the client (web home page).
Placement.WEB_HOME_TAKEOVER:
{
'experiences': [
Experience.WEB_MANDATORY_AUTOBAHN,
Experience.WEB_FIRST_PIN_EXP,
Experience.WEB_FIRST_PIN_USER_ED,
Experience.WEB_YOUR_BOARDS_USER_ED,
Experience.WEB_FAST_FOLLOW_USER_ED,
Experience.WEB_FIND_FRIENDS_USER_ED
],
'cooldown': SESSION_LENGTH # 2 hour cooldown - session length
}
There could be many eligible experiences on the home page placement. The framework resolves this by guaranteeing only one experience can be shown on that view.
Be fast or fail fast
The experience framework is the gatekeeper in determining what experience a Pinner should see, so at it’s peak it can see about 50k decision requests/second and growing. Since in some cases the view needs to synchronously call the Experience Framework before rendering, it needs to be fast or at the very least fail fast.
The major bottleneck in our case is I/O, i.e accessing persistent user and experience state data. We addressed this by:
- Storing all our data in an HBase cluster that’s highly optimized for retrieving state data
- Minimizing the number of calls to HBase
- Making use of gevent
Luckily we also applied much of our past learnings when optimizing HBase for fast online reads. As a result, we were able to achieve an upper90 latency of 30-40ms.
Even with fast response times, it’s important to avoid making any unnecessary backend calls. In order to achieve this, our clients periodically pull down and caches all displayable experiences and uses that to decide and render an experience whenever possible. However, keeping this “state of the world” cache up to date can be tricky.
As a last resort, if the decision engine takes too long to respond, we fail fast. This means in the worst case the Pinner will experience the default user experience, which is not ideal, but allows for the Pinner to continue using Pinterest.
Today the Experience Framework powers the majority of experiences on our website. The framework is also steadily powering experiences on our mobile apps, which is exciting because it enables us to dynamically render experiences as we run experiments without pushing a new release. You can expect to see better and improved experiences coming to a Pinterest app near you.
If problems like this interest you, the Pinterest Growth Team is hiring product-minded hackers to help billions of users worldwide discover the things they love and inspire them to go do those things.
Daniel Chu is an engineer at Pinterest.