We launched Guided Search last year to give Pinners an exploratory search where they can discover the best ideas by clicking different guides to filter results. We’re continuing to make updates, such as recent improvements to show more personalized results based on who’s searching and building a smarter platform to understand queries. Today searches derived from guide clicking is one of the major sources of our search traffic. In fact, the number of guides clicked per day has tripled in the last six months, and we’re seeing patterns of its momentum. Guides change based on engagement, so the more people search and Pin, the better the experience gets. In this post, you’ll learn how we create and rank guides, as well as gain insights into trends around how Guided Search is being used for discovery every day.
Who’s clicking guides?
On average a Pinner clicks 3.6 guides daily when they use Guided Search. As for geography, Pinners outside the U.S. click guides more often than American Pinners. For example, Pinners in Argentina, Australia, Brazil, Canada, France, Germany, Italy, Japan, Mexico, Netherlands, Philippines and the U.K. are more likely to click a search guide than those in the U.S. Among these countries, Mexico has the highest guides click rate, where users are 46 percent more likely to click a guide.
The topic of the search also plays a role in whether or not a guide is clicked. For instance, Pinners who search for topics related to Celebrities, Fitness, Health, Home Decor, Humor, Men’s Fashion, Photography or Women’s Fashion are more likely to click a guide than those who search Gardening or History. Fitness related searches are 56 percent more likely to see guide clicking than Gardening related searches.
In general, men are more likely to click guides than women. We found men click guides most often when searching the topics of Art, Cars, Fitness, Health, Men’s Fashion, Outdoors and Shopping. Women Pinners tend to click guides when searching for topics like Food and Drink, Home Decor and Technology.
When and where are guides being clicked?
Guided Search launched on mobile first and was designed with a small screen in mind and optimized for tapping instead of typing, so it’s no surprise guides are clicked more often on mobile than web. iPhone has the highest guides clicking rate, followed by Android phone and iPad. In fact, iPhone users are 50 percent more likely to click a guide than those on desktop.
Additionally, Pinners are more likely to click guides during weekends than weekdays.
How are search guides created?
When we first launched Guided Search, we started from organic search logs. Since Pinners typically refine their search queries by adding or changing words, we wanted to find a way to extract these refinements from search queries.
We built a model to collect search queries in corpus. Queries are processed using a TF/IDF algorithm to get the most unique queries. Then, an entity extraction model is applied to the query corpora. Queries are partitioned into entity and guides. For example query “Red Nike Shoes” is extracted as entity “Shoes” and guides “Red” and “Nike.” We built a system to group synonyms, detect typos and remove porn or spammy terms. Both guides and entities are processed in the system to avoid showing inappropriate words.
Search guides generated in this way can cover 49 percent of Pinterest search traffic. Most of the search queries that show guides are popular queries. This is good coverage, but we wanted to push its boundary. So in September we started looking for a better way to generate guides for long-tail queries. If you draw the search pipeline vertically, you can imagine our first approach is top-down where guides are generated from user queries from the top. Instead, we challenged ourselves to try solving the problem from bottom-up.
This was our second approach to generate guides from search result Pins. On Pinterest, each Pin is hand-picked and labeled by Pinners to a board, and so the meta information associated with Pins is of a high quality. We tag Pins with annotations generated from meta information including container boards’ data, Pin descriptions, interest categories, linked third-party web page text and meta information of similar looking Pins. Figure 3 shows one Pin from the results of the search query “Outdoor Living” and the annotations generated for this Pin. We aggregated annotations of search result Pins into annotation corpora for each search query we saw. We built a system to partition Pin annotations of each search query so that each partition represents a meaningful subset of the query’s search results. For example, query “Outdoor Living” has guides generated from annotations “DIY,” “Decks,” “Patios,” “Ideas,” etc. Guides generated in this way can guarantee the composed query (e.g. “Outdoor Living DIY”) has enough high quality search results. Figure 4 depicts the architecture of guides generation.
After its launch, the search guides coverage was improved to 73 percent (as shown in Figure 5). Each search guide is associated with a cover image based on a Pin from the composed query search results. For example, the cover image of guide “Chicken” for query “Recipe” is a Pin in the search results of “Chicken Recipe.” We use “the most interesting” Pin as the cover image, which is selected based on several factors including the number of times a Pin is searched for, pinned and clicked on, and color tone. Each guide is associated with a list of cover image candidates which we dedup for each query dynamically, so Pinners are less likely to see the same image used for different guides of a query.
We store the meta data of queries and guides in QueryJoin to serve other features such as search query expansion and rewrite.
How are search guides ranked?
We’ve come a long way in the ranking of search guides. Today we have a sophisticated scoring and ranking system to order guides, and the ranking of guides for each query is calculated based on following scores:
- Interests to guides. How do Pinners click each guide of a query? The more interest a Pinner shows to a guide, the higher the guide’s rank.
- Quality of the results of composed queries. How confident we are with the search results after Pinners click a guide? The confidence score is calculated based on how Pinners click the result Pins and how often they add them to their boards. The score also considers the quality of third party web pages that the result Pins link to. In other words, the more Pinners like the search results, the higher the guide’s rank.
- Location. We started ranking guides by localization last December, and have since seen a 5 - 10 percent increase in guide clicks in many treatment countries, which shows us the search guides are more relevant and useful to users across the globe. As part of localization efforts, we also calculate guide location scores based on how much interest Pinners of various countries show to each guide. Figure 6 shows guides ranked differently to users in the U.S. and U.K. For example, Pinners in the U.K. have more interests in “London Street Styles” than “Parisian Street Styles.”
- Gender. In general, male Pinners have different interests in guides than females, and so we rank guides differently based on what’s trending for each group. Gender scores are orthogonal to location scores in ranking. For example, male users in Mexico see guides ranked specifically for their demographic.
- Current trend. We built a time sensitive scoring function to detect the current trend of users’ interests in guides. This function applies a recency boost to guides that have a momentum in ranking. If a large number of Pinners are interested in a guide in a short amount of time, this guide becomes a popular guide. Popular guides can be boosted to a higher rank for days. Once they lose their momentum, meaning less people are engaged in this guide, the function quickly ranks the guide to a later position.
- Spam detection. We detect spammy Pins, search queries and users, which are removed from guides ranking.
Since its launch, Guided Search has become an important driver of Pinterest search traffic and Pinner engagements. The number of daily searches on Pinterest also has greatly increased, with a 25 percent uptick in searches per-person. We’ll continue to make updates to Guided Search to make guides and results more personal and localized, with increasingly higher quality. Check back for more on these updates throughout the year.
If you’re interested in building search and discovery products like Guided Search, join us!
Kevin Ma is a software engineer at Pinterest on the Discovery team.
Acknowledgements: This technology was built in collaboration with Rui Jiang, Yuliang Yin, Pei Yin, Alex Bao, Yuan Wei and Ningning Hu. People across the whole company helped launch the features with their insights and feedbacks.
For Pinterest engineering news and updates, follow our engineering Pinterest, Facebook and Twitter. Interested in joining the team? Check out our Careers site.