Quartz Case Study
In 2016, Quartz, the digitally native news outlet for business people in the new global economy, broke new ground by re-imagining the news as a conversation. Its mobile app lets readers enjoy the top headlines of the day through a seamless chat experience. It was a defining moment for the media industry—and for conversational interfaces.
Two years later, the Quartz once again pushed the envelope with the release of Quartz for Facebook Messenger —a bot experience that takes conversational journalism to the next level. The Quartz bot provides a personalized messaging experience, picking up cues on when to send users content and remembering their preferences over time.
Whereas most bots struggle with retention (Forbes estimates the industry average after one week as 4%), Quartz for Messenger users have remained loyal well past that milestone, with a retention rate over 27%. According the Quartz data analyst Karen Hao, there are several reasons for this success: "We play to our strengths as storytellers, we offer brand-new experiences that cannot be attained anywhere else, and above all, we respect our users' time."
Quartz uses Chatbase Virtual Agent Analytics to keep a close eye on its retention rate. Since launching the bot, it also uses Chatbase to carefully tracks other important metrics in order to establish a performance baseline. For example, it uses Chatbase to:
Resolve not-handled messages. The Chatbase Messages Report offers visibility into any failure points across the conversational experiences, as well as the most common user requests. According to Hao, the report "allows us resolve bugs, improve our NLP capabilities, and produce more content in alignment with user interests. For example, we're already seeing user requests for news from specific regions or about specific topics."
Minimize exit rates. Using the Session Flow tool, Quartz evaluates where users have the highest exit rates. That can often lead toward reworking a specific conversation or rethinking what types of conversations users most enjoy.
Maximize completion rates. Quartz uses the funnel analysis tool to identify which experiences have the highest and lowest completion rates. That helps it learn from the experiences with the best completion rates, and apply those learnings to the rest. As Hao puts it, Quartz is always asking itself "How do we rework experiences with low completion rates to be more engaging, informative, and seamless?"
Hao says that with help from Chatbase, Quartz expects to achieve even higher retention rates over time. And in the coming months, it plans to iterate the bot's design and do careful A/B testing to bring an even better experience to users.chevron_left Back to Case Studies