Data-driven Documentation

Non-data folks in the house, this guide will help you understand how you can use data to create better, more relevant help docs for your customers and external stakeholders.

That said, some of the ideas here can very well be applied to creating documentation for internal stakeholders, so data folks in the house, this might be useful for you too.

Okay, so what kind of data can help create better docs?

In the context of this guide, we’re talking about data from the following sources:

• Search queries from Google
• Questions from communities and support tickets
• Search queries from internal sites

Matthey Guay has written a crazy amount of content for Zapier and others to help people do more with the tools they use. So when he offered to share some tips with the DB community, I was like — hell yeah!

Thanks Matthew, over to you.

Is documenting every little thing your product does worth the effort? Is it even practical to do that and ensure that everything is up-to-date?

Now, I’m not asking you to throw up your hands and admit failure — I’m here to offer a happy medium, one that requires a bit of research and a bit of patience.

You’ve got to let your audiences lead you.

Start with Google

One day, when your product is established and your support inbox is full of eager customers, you’ll have plenty of data to guide your documentation. You’ll have to rely on your competitors until then.

You likely already know some of the common struggles with your competitors' tools. You've built your app, in part, to solve them. The rest you can find with a bit of searching.

You can find what people are searching about your competitors from Google’s suggested queries.

Say you’re building a to-do list: Search “how do you trello” to see a list of common questions, with people asking how to delete boards, bold text, filter cards, and more. Repeat that for other competitive apps to build a more extensive list.

To dig deeper, use Google Ads Keyword Planner where you can find search volume and estimated ad cost (a proxy for how difficult it is to rank first on that search page) for keywords about your competitors.

Additionally, you can use paid SEO tools like Ahrefs for more detailed search query information, or Answer the Public, a free site that surfaces questions around any topic. These tools are designed for marketers but are just as useful when you need data on the problems your audience typically faces.

Find insights in real questions

There are many places to do this depending on where your audiences hang out as well as your customer support channels.

Communities

I’ve found a lot of success in running searches on sites like Stack Overflow, Reddit, Twitter, and even Quora for that matter for question words like “how” followed by a competitor’s product name.

Note those questions; they could be things you need to make sure to document or new features you could add to lure people away from the competition in the future.

On StackExchange, as in the screenshot above, you can sort by most upvoted and find the most-discussed questions — which likely indicates how many people had that issue or want that feature.

On Twitter, you could approximate something like that by searching for tweets with at least a certain number of replies or likes using Twitter Advanced Search.

Once you’ve built out your issue list, prioritize it by how often you see similar questions pop up across search and social media — this can be a great starting point to let data inform what you document.

Support Conversations

Then come the support emails. So many emails (and so many opportunities to understand what your audiences need).

People will ask the most basic questions; you’ll soon be able to detail your app’s pricing plans and how to reset passwords in your sleep. They’ll probably complain about things you’ll never fix and ask for features that’d never make sense to add.

But they’ll also point out things you should be documenting.

The first few weeks I worked on documenting Flow, then a brand-new to-do list app, I wrote little documentation. Instead, I kept a spreadsheet listing all the things people asked about, and standardized my replies to those that came in the most often.

Then I’d expand on those emails and turn them into documentation — paired with email templates to start replying to those standard queries in a couple of clicks.

It won’t make sense to document everything people ask. It will, though, quickly become apparent that 20% of your questions are about one topic, 10% about another, and so on. Document those questions, turn the docs into email templates, and with luck, you’ll find that only the most difficult, setup-specific questions are left to answer manually.

“Much of the best documentation comes from looking at what your support tickets are telling you,” wrote Jesse Parker in Zapier’s original guide to building an effective knowledge base.

Zapier’s app integration platform proved difficult to fully templatize support replies, but the more common issues around billing and account issues were easy enough. And the more thorny tickets still revealed broader issues the team could tackle in more detailed guides and documentation.

You couldn’t solve everyone’s issues, but with the support ticket data in hand, you had a much better shot at building guides that’d cover the majority of use cases.

See what people search on your site

Then there are the shadows — the things people wonder but never ask, the questions from users who are most likely to bounce to the competition.

These questions are likely to only ever be seen by the search dialog on your documentation site. And that, my friend, is a goldmine for your documentation efforts.

If you’re using an internal search tool like Algolia, or are using a help desk app that shows search results, check it every so often to see what people are searching for. You might even be able to find the data via your site’s Google Analytics or Google Search Console, if nowhere else.

Pay special attention to the queries that returned no results — or for that matter, those that return answers that don’t really match the query.

If the same questions keep popping up, that’s the list of questions you should answer with your documentation. You’ll still never answer every possible query, but with time you’ll make sure most people get an answer on their own most of the time.

Bonus: Write your competitor’s docs

Documentation is a never-ending challenge. As long as your team is improving your product and adding new features and integrations, the ways it can break and trip up users will continue to multiply.

Data will keep things manageable though.

You can check in on your search and support ticket data every so often to see what new docs are needed, or what older ones need to be updated.

And if you run out of things that need your attention, you could shift gears to start writing about the competition.

For there’s a fine line between documentation and tutorials. Say you know there’s something your product does better than the competition — you could write a detailed guide about how to accomplish that task using your competitor’s product, directly answering the question people had when they searched about that topic.

Of course, that’s not all.

In the same guide, you could also show how much simpler it’d be to achieve the same results using your product, tempting readers to take it for a spin.

Great documentation, it turns out, can actually be an effective customer acquisition tool as well.

If writing is your thing, I'm building something new to help folks write better: Reproof.