I’m at Write the Docs today in Portland and will be posting notes from sessions throughout the day. These are all posted right after a talk finishes so they’re rough around the edges.
James started things off after lunch. He works on support.mozilla.org and the Mozilla Developer Network.
James started talking with a short history of SUMO, which Michael talked a bit about yesterday. Through a series of redesigns they got to the design they now have. In that process they worked through solving the problems of earlier iterations.
When they tried solving this problem they lacked a good bit of data. What they did have, though, showed very low “helpful” scores on articles. They also had high exits from searches, re-searches, and high bounce rates.
One of the first things they did was have someone dedicated to the web side of support. They started with an heuristic evaluation and worked with a user experience expert on improving things. One thing they discovered in this was that if people got to the right article the helpfulness scores were very high. Outside of that, though, the scores tanked. They knew they had an information architecture problem.
They set out to analyze the current information architecture of the site. The first step was the manually look through the docs. They looked at what articles they had, where they were linked from, and the taxonomy that existed. To help with this they did a card sort, a means to guide users to generate a category tree.
With the map they had from the card sort they used Treejack and limited the user testing to just displaying the title of docs. The goal for users was then to say, “This is where I will find my answer.” With their current architecture of the time the success rates were as low as 1%. That’s bad. With that, though, they now had data. They had something they could work with and could optimize. What they found was interesting. Some articles were missing, some were badly named, and some had other issues.
Their user experience people had a few ideas. They proposed and tested a few solutions. This took their success rates in user tests up to highs of 92%. One task specifically went from a 1% success rate all the way up to 86%. With Treejack they were able to run all these tests by focusing just on the titles. It meant they could test quickly without having to rearrange or rewrite all of their docs.
At the end of things 10% more people were coming to the site and finding their answer. They tracked this by graphing the rate of “helpful” scores on documents. That 10% meant 7.25 million more people a year found the solution.