This year the SUMO team is focused on learning what to improve on our site. As part of that, we spent January setting support.mozilla.org up for A/B testing and last week we ran our first test!

The goal of the test was to run a series of experiments on individual Knowledge Base articles to:

• Improve navigation from KB article to KB article (in-article suggestions)
• Improve design of KB articles to ensure users better understand content and can engage with content faster

The two tests we are running are trying a bunch of different things, such as screengrabs, video clips, highlights, better feedback options on articles, and better navigation.

Version A: Breadcrumbs

• Screengrabs
• Ratings at different parts of the page
• Highlights
• On both experiments we have a section of related articles at the bottom.

 

 

A breadcrumb menu should make it clearer to users where they are.

 

 

 

 

Feedback points through up/down icons with a follow up question to understand to allow for more feedback.

 

 

 

 

Highlights in the text to help the user see the important areas.

 

Version B: Hamburger menu – Categories

• One rating at the end of the page
• No highlights in text
• On both experiments we have a section of related articles at the bottom.

 

Hamburger menu to allow for users to focus on the content not the menu.

 

 

 

 

Drop down to see wider menu.

 

 

 

The test will run for the next 2-3 weeks and we will report back on here and our weekly SUMO meeting on the results and next steps.

The test is currently serving for 50% of visitors and you can ‘maybe’ see the tests by going here or here.

SUMO staff team

https://blog.mozilla.org/sumo/2019/03/22/sumo-a-b-experiments/