Building a community
During my time at an educational software company, I lead design and research on Community, which involved (among many other things) Forums.
This article is relatively complete, but it's missing several design and process details, of which I am in the process of adding. In the meanwhile, you can also check out this Figma document for the example work detailed in this project (and on mobile, too).
Still to flesh out:
- Mobile design considerations
- Quality checking methods
- Process work and wireframes
- Feature details and final screenshots
Background
Our company (we'll call it "A Cloud Company") had an existing in-house solution, but it was outdated both technically (using Angular 1.0) and from a design perspective. User experience was significantly different than the rest of the platform, and the way we grouped content caused many duplicate topics.
The goal of forums work from both a business and user perspective is increasing engagement and discovery, which are constructive with one another.
Engagement is (obviously) from a business perspective good for the "stickiness" of the platform. However, forums are also a "network" and benefit from network effects. More people who find use in our community will lead to more contributions and ultimately more quality answers.
We deliberately chose from the start to keep all information on our forums open for any anonymous user to access. It's good for SEO, and it's a great way to contribute something back to the community, which is hard to come by these days - reliable hosting of valuable information.
Problem space
Several issues plague the old forums, both in overall architecture and visual design. Some lowlights are listed below.
- Visibility of the feature is poor and users aren't always aware that forums exist
Opportunity: how can our platform integrate the forums as a valuable resource into course content?
- Some learners report negative experiences interacting with other users.
Opportunity: what ways can we design social tools to maximize positive interaction while making negative interactions more difficult?
- Legacy forums feel inactive and dead.
Opportunity: while this is partially due to poor visibility, there is no indication of activity stats anywhere on any community page.
- Searching for the right content is nearly impossible.
This is another design flow and engineering problem.
- Staff have few moderation tools.
Opportunity: interview current moderators and flesh out what is needed for a moderation platform.
Competitive analysis
Before developing anything, our team investigated several solutions available on the market, including Vanilla Forums and Discourse. Ultimately, these solutions were inadequate with the future development we planned. Eventually, we wanted to work site content into forum discussions. For instance, showing a forum question with the video lesson it is about in context. The off-the-shelf products didn't easily allow for this.
We analyzed about 15 competing and comparable services resulting in a list of tablestakes. Among the most frequent features were notifications, peer moderation, categories, and media uploads. These tablestakes are a few of what user expect when using a forum.
However, just because something appears frequently in other products does not mean it will work for our product. These tablestakes served as a useful jumping-off point into what we should explore.
Student surveys
I sent surveys out to users of our product utilizing our in-app messaging. In this survey, I asked what learners like and dislike about their current community experience. Some highlights (and lowlights):
- Learners loved the environment of helpful students and appreciated staff engagement.
- The structure in which we present community content is confusing.
- Forums suffer from significant performance issues.
- Response times are the biggest concern to users. "When will my post get a response?"
Moderator interviews
I also talked with staff to identify moderation issues they face.
- Moderators had issues finding the right content, using Google with
site:acloudsite.example
over our built-in search. - It is impossible for them to tell what video lesson to which a question is referencing.
- Composing a message is painful & unreliable. Huge paragraphs and significant work can be lost with a mis-click.
- The current forum structure is unmanageable, leading to constant content duplication and hidden resources.
- There is a serious lack of moderation tools available. In fact, there's not even an easy way to remove posts, so moderators often just edit offending posts to say "[[redacted]]". Not a great system!
Stakeholder interviews
I conducted interviews with five stakeholders to guide forums prioritization and features, and observed several patterns:
- Legacy forums feel outdated.
- Don't re-invent real-time chat (we use Discord already).
- Forums should be a knowledge repository.
- B2B users could use a personalized instance of forums as a knowledge base.
- Building self-sufficiency with user generated content is key.
- B2B learners don't think of themselves much differently than individual learners. While their company provides the account, they still learn and communicate as individuals. This was later proven by multiple studies after our Pluralsight acquisition.
- Inclusivity is key, our community has to be welcoming and friendly. Psychological safety is important. Students shouldn't fear asking a question.
- Users are not returning to the platform regularly. Forums can be a driver for more routine engagement, from gamification to harnessing the intrinsic desire to help others.
- Legacy content needs to be considered when moving over to a new platform, so we don't mince all the good content.
Summary
- Everyone seems pretty unified in that forums should be a knowledge base. Social stuff belongs on channels like Discord.
- The biggest issue with forums as of writing is that it feels separate from the rest of the platform. A tighter integration with learning content would be great.
- Students categorize content based on cloud vendor (e.g., AWS, GCP). One possible explanation is our platform has a majority of single-cloud learners. It makes sense to structure content around that pattern, especially in forums.
- Staff can really use some better tools. What we released/are releasing will help staff find what they are looking for more easily, but robust moderation tools are still needed.
Personas
Personas are a useful tool in visualizing the different users of forums so that flows can be designed around a variety of important needs.
- Learners compose the majority of active forums users. They use forums to augment their learning experience.
- Volunteer moderators are users who have demonstrated interest and ability to help administrate. Dubbed "Volmods", these helpful users answer questions and remove spam in exchange for a free yearly subscription.
- Staff instructors use forums to answer questions and address problems with content directly to learners.
- Marketing staff use forums to drive traffic to A Cloud Company via search engine results.
- Anonymous users arrive at A Cloud Company forums for information via a 3rd party search engine results.
- While business admins aren’t discussed here, and there are no special considerations with this pass, it is useful to think about how to expand forums to better suit B2B in the future. This was taken into account when designing the final structure of new forums.
Beginner Ben
Behaviors
- Ben is a beginner to the platform and to cloud computing
- He doesn’t know what he doesn’t know. In other words, he has no idea where to start.
- He mainly sticks to learning content, consuming video lessons and quizzes.
- He's focused on getting certified to assist with a career change.
- He arrives at forums by clicking a linked page on the video lesson page.
Needs
- Ben needs to understand a confusing concept presented in a lesson so he can continue their learning.
- Ben needs to understand unknowns around exam expectations and experiences so he can successfully pass their certification test.
- Ben is confused on how to find content and search does not return helpful results. This results in choice overload.
Instructor Ivana
I’m supposed to be making content, not refreshing the screen all the time!
Behaviors
- Ivana creates video lessons and other content for A Cloud Company.
- She is focused on answering questions, helping students, and keeping high content ratings
- She occasionally removes abusive or low-quality posts, but only when she comes across them while answering other questions.
Needs
- Ivana needs to eliminate knowledge gaps in her learning content and improve lesson quality.
- She can’t distinguish different types of posts. Students will posts celebrations, questions, complaints, and bug reports all in the same channels.
Drifter Dae-sung
Behaviors
- Dae-sung is an intermediate cloud engineer at a medium agency.
- He does not have an A Cloud Company account.
- He has very specific and targeted questions.
- He's Focused on obtaining information first and foremost, and is not necessarily interested in creating an account unless he needs to do so.
- He arrives via search engines like Google.
Needs
- Dae-sung is running into an error with a Wordpress installation on AWS and needs to diagnose the error before she can continue building out a client’s site.
- He is gone once he gets his info and misses an opportunity to delve deeper into the problem with A Cloud Company content.
User journeys
From these personas, I created journeys detailing different aspects of the forums experience, and how each persona would react and feel. This was the basis for initial sketches and later designs.
IA
The legacy forums information architecture made it impossible for users to find the content they were looking for. Every discussion must belong to a specific single category called a "room" focused around a specific course on our platform. Constant growth of content led to nearly 500 disparate rooms.
I conducted both internal and external card sorts via Optimal Workshop to determine a user's ideal forums structure.
Dendrograms
These results were normalized and synthesized into dendrograms, which visually show the grouping of agreement, as well as agreement between internal and external participants.
Internal staff
External users
Interpretation
Despite considerations made to avoid obvious groupings, users still group content overwhelmingly by cloud vendor (AWS, GCP, Azure). This can be explained by user goals and learned behavior. Students tend to study one vendor at a time, so categorizing everything by vendor makes natural sense to them. Additionally, our site search already delineates most content by vendor, leading to an expectation for that pattern to continue elsewhere.
Old structure
The legacy forums structure does not support what users confirmed during the card sort exercise. With a 1-to-1 relationship between a specific course and a post, students create duplicate topics across related courses.
In this example, similar posts are repeated in every course room. (This is also the first time i have used the "tape" tool in FigJam.)
New structure
New forums allow filtering and searching by vendor (AWS, GCP, Azure), and remove the necessity to map a post to a particular course. Now, a course can be linked to a post, instead of the other way around.
More importantly, all content is easily accessible under the "Azure" tag, instead of only being associated with a course. This relationship allows for a more natural and flexible discovery experience that is based off of user expectations.
In this example, duplicate posts are removed. Instead only one post is needed.
Design studio
I ran a design studio (also known as a co-design session) with the team. This activity is fantastic for aligning entire teams, and gives everyone a chance to apply their individual expertise to the product.
Design studios are comprised of a few basic steps .
- A brief overview of the problem we are trying to ideate solutions for
- Sketching various solutions to problem statement
- Going over sketches and ideas as a group, learn from individual explorations
- Pair refinement
- Converge on ideas
Takeaways
- Incentivizing replies to older posts without replies can improve engagement and resolution time.
- Filtering and other discovery options help improve visibility
- The time frame of when to receive an answer is extremely important to users
Included an image below showing one of the solutions the team came up with in greater detail, where it shows the expected amount of time before a question is answered. Outputs of these activities are saved in Miro, image shown here is an example of one of the sketch outputs from the team.
Team planning
Often in agile teams, designers work towards the "MVP" with development, planning the minimum that could be potentially considered a product. Features that would take more design thinking or development discovery are put in the "future release" pile - but the problem is that pile only grows, and is eventually forgotten like an old landfill. User centric design involves designing the best experience possible, not the best experience within current constraints.
Phases
With this in mind, I planned to have development have about a month of lead time (a dev cycle) to give product proper time to discover, research, and execute these solutions. I designed ideal states for forums, and then retroactively sliced it up into manageable phases for development, which were subsequently planned out. No more kicking the can down the road!
Prioritization
The team had to deliver everything needed to transition off of legacy forums completely as an "MVP", which was relatively large. Along with that, the team committed to adding some additional improvements which were prioritized together. Along with PM insight, we created a priority map for search & forums features.
Feedback loops
To ensure we gathered feedback during and after release, we planned what relevant data should be sent out to amplitude and FullStory, with proper precautions in place to ensure GDPR compliance. This was done at the beginning of our planning.
Search
Top Result
Top result highlighting
In-context results
Context without leaving search
Views to indicate activity
Read a discussion
- Eliminated comments in lieu of top-level replies in the thread
- Eliminated voting, and esp. Downvoting
- Emoji reactions meant to replace the need for some short “thanks”-type comments
- Shows course at top of discussion if there is an association (but this is not the main way to find content)
- Also shows keyframes of where in time this discussion was asked, video preview
Engaging in discussion
Create a discussion
- Old create discussion
- Needed to create within a room
- Required a lot of steps meant to root out duplicate conversations, but was just needlessly complex.
Landing page
Legacy
New designs
- Built around personalization
- Latest activity from topics followed
- Can follow topics
- Items personalized by lesson consumption
- Get user engaged with actual content instead of categories/rooms
- Highlight activity of forums and make it feel as lively as possible
Future
- We wanted to build upon learn integration and more visually show discussions without leaving the context of a lesson. This sketch is showing questions represented as captions underneath the video scrubber.
- Forums can be even further surfaced on other lesson pages, like the course overview. Students spend most of their time with our learning content, so showing posts that are useful to them in their period can be a good way to obtain more users.
This project is locked. To see a full case study, enter the password to continue.
To get a password, please message me.