For many people, the premise that an interface should be tested and undergo comprehensive research is no longer surprising. Larger, more conscious organisations (such as OTP eBiz) are starting to supplement the development process with strings of interviews (including interviews to assess mental models)

Research and testing is recommended throughout the entire development process; situations where we want to perform a specific measurement on an existing interface can also arise after development has been completed because either we perceive a hiccup or our users tell us about it. Alternatively, we may find a flaw with the interface that could have been remedied much easier and at a lower cost if we had noticed it long ago, at an earlier design and development stage.

We have compiled a list of some of the free and low-cost solutions available online to test interfaces and collect related user feedback. There is an endless choice of solutions, so we will only look at a few and adopt a practical approach to the matter.

As Jacob Nielsen, the pioneer of UX design said, the greatest challenge when designing certain applications, websites, etc. is the structuring and naming of information. There are various applications for structuring the contents found on an interface. Pioneers of the field developed multiple methods to solve these problems. We are introducing some of these below:

Card sorting is an excellent, tried and tested method for grouping information found on an interface and categorizing the groups. During this process, the moderator asks participants to structure and group the planned contents of the website in a way that is logical for them. The different groups can be named and renamed. Card sorting offers insight on user opinions about the contents of our website and also on the distribution of the planned information that is considered logical by users. Sorting can be open or closed. In the former case, users sort the contents themselves and form and name the groups. In the case of closed sorting, members of the focus group sort content elements in predefined categories.

This method offers the advantage of helping to create and organise website structure and also sorting the contents and naming the categories. Because the contents are created by the users themselves, the structure of the future website will presumably be logical for them.

Sorting can be performed in person and in groups or one-on-one interviews. In these cases, we can also collect more in-depth qualitative data and investigate the reasons.

If we are focusing on quantity and quantitative data instead, in other words want to work on a larger sample, it is recommended to opt for an online solution that is easy to understand and access for users. In these scenarios, we use the OptilmalSort software.

Optimal sort
This allows us to test whether the number and name of menu items is actually optimal for users, not only us.

Benefits:

• If used online, it has the greatest reach (hundreds or thousands of users). It can also be used off-line on a smaller sample and in this scenario, give us qualitative data.

Drawbacks:

• When opting for online data collection, we do not know the reasons. It is best to combine the two methods and to back-test something that we have discussed on a smaller sample on a larger sample.

Pro use #1: 

• We asked participants to group contents under the menu items that we defined.

Pro use #2:

• We asked participants to group contents and to name the groups. We also discussed with them their reasons to gain insight about their thought process.

Beside this way of content categorization, we can earn great deal of knowledge from the analysis of web page traffic. In this field there are a wide variety of tools to be applied as well. With the help of these tools, we can learn about the source un-exhaustively, which is, what was the kind of the source page from where our users arrived to our website. This is especially important from marketing and SEO perspectives because we can learn where it is worth to place more ads.

Moreover, we can learn the ratio among platforms on which our website has been visited. That can be particularly useful, if we need to choose which platform is more urgent to develop. In addition we can also analyse the frequency of clicking on a given point and the covered road our users completed throughout their visit on our webpage. Another great tool is the automatically generated heatmap that can show us the most prominent and the least visited areas on the screen. Simple screen recordings, however, provide us valuable information about the pain points of our users.

Google Analytics
Google Analytics is the most widely known and most often used analytical solution that offers much more than merely measuring user traffic, its most popular usage. Analytics can be used for more complex measurements and analyses besides measuring basic traffic data (pageviews, bounce rate, session duration, etc.). For example it can be used to measure how many people clicked on a specific button, making it quite flexible and practical, rightfully giving it the status of the Swiss knife of web measurement.

Pro use #1: 

• Set a custom event and you will see it together with the page views.

Pro use #2: 

• Success on the web is also measured by money. Analytics is just as apt at calculating revenue per user as the number of visitors.

Benefits:

• Varied and diverse capabilities
• Universally used, hence data is comparable
• Free for small to medium traffic sites
• Any (javascript) event can be flexibly measured
• Backed by a relatively safe parent company

Drawbacks:

• Very costly for high traffic websites (over 1 million daily views)
• Data is only available with a one day lag
• Data is not stored in encrypted form

Pro use #1: 

• Set a custom event and you will see it together with the page views.

Pro use #2:

• Success on the web is also measured by money. Analytics is just as apt at calculating revenue per user as the number of visitors.

Hotjar
Hotjar is a simple tool for measuring user behaviour. It shows data graphically in a more visual manner compared to Google Analytics, transparently showing us how users use the website.

Although the website is known for its excellent heatmaps, the greatest advantage is to essentially see user sessions captured on video. Every UX test should begin by watching videos of randomly chosen users navigate the website for an hour or two. Flaws, shortfalls and misleading elements can thus be easily identified.Traditional heatmaps follow the user’s mouse movements and clicks to allows us to easily visualise how users navigate the website.

The site also provides tools for surveying and involving users. These are simple tools and not nearly as well-equipped as a laboratory-based solution, but may nevertheless be useful for addressing issues from time to time. Needless to say, eye tracking tests cannot be performed, but they are actually not necessary in many cases because clicks reflect actual use. So this tool can supplement often costly laboratory measurements.

Benefits:

• Heatmaps are easy to use and understand.
• Video recording of the user’s work session, which is essentially equivalent to observing the monitor from the perspective of the user’s head.
• Free for lower traffic websites (2000 page views/day).
• Excellent mobile support.

Drawbacks:

• A simple tool, however it does not allow more in-depth and complex research.
• Only available in English, although the interfaces accessible to users can be completely tailored to the website, including the text.
• Costly for larger pages, so it is recommended to work based on a smaller sample.

Pro use #1: 

• Watching user’s video, browsing the website and closely observing where they reach obstacles.

Pro use #2:

• Removing all unused elements from the website based on the heatmap.

Possible alternatives:

• Clicktale
• Decibel Insight
• Mouseflow
• Zarget

Screen capturing software
It is a frequently recurring need to make user tests screen recordings as it is useful to watch them later on, since we want to analyse them in details, which is discovering new drop-offs or just making sure that our user’s focus is directed to where we intended it. For these purpose we mostly prefer to use the tools below:

Skype
If one wants to research something remotely (rather than in person), the screen sharing function of chat applications (Hangouts, Skype) can recreate full mini laboratory tests for just pennies.

We prefer to use Skype for this function because almost everyone has it on their computer. This makes things easier for both the client and the test subjects. It also has the advantage of allowing testers to work on their own computer, in their own environment so they don’t have to struggle with getting acquainted with the mouse or mouse pad during testing. In their own environment, users will navigate more freely, as they would normally do; meanwhile we can observe actual use.
It also offers an advantage in terms of research organisation logistics, allowing us to work more easily with users in rural areas or even villages who are thus spared from having to travel – only a good Internet connection is needed.

Benefits:

• Users can stay in their natural environment which is naturally decreasing their stress level and prevents any discomforts they may feel in a lab setting. Moreover, they don’t have to adopt to a new situation, she can use her own devices that she knows the best.

Drawbacks:

• In case of smaller bandwidth the recording’s quality is poor.

Pro use #1: 

• Once the main issues with the website have been identified, it is recommended to test the solution plan. We can publish the website immediately if we want to, but it is recommended to sit down.

Pro use #2:

• We can record video on the user’s own device. This yields more accurate results but often comes up against logistic hurdles.

Lookback
Lookback is a popular solution to get to know how members of our target group (even within our own environment) use the given interface. With this tool, we can follow the “use” process of a given website or application. Lookback allows us to gain insight on how the interface- application is used in the users’ natural environment. In other words, designers can get to know exactly how members of the target group use the product.

The app is available for both iOS and Android and we often use it to test various apps under development as well as existing websites. After downloading the app (here for example:), we need to register and can send a link to the tested interface to testers. If they click on the link, the app will record everything that happens on the screen plus any external noises. The tester can also test alone at a chosen time, and live testing and broadcasting (live mode) is also possible.

On the recording, the software also indicates what the user has just tapped/clicked. The recordings can be uploaded to the Lookback website and can be viewed subsequently, and we can also perform administration of the events as needed. It can be used to test prototypes, apps and websites on both mobile (iOS, Android) and desktop.

We can glean practical information by collectively looking at the recordings and can then contact our testers for some qualitative sampling when discussing their experiences and positive and negative impressions. This can be done either individually or in a group.

Benefits:

• Simple use on every possible platform (iOS, Android, desktop), broad testing opportunities for prototypes, completed websites/apps or websites/apps under development. Total compatibility across the board.

Drawbacks:

• During use, the recording stops when the mobile phone’s screen shuts off if the tester was not navigating the interface while discussing something. To prevent this, we recommend turning the screensaver off. It also runs on certain Android devices

Pro use #1: 

• Performing data analyses based on what we have seen, highlighting key moments.

Pro use #2:

• After analysing the recordings, contact the test subject and discuss their experiences; i.e. supplement the survey with qualitative data collection.

Silverback
Silverback also records activities performed on the screen. The application records audio and also records the tester in the bottom right-hand corner, minor details of the tester’s reactions and nonverbal communication that can be captured on camera. The software indicates mouse movements and clicks on the recording. Silverback can be used to conduct complete research on large samples.

Benefits:

• Important parts in the recordings can be highlighted and these can be exported  separately.

Drawbacks:

• QuickTime Player is capable of screen and click recording as well, thus, if you do not need the highlighting function specifically, you can use it without downloading anything else.

Pro use #1: 

• Free use with a demo version (this generates recordings featuring a watermark).

Pro use #2:

• Paid version with no watermarks. On the video extracts, we can highlight the most important events for our research.

Collecting user opinions

If you want to identify the potential flaws of your website, show it to users. Generally speaking, do not be afraid to inquire or ask. Anyone in our surroundings who is not involved in the development can be a wonderful and useful helper. A co-worker, partner, child or any other user is capable of offering new perspective. In this case, we are not speaking strictly of testing but of the more general and fundamental aspects of the interface.

However, if you choose not to involve the people surrounding you in the process, you can opt to use a service. There are several services offering help by asking simple questions from users who are unknown to us but can offer assistance in testing. https://www.surveymonkey.com/, https://usabilityhub.com/ , https://www.uservoice.com)

Benefits:

• A great amount of data can be collected very quickly and with the help of online surveys creating programs, data analysis and visualization are easily done.

Drawbacks:

• It is very easy to make methodological errors. It is worth to familiarize with the subject in advance and study methodological books (please see suggested literature below).

Pro use #1: 

• Ask for user’s attention for a short time to answer a few simple questions. This can be in the form of quick guerrilla testing or a short interview.  We do not want to get answers to big questions, but rather many small ones. Sample questions: “Where do you think registration is on this website?” “Where would you start process XY?”

Pro use #2:

• We can not only investigate simple issues, but also quite complex ones regarding the interface. In these scenarios, data collection can be more in-depth and complex. We can ask the questions following a strict, fixed script or a more relaxed one. The latter may be warranted if we want to drill deep on certain topics. This of course requires a greater number of testers. However general experience shows that interface testing with 5-8 users often identifies more than 80% of major UX flaws.

We can perform better if we read the literature on testing and practice regularly. For starters, refer to Steve Krug’s book, Don’t Make Me Think or or Jakob Nielsen’s piece.

More reading:

• Jacob Nielsen: Why You Only Need to Test with 5 Users
• Steve Portigal: Interviewing users
• Russ Unger – Carolyn Chandler: A Project guide to UX design
• Earl Babbie: The practice of social research, Cengage Learning Asia Pte Ltd Singapore, 2016.