{"id":10271,"date":"2017-12-20T00:15:16","date_gmt":"2017-12-20T05:15:16","guid":{"rendered":"https:\/\/www.invespcro.com\/blog\/?p=10271"},"modified":"2024-09-03T14:15:56","modified_gmt":"2024-09-03T14:15:56","slug":"usability-metrics","status":"publish","type":"post","link":"https:\/\/www.invespcro.com\/blog\/usability-metrics\/","title":{"rendered":"Usability Metrics: A Guide To Measuring The User Experience"},"content":{"rendered":"<span class=\"span-reading-time rt-reading-time\" style=\"display: block;\"><span class=\"rt-label rt-prefix\">Reading Time: <\/span> <span class=\"rt-time\"> 7<\/span> <span class=\"rt-label rt-postfix\">minutes<\/span><\/span><p><span style=\"font-weight: 400;\">At Invesp, while hiring for a senior UX\/UI position, we screened 20 resumes, conducted 10 phone interviews, and held one face-to-face interview.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Despite impressive design claims, the candidate struggled with usability guidelines and when we asked\u2014how do you measure the usability of a product you created?<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Complete silence.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">The basics of <\/span><a href=\"https:\/\/www.invespcro.com\/cro\/\"><span style=\"font-weight: 400;\">CRO<\/span><\/a><span style=\"font-weight: 400;\"> lie in measuring usability. Without it, we can\u2019t design and offer a superior visitor experience. It also improves user satisfaction by providing an intuitive interface, reduces errors, and lowers support costs.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">A product with poor usability can lead to user frustration and lost conversions, making usability an enhancement and a fundamental element.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">While data is essential, turning it into actionable insights requires a deep understanding of usability. This article will explore the fundamentals of usability and how to track usability metrics.\u00a0<\/span><\/p>\n<h2><span style=\"font-weight: 400;\">Usability Framework<\/span><\/h2>\n<p><span style=\"font-weight: 400;\">According to the usability framework, \u201cusability is a multidimensional concept that aims to fulfill a certain set of goals, mainly effectiveness, efficiency, and satisfaction,\u201d and without these goals, usability cannot be achieved.<\/span><\/p>\n<figure id=\"attachment_98781\" aria-describedby=\"caption-attachment-98781\" style=\"width: 700px\" class=\"wp-caption alignnone\"><img fetchpriority=\"high\" decoding=\"async\" class=\"size-full wp-image-98781\" src=\"https:\/\/www.invespcro.com\/blog\/images\/blog-images\/image1-28.png\" alt=\"Usability Framework \" width=\"700\" height=\"481\" srcset=\"https:\/\/www.invespcro.com\/blog\/images\/blog-images\/image1-28.png 700w, https:\/\/www.invespcro.com\/blog\/images\/blog-images\/image1-28-300x206.png 300w\" sizes=\"(max-width: 700px) 100vw, 700px\" \/><figcaption id=\"caption-attachment-98781\" class=\"wp-caption-text\">Usability framework<\/figcaption><\/figure>\n<p><span style=\"font-weight: 400;\">Here\u2019s a brief overview of all the major core components of usability testing and how they relate to the overall user experience:<\/span><\/p>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><b>Effectiveness:<\/b><span style=\"font-weight: 400;\"> This term refers to the accuracy and completeness of the user goal achievement. It&#8217;s measured by factors like:<\/span>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><b>Success rate or completion rate: <\/b><span style=\"font-weight: 400;\">The percentage of users who successfully complete a task.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><b>Error rate: <\/b><span style=\"font-weight: 400;\">The number of errors users make while using the system.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><b>Task time:<\/b><span style=\"font-weight: 400;\"> The average time it takes users to complete a task.<\/span><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><b>Efficiency: <\/b><span style=\"font-weight: 400;\">Refers to the resources exhausted by users to ensure the accurate and complete achievement of the goals. Key metrics include:<\/span>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><b>Task time:<\/b><span style=\"font-weight: 400;\"> Again, it is related to how quickly users can complete tasks without errors.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><b>Number of actions:<\/b><span style=\"font-weight: 400;\"> The steps users take to complete a task.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><b>Learnability: <\/b><span style=\"font-weight: 400;\">How quickly users can learn to use the system.<\/span><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><b>Satisfaction: <\/b><span style=\"font-weight: 400;\">It refers to the subjective thoughts of the user regarding their attitude, level of comfort, relevance of application, and the acceptability of use. It&#8217;s often measured through:<\/span>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><b>User surveys:<\/b><span style=\"font-weight: 400;\"> Gathering feedback on user attitudes and opinions.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><b>Observation:<\/b><span style=\"font-weight: 400;\"> Watching users interact with the system to identify pain points and positive experiences.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><b>System usability scale (SUS):<\/b><span style=\"font-weight: 400;\"> A standardized questionnaire to measure perceived usability.<\/span><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p><span style=\"font-weight: 400;\">How well a product works depends on where and how it&#8217;s used, who&#8217;s using it, and what they&#8217;re using it with. To measure how usable a product is, we look at how well people can use it to reach their goals, how much effort it takes, and how happy they are with it.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">In light of the three goals mentioned earlier, we\u2019ll review the different key usability metrics used to measure each goal. We&#8217;ll focus on measuring how often users can successfully complete tasks, as this gives a good overall picture of the product&#8217;s performance.\u00a0<\/span><\/p>\n<h2><span style=\"font-weight: 400;\">Usability Metrics: Measuring How Well Your Product Works<\/span><\/h2>\n<p><img decoding=\"async\" class=\"size-full wp-image-98782\" src=\"https:\/\/www.invespcro.com\/blog\/images\/blog-images\/image2-11.jpg\" alt=\"Measuring Usability Metrics\" width=\"501\" height=\"334\" srcset=\"https:\/\/www.invespcro.com\/blog\/images\/blog-images\/image2-11.jpg 501w, https:\/\/www.invespcro.com\/blog\/images\/blog-images\/image2-11-300x200.jpg 300w\" sizes=\"(max-width: 501px) 100vw, 501px\" \/><\/p>\n<h2><span style=\"font-weight: 400;\">Effectiveness Metrics: How well users achieve their goals.<\/span><\/h2>\n<p><span style=\"font-weight: 400;\">You can measure effectiveness using two usability test metrics:<\/span><\/p>\n<h3><span style=\"font-weight: 400;\">Success rate (also called completion rate)<\/span><\/h3>\n<p><b>Success rate or completion rate <\/b><span style=\"font-weight: 400;\">refers to the percentage of users who were able to complete the tasks successfully.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Despite not revealing how tasks were performed or why users fail, this metric remains critical <\/span><a href=\"https:\/\/www.nngroup.com\/articles\/success-rate-the-simplest-usability-metric\/\"><span style=\"font-weight: 400;\">and is at the core of usability<\/span><\/a><span style=\"font-weight: 400;\">.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">The success rate metric can be measured by assigning a binary value of 0 and 1 to the users, where 1 is assigned to those who successfully complete the task and 0 to the ones who fail to do so.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">To calculate your success rate, divide the total number of correctly completed attempts by the total number of attempts and multiply by 100.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">The completion rate (or success rate) is easy to measure but has a pitfall: users may stop midway or complete tasks incorrectly. How do you score partial success?<\/span><\/p>\n<p><span style=\"font-weight: 400;\">For example, a user\u2019s task is to order a box of dark chocolates with a card for Mother\u2019s Day. Scoring might seem simple: success if the mother receives the chocolates and card, failure if she doesn\u2019t.\u00a0<\/span><\/p>\n<p><b>But consider these scenarios:<\/b><\/p>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">I ordered a box of chocolate but not the dark one (white, milky, or a variety of these), along with the card.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Ordered the right chocolate box without a gift card<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Ordered more than one box of chocolate by mistake, and a gift card<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Ordered a box of chocolate but didn\u2019t add delivery information or an address<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Ordered a box of chocolates and a gift card successfully but to the wrong address<\/span><\/li>\n<\/ul>\n<p><span style=\"font-weight: 400;\">Each scenario shows partial success or failure, requiring evaluators to use personal judgment. Different evaluators may score these scenarios differently. To mainstream scoring, determine the important aspects of the task and assign scores accordingly.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Success rate remains the simplest usability metric and the easiest among the whole range of these usability signals, mainly because it\u2019s quick and easy, does not require much preparation and time to collect, and, most importantly, helping with tracking usability metrics and progress within your system.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">However, as one of the general areas commonly used by marketers and designers to see the big picture of how well their system is doing at the level of user experience, this does not change the fact that it remains subjective.<\/span><\/p>\n<h3><span style=\"font-weight: 400;\">The Number of Errors<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">This metric provides an idea about the average number of times where an error occurred per user when performing a given task.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">These errors can be either slips, where the user accidentally types the wrong email address or picks the wrong dates when making a reservation or booking a flight, or mistakes, where the user clicks on an image that\u2019s not clickable or even double-clicks a button or a link intentionally.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Normally, any user of any interactive system may make errors, <\/span><a href=\"http:\/\/usabilitygeek.com\/usability-metrics-a-guide-to-quantify-system-usability\/\"><span style=\"font-weight: 400;\">where 2 out of every 3 users err<\/span><\/a><span style=\"font-weight: 400;\">, and there is absolutely no such thing as a \u2018\u2019perfect\u2019\u2019 user interface anyway. The rationale behind using this metric is not to eliminate errors but rather to lessen their numbers within the system.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">To help you measure and ensure obtaining great diagnostic results, it is highly recommended to set a short description where you give details about how to score those errors and the severity of a certain error to show you how simple and intuitive your system is.<\/span><\/p>\n<h2><span style=\"font-weight: 400;\">Efficiency Metrics: How much effort users put in.<\/span><\/h2>\n<h3><span style=\"font-weight: 400;\">Time-Based Efficiency<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">Referred to as time on task, this metric measures the time the user spends completing the task or the <\/span><a href=\"http:\/\/ui-designer.net\/usability\/efficiency.htm\"><span style=\"font-weight: 400;\">speed of work<\/span><\/a><span style=\"font-weight: 400;\">. Consequently, this means that there is a direct relationship between efficiency and effectiveness, and we can say that efficiency is actually the user&#8217;s effectiveness divided by the user&#8217;s time spent.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">I will use an example to simplify the calculation of the time-based efficiency.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Let\u2019s consider, for example, that we have 3 different users performing the same task. Two managed to complete it successfully in a considerable time\u2014-2,3 seconds, respectively\u2014while the third user took 7 seconds and never finished the task.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Now that we know the number of tasks is N=1 and the number of our users is actually R=3, and we also know the time spent on each one, it becomes easy to determine the value of our Time-Based Efficiency.<\/span><\/p>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">N= Number of tasks (in this case N=1)<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">R= Number of users (in this case R=3)<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Nij = The result of task i by user j; if the user successfully completes the task, then Nij = 1. If not, then Nij = 0<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Tij = The time spent by user j to complete task i. If the task is not successfully completed, then time is measured till the moment the user quits the task<\/span><\/li>\n<\/ul>\n<p><b>Using the time-based efficiency equation, we end up with something like this:<\/b><\/p>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">User 1: Nij = 1 and Tij = 2<\/span><span style=\"font-weight: 400;\"><br \/>\n<\/span><span style=\"font-weight: 400;\">User 2: Nij = 1 and Tij = 3<\/span><span style=\"font-weight: 400;\"><br \/>\n<\/span><span style=\"font-weight: 400;\">User 3: Nij = 0 and Tij = 7<\/span><\/li>\n<\/ul>\n<p><b>Time-based Efficiency = (1\/2+1\/3+0\/7)\/1*3 = 0,71 goals\/sec<\/b><\/p>\n<h3><span style=\"font-weight: 400;\">The Overall Relative Efficiency<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">This is actually measured through users who successfully completed the task in relation to the total time taken by all users.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Let\u2019s consider that we have 2 users, and each one is supposed to complete a different task. The first user has successfully completed the task (1) yet failed to complete the task (2). The second user failed to complete task (1) but completed task (2) successfully.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">In this case, the overall efficiency can be calculated as the following:<\/span><b> The Overall Relative Efficiency = E = ((1*1+1*1)\/ (2*2)) *100% = 50%<\/b><\/p>\n<h2><span style=\"font-weight: 400;\">Satisfaction Metrics: How users feel about the experience.<\/span><\/h2>\n<h3><span style=\"font-weight: 400;\">Post Task Satisfaction<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">Once your users have finished the task, and it doesn\u2019t matter whether they complete it successfully or not, it\u2019s time to hand them over a questionnaire to have an idea about the difficulty of the task from the user\u2019s point of view.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Generally, these tasks consist of 5 questions, and the idea behind them is to give your users a space to judge the usability of your system and measure user satisfaction.<\/span><\/p>\n<h3><span style=\"font-weight: 400;\">Task Level Satisfaction<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">This metric helps investigate users&#8217; general impressions of the system.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">To measure the level of satisfaction, you can use the smiley scale method. In this method, the user is expected to choose one of the five smileys to reflect user interaction satisfaction or lack of satisfaction, enabling you to track test-level satisfaction.<\/span><\/p>\n<figure id=\"attachment_98783\" aria-describedby=\"caption-attachment-98783\" style=\"width: 800px\" class=\"wp-caption alignnone\"><img decoding=\"async\" class=\"size-full wp-image-98783\" src=\"https:\/\/www.invespcro.com\/blog\/images\/blog-images\/image3-22.png\" alt=\"Usability metrics \" width=\"800\" height=\"265\" srcset=\"https:\/\/www.invespcro.com\/blog\/images\/blog-images\/image3-22.png 800w, https:\/\/www.invespcro.com\/blog\/images\/blog-images\/image3-22-300x99.png 300w, https:\/\/www.invespcro.com\/blog\/images\/blog-images\/image3-22-768x254.png 768w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/><figcaption id=\"caption-attachment-98783\" class=\"wp-caption-text\">Smiley face rating scale<\/figcaption><\/figure>\n<h2><span style=\"font-weight: 400;\">Over to You!<\/span><\/h2>\n<p><span style=\"font-weight: 400;\">In light of the conceptual framework we discussed earlier, the user experience is highly influenced by everything around it.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">In the digital world, especially\u00a0<\/span><span style=\"font-weight: 400;\"><span style=\"box-sizing: border-box; margin: 0px; padding: 0px;\">when it comes to user experience and system\u00a0<a href=\"https:\/\/www.invespcro.com\/blog\/usability-testing\/\" target=\"_blank\" rel=\"noopener\">usability<\/a><\/span>, it\u2019s all about how accurate and exact your numbers and stats are.\u00a0\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Since there\u2019s so much qualitative data and many uncontrollable variables, usability metrics are a great way to combine this data in a quantitative and qualitative manner. This helps you understand your system&#8217;s performance against its objectives and goals, ultimately providing a better user experience.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Once you have collected the metrics, it\u2019s time to use them to form a conclusion about your system&#8217;s overall usability and make sound, data-driven decisions about future changes to improve the user experience.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p><span class=\"span-reading-time rt-reading-time\" style=\"display: block;\"><span class=\"rt-label rt-prefix\">Reading Time: <\/span> <span class=\"rt-time\"> 7<\/span> <span class=\"rt-label rt-postfix\">minutes<\/span><\/span>At Invesp, while hiring for a senior UX\/UI position, we screened 20 resumes, conducted 10 phone interviews, and held one face-to-face interview.\u00a0 Despite impressive design claims, the candidate struggled with usability guidelines and when we asked\u2014how do you measure the usability of a product you created? Complete silence. The basics of CRO lie in measuring [&hellip;]<\/p>\n","protected":false},"author":3,"featured_media":98784,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[18],"tags":[87,245,109],"class_list":["post-10271","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ux","tag-general","tag-intermediate","tag-resource"],"_links":{"self":[{"href":"https:\/\/www.invespcro.com\/blog\/wp-json\/wp\/v2\/posts\/10271","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.invespcro.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.invespcro.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.invespcro.com\/blog\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.invespcro.com\/blog\/wp-json\/wp\/v2\/comments?post=10271"}],"version-history":[{"count":2,"href":"https:\/\/www.invespcro.com\/blog\/wp-json\/wp\/v2\/posts\/10271\/revisions"}],"predecessor-version":[{"id":98785,"href":"https:\/\/www.invespcro.com\/blog\/wp-json\/wp\/v2\/posts\/10271\/revisions\/98785"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.invespcro.com\/blog\/wp-json\/wp\/v2\/media\/98784"}],"wp:attachment":[{"href":"https:\/\/www.invespcro.com\/blog\/wp-json\/wp\/v2\/media?parent=10271"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.invespcro.com\/blog\/wp-json\/wp\/v2\/categories?post=10271"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.invespcro.com\/blog\/wp-json\/wp\/v2\/tags?post=10271"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}