App Analysis with a Larger Than Usual Number of Usability Experts

Mobile app development and usage is booming in India; however, it is still in the early stages of adoption especially among the rural farming community. This research attempted to identify the usability problems of a selected agricultural mobile app, NaPanta, developed for Indian farmers. A twofold method was used to understand the usability and functionality of the app. Phase-1: the app was reviewed by experts (n = 18) using cognitive walkthrough and heuristic evaluation for usability and performance analyses. Phase-2: field studies with real users (n = 53) were conducted to compare and validate findings from the low-cost usability studies of phase-1. This paper discusses methods and findings of phase-1. A mixed-methods approach was used to perform qualitative and quantitative data analysis. Thematic analysis and descriptive and inferential statistics were calculated. A total of 90 usability problems within 25 themes and two overarching categories were found. Lack of search functionality and lack of consistency topped the list. This study also contributes to usability research by providing interesting insights on the number of experts required to identify most of the usability problems in comparison to the magic number 5. Results indicate our study required more than 5 participants to find 80% of the problems.