Mouse tracking

Mouse tracking (also known as cursor tracking) is the use of software to collect users' mouse cursor positions on the computer.^[1] This goal is to automatically gather richer information about what people are doing, typically to improve the design of an interface. Often this is done on the Web and can supplement eye tracking in some situations.

When mouse tracking takes place without the user's consent, for example on a website, there may be privacy implications.

The computer mouse was first invented in 1968 by Douglas Engelbart.^[2] The term mouse tracking originally referred to how movements were captured and transmitted to the computer. For example, the original tracker ball mouse used a metal bearing pressed against two rollers to track movement.^[2] Much research and technology has gone into what type of tracker provides the most accurate depiction of the user's movement.

With the advent of the World Wide Web, mouse tracking was expanded to include click data. Researchers and developers would track and record each time a user used the mouse to click something on the website, as well as the location of the event. Web developers use these mouse clicks to assess what information users are interested in and how they interact with a page. Additionally, advertisers are interested in click data in terms of banner advertisements and where to place their ads on pages to get the most click-throughs.

More recently, the term mouse tracking has been expanded even more to develop a much broader area of research in helping understand the human-computer interaction (HCI). This development began with eye tracking. While eye tracking has been around since the 1800s, it was not used in HCI until the 1980, primarily to help answer questions about how users search for commands in computer menus and to develop systems to help disabled users.^[3] More recently, eye tracking has been used in usability testing on web pages to understand a user's point of focus as well as test the usability of different features of a site, such as dropdown menus.^[4] This information can influence Web design so it meets the researcher's goals yet is user friendly.

The problem with using eye tracking in usability testing is the required hardware and then expense. Additionally, eye tracking is limited to small sample sizes and abnormal browsing environments. Mouse tracking, on the other hand, is inexpensive and the data can be collected from any computer. It is in this capacity that mouse tracking was re-invented in HCI research. Eye tracking researchers in the late 1990s noticed patterns between the eye and mouse movements.^[5] Based on these findings, researchers who had been tracking click data realized there might be more to learn from the mouse. In 2001 Mon-Chu Chen, John Anderson, and Myeong-Ho Sohn at Carnegie Mellon University, began explicitly investigating whether tracking mouse movements could be used as a proxy for tracking eye movements. This research has continued through the 2000s and to the present.^[6][7][8][9] The general findings in the research are that the correlation is not one to one, but there is a relationship between eye and mouse movements, which, in turn suggests mouse movements can in fact be used to determine a user's focus of attention. More recent research has shown that the correlation depends strongly on the user behavior at that time, such as whether the user is reading with the mouse, moving it to perform a click, or leaving it idle.^[10] Furthermore, the mouse position actually correlates better with past eye-gaze positions, meaning that people will typically look somewhere before moving their mouse there about 700ms later.^[10] Generally, tracking mouse positions can lead to a vast improvement to understanding the user compared with relying on mouse clicks only. In other words, click data informed researchers of a users' primary focus of attention, or their end choice. However, looking at all of the mouse movements can inform the researcher to other options that were of interest to the user but were not selected by clicking, which can lead to better overall understanding of the user's thought process.

The most recent research in this area is using this knowledge to improve websites and applications. Specifically, researchers are trying to parse out what different individual movements mean as well as beginning to use mouse tracking in usability testing to improve products and pages.^[11]

JavaScript is a scripting language which supports multiple programming styles. It is typically client-side, and does not require constant downloads from the website. JavaScript is implemented as part of a Web browser and is supported by all the major web browsers, including Internet Explorer, Firefox and Safari.

Therefore, using this language, Web developers can track user's mouse movements simply by entering lines of code on a page. It does not require any additional software to be installed on the user's computer; they only have to have JavaScript enabled for the researcher to collect data from the webpage. Mouse tracking using JavaScript has been deployed on high-traffic websites such as search engines ^[12] to collect mouse movement data without affecting the user's computer performance.

Current mouse tracking tools provide a variety of data including the location of the mouse (in terms of pixels), time stamps, any time the mouse hovers on a link of interest, mouse clicks, time spent in areas of interest, and duration of hovers. Additionally, some tracking tools provide more high level analyses, such as heat maps and playbacks which can retrace the mouse's trajectory.^[6][13][14] An example of an output log is below:^[15]

141.84.8.77 2006-09-01,18:44:07 serverdata 8
141.84.8.77 2006-09-01,19:44:08 8 load size=1047x529
141.84.8.77 2006-09-01,19:44:08 8 mousemove coord=283,2
141.84.8.77 2006-09-01,19:44:09 8 mousemove coord=257,125
141.84.8.77 2006-09-01,19:44:10 8 mouseover coord=247,152 name=f dom=abae
141.84.8.77 2006-09-01,19:44:13 8 select radio id=lgr value=lr%3Dlang_de dom=abaecabaac
141.84.8.77 2006-09-01,19:44:16 8 click coord=374,187 name=q dom=abaecaabb
141.84.8.77 2006-09-01,19:44:17 8 keyPress key=H
141.84.8.77 2006-09-01,19:44:17 8 keypress key=a

Mouse movements can be used to infer a user's intent and focus while browsing a website. By using mouse movements in usability testing, researchers can determine if users are confused, if their expectations are met, where their attention is focused, and much additional information. This tool can be especially beneficial in conjunction with other techniques used in usability testing, such as think aloud procedures, as this information can lead to a better mouse movement model.

Tracking mouse movements can be used to adapt interfaces in real time based on respondents' interests. Researchers can use information, such as where respondents hold the mouse for an extended period of time and the trajectory of the mouse, to assess their level of interest in that object.^[12][16][17] The knowledge gained from this can be used to re-sort search criteria based on individual relevancy and suggest other objects, products, or information that might be of interest to the user.

Mouse tracking allows Web developers to view the behaviors of actual users in their natural browsing environment instead of in a laboratory. By tracking where the mouse is located, designers can evaluate the ease of use of their Websites. Specifically, they can see how difficult it is for users to find and use certain features, such as scroll bars or dropdown menus, or to locate important links. Additionally, developers can see what parts of the pages users are most interested in, which can influence page layout if they are not focused where the designer wants them.^[13]

Each computer user has their own unique way of using the mouse, which can be used as a biometric identifier.^[18][19][20] An example of how mouse movements can be used for online security is as follows. Some people rarely engage the mouse until they need it to complete an action, while others are very active with their mouse and use it to read along with text on a page. For users who are active with their mouse, researchers have successfully been able to "learn" a user's typical behavior through a supervised learning method.^[21] Once this behavior is learned, it can be linked to an individual's account. If the behavior of a user deviates significantly from that user's learned, typical behavior, they can be locked out of the system until their identity is verified. This is another way of ensuring a user is who they claim to be.

Mouse tracking has been used in education to help understand the impact of reading on a computer as opposed to paper and propose ways that reading on a computer could be adapted so understanding and learning were easier.^[22] It has also been used to identify off-task behaviors in tutoring settings and in physics to understand how students perceive and process multimedia representations of real experiments.^[23][24]

• Click tracking
• Eye tracking
• Macro recorder
• Keystroke logging
• Usability testing
• Recorded Future

• Mouse movements reveal your behavior • June 23, 2021 • University of Luxembourg
• Monitor your Mouse Clicks