For a long time, proponents of eye-tracking have said that the amount of viewing a link receives directly effects its likelihood of being clicked.  Well, it turns out that this is particularly true in the case of search results.  Recently, Eyetools and Sendtec released a study showing that decreased viewing on a search results page has a huge impact on click-through conversions. This is true for both paid and organic listings. 

For example, the data shows that if overall viewing of a link falls from 80% to 60%, the initial probability that the user will select the link decreases by over 50%! This helps to explain why even small drops in page position for organic search links has such a significant impact on a site's conversion rate. 

In addition to emphasizing the need for ads and organic listings to have high page positions, this result also means optimizing link performance is critical since not everyone can be a top organic or adwords listing. 

I've included the excerpt below in the hopes that it will open a dialogue about the effect of increased viewing for other kinds of web pages. 

----------------------- REPORT EXCERPT -------------------------

The graph shows an important correlation between increased link viewing and the probability of being clicked. [...] The page elements which received the most clicks were those seen by the most users.  As user viewing decreased, so did the number of clicks.  [... Our data shows that] decreases in viewing will have a disproportionately large negative impact on click-through.

A number of other studies have explored various factors which influence a user’s decision to click including link text, description, and perceived link relevance. Our study shows that percent user viewing can be an excellent predictor of user click behavior for online searches.  Our study supports the logic that users must first see a link before they can make selections based on text descriptions or relevance. There are many factors which contribute to whether or not a user decides to select a link, but all things considered, eye tracking data is a reliable and efficient way of understanding why people do and do not click.

It is important to note that this data is specific to search results and should not be used in conversation about standard web pages. Eyetracking has proven to be very helpful in directing redesigns of web pages, landing pages, and emails with great increases in conversion rates, but this data presented here is specific to search result pages and should not be generalized to other pages.

Update, June 2010: Sadly, it seems that we outlived SendTec, our partner on this project. It appears that they closed down in 2009, taking the report down with the rest of their website.

You can check out the CBS news video at http://cbs5.com/video/?id=27290@kpix.dayport.com (alternately, try http://cbs5.com/video/ and search for "yahoo search" if the video doesn't load)

About 1 minute into the video, Yahoo shows an eyetracking heatmap generated by Eyetools software and talks about how Yahoo used eyetracking data/heatmaps to optimize the Search Assist design. They're calling it "the biggest consumer improvement to its search engine... ever".

(Begin self promotion: contact us if you'd like to optimize your webpages, landing pages, and emails. We've helped large and small companies; we can help you.)

We test a lot of emails at Eyetools... newsletters, B2C, B2B, membership correspondence, etc.  One of the more common layouts we see used by large retailers is a print-advertisement style product layout.  In general, these templates never test particularly well.  We have been using one of these emails as our simple report example for a while now.   I thought the report content might be of interest to blog readers, so here it is.  If there are questions regarding the results, please let me know.

The PDF of this sample report can be downloaded here .

Note: This is a brand modified sample  The email was received through a Fortune500 retail company’s opt-in email list.  The branding was removed, but basic layout and graphical elements were retained.  The analysis has been kept simple, but shows the benefit of even a quick eye tracking study.

____________________________________________________

Eyetools Sample Email Report

Overview

Diagnosis

Critical Messaging

The words and images highlighted in the red/yellow areas define the "window of opportunity" this email has to grab its readers. Everything outside of that "window" was not looked at by most people (and if they don't read it, they can't be affected by it ).

If the critical message isn't being read or isn't clear, then focus initially on rewriting the areas with the highest concentrations of reading (the red, then orange, then yellow areas).

Diagnosis

The message read by most people (at least 60%) was:

The majority of viewers did not get a cohesive message which included a specific sales pitch or call to action.  If this page layout is maintained, improved copywriting is vital to improving read through in each of the top section of the page. 

Did They Scroll?

The red lines show where each person
stopped scrolling. It is a myth that everything has to be above the fold, but data has shown that clutter and bad design will greatly reduce scrolling and visual hotspots below the fold.

Diagnosis

Most people are scrolling, which is good. This means that this email has the potential to do even better. 

However, the 33% that stopped scrolling early in the email, did so before reaching the individual sale item listings.  The top 33% of this email is comprised of 4 major horizontal elements, each with a “banner ad” composition.  Viewers skimmed the ads, becoming less interested, and less patient at each subsequent section.  Much of this result is due to poor copywriting (See “Critical Messaging”). 

In the page structure at the top of this email, viewing is helped by the fact that each section heading is aligned on approximately the same vertical axis.  However, as soon as the 3 column format is introduced, common viewing areas decrease dramatically.

Improving the structure and copy in the top section of this email will likely result in increased scrolling by a larger percentage of viewers.

Identifying Design vs. Text/Copy Problems

The green and blue areas highlight those areas read by people who engaged the email more. Because the blue areas indicate where viewers read content quickly, the placement of important text becomes increasingly crucial, particularly in areas below the “fold ” (the yellow dotted line).

Broadly speaking, if a critical area does not have a strong hotspot (orange/red) on it then The design is not guiding the users’ eyes effectively. If, however, a hotspot exists and users are starting to read the text but do not get to the critical messaging, then the area has a copy/text problem.

Diagnosis

Poor read through in the top section of this email is due to poor copy.  The heading are high contrast and catch the viewer as they scan the page.  However, the first 2-3 words or each headline are relatively uninformative (i.e. “Are you ready”, “Watch the”).

A 3 or more column layout is generally suboptimal for viewing, especially in an email.  In this email 80% of participants ignored the right most product column completely.   When viewing this email, participants selected one of the 3 vertical columns and skimmed downward.  This proved problematic for sale items spanning 2 columns.
In the lower page structure of this email, the product pictures most effectively caught the eye of participants.  The outlines around each product primarily disrupted visual flow by creating mini “banner ads”.   When viewers see many small distinct units, they are more apt to view one well, and skip several others (even along the same vertical or horizontal axis).

The “Tax Center” section near the bottom of the email was most effective due to it’s large, informative header.  This element performed surprisingly well.

User Feedback

User feedback was not collected on this email because it was run only for sample purposes.

Part V:  Time and the Heatmap

In my previous post, I mentioned that heatmaps do not have a time component. Several people have asked me to discuss this topic in a little more detail, so here we go.

Important point #1:

A heatmap represents which content was seen by a group of participants. A heatmap helps to answer the questions “Where did users look?” and “Where didn’t users look?”

Important point #2:

If you are analyzing a heatmap, you are momentarily saying “Ok, in this instant we don’t care how long people looked at things, we just want to know what they saw.”

A heatmap does not answer the questions:

•    How long did someone look at my page or page element?
•    Did users look away and look back?

To further clarify these points, here’s a general overview of how a heatmap is created (click illustrations for a larger view):

Step 1:  Collect data A single participant views a web page. We record her eye movements as she browses. This gives us data which can be represented in a gaze replay animation. Now imagine that the animation is composed of a stack of stills, just like a flip book. Each page of our eye tracking “flipbook” contains the X,Y, and Time coordinates of a single fixation.

Step 2: Collapse individual data sets along the t-dimension Keep imaging our eyetracking data as a flipbook (3D data structure).  Now we will collapse the stack along the time axis.  In its most basic form, the calculation will take the area of a web page and note every location where the participant fixated.  By collapsing the time dimension, we remove it from all further calculations (poof!)  This means that someone who fixated the center of a page for 100ms will end up with the same fixation summary as someone who stared at the center of the page for an extended period of time.

Of course, the algorithm for doing this computation is more complex than the example I just presented. Any heatmapping algorithm should take into account peripheral vision, microsaccades, blinks, fixation duration, “bad data”, ocular drift, dynamic page behavior, etc.   For now let’s continue with the simplified example. Steps #1 and #2 are completed for, oh, say 15 people total. This gives us 15 individual fixation summary plots.

Step #3: Compute an average viewing value for each pixel of the webpage Again the algorithm which handles this step is more complex than what is presented here, but the basic idea is… For every pixel of a web page, the system asks “how many people saw this pixel?”  If 10 people saw the pixel out of a group of 15, then the algorithm says “66% percent of people saw this pixel… color it yellow on the plot.”

This averaging algorithm outputs a heatmap showing what percentage of participants saw each page element.  Handling the data in this way keeps any single individual from biasing heatmap percentages.  For example, imagine that 14 people only looked at the center of a page, and 1 person looked at the entire area of the page.  Even though 1 participant viewed a much larger area than anyone else (and probably spend a goodly amount of time doing it), she still only represents 6.7% of the 15 person group.  The resulting heatmap would then show that 100% of people fixated the center of the screen, and less than 10% looked elsewhere.

Other questions that have come up:

If you make a heatmap from only the first 10 seconds of viewing, doesn’t that add a time component to the heatmap?

The answer is no.  The method for computing a heatmap is the same no matter how large the time sample used. When you slice for specific time intervals, you are just selecting a specific group of fixations to include in the calculation. The resulting heatmap still has no time component.

Are heatmaps created from time slices more valid or informative than those created from full experiment sessions?

I suppose that depends on the kind of information you want to get from the plot.  If you want to see where people looked in the first 10 seconds, a time slice heatmap is appropriate. If you are trying to understand an order for page element viewing, a heatmap is probably the wrong analysis tool. If you want to see where people are looking over their entire experience with the page, a full session heatmap is the way to go.

This is the last planned installment of our “But what does it all mean?” series. However, the discussion is still open, so if anyone is interested in other topics, just let me know.

Article by Teresa Hernandez - Eyetools, Inc.
Illustrations by Boyd Richard - Eyetools, Inc.

Part IV:  What is a scan path?

I believe I referred to “scan paths” or “fixation patterns” in earlier entries.  Understanding scan paths are critical to interpreting eye movement results.

A scan path is a sequence of eye fixations made when someone samples a visual scene. Put simply, a scan path is the path our eyes follow when looking at an image.

One scan path image shows the sequence of fixations made by a single user.  Please note that individual fixation patterns are very noisy. However, if you look at a number of these traces, patterns can emerge. What we look for in individual scan paths are “orderly viewing patterns”.  In other words, we try to determine whether or not a person’s order of fixation follows a logical flow through a page.

At Eyetools, we produce scan path images which look like the image to the right.  A key to this can be found on our website. 

Qualitative analysis of scan path patterns is critical to any (usability targeted) eye-tracking study. Scan paths can be especially useful when presenting to design, marketing, and usability groups.  They very effectively illustrate how individual users are navigating the page.  However, when doing your basic analysis, experience and a working knowledge of eye movement/scene search literature will get you the most reliable read from these results.

There are many ways to discuss gaze traces.

Here are just a few examples of the imagery I've found works well when explaining traces:

• Think of view flow as a stream.  Are users flowing smoothly and unhindered through page content? Are things like ads or section breaks helping to guide the eye, or is the design placing a big boulder in the stream forcing users to detour around a section?

• Is there a "spaghetti effect"? Do many individual scan path images look as though someone dropped a handful of spaghetti on the page? (splat!) If users are searching a page at random, they are probably lost and aren't internalizing the message your page is trying to convey (at least not in the most efficient manner). 

• Does viewing suffer from a "pendulum effect"?  Do viewers use a page element as an anchor point and oscillate back and forth in multiple directions?  (Think Foucault's Pendulum) This pattern can result in decreased read-though of content, and has a number of other implications.  However, the severity of this problem really depends on the individual page content, design/client priorities, etc.

From this point on, any number of other metrics can be considered for data analysis.  The data you choose to examine more closely all depends on what you want to learn from your study.   

Continuing the Discussion: Methods and More

A note on software:

At Eyetools, we use in-house software to analyze our data. This has allowed us to develop tools that many commercial software systems don't readily provide. I'm not entirely familiar with the limitations of commercial and open source tools, but I assume that a large portion of stage 1 (heatmap analysis) and stage 2 (qualitative scan path analysis) is very common place.

But regardless of what kind of analysis algorithms you use (commercial, in-house, open source, etc.) it's important for the experimenter to have a solid understanding of eye movement terminology, experimental methodologies, and eye tracking algorithms.   After all, it helps to know (1) what you are actually measuring with an eyetracker, and (2) what the measurements mean, and (3) what the limitations of your hardware and software are. (Well, that's true in any field isn't it?)

A note on methodology:

Task instructions and participant preparation (priming) are important concerns in any behavioral or survey style study.  Eye-tracking is no different.  Rather than discuss methods at length, I’ll just pop in some pointers to other discussions and resources. 

•    [pdf] Granka, L and Rodden, K (2006). Incorporating Eyetracking into User Studies at Google. ACM Conference on Human Factors in Computing Systems.
•    [blog] Matteo Penzo's  Introduction to Eyetracking: Seeing Through Your Users’ Eyes
•    Suman, S and Bradburn, NM (1983). Asking Questions. San Francisco: Jossey-Bass.

Part III:  What is a heatmap… really?

A heatmap can be defined several different ways, but as they say, all roads lead to Rome. I find the 2 most useful ways of describing a heatmap are:

1 – A heatmap is an aggregate representation showing what areas a group of people fixated when viewing a webpage or email. The image shows where the sample group looked or didn’t look.  Well produced heatmaps should also take into account a reasonable percentage of peripheral vision. 

And

2 – A heatmap is a probability map which tells you what page content users are most likely to see. This is only true when the number of individual sessions used to create the heatmap is large enough.  At Eyetools, we recommend 15 people in most cases.  This number is based on what we have found gets our clients reliable results and the best cost-benefit ratio (bang for their buck).

Heatmaps are striking images, and very informative analysis tools.  However, they are just the tip of the iceberg when performing a proper analysis of eye movement data.  This is why companies who specialize in eye-tracking research have begun popping up. 

What can’t you learn from a single heatmap?

•    View order. Are users scanning the page in the most efficient way possible?  Are they forming a coherent message or are they lost? Sometimes a heatmap can be covered in "hot spots" but read through was very low.  More analysis is needed.

•    Copy vs. Layout Problems.  Heatmaps can help experimenters determine whether a page’s layout or copy is responsible for decreased viewing.  However, further analysis is needed to make a complete diagnosis.

•    How to improve page performance.  This is really knowledge that can only be gained through experience.  Viewing patterns can be affected by a number of different factors.  Heatmaps can help you to understand if there is a problem, but not how to specifically go about making changes.  (That’s why we're here. Please excuse the shameless plug.)

•    Variability.  By definition heatmaps do not have a measure of variability (error bar) associated with them.  I have a few thoughts, and (some rather interesting findings) about the variability in heatmaps and other eye-tracking metrics, but that is another discussion all together.  If anyone is particularly interested in discussing analysis at this level, let me know.

•    Time Components. Heatmaps do not have a time component. Time spent looking at a page element, or the page as a whole, is not represented at the heatmap level. 

Part II:  What can you learn from eye-tracking data?

People often ask me what exactly they can learn from eye tracking.  I have a short version answer which is:

We track:

•    Where people look
•    Where people click
•    What people ignore
•    And we discover why they decide to click ...or not click.

Why does it work?

•    Your eyes are hardwired into your brain and the eyes cannot lie.
•    Eyes can’t be "put down" like a mouse between clicks.
•    Eyes + clicks + subjective questions give a comprehensive view into the user experience.

This explanation is pretty common, but in reality, you can gain much more insight into the user experience from these kinds of studies.  Some advantages of eye tracking studies include:

•    Biometric measurements are more accurate than user feedback.  User feedback is invaluable, but notoriously unreliable.  Feedback regarding feelings, opinions, etc. must be acquired through survey methods.  However, measuring natural behavior gives a much more accurate picture of a user’s immediate experience than asking them after the task is complete.

•    Eye-tracking data can be used to accurately predict user feedback. This is true for questions regarding ease of use, ease of navigation, etc. All of these items also affect user confidence in the site and company.

•    Site navigation patterns can be mapped.  For instance, we can map out common navigation paths from a homepage to an interior page, and can begin to understand why some are being used more than others.

•    Results are representative of a “natural work environment”.  In other words, eye tracking does not require that a moderator be in the room with the user. Obviously the studies are done in a controlled environment, but not having other people in the room makes the experience very comfortable for users.

•    Viewing order of pages and page elements can be established.  We begin to understand what information is most likely to be seen/missed by users, and in what order.  For example, are users viewing a “Free Trial” offer 1st or 15th when seeing the page? Now you can know.

•    DHTML elements can be tracked separately.  The visual effectiveness and frequency of use of DHTML elements can be studied.

•    Short iterative testing can also be implemented. Because we can test prototypes (yes, even jpeg mock-ups), short eye tracking tests can be used to modify designs quickly.  This kind of testing is not a standard use of eye-tracking, but is proving to be very effective.

•    Page element placement, copy, etc effects can be ranked.  Should I change my header text? Move a menu to the right rail? Change the icon size or background? Certain layout changes can be ranked as to how much effect they will have on viewing patterns.

One point to keep in mind is that eye-tracking, like all other usability tests, is not going to provide all of the answers by itself.  Eye-tracking is a valuable and powerful tool when implemented alone.   But if you want the best over all picture, use it in conjunction with other usability tests, and as a part of an iterative process.

Does eye tracking measure visual attention?

Yes, eye tracking can estimate the areas of an interface which receive visual attention.  The “bouncing around” of the eye trace shown in the video is created by a series of fast eye movements called saccades.  When your brain is planning an eye movement, it shifts covert attention to the eye’s destination [ 1].  The attentional shift and saccade movement have been shown to be inseparable [ 2 ].  And, of course, once you fixate something, your visual system starts processing the image.

Does this mean that we remember 100% of everything we fixated on a screen? Definitely not.  Our brains can suppress images, or use the visual signal to inform any number of low level cognitive functions. Does it mean that what we fixated has had an opportunity to directly affect our experience with an interface? Absolutely.

Gaze trace information helps us to understand the areas of a page which most helped to form a user's visual experience with the site or email.

References:

Here's an EXELLENT introduction to eye tracking and usability:
Matteo Penzo's Introduction to Eyetracking: Seeing Through Your Users’ Eyes

[1]   Shipp S. (2004) The brain circuitry of attention. Trends Cogn Sci. 8, 223-30.
[2]   Peterson MS, Kramer AF, Irwin DE. (2004) Covert shifts of attention precede involuntary eye movements. Percept Psychophys. 66, 398-405.
[ref]   Liversedge SP, Findlay JM. (2000) Saccadic eye movements and cognition. Trends Cogn Sci. 4, 6-14.

Part I:  Misinterpreting the data

In 2000, the Poynter Institute released their first study analyzing how users view online news websites. Yet, 7 years after eye-tracking made this first major impression on the usability and marketing industries, there still seems to be a lot of confusion over what eye-tracking data can actually tell you about how users interact with your site.

Vague descriptions of methodologies and misinterpretation of eye-tracking data has lead to skepticism about the validity of eye-tracking in usability and marketing research.

Getting answers to common questions

In this next series of blog entries, I thought I’d take a shot at dispelling some small fraction of the confusion surrounding eye-tracking research.  Over the next few weeks I’ll address some recurring questions I get about our research, and the optimal use of eye-tracking studies.

Questions like:

•    What is a heatmap… really?
•    How to read a scan path… and what is a scan path?
•    How do you get the most out of eye-tracking analysis? (What many commercial software packages won’t tell you)
•    Basic eye movement terminology and why is it important when interpreting results?

Bad web design is not a good thing.

Just to start us off, I thought I’d share one of my favorite misinterpretations of eye-tracking data.  This originally appeared in a blog entry last year:

I think web surfing is a hunting activity. The eye is looking for anamolies, for things that don't belong. (That might be why the word anomaly, spelled wrong in the previous sentence, got your focus). […] One of the takeaways is that bad web design might actually be a good thing! Slightly bad design isn't familiar. It's off. It demands attention. (Very bad design demands the 'back' button, of course).

What I learned from eyetracking -- Seth Godin

I have a love/hate relationship with Seth Godin’s article. I love it because it is a perfect cautionary tale about why we should take the time to stop and understand data.  Quick assumptions (especially based on eye-movement recordings) can lead to some surprising conclusions.  This erroneous interpretation has gotten quite a bit of attention, and has even been mentioned in meetings I’ve had with several designers.

So just a few notes to get us started:

•    Bad web design does not encourage viewer attention. It discourages the user from making an effort to understand web content, and only succeeds in getting users lost and frustrated. 

•    Novel interface design does change looking patterns.  However, as long as a website or email is well designed and intuitive, users will learn to navigate it quickly. 

•    Individual search patterns should almost never be considered alone. The video is interesting and fun to watch, but cannot by itself give useful information about how a broad range of people view the site.

•    Individual gaze plot data is always noisy.  This is because we normally move our eye 3 times a second.  A group of gaze plots must be examined to find patterns in page viewing.

Useful References:

Poynter Studies
2005 Enquiro, Eyetools, Did-It Study -- Google

E*Trade's homepage from January 2001

Visual dead zones reduced E*Trade's ability to communicate with their target customers, because low percentages of people read their promotional content.

New Eyetools EyeTracking Study verifies the importance of page position and rank in both Organic and Pay-per-click search results for visibility and click through in Google.

(PRWEB) -- A joint eye tracking study conducted by eye tracking firm Eyetools and search marketing firms Enquiro and Did-it.com has shown that the vast majority of eye tracking activity during a search happens in a triangle at the top of the search results page indicating that the areas of maximum interest create a "golden triangle."

These eyetracking results apply to Search Engine Optimization and Marketing.

Read more...

Eyetools Heatmap of people reading
the new Washington Post homepage
(launched February 15, 2005)

Last week, the Washington Post announced a new homepag, and here is the data showing how it performs: an Eyetools Heatmap of a group of 19 new visitors viewing the new page; along with what we can learn from its design.

Fast summary:

• Top half of page — good readable design.
• Bottom half of page — bad example of line-spacing and white-space discourages reading.

Guiding a website redesign with Eyetools eyetracking data offers these advantages:

• Objective, visual feedback to your designers and copywriters about what works and what doesn't,
• Test before launching — don't launch a mistake that loses traffic,
• Remove "opinion-based discussions" (and guess-work) about what is seen.

An Eyetools Heatmap of people reading one of our blogs

People interacted with this nav differently depending on other page elements.