Pixel Inference Attack: The Human Side Channel

Ron Masas
5 minute read

Our behavior is extremely predictable when it comes to how we use the web.
When a popup jumps at us we close it. When we see a captcha, we solve it.

It’s easy to underestimate the information that can be leaked by knowing the color of a single-pixel on a cross-origin page. While the Same Origin Policy prevents scripts from accessing different origins, it does not limit what is shown on our screen.

In this post, I'm going to talk about attacks that exploit the users themselves as a side-channel. This is done by tricking users into unknowingly disclosing private information by taking advantage of their visual system.

Most of the attacks I’m going to discuss relies on the dangerous combination of humans, CSS, and the lovely iframe tag.

Thanks to the X-Frame-Options header there aren’t that many sites that both reflect valuable information and can be used in an iframe. However, the fact that there aren’t many of them does not mean the biggest social network in the world isn't one of them.

Allow me to introduce, the Facebook like button. It’s an iframe that reflects whether or not you like something.

The like button contains many pixels that leak its state. For example, the pixel color in the tip of the thumb of the like button “off state” would be light blue if the currently logged Facebook user liked the tested page or white if he doesn’t. With a bit of CSS magic, we can cut, scale, and normalize the colors so that we have a square in whatever size we want that is either black or white depending on whether the current Facebook user likes a given page.

Since we know the color must be either white or black, we can simply position two elements on top of it, one black and one white (see the illustration below) and let our user click the one he can see, unknowingly disclosing the pixel color to us.

Facebook Pixel Extraction

I made a proof of concept that leaks whether or not the current Facebook user likes the Nike Facebook page.
Please note, you must be logged in to Facebook for the attack to work, the attack does not work on iOS.

While writing this post I thought about a cool, yet not so practical variation of this attack. The idea was to use the user webcam to figure out the pixel color by measuring the light the screen emits.

This obviously assumes the user has a webcam, and that he’s given a rogue site access to it. On top of it, the technique wouldn’t work in some lighting conditions, or if the screen light does not affect what's being recorded.

With that said, once we have access to the camera, we can continuously leak information without any further user interaction.

Theoretically, I believe this method could be used to extract arbitrary text from any embeddable page by performing multiple tests on each character. Once you identify the character you essentially also solve the "character distance problem".

With that said, extraction will be slow, my estimation is that each character would take between 1-5 seconds.

Below is my proof of concept for leaking whether or not you like the Nike Facebook page using your camera.

Leaking Your Browser History

Links can have different colors based on whether or not you’ve visited the page they reference. Unfortunately, this behavior can be easily exploited.
With a bit of CSS, we can turn the link into a single pixel reflecting this boolean result, and then, we could use one of the techniques I’ve discussed above to trick the user into leaking it.

Evil Captcha

When an embeddable page renders personal and or sensitive information in a predictable location, a fake captcha can be used to trick users into leaking their own information.

The fake captcha is simply a bunch of iframe elements that were transformed to show a single or a group of characters.

When leaking numbers, this attack works perfectly.
This is because numbers have the same width, which means we can reliably show each number in its own iframe.

However, English letters have different widths, which means we can not reliably create iframes that show just a single letter. I overcome this limitation by showing a chunk of characters each time, purposely overlapping a few pixels for each chunk.

I recently found out Google Docs documents can be embedded. As you can see below, when the viewing user does not have sufficient permission to view the page a "request access" page is shown with the user email address at the bottom.

I made a proof of concept showing how this could be abused to leak the user email address. You have to be logged into your Google account for the demo below to work.

To perform this attack we have to find an embeddable page that reflects a valuable user state in the form of a pixel color in a predictable location.

For example, a page that has user-related content that affects the position of the pixel we wish to leak would make this attack impossible. This is because we wouldn't be able to eloquently cut and scale the iframe to show just that one pixel.

Closing Thoughts

I think that some of the attacks I've discussed are highly practical and should be taken into consideration when creating
embeddable pages.

Currently, rogue sites could use those types of attacks to incrementally collect private information about their users.

  • Facebook should at the minimum allow users to disable the like button statefulness.
  • Browsers should disable or at least give the option to disable the coloring of previously visited links.