Teaching users to get hacked, or why HTTP forms are bad for the web

This morning I was scanning TechCrunch while still in bed, and I saw a news item about a hot new Silicon Valley startup, Adgregate Markets, that has either solved a really vexing problem in online security or has made the situation a lot worse. Given what we’ve been working on recently at Redfin, I’m pretty sure it’s the latter.

Backing up: for the last few days, we’ve been revamping our registration and login process at Redfin to make it easier for users, and I must admit that what we’ve found has depressed me to no end. We want to put an inline form that lets users log in without ever leaving the page, but it turns out that it’s very difficult to do this reasonably securely. Much more depressing has been the realization that many sites that I use daily, sites I respect, have abysmal security around login and registration. Our awesome summer intern Aditya said after a long discussion of the myriad ways to hack the login process: “Man, I’m never banking online again.” He then confessed that he’d never banked offline (and didn’t know how), which made me feel impossibly old.

Dr. Evil

The topic that most scared Aditya was “Man In The Middle” attacks and the impossibility of providing a reasonably secure experience from an HTTP page. The problem is this: suppose someone evil has set up a wifi router in your favorite coffee house and that he has control over that router. Because all of your traffic is going through that wifi router, Mr. Evil can not only listen to every packet that is transmitted between you and the Internet, he can also modify any packet. Now, on an HTTPS page, that’s not a problem because (a) the packets that the router can see are encrypted and (b) the browser will alert the user with a big scary warning if the any of the packets have been modified along the way. HTTP pages, on the other hand, will act completely normally if Mr. Evil mucks around with them.

What this means is that any form on an HTTP page is inherently unsafe, even if it posts to an HTTPS page.

Let me repeat that: ANY form on an HTTP page is inherently unsafe.

Why? Because as a user, you have no idea if the form at http://awesomesite.com came from awesomesite.com or from Mr. Evil’s nefarious lair. And if the form did come from Mr. Evil, chances are that it will send your credit card number back to Mr. Evil (and your username and your password and your SSN and your mother’s maiden name…). Worse, it could send the credit card info to Mr. Evil before continuing and sending it along to awesomesite.com. Neither you nor awesomesite.com would be any the wiser.

Many web developers mistakenly believe that an HTTP form (on http://awesomesite.com) that posts to an SSL URL (https://awesomesite.com) is safe, because the communication channel for the post is secure between client and server. But Mr. Evil can change http://awesomesite.com freely, so he can make the form post to anywhere he wants. This is why we tell users again and again rule #1 of web security: Never enter confidential information, especially credit card numbers, unless you see the SSL lock in your browser.

Which brings us back to Adgregate Markets. Adgregate is a company that launched at the TechCrunch 50 yesterday, and they have a product that creates interactive banner ads that allow the user to complete a shopping checkout process completely inside an ad, without leaving the host page at all. They say that this both increases conversion rates, which is a Good Thing for advertisers, and increases page stickiness, which is a Good Thing for publishers. The panel at TC50 reacted positively to Adgregate, and the fact that their business model takes a cut of every sale that happens through their ads is certainly attractive.

There’s a problem here, though, and it’s packaged neatly in Adgregate’s marketing copy: “provide(s) secure transactions on both secure and non-secure pages.” As I said above, I think this simply isn’t possible. If the host page is HTTP, then Mr. Evil can swap out Adgregate’s Flash ad for an ad that looks exactly like it but sends all of the info to Mr. Evil’s server, not to Adgregate. Users will be none the wiser until a laser death ray shows up on their MasterCard statement.

On the other hand, maybe Adgregate has figured out how to securely conduct a conversation on an HTTP page. Many, many smart people have tried for over a decade to solve this problem and failed, but it’s possible Adgregate cracked that nut. If they have, though, it would arguably be a better business model to patent and sell the intellectual property. There are a lot of people who would pay good money for it.

As I write this post, Adgregate’s running third in a prediction market for winning best in show. My amateur prediction is that they won’t win, but that they will probably come out of TC50 with a good boost in publicity. My fear, though, is that they will start to find success with their “secure” forms on HTTP pages, and that they will succeed at convincing users that those forms are safe. (To be fair, Adgregate is far from the only company that is attempting to convince users of the safety of HTTP forms, a fact we discovered in the course of our research on inline registration. To my horror, I found out that even my bank encourages users to enter their password into a non-SSL page.)

If users become inured to entering confidential info on to HTTP pages, then everyone in the consumer web space loses. Because when those Adgregate ads get hacked (and if they become at all successful, they will be hacked), people will lose some degree of faith in the consumer web. And after Mr. Evil has gotten you once, it’s hard to believe that he won’t get you again.

(Dr. Evil pose photo used under Creative Commons from flickr user caitlinator.)

Discussion

  • http://blog.jasonbrackins.com jason

    Based on the examples on their marketing page, if they’ve solved it, they’ve done so by avoiding the problem. Their ad forms are all flash widgets, and those are served up over https.

    In theory, since flash is sandboxed, it doesn’t matter what’s going on in the surrounding page. I wonder if the same principle holds for iframes.

  • http://www.redfin.com Sasha

    @Jason:

    While it’s true that the Flash could be served up over HTTPS, that doesn’t matter because Mr. Evil can change the HTTP page that includes the Flash to pull the Flash from anywhere he wants, including Mr. Evil’s website.

    The problem is still not solved because the Flash control’s *hosting page* is not secure.

    Cheers,
    Sasha.

  • http://blog.jasonbrackins.com jason

    right. duh.

    I think I successfully illustrated the point that these vectors are non-intuitive for your average web dev.

  • http://www.isecpartners.com Chris

    This is a good post, but remember that hosting the form and supporting page over HTTPS does not completely mitigate the problem. Most sites return the user a cookie after successfully validating the user’s login credentials. This cookie will then be sent by the browser on any subsequent request.

    Websites that use HTTPS exclusively for login and then HTTP for the rest of the site will still have the cookie being sent in the clear. A MITM monitoring the network will simply wait for the user to login and then grab the session cookie. Once the attacker has the cookie, they can use that within their own browser to interact with the site as the user. The attacker will not have access to the username and password but will have complete access to the site as the victim user for the lifetime of the cookie. Banking sites and other sites that value security are hosted entirely over HTTPS and mitigate this problem by setting the “Secure” flag on any cookie that is sent. This flag causes the browser to only send the cookie over HTTPS and never in the clear.

    Active MITM attackers can also modify the HTTP page directly and insert malicious Javascript that would execute in the context of the site. This Javascript enables the attacker to interact with the site as if they were the user. See BEEF for an example of how sophisticated this form of Javascript attack can be.

    HTTP should never be used for any site that wants some form of provable security. HTTP provides no integrity or confidentiality guarantees and it is impossible for the browser to distinguish between interactions with an attacker and with the legitimate site.

    For more information on proper session management and some of the pitfalls, read Chris Palmer’s excellent whitepaper: http://www.isecpartners.com/files/web-session-management.pdf

  • http://www.redfin.com Sasha

    Chris,

    I totally agree. Many sites, like Amazon, allow access to parts of the account that they don’t consider to be confidential (i.e. browsing history and “you may be also interested in product X” recommendations) over HTTP, while reserving HTTPS for all important information (usernames, passwords, CC#s, addresses, etc.). I think this is a reasonable compromise.

  • http://www.isecpartners.com Chris

    Sasha: Only if they use a different cookie for that portion of the site. If they use the same cookie then it doesn’t matter.

  • databyte junky smurfy david

    It IS possible to make secure forms with flash.
    this is in response to this quote:

    “If the host page is HTTP, then Mr. Evil can swap out Adgregate’s Flash ad for an ad that looks exactly like it but sends all of the info to Mr. Evil’s server, not to Adgregate. Users will be none the wiser until a laser death ray shows up on their MasterCard statement.”

    Now, the above quote is true because Adgregate’s flash is cached, that is how Mr. Evil can get it and swap it out.

    It is possible however, to hinder the cacheing of your flash files.
    This is done by calling that flash file via your favorite server code, and first giving no cache headers before printing out the binary flash data.

    It is not possible to put a flash on an html page via any server code directly (although php as whats called ming that is heavily outdated) so you must use a “loader” or “dummy” flash to call out to the server via the loadvars class or loadclip class.
    The “loader” flash replacing itself with the called flash file.
    In this way the only file that gets cached is “loader” flash.
    Now, its not good enough to have the file not be cached. The files name must also be effectively hidden from Mr. Evil.

    To accomplish this the call to the server code that will print the “real” flash needs to be coded or encrypted. You can simply make the call to the server be like: loader.php?myfile.swf,
    but inside loader.php myfile.swf really stands for
    realfile.swf…of course there are code security issues one must deal with in regards to whatever server side code you use. And it would be wise to use some sort of encryption on both the fake and real filenames making it even harder for Mr. Evil.

    My explanations are brief and do not cover every detail but I hope that you can draw a general idea of the methodology.

    So using these two methods one can 1. Make a flash not cached by the browser (and therefore not decompiled by decompilers) 2. Keep the real name of the file hidden completely (from the browser and any sniffers)

    A Flash form based on this system is far superior to the standard HTML based form.

    To see an example of the no-cache methods being used to not cache images loaded by a swf you can visit http://www.achronicleofike.com

  • http://www.redfin.com Sasha

    @DJS David:

    I’m not sure I follow your comments completely, but I think that what you’re describing won’t protect from the kind of Man-In-The-Middle attack I’m describing.

    First off, the attack has nothing to do with the cacheability of the Flash file. It has to do with the fact that Mr. Evil controls a router that is in between your browser and the server with the HTML page that is serving the Adgregate ad.

    If that HTML page is served over HTTP, Mr. Evil’s router can change the page’s code completely *before it gets to your browser* and have the page load whatever Flash file Mr. Evil wants you to load. That Flash file could be from adgregate.com, but it could just as easily be from http://www.evil.com or anywhere else, and the user will be none the wiser.

    I can’t really see how changing Flash cacheability or keeping the Flash file name hidden would mitigate this sort of attack. Can you help me out?

  • dj smurfy d

    Yes yes you are right. I did not explain myself completely, I have neglected to include the important step 3.

    The caching of the file comes into play because the actual data that Mr. Evil will be able to see and change is the source code of our HTML files and any cached flash files.
    (I’ll expand on this below)

    However the actual data he will see is the “loader” file and not the “real” file. Eg. the movie name parameter in the object tag we use is the name of the “loader” swf.
    If Mr. Evil puts another value in that parameter to load his file, our “real” file will never get loaded. And his file would have to be an exact representation of our “real” file in order to fool our server and our users, this is nearly impossible because Mr. Evil does not even know the name of our “real” file let alone the code within it. (refer to my previous comment)

    Now, the code within our “real” file, therefore, must be married to our HTML page.
    So that if the code in the HTML is not what it is supposed to be or if the “loader” was never called
    the user will see no flash data or be alerted of the change.

    This is the most complex step three, which I must add does not work on every system.
    Step three is simple to implement if all Mr.Evil tries to do is swap out the file he can see with one of his own.

    Before I explain the third step let me talk about how Mr Evil can actually see our data.
    (I’m not well versed on how its actually changed)
    (but now I will expand on what I said earlier)
    In a packet sniffer when a flash file is called upon and given to the browser from the server the data is truncated and hard to understand but the active-x headers go with it and this is how Mr. Evil can know a flash is being transmitted.
    When using the non-cache methods (in my previous comment) the transfer of such a flash is even more truncated and in fact it does not resemble a standard cached flash passing threw at all.
    So Mr. Evil would have to be one of the craftiest and smartest hackers in the world in order to see everything clearly from just looking at the packets…but lets assume he is that smart.

    Now, if Mr. Evil passes his malicious swf where our “real” swf is being transmitted the marriage between the HTML and the flash will be broken because the code inside our ‘real’ swf will no longer go out to the HTML page.
    If he only changes out the “loader” swf
    the HTML page can realize this.
    (this is assuming Mr. Evil is not doing any real-time attacks although using xml sockets one can have a real-time relationship with the page)

    If Mr. Evil changes the actual HTML source code.
    He has complete control of our site anyway and there is no real defense against this except to stay away from simple looking flash files that are easy to recreate, and to use HTTPS.
    Also against this one must rely on flashes that call upon other flashes,(my other comment)
    if Mr. Evil changes the whole HTML code and loads his own swf, that ‘evil’
    swf will not be able to load subsequent files, because Mr. Evil does know the names of those files. He would have to create his own flash file system in order to fool our users. But he would not have our images or swfs to use in order to do this. (he would have to rely on screen captures or something else which is normally of far less quality than the originals)

    Alright let me just explain step three from easiest to hardest, theres no better way than just to explain the steps and I feel I’m rambling on.

    easy step three:
    (doesn’t guard against all possibilities eg. full HTML code takeover)
    basic javascript flash swap guard:
    (steps one and two are generally outlined in my previous comment)
    now our “loader” file is the actual file
    in our source HTML, in the object tag,
    so its url is for example myflash.swf,
    however when it replaces itself with the “real”
    swf the new url of the flashes root accessible by the DOM object will be the url used to load the “real” swf for example loader.php?other
    (this url should not include the letters swf)
    so with javascript we can setup a function to look for the root url of the flash which should not be
    myflash.swf after the “real” flash has been loaded. (this function will need to be on a timeout to give the real swf a chance to load)

    If Mr. Evil swaps out the “loader” swf for one of his own and that evil swf does not load our “real” swf,
    (the call to loader?other can be seen and Mr. Evil can put this call in his evil swf, but by doing so
    it will replace his evil swf with our “real” one
    making the attack useless)
    the url of the root will be wrong and js can alert the user that the file seen is not the correct one.

    To actually do this requires some crafty try catch statements in relation to the way each browser sees the flash object. And one must also use the most modern way to embed flash, which does not use the embed tag at all but rather includes the data parameter for non MSIE browsers.

    The JS method used for this is TGetProperty(“/”, ’15′)
    (you must use a var that stands for 15)
    this method has been around since flash 4.
    More info can be found on Adobe.com, although the info is old and hard to find. Try searching for the old Macromedia notes “flash calling methods from javascript.”

    As you can see “easy” step three is somewhat involved. To make the system more secure your “real” flash needs to also have a marriage with your server side code. I will not get into it now as i have typed alot already and I’m refraining from writing any actual code here, but the tactics are similar to the js method.

    All in all those reading this blog should come away with the fact that true security on the web means a constant HTTPS connection(period)

    However using flash correctly can add some security in case you can’t afford HTTPS (like most of us)

    I hope I’ve helped a bit.
    Please note that some of these methods are brand new 2009 stuff.
    What I’ve coined as strict flash 0.0
    and the pigindyserf method will be published on the web soon for all to learn in detail.