Piping Real Estate Data

For a long time at Redfin, we have been casting about for an easy way to incorporate for-sale-by-owner listings from Web sites such as craigslist, or to supplement a real estate search with blog posts about the neighborhood being searched.

The problem has always been that craigslist, blogs and other Web sites aren’t structured databases the way that MLS databases are. The columns and rows of the MLS database make it easy to filter listings by, for example, the number of bedrooms or bathrooms the consumer wants to see, whereas the Web site feeds we get are just a bunch of text, which is hard to sort out once it reaches your browser.
Enter a new technology, Yahoo!’s cool new Pipes service, which lets us mash-up Web site feeds with MLS data. Yahoo! Pipes helps to turn web sites into databases, so that we can filter and manipulate the data from web sites the way we filter data and manipulate data from databases. With Pipes, one search on “La Jolla 3-bedroom homes” could return blog posts on property prices in La Jolla, La Jolla for-sale-by-owner listings from craigslist as well as data from the San Diego MLS.
Isn’t that exciting?

Redfin’s pioneering idea, that a real estate search site isn’t just a front-end to an MLS database, but a portal to other, more objective, data too, such as tax records, appraisals, property outlines and maps, can now be radically extended to encompass the entire Internet much more easily. What this means is that soon consumers may see a lot more information about a property from a lot more sources, not just the basic marketing details that the industry has traditionally wanted people to see.

Bonus link, from a friend of Redfin: Grenada national police band plays the Taiwanese national anthem in a Chinese-built stadium.


  • Milke B.

    The thing I don’t understand is if the MLS is owned and payed for by the real estate companys and agents that put it together in local markets, why are they forced to give it away to others. How about M.S., software companies,data mining companies, and record companies. Why are their software programs and info protected for them!!! Shouldn’t we all be open or free.??

  • http://www.redfin.com Glenn Kelman

    This post was really about a technology that would allow any brokerage to be more innovative in how it combines MLS data with other information on the Internet.

    But to answer your question: any business has the right to protect its intellectual property. But the listing used to market the property is not what’s being sold; the actual house is what’s for sale (the MLS does not own the property). So limiting access to the listing isn’t a legitimate protection of intellectual property, or at least it isn’t in the same way as limiting access to a video game or a song.

    Moreover, when it’s not just one business, but virtually all the businesses in an industry that work together to control how information is shared with consumers and by whom, it seems natural to wonder whether this cooperation could interfere with a free, competitive market.

    Really, I can’t believe we’re even arguing about this: why wouldn’t it be an indisputably good thing for consumers to see more information about a listing?

  • tiptoeinseattle@lightloafer.com

    Hey Mike B:

    “companys” is not a word, however “companies” is.

  • Jerry

    You’re confusing the fact that the MLS is “owned & paid for” by the realtors (of which Redfin is one) with the concept of intellectual property, such as copyright, in the underlying data.

    In the US, there is generally no copyright in collections of data because very few of them have the necessary level of creativity to qualify for copyright protection. Even if there were, however, the author has copyright and the MLS is not the author. The last time I sold a house, I filled out the MLS form, which would make “me” the author of the data and the one entitled to copyright. Not the MLS or the listing agent.

    As a home seller, I want data about my home to be distributed to as many people as possible, through as many channels as possible. If the MLS and/or the listing agent restrict dissemination of that data, they are acting against my best interest; in essence betraying their fiduciary duty to me.

    One last thing: The MLS doesn’t give anything away for free. It’s my understanding that Redfin pays for the MLS data just like all the other realtors. In the past, the realtors and brokers have tried to deny MLS access to discount brokerages. When a group of companies connive together to restrict competition, that’s a violation of federal anti-trust law. Which is why the DOJ is suing the National Associatin of Realtors and investigating the entire real estate industry.

  • Brad C

    How exciting about pipes! I’ve been playing around a bit with Amazon’s Mechanical Turk web service, and I wonder if there is trivial work involved in identifying housing info that may be more complicated to tackle from a software perspective than what a human would be able to do. By breaking up a project into a thousand tiny steps and farming it out, companies are able to pull content together, identify items in photographs, choose the ‘best’ photograph, and scan hi-res photos of the Pacific in search of a lost software engineer.

  • http://www.rentspider.com Cid Dennis

    I really like pipes. With it you can create some really good data feeds for news with out doing all the programming.

  • http://www.mysharm-el-sheikh.com Red sea real estate

    Realy I waned to thank you for your article “Piping Real Estate Data” it gives us useful informations as we interested in reai estate and properties in a very important region it,s Sharm El Sheikh in Red Sea , Egypt…
    Hope to have your iformations usually in our sit
    becausr we not onlel sell real eastate & properties but build and construct it
    Thanks again

  • http://virtual.marketing.realestate.googlepages.com Realestate_Agent_US

    We have Triple net (NNN) leased properties database covering all US regions – for more information visit http://virtual.marketing.realestate.googlepages.com