Sometime in the wee hours of the night last night, Redfin upgraded our website to show photos, prices, and all the gory details on every property sold for the past two years, in almost every area we serve. So far, we’ve loaded up 9 million photos of 1.4 million recently sold properties, quadrupling the amount of real estate information we store.

And since we turned on our data slurpers full blast for new past-sales records, the data will keep piling up: within 15 minutes of an agent’s taking a property off the market, you can expect to see the pictures and the price on Redfin’s site. This investment in near-real-time sales history continues Redfin’s multi-year journey into Freakish Depth, which is our strategy for taking property data to ridiculous extremes of accuracy, freshness and depth. We meet our customers exclusively via our website, and if the site isn’t trustworthy, why would anyone expect the service to be?

Yes, OK, You Do Have to Register…
The only hitch is that outside of the Seattle and Washington DC areas you need to register before going deep. OK, so requiring registration is not normally our style — we’re not looking to spam you or call you out of the blue — but there’s a reason for it. The Multiple Listing Services that share this data with us first want to make sure we’re not posting it everywhere willy-nilly, limiting it instead to the folks who are serious.

But It’s Worth It
Once you’re registered, you can do all sorts of fun stuff, like checking out all the $1M+ homes that sold last week in Los Angeles, San Francisco or Boston. Or do your own search on recent past sales: visit our search page and look underneath the Search Listings button for a More Options link.

Clicking that link opens a big search dialog where the options for searching past-sales records appear bottom left:

After you run a query on past sales, the map fills up with pretty baby-blue icons representing homes that recently sold. Just as with listings, pictures and basic details pop up to the right:

If you click through the thumbnail picture for more details, you get a staggering amount of detail on the property that just sold, usually all the information the seller’s agent provided when listing the house in the first place — which we then pair with parcel outlines, tax record, automated appraisals and all sorts of other goodies too numerous to include in the screenshot.

Before the upgrade, we only showed public records of recent sales: the most basic facts about the property’s bedrooms and bathrooms, the price, no pictures, all after waiting two to twelve weeks for the records first to get recorded by the county, then aggregated by a data-collection service and then syndicated out to sites like ours. What we have up today is a big improvement.

The Killer Application
But now that we’ve published a near-real-time feed of pictures and details of virtually every recent sale, there’s still the question of what you as a consumer can do with that data. And the first, most obvious application is a Comparative Market Analysis, or CMA. If you’re trying to price a property, you can usually establish a reasonable range by putting together a list of similar properties that sold in the area over the past month or two — use the photos to exclude all the dumps, unless you’re buying or selling a dump — with the same square footage plus or minus 10% — and usually the same number of bedrooms and bathrooms plus or minus one. Then take what each of the comparable properties cost per square foot and multiply it by the square footage of the property in question, sorting the resulting prices from high to low. This is the starting point for any real estate professional to think about a property’s price range.

Now you can take this on yourself, using the same data that agents have, even if later on you still want an agent’s help corroborating which comparables you picked and analyzing the results — Redfin agents put together CMAs every day, and don’t expect to stop any time soon. But even if you get a little help, it’s nice to be able to see for yourself what the agent can see, and to go as far as you like on your own.

The Other Big Feature… Trackbacks
The other big new feature in this release is blog trackbacks. Any time a blogger links to a listing on our site, we now link back to the blogger, so consumers can easily see what everybody on the Internet is saying about a house. If a blogger reviews a condo in a new development, any link to that listing on Redfin will generate a reciprocal link from the listing on Redfin back to the reviewer’s site, along with a snippet of text from the review.

Tying together everything that’s being said about a property can, we hope, socialize the lonely, scary process of buying a home, and it will almost certainly boost the quality and quantity of online discussion. The same trackback technology that almost overnight created the blogosphere’s ecosystem of criss-crossing links  can now nurture the growing number of real estate bloggers — many of them agents — who use Redfin and other brokerage sites as a reference for their real estate discussions.

For the industry, both of these features are a little bit daring. Real estate agents don’t like the idea of some crazy blogger trashing their listings, and most everywhere they have preserved the option for sellers to opt out of any type of blog integration or online commentary. We think opting out is a very bad move, just because social media drives traffic like nothing else to a listing — but we can’t argue with giving the sellers a choice.

Really, it’s impressive that we got this data at all. A long time ago and with plenty of the evidence to the contrary, we made a bet that the industry would become more open, that we could challenge the status quo as a broker without being squashed, that complying with local MLS rules would be worth the extra hassle so we could get information directly from other brokers. This turned out to be a very lucky bet.

You can argue that this version of Redfin would never have happened without last year’s Department of Justice settlement with the National Association of Realtors, which gave online brokers the right to share any data with our customers that an agent can divulge to a customer face to face. Both trackbacks and the newly available past-sales data would not have seen the light of day if not for this settlement. But the hole in that argument is that MLSs not even governed by the settlement nonetheless have agreed to many of the same terms.

We still have plenty of gripes, but the industry is changing for the better. So in addition to the thanks we owe to all the Redfin folks who worked on this gargantuan effort — hats off to the whole team —  we also wanted to thank the other brokers for agreeing to a more open data-sharing policy. We’re excited to see what consumers think of the new policy and the new site.

Update from Matt: When we first released the site this morning we experienced a lengthy site outage. We’re very sorry for the inconvenience.

I remember running down a hallway and bursting through the double doors of a closed conference room to tell all the Redfin bigshots the big news. Our dignified compliance manager, Mary Black, flushed with an unholy glow, had somehow gotten there ahead of me.

And now, hardly a week later, Redfin has shipped a new version of its site that lets Boston consumers use Redfin the way everyone else does: without having to register your name or email address. This means consumers can get all the information about Boston-area homes for sale without wondering when a real estate agent will call, or getting buried in spam.

Why Registration Is a Big Deal
Why is registration such a big deal? Well, imagine if you had to register with Google before you ran a search. And imagine if Google was in an industry notorious for using that information to strap you into a gigantic drip-marketing system?

You would say what most Boston-area consumers have said to our website: no thanks. The graph below, taken from yesterday’s presentation to Redfin’s board, shows the results. Boston traffic is the green line, which after a year of toodling along, just got passed by Chicago (orange line) in its second month of operations. Pathetic!

But that’s all going to change. Now that we’re the first site to offer complete registration-free access to all the MLS homes for sale, we hope that our Boston traffic will shoot through the roof, and that our business there will too. We’re gearing up a big marketing campaign next week.

Many thanks to Kathy Condon, John Breault and the entire MLS Property Information Network Board for taking such a huge step forward.

What Does This Mean? The Big Picture
A long time ago, Redfin made a big bet that we could work within the system as a broker, showing all the homes for sale even as we changed how consumers worked with a Realtor and what they had to pay. For years that was a crazy bet. Maybe it still is.

But we’re seeing MLSs across the country negotiate a truce between brokers of all stripes so consumers can get more information about listings. That’s good for consumers, good for Redfin and, at a time when people have wondered whether MLSs and brokers would change with the times, good for the industry too.

Redfin Boston supporters, spread the word!!! And gentle Redfin blog readers, what do you think? Is the Boston-area MLS decision part of a bigger trend? We’ll keep you posted on what happens to Boston traffic.

Some of you may have noticed that San Diego listings on Redfin.com haven’t been updated over the weekend. Sandicor, the San Diego area MLS, has been having some issues that mean it can’t export listing data to MLS-powered websites like ours.

We’ve talked to the folks over at Sandicor and they’re working hard to resolve the issues. They run a tight ship, so we’re confident the data will be moving again soon.

San Diego listing data highway: temporarily closed.

It bothers us whenever the data on our site isn’t as fresh as possible. We’re a member-broker of the MLSs in each of our markets so that we can get the most accurate and up-to-date listings on Redfin.com. We update our site with new data from most of the MLS databases every 15 minutes.

But sometimes problems come up. You should know what you’re getting on Redfin.com, so here’s what you can expect:

We’ll update listings on Redfin.com every 24 hours, or sooner.
This applies to our MLS listings, as well as foreclosure and for-sale-by-owner (FSBO) listings. An update means the latest data from our listing data providers is on Redfin.com. For sources of data that change less often, like tax records and school information, you can expect the data to be updated less frequently.

When we can’t, we’ll tell you why.
Whenever our listing data is more than 24 hours old, we’ll post a note in the forums for each market that’s affected, explaining the issue and giving you some idea of when it will be resolved.

And then we’ll fix the problem ASAP.
If the problem is on our side, you can rest assured that we’ll attack it like rabid dogs until it’s fixed. On weekends, we may be slower to discover problems with our data, since our developers personally monitor the data feeds and they occasionally, but only occasionally, take part in non-development related activities – i.e. “personal life.” If it’s our partners who are having trouble, we’ll give them whatever help they need.

We simply want to build the best real estate website in the world, and as part of that we’re making this public commitment on our listing data. Comments are, as always, welcome.

Photocredit: sillygwailo on Flickr.

The policy was suspended while the lawsuit lumbered through federal court. And in the interim, Redfin was able to cite the lawsuit in convincing investors that we could compete straight up, broker to broker, without losing access to all the listing data controlled by other brokers.

And it’s still why Redfin, alone among the major new websites, has had all the broker-listed homes for sale: we’ve been able to become members of the Multiple Listing Services (MLSs) that Realtors use to share data, and have made our peace with its other rules.

But plenty of folks wondered what would happen to Redfin when the NAR suit settled. We wondered too. Well, today the suit settled. When I first read the NAR press release, I suddenly remembered what Billy told his platoon of mercenaries at the beginning of “Predator”: “We’re all gonna die.”

The National Association of Realtors proclaimed a stunning victory, first because it didn’t have to admit to any wrongdoing, though this is a standard feature of many settlement agreements; and second because the NAR also said that it didn’t have to pay any money, though this is hardly what the Department of Justice was after.

Greg Swann at Bloodhound, quoting Hamlet Macbeth, rightly said so what.

But the proposed settlement agreement did result in a major change, the permanent repeal of the Internet Listings Display policy that would have allowed brokers to selectively withhold listing data.

So for the consumer (and for Redfin too), the settlement is good news: an MLS can’t discriminate against Redfin or any other broker because of our business model or our technology. Any information that can be whispered by a real estate agent to his client — such as how long a home has been on the market, or how its price has changed over time — can be distributed by Redfin through its site. Hooray!

But the NAR wasn’t about to set the data free willy-nilly, especially when its member Realtors are accountable to home-sellers who want to see their homes marketed, not discussed or criticized. For one thing, the DoJ protections only apply if we ask site visitors to register, which turns off about 90% of the people who visit a real estate site (how would Google have grown if it required registration to search?).

Beyond that the NAR claims that “the new policy protects sellers from having false or other unwanted information about their listings appear” on sites like ours. We wondered what that meant. According to the exhibits in the settlement agreement, a seller can opt out of:

“1. allow[ing] third-parties to write comments or reviews about particular listings or displays a hyperlink to such comments or reviews in immediate conjunction with particular istings, or
2. display[ing] an automated estimate of the market value of the listing (or hyperlink to such estimate) in immediate conjunction with the listing.”

The automated estimate mentioned in the exhibits is exactly what we’ve integrated from Zillow, eppraisal and Cyberhomes. And the online discussions are something we’ve tried to host before, too. We suspect that some brokers will include such prohibitions in their standard listing agreements, so that many sellers will opt out.

Ultimately, we think that ducking a conversation like this is just sticking our heads in the sand. We can understand why the NAR took the position it did, but in the final analysis it marginalizes Realtors, and limits our ability to connect buyers and sellers.

People will talk about homes online, and they’d rather do it on brokers’ sites, where all the listings are available. But if they can’t talk here, they’ll go somewhere else.

So all in all, we were glad to see that the settlement protected all brokers’ access to data. We just want to make sure we can still do something meaningful with the data, too.

Bonus link: The NYT gets snarky about Paris Hilton, AGAIN…

“Great question,” I said as real estate inventory has somehow become one of my favorite subjects, second only to “how popular I was in high school,” gossip, my many grudges, and what different drugs are like. “We have all the broker-listed homes for sale.”
“And that’s ALL the homes for sale?”
“Well, technically there’s always a few sold by the owner, without a broker. It’s called FSBO.”
“Yeah?” Conan said. “You don’t have those?”
“No.”
“Is that it?”
“And there’s also stuff in foreclosure, that’s been repo’d by the bank, before the bank has hired a broker to put it in the MLS.”
“Yeah?” Conan said. “Is THAT it?”
“And there’s always the possibility that you could sell your place to your cousin, without ever putting it on the market. We wouldn’t have that either.”
“So basically you’re saying you kind of suck,” Conan said. We had now come to very familiar ground in our 20-year friendship.
“Yeah,” I said. “But everyone else sucks too. Many suck worse.”
In the background, I could hear Conan typing in his first search.

A few minutes after our call ended, the U.S. real estate market collapsed, putting more inventory in a gray market of foreclosures. And we soon began working with I-don’t-have-an-office-number-just-a-cell-phone middle-men, data-scrapers and offshore aggregators to get our hands on every home for sale we could find, minus outdated garbage, copyrighted information, stuff that wasn’t really for sale, and teasers where consumers would have have to pay someone else for the address.

This culmination of this effort is the latest version of our site, live since 6 a.m. this morning, which has bank-owned foreclosures and for-sale-by-owner listings alongside all the listings from the Multiple Listing Service (MLS). It’s a little controversial, and lots of fun to play around with and of course we think it’s beautiful too.

Not that even now the new version of Redfin has all the homes for sale – we only had enough money to get data on foreclosures once the bank owns them, but not before, when they’re up for auction. For the markets we serve, we should have more real estate listings than anyone else. And the foreclosure data we do have is unique, because we give the foreclosures’ actual addresses and bank contact information, which other sites require you to pay for. Redfin is the only site we know of that aggregates bank-owned properties for free.

We’ve hit up more than 50 for-sale-by-owner sites for their inventory, with only two major gaps: Zillow and craigslist. Zillow is, we hope, coming soon. Someone offered us a craigslist feed, no questions asked. But when we emailed craigslist about it, Internet god Craig Newmark confirmed this was a violation of the terms of service. What was incredible about his reply was that it took exactly three minutes and it came from The Man Himself (Craig Newmark rocks).

On other fronts, we’ve fixed up our URLs for Google, and juiced up our MLS integration again, in both Boston and Southern California, so that we can get comprehensive inventory updates every 15 minutes (our only laggard MLS integration is now in Ventura County). While we were at it, we grabbed more open-house data, let people search for open houses, and upgraded our email notifications to tell people when their favorite listings change prices or sell. Next up, the data team is hoping to get photos of very-recent past sales.

But for now we’ve also made it more obvious how to search for data on past sales, a sweet feature that Redfin has always had but which no one ever knew about because it used to be hard to find. No longer! For the markets we serve, you can easily see what any home sold for between last week and 20 years ago (the lag is often more than a week, depending on how long it takes the transaction to record).

What does this all add up to? As our press release explains, the new version takes us deeper and deeper into Freakish Depth, which is our strategy to build the best real estate site by getting the best data. Because we’re a broker in the thick of doing deals, we have access to data as it’s being recorded. And unlike lead-generation sites, we want people to use Redfin to go all the way, getting everything they need to buy the home without having to talk to anyone or pay anything.

The strategy started in January with super-low-latency-MLS integration, bird’s-eye views, two new estimates, Excel integration, neighborhood boundaries and comparable listings. Then we got deeper with listing statistics, which analyzes where the traffic for a listing is coming from, and how inventory levels in the neighborhood have changed.

A gratifying subplot of all this is that the MLSs with whom we have sometimes jousted worked with us on this release. One MLS guided us on how we could show different types of inventory without violating the rules. And another provided the square footage data we’d been asking for. It’s still a delicate balance, sharing data among competitors, and working within a common set of rules to protect the different brokers’ clients.

Which makes us all the more careful not to screw things up. For years, we’ve wanted to add inventory to Redfin’s site, but have hesitated out of respect for the brokers who share their listing data with Redfin, many of whom have legitimate concerns about properties for sale directly from banks or owners. In the end what swayed us was:
a. the consumers who want above all else to see all the homes for sale,
b. the market, where in places like San Diego 40% of the sales have been foreclosure-related,
c. guidance from some of the MLSs we work with.

Hopefully, we’ve found a way to show consumers more information without causing any harm to other MLS members. We’re going to roll out flat-fee services in the next few weeks to work with buyers on for-sale-by-owner and foreclosure properties, but we won’t help anyone go around a listing broker if one has been hired.

Many thanks to all the engineers, product managers and customers – a San Francisco focus group talked us into buying the addresses for foreclosures — who came together to create this site. We worked awfully hard on it, some of us through occasionally daunting circumstances, and are anxious to see how you like it. We hope there aren’t too many bugs. And please, spread the word!

*PS: we may add auction properties in the future but in that case we would have to require users to pay an extra fee to see the address.