Publishing MLS data that shows that Redfin got a better deal for buyers than agents at other brokerages sparked a riot yesterday: here, here, here, here, here, here, here, here, here, here and here. We also showed up in Freakonomics (holy cow!).
There has been a healthy discussion about how to interpret the data: whether Redfin agents negotiate better, for example, or its customers tend to seek better deals. We think both factors contribute to our success, and we love the debate.
But attacks on the main finding, that Redfin customers tend to pay less for properties above and beyond the commission refund, have been flat-out wrong:
The date range was arbitrary (see comment #5): the date range we chose was from February 6, 2006 to February 5, 2007; exactly one year from the launch of our home-buying service. If the date range had been January 1, 2006 to December 31, 2006, as critics suggested it should have been, our negotiating advantage would still have been .769%. It seemed less fair, not more fair, to include January 2006 data when Redfin Direct was not available in January 2006, but the overall result still favors Redfin.
Redfin sales increased as the market softened, skewing its advantage (see comment #6): if we analyzed only the last 90 days of the time period studied, the Redfin negotiating advantage would have been 1.10%, as opposed to the .904% we calculated for a full year. Redfin’s negotiating advantage over other brokerages actually increased, not decreased, with deal volume.
The data are not statistically significant: based on a p-value calculated from the MLS data set, the likelihood that Redfin’s advantage is entirely due to a small sample rather than a legitimate difference is less than 3%.
An NWMLS report contradicts the NWMLS data Redfin cites: an NWMLS report states the median final price of King County homes sold in 2006 was 81% of the median listing price of King County homes listed in 2006. No one believes that a typical home sells for 19% below its list price; to verify this, Redfin retrieved from the MLS every record of a house or condominium sale that closed in 2006; of the 37,185 transactions, only 49 (.13%) closed at a discount of 19% or more. If that report were correct, a discount of this size would have been 385 times more prevalent than it actually was. We have called and written the NWMLS, which is verifying its own report; we will notify you when the source data or the methodology becomes available. If this report proves us wrong, we will say so.
The data are impossible to replicate: we published a methodology for replicating the data that several complete NWMLS neophytes were able to follow. Nonetheless, we are now offering to share the data, with addresses and other private information removed, until the NWMLS objects. We already have sent the data to folks from Rain City Guide, Three Oceans Real Estate, Bloodhound Blog and 360 Digest, all of which have been strongly critical of Redfin in the past; if there is an error, one of these bloggers will find it.
Having challenged us, we would ask at this point that our critics report their findings, whether there is an error or not.