Comments on: Elephant versus Dolphin: Which is Faster? Which is Smarter?

By: Logan Bowers

Logan Bowers — Fri, 09 Nov 2007 05:23:45 +0000

On the Partitioning methods, you can also partition randomly, e.g. of n servers, each has 1/n properties. Your overall load is higher since the query has to be prepared separately on each server, but your total throughput is N times greater so (theoretically) your query finishes in nearly 1/N the time. For a small reduction in efficiency, you can significantly reduce visible latency to the user.

By: Durch Marlowe

Durch Marlowe — Thu, 08 Nov 2007 20:15:32 +0000

#1 is easy. Oracle is a great database for this. Unfortunately you have to deal with sales guys.
#2 isn’t that every Web 2.0 company’s knee-jerk reactions. Don’t they have a name: “sharding” for that. Seems like a band aid.
#3 is cool. If you can formulate your query in something other than SQL (i.e. XML), you can split the query quite simply. The problem is assembling the result set. That is a major flaw of something like this. Databases are good at merging records. No one, to my knowledge has written a thing that efficiently merges records asynchronously arriving from processing node (unless Google does that).
#4. Elastra — interesting company, but still thinking small and database focused. There are parallel problems like yours that need to be addressed and they are beyond the DB technologies that exist today. Hopefully someone gets to it one of these days, especially now, with virtual computing and web-based CPU farms being all the rage.

By: Michael Smedberg

Michael Smedberg — Thu, 08 Nov 2007 17:16:46 +0000

Yup, we’ve thought a little about these issues, and had some internal debates on how to do partitioning.

As you point out, our business naturally partitions geographically. There are a few possible approaches to partitioning that I’m aware of:

1. Let the DB do partitioning, as you mentioned
2. Partition by market. We currently break the world into markets (the San Francisco Bay Area market, the Seattle market, etc.) Real Estate agents are assigned to markets, most users are searching for a home within a single market, and we have “home” pages for each market (e.g. http://sfbay.redfin.com/ or http://seattle.redfin.com/.) However, as you point out, markets are NOT of equal size- it’s a convenient way to partition, but not necessarily a very smart way.
3. Handle partitioning in custom logic. For example, we could write our own partitioner, and direct queries to the server that corresponds to the region being searched (or even to all partitions in parallel.)
4. Use a service like Elastra (http://www.elastra.com/), which provides something like the “on-demand access to an arbitrarily large number of machines to perform the processing” that you’re proposing.

For now, Redfin probably doesn’t need to do any of those things. The combination of good GIS indexing, 64 bit machines, and cheap RAM lets us handle pretty big data volumes (tens of gigs) without sweating. We can probably double the number of markets we serve without worrying too much about partitioning. Once we approach hundreds of gigs of data, though, we’ll definitely have to think about this more carefully.

By: Dutch Marlowe

Dutch Marlowe — Thu, 08 Nov 2007 07:00:21 +0000

First of all, good points on GIS support in Postgres. I always liked a database that let me compute spheroid distances based on the Geodetic survey of my choice (big fan of the 1980 ). Yeah, Oracle can do it, but who wants to blow $20K on a server license.

One interesting thing I never got to doing while playing with Postgres 8.1+ was the ability to mix Postgres query partitioning (http://www.postgresql.org/docs/8.1/interactive/ddl-partitioning.html)
with spatial filters.

In theory, your listing are much more dense in some areas than in others. If you could partition on arbitrary geometry and then have the query split among resources as it executes, you could create this giant query computer that really focuses on geography as a way of keeping query times sane. Your geographic partitions can be very large (”The Midwest”) or very small (”The Upper West Side”). Now imagine data for the midwest sitting on one machine and data for the UWS sitting on another machine. The machines have the same resources (memory, CPU), but cover vastly different areas because there are more listings in one than the other. They individually perform well and can be easily re-partitioned (split Midwest into 3 “query machines” for example). Performance is linear on the machines as long as they are similar. Performance on an area that does not cross geographic boundaries split between machines is linear. Performance when someone spans boundaries is NOT #machines X individual machine peformance. It looks more logarithmic. Now you have the “Google” of geographic searches.

You should be able to arbitrarily scale this system ad nauseum.

Now, if you only had on-demand access to an arbitrarily large number of machines to perform the processing…

By: Michael Smedberg

Michael Smedberg — Tue, 06 Nov 2007 18:05:52 +0000

Well, listing perf is already pretty good for most users (almost always under 200ms), so there isn’t as much of room for improvement. But yes, I do expect to see performance improvements on the listing end.
As the number of listings grows (i.e. as we move into more markets), this will be a bigger effect, and the effort of indexing listings well will be more apparent.

By: Glenn Kelman

Glenn Kelman — Tue, 06 Nov 2007 17:55:59 +0000

Michael, do you think performance will be improved much by having Postgres run LISTINGS search?