Now, I know what you’re thinking. For a core back-end technology like a web server, why use JavaScript, a technology that for much of its history has been used to run cute little animations within your browser?

Well, it turns out that JavaScript is very good at non-blocking programming, setting boundaries on what a program can and can’t do so that the little animations running on your favorite web page don’t lock up your whole computer. And it’s good for callback-style programming, so a program can wait for another program to do its thing without using up a lot of resources in the meantime.

This turns out to be a perfect solution for a web server handling a large number of users doing a lot of little things in real-time, like Twitter tweets or Facebook updates or even comments on properties, favorites and x-outs. The problem these programs face isn’t using your computer’s whole processor to solve an advanced physics problem. No, their problem is mostly just storing in a database a fire-hose of tiny bits of data, and zipping information between all the other computers being used to keep a site like Twitter or Facebook up and running.

Well it turns out that JavaScript, the humble scripting language once largely confined to the web browser running on your laptop, is a perfect framework for a lot of these back-end communications. Node.JS, the web server Ryan Dahl created in JavaScript, can easily ask a database to store a tweet without actually waiting for the database to store it; the server just moves on to the next task. This allows the web server to a do a lot of little things very fast, for very many people, which is exactly what a web server often has to do.

Ryan’s central insight in a world of networked computers is that it’s never your computer’s processor that slows you down anymore, it’s the inputs and outputs used to connect the computer to file servers and other computers. So rather than just trying to be computationally efficient, Node.JS focuses on being efficient in how a program deals with streams of information.

Even though a big website may use different web servers that have better security or more institutional support, there might be a place alongside those web servers for a Node.JS server as a hub for sending requests to other databases and servers. Suffice it to say, Ryan Dahl’s talk was a fascinating exploration of how technology may change to support real-time streaming functions.

And Ryan was a dazzling speaker, somehow both opinionated and scrupulous. Among the highlights:

On V8, Google Chrome’s JavaScript engine: V8 is a masterpiece of engineering. Google took the 14 best VM engineers and locked them in a closet in Denmark. They were given the JavaScript spec and then told to make it fast.

On Node.JS’s performance as compared to other web servers: This should be shocking to you. You should be urinating right now. Or getting angry. It shocks me.

On the performance problems created because the popular programming language Ruby on Rails doesn’t make it easy to talk to a database while doing other things : It’s the year 2010, we’re using Rails, and when you access a database, it stops, the world stops for who knows how long, the database might be in LA, and it takes 2 seconds to respond.

But don’t take my word for it. Read the we-typed-as-fast-we-could transcript of his talk! And for a list of upcoming talks, visit the event’s Facebook page.

The post "You Should Be Urinating Right Now" (A Real-Time Web Server) appeared first on Redfin Real Estate Blog.

Since Ryan created Node.JS, he’s the perfect person to talk to about alternative approaches to web servers, and his talk is one of several we’ve hosted recently on building large-scale web applications. Together with Pandora, Digg and Greylock, we’ve put together a whole speaker series on topics as diverse as Clusto, Cassandra and HTML 5 vs. native mobile applications.

Now Wednesday’s talk is at 6 p.m. in Redfin’s San Francisco offices, 44 Montgomery Street, 13th floor. We’re ordering pizza and beer, but we need to know for how many. Please let us know if you’d like to come by visiting the event’s Facebook page… so far, the turn-out is looking fantastic, so it should be a good time.

The post The Great Ryan Dahl to Speak Wednesday on Node.JS appeared first on Redfin Real Estate Blog.

Now Redfin’s very own Sasha Aickin just published the slides for a  discussion he led last month, about whether to use a web standard like HTML5 for building mobile apps in lieu of building separate proprietary apps for the iPhone, Android, BlackBerry and other devices.

Like everything Sasha writes, it’s funny and thoughtful. The basic argument is that you should use HTML5 unless you:

1. you need to use the mobile device’s camera, accelerometer or notifications,
2. you need the app to actually, you know, look and feel really cool
3. you need to make money from the app, especially by charging for it
4. you’re worried that no one will be able to find your app outside of an app store like iTunes

Up next are talks about Node.JS, Cassandra, Clusto and scaling an engineering organization. You’re all invited!

The post HTML 5 vs. Native Applications appeared first on Redfin Real Estate Blog.

Software Developers as Business Analysts: Jeff emphasized the importance of letting developers access usage data, so they can figure out on their own what optimizations need to be made to the site. While the query language of traditional relational databases doesn’t do much beyond pull the data out, Hadoop allows developers to build their own analytical tools, or extend those of others, using a more powerful all-around programming language like Java.

The Rise of the Machines: Jeff observed that the amount of data being stored by computers is exploding, just because most of it is now automatically captured by machines. You can change one setting on a web server and increase the amount of data you capture about what people do on your website by a factor of 100 or even 1,000. When Jeff worked at Facebook, the company was generating a terabyte of data — a trillion bits of information — every day. And this was in 2007.

Microsoft Has Started to Do a Lot Right: While Jeff was hardly impressed by the packaged software bought by businesses less enterprising than Facebook — Gordon was too polite to include Jeff’s unexpurgated opinions — Jeff reserved special praise for all the analytical tools Microsoft offers with its database, SQL Server. As Jeff said, “It’s kind of scary that Microsoft has started to do a lot right within the last 5 years.” If you do end up working with a relational database rather than Hadoop, Microsoft’s a good choice.

To read the entire summary of Jeff’s talk, visit our devblog. Or check out other engineer-to-engineer talks, such as Twitter talking about its use of Scala. The next big talk is our own Sasha Aickin, dishing it out next Thursday on the merits of HTML5 vs. proprietary mobile applications. Everyone’s invited! And many, many thanks to Jeff for a fantastic talk.

The post Jeff Hammerbacher on Hadoop, Facebook and a Surprising Bit About Microsoft appeared first on Redfin Real Estate Blog.

Alex is an interesting person. He has a tattoo of a Chinese character on his left arm to remind him that change is constant. He was the second or third engineer at Twitter, joining in 2007 to build out the programming interface that introduced real-time to the rest of the Web, and now he is one of the people leading the charge to keep the service up and running at terrifying speed and scale. He has strong opinions about San Francisco, a city he left for good last night. He was driving several cats up to Portland, but had never heard of Mitt Romney’s preferred method of animal car transport.

What I’ve always found most curious about Twitter is that the service is innovative not just in what it does, but in how it does it: despite facing more reliability and scalability pressure than any modern online service, Twitter has embraced a somewhat obscure programming language called Scala to solve its unique problems. This seemed like a big, scary bet (offset by the fact that Scala can use Java libraries and tools). As Alex discussed in his talk, Twitter today employs as many as half of the world’s Scala programmers.

What emerged in the discussion was a fascinating exploration not only of Scala, but also of how any new company has to approach building a large-scale web service: the decisions it makes, the tools it builds, the architecture that it evolves. We invited folks from all over the San Francisco developer community, and many came — from as far away as Apple. By the end of it, we were all deeply impressed at the self-confidence, openness and thoughtfulness of Twitter’s engineers.

Thanks to Alex for coming by on what must have been a hectic day, to the folks at Greylock for their help in putting this together with Digg and Pandora, and to everyone who visited us. You’re all invited to Jeff Hammerbacher’s talk on Hadoop, which should be equally good, since Jeff conceived of and built the data team at Facebook. For now, check out the summary of Alex’s Scala talk; it’s interesting even for folks who have never written a line of code to understand how one programmer thinks about a very big problem.

The post Under the Hood at Twitter appeared first on Redfin Real Estate Blog.

But it wasn’t until our last speaker, Neil Roseman, spoke about managing engineers, that we realized how few programs there are for engineers to talk to engineers. And there’s plenty to talk about: how we’re using the latest technologies, and what decisions we’ve had to make when hiring folks or choosing platforms.

So we hooked up with Greylock Partners, Pandora and Digg to put together an all-star line-up of engineering leaders to lead a monthly discussion in San Francisco on topics that really matter to engineers. The idea is to share practical knowledge and have fun.

We’re kicking the program off with an April 30 talk by Alex Payne, one of the lead engineers on Twitter’s developer platform and back-end infrastructure, about what he likes and doesn’t about using Scala. We’ll host the event at Redfin’s San Francisco office, 44 Montgomery Street. To sign up for this talk or to be notified of others, visit the Facebook page we’ve created for the events.

The rest of the talks will occur mostly monthly, on Thursday evenings at 6, with hosting duties shared by our friends at Digg. Here’s a list of who’s speaking so far:

• Alex Payne, platform & infrastructure engineer, Twitter: Scala
• Jeff Hammerbacher, founder, Cloudera: Hadoop
• Sasha Aickin, head of user experience, Redfin: HTML 5 vs. Native Mobile Apps
• Tom Conrad, CTO, Pandora: Smartphone Platforms
• Rasmus Lerdof, PHP Inventor: PHP Performance
• Jean-Luc Vaillant, CTO, LinkedIn: Scaling Engineering
• Ryan Dahl, Joyent, Inventor of Node.JS: Node.JS

Many thanks to Greylock’s Aki Taha for helping us put the series together, and we look forward to seeing everybody there! To come to the event, you must RSVP via the Facebook page at least 12 hours in advance, so we know how much pizza to order. See you on April 30!

The post Engineer-to-Engineer Tech Talks: The Stars Come Out appeared first on Redfin Real Estate Blog.