Building log cabins

Your applications often try to speak to you, but do you hear them?

Many systems have logging, and when you start scaling out, you even have more systems complicating the matter. Getting a handle on tracing a message or event through even just 3 servers can be really tedious, nevermind on more machines. So I built logcabin. It's made out of all the logs on all our servers everywhere.

This talk will be a review of how to get started, an overview (plus demo) of what you can do with it, scaling tips, some Q&A about anything people may want to ask. It will be specifically focused on a python-rich (Ubuntu) environment, including diversity across regular twistd logging, custom application logging, and any other components which might fit into your application logic (nginx, dpkg, mail logs), and how it can assist in distributed fault tracing.