OpenStack Summit May 2015 Vancouver

 

OpenStack is incredibly complex with tons of moving parts. When we run tests against a running OpenStack in the gate sometimes things fail for reasons unrelated to the proposed patch. These failures, sometimes refered to developers as "flaky failures", are normally an exposure of interesting timing problems in OpenStack and they can often be very tricky to debug. However, we've found that almost all of these issues and the difficulty we have in debugging them also applies to real deployments.

 

This talk will go over the mechanics of debugging gate failures. Including topics like: common failure patterns to look for, how to trace through (and improve) OpenStack logs, and writing elastic-recheck fingerprints.

 

As part of the discussion comparisons will be drawn to show how similar the process of debugging gate failures is to production OpenStack issues. To reinforce that anything we see in the gate most likely is being dealt with by users of OpenStack outside of the developer community.