Emanuil Slavov

Head of QA at Komfo at Komfo


The Ultimate Feedback Loop: Learning From Customer Reported Defects
Friday 15:40 - 16:25

Your rating:

“In God we trust; all others bring data.” W. Edwards Deming

A defect reported by your customers is the most expensive one. Everyone in the developer<->customer chain is getting paid for rework, and wastes time for something that should have beed done right in the first place. Beyond pure money loss, those defects are an embarrassment for any organization. They passed all the quality gates. But the greatest cost that cannot be measured easily is the loss of reputation.

It comes as a great surprise then, that almost no company investigates the defects reported by its customers. The companies try to quickly patch the problem and move forward. It’s a shame as a great deal knowledge can be gained about the system that produced the defect in the first place.

We’ve analyzed more than two years of customer reported defects data. Even though we thought that each defect is unique snowflake, some obvious patterns emerged quickly. We were able to debunk some widely believed software dogmas that were not working for us. We figured out which of the techniques listed were helping us or not to lower the defects count:

  • Following the software testing pyramid guidelines?
  • Switching the backend from PHP to Java?
  • Writing a simple unit test, where there was none?
  • Writing a simple integration test, where there was none?
  • Performing mutation testing?
  • Focusing test engineers to use specific techniques?
  • Using static code analysis?
  • Determining the typical profile of a method thats likely to contain an error?
  • What are MTBF, MTTD, MTTR and do they matter?

Each company is different, what works in one situation will not work in another. But we all need to learn from the most expensive kind of defects. This is very powerful feedback mechanism that should not be wasted. We’ll share our experience in building a simple framework for analyzing such defects and well as tips and trick so that you can build a similar program in your organization.


Emanuil Slavov is head of QA at Komfo, where he deals with the ever-changing landscape of social networks and works with dev, ops, and test teams to implement automated testing, static code analysis, custom build systems, development tools, continuous integration, and delivery pipelines. For more than 15 years, Emanuil has specialized in software quality in industries including online money transactions, secure file transfers, email and network security, and antifraud, as well as various legacy and backend applications. He blogs at Emanuilslavov.com.

Create account

You need an account to save events or rate talks.

Already have an account? Sign in

Sign in

Sign in to save events or rate talks.