Characterization of operational failures from a business data processing SaaS platform

This paper characterizes operational failures of a production Custom Package Good Software-as-a-Service (SaaS) plat-form. Events log collected over 283 days of in-field oper-ation are used to characterize platform failures. The char-acterization is performed by estimating (i) common failure types of the platform, (ii) key factors impacting platform failures, (iii) failure rate, and (iv) how user workload (files submitted for processing) impacts on the failure rate. The major findings are: (i) 34.1% of failures are caused by un-expected values in customers' data, (ii) nearly 33% of the failures are because of timeout, and (iii) the failure rate in-creases if the workload intensity (transactions/second) in-creases, while there is no statistical evidence of being in u-enced by the workload volume (size of users' data). Finally, the paper presents the lessons learned and how the findings and the implemented analysis tool allow platform develop-ers to improve platform code, system settings and customer management.