As part of introduction of PCI Proxy as an alternative form of payment card storage in our products, we had to change how payment cards are processed in our code due to different technology being used. During this change, we introduced a bug in Distributor booking engine that was unfortunately missed during our code review process and QA testing. The bug prevented a reservation from being created when no payment card details were required during checkout due to sending invalid values through API call to our server. Checkouts that require payment cards (no matter which payment card storage is being used) were not affected with this bug.
Within our error reporting this bug was identified rather quickly on production and fixed in our code. However, the severity of the problem was underestimated by developers and rather than being identified as a blocking issue and hotfixed immediately, it was only scheduled for regular weekly deploy. This further prolonged the existence of the bug on production.
We have identified the bug and fix was deployed as soon as we realized severity of it.
In order to prevent any future similar bugs preventing from reservation being created, we have added several automated tests for creation of reservation with different configurations of Distributor, including one without a payment card details required. These tests are run automatically by our integration server for every change in the code and any failure in them will prevent the change from being introduced to production environment.
We will also have internal review and evaluation of our error reporting process, and we will focus on improving identification of severe errors that should be addressed and fixed without delays.