#100 Database interference between multiple Halef instances causes extreme latency

[David Suendermann-Oeft]

A number of calls that were placed since the evening of Feb 9, suffered some extreme latency. An example is

ad33957b56086e81ffb868c0dbe1ff3d@10.61.82.235

Latency was between 15 and 20s, and the VAD caught background noise rather than the speaker's voice. Halef was practically dysfunctional. It turned out that the root cause is another Halef instance (the dev instance) writing an extreme amount of log messages into the DB (several million records per day) which strongly compromised the performance of the production Halef instance.

It should be investigated how this can be mitigated. Why would this happen given DB logging is asynchronous and making use of a queue?

Yours,

DSO