For the past several weeks, we’ve received and acted on reports of lag in the Sanctum of Domination raid, particularly during the Sylvanas encounter. We’ve been working hard to understand and resolve the issues, but we overlooked telling you that. We apologize for that oversight, and we will strive to do better in the future.The most common complaint we received was that there was a burst of lag when Sylvanas is first engaged. Our server performance group dug into those reports, and we found the cause – there are a higher-than-usual number of calculations being done at the start of the fight to facilitate the complex mechanics of the encounter. We’re looking into ways we can potentially improve this, but we have to be careful to not change the design or mechanics of the encounter. While this is something that we think we can improve over time, there will still be a few seconds of server lag on pull for now.Outside of that moment, our main challenge in diagnosing these lag reports came from the fact that lag was reported in almost every phase of the fight, and we were unable to reproduce that lag in our test cases. In addition to examining our own internal testing, when we utilized our performance analysis tools on actual live raids that reported they were experiencing lag, the server always appeared to be functioning smoothly. This guided our focus elsewhere.What we eventually discovered is that the Sylvanas fight in particular is very messaging-heavy, the mechanics involved require sending many frequent updates to the player’s game clients, which in itself isn’t a problem. However, we discovered that an unrelated change to how we organize and distribute server processes across our server hardware, combined with accumulated changes over time to how our hardware server infrastructure is organized, resulted in our server processes being more concentrated across our hardware.We believe all of these factors combined to make the messaging overload happen more commonly and cause lag.With our most recent weekly maintenance, we’ve deployed several changes to how we distribute server processes across our infrastructure, which should alleviate this potential source of lag. Please let us know about your experiences in Sanctum of Domination going forward, especially if you can capture video of any new misbehavior.Thank you very much for your valuable feedback!
