PUBG Scoring Systems - Can we agree to disagree?
With the official PUBG leagues approaching rapidly, and no clear preferred scoring system at present, what is ideal?
At the first PUBG Gamescom event, the question was not how to quantify the results of a team, but if running a LAN server with 100 players on a poorly optimized game was going to run problem-free. Of course, times have changed, PUBG Corp is pushing for official leagues in multiple regions and the rules are (mostly) agreed upon. One thing Starladder, OnGameNet, PUBG Corp and other organizers are struggling to agree on is balancing the merit of placement and kills.
Popular scoring systems
On the most unforgiving end, OGN gives out 10 points for first place and no points for any other team. The assumption is that teams that survive for longer get more kills, which are worth 1 point each, however in the case of M19, who are infamous for being as subtle as possible, this is far from true.
Next is the Korean scoring, used in the only officially sanctioned league - the PKL. The winner receives 8 points, the runner-up receives 4, and 3rd and 4th receive 2, with 1 additional point per kill, in the PKL format, teams are then given weekly points based on their overall placement, from 100 for 1st down to 0 for any team that fails to reach the 16 team final.
The intermediary scoring system, referred to as V4 in some discussions, allocates a different amount of points to all participating teams as seen below, it is unclear how many points will be awarded per kill, or if any will be awarded at all. When I test it later, I will assume 8 points are awarded per kill.
The last scoring system, referred to as Unified scoring as the organizers Auzom and GLL agreed to use it for all their events earlier this year, is the most lenient. 16 points are awarded per kill and points are distributed as seen below.
Acquiring a dataset
In order to compare the scoring systems we need some data to test them on. A fortnight ago, at the OGN Super League Europe Invitational, 24 games were played across the 3 maps - Sanhok, Miramar then Erangel - I'm going to use the data from the 8 games on Sanhok. OGN doesn't record the exact placements of teams as they only award points to the winners, fortunately, Liquipedia does and lays out the results in a format that can be easily copied into a spreadsheet.
After about an hour of tinkering I was able to produce some usable statistics on this spreadsheet here:
Full Comparison of PUBG Scoring Systems
Inside the mind of a data miner
My first observation was that there is almost no difference between the placements of teams using V4 and Unified scoring, and Korean and OGN scoring. This was sort of expected as there are few fundamental differences between them, however we can now divide the systems into classes. The first I will call “Naïve”, and the second “Stingy”. Given that V4 is supposed to be a compromise between Korean and Unified scoring the fact I’m classifying them alongside either is disappointing.
Stinginess is quantifiable - I'm going to do it by comparing the standard deviations of the percentages of the highest score for each scoring system, I'm going to exclude the top scorer (Na`Vi) as they are an outlier.
In theory, using the standard deviations, we could create a scoring system that is an exact compromise between Unified and V4, time to crunch some numbers, again.
A disclaimer
I acknowledge that creating a scoring system based on parameters as simple as these will not provide a perfect scoring system. For anyone with an interest in data science, don’t artificially create a system that fulfils parameters, computer scientists haven't spent decades thinking of ways to prevent overfitting for me to ruin it.
Creating a theoretically balanced scoring system
In reality, it only took a few minutes to generate a balanced scoring system:
This gives a standard deviation of the percentage of the highest score (feel free to come up with a shorter name) of 13.9%, and, in my opinion, it solves both the problem of kills being worth more than places in Korean scoring and teams being rewarded for dying less quickly in the early game in V4 and Unified scoring.
The data I created is public, so please use it and make sure to send your thoughts or any additional observations to me via social media (@sigmoyd).
