Personal blog and homepage.

Why it took so long to update CheatCommands Mod II commander abilities

A look into how complicated the underlying problem really was and why the previous solution ultimately failed.
coh2 modding

Table of contents

Introduction

The problem

Original solution, part #1

New solution, part #1

The bigger problem

Original solution, part #2

New solution, part #2

Setup step

Per-ability tests

Conclusion

Introduction

CheatCommands Mod II (referred as the mod for the rest of the article) is a popular utility mod for Company of Heroes 2. It allows users to take almost full control of the game with powerful tools. These tools include ability to spawn almost any unit, building, and commander ability.


tightrope's "COH2 Cheat Commands Mod 2 Guide" provides a more in-depth introduction to the mod.


Video game mods usually have to stay up to date with the base game to maintain functionality. This is especially true for the mod because it contains pre-defined lists of units, buildings, and commander abilities. Whenever an update is released for the game, the mod may potentially reference a unit that no longer exists or miss a newly added commander ability.


Issues like the ones mentioned above may lead to expressions of disgruntlement from the mod's users, and rightfully so.

The problem

Company of Heroes 2 is a large game. It has 5 factions, 80 commanders (71 that can be considered accessible & non-duplicate), and 215 distinct commander abilities. The mod provides access to all of these commander abilities and any of them may change in any update to the game. Maintaining the list of commander abilities quickly becomes a daunting task. I've always hated repetitive tasks and making sure 215 commander abilities remain functional after every CoH2 patch certainly fits that description.

Original solution, part #1

Naturally in this situation any software developer would seek to find out if a list of commander abilities could instead be automatically generated. This way there would be some level of guarantee of not making a mistake or 2 while manually updating the list and the mod would be easier to keep updated and functional after each CoH2 patch.


I originally (several years ago) chose the official Company of Heroes 2 Modding Tools as the source of data for generating a list of commander abilities. This was an easy choice to make because the modding tools provided all the game's attribute files in standardized XML format.


Eventually the modding tools became an unreliable source of data because they are a separately managed package. Whenever the game is updated, the modding tools are usually updated. This used to be case but around when CoH2 received a 64bit update (CoH2 used to be a 32bit application), the modding tools have been seriously lagging behind. As of October 2nd, 2021 the modding tools last received a public update on April 12th, 2020. Relying on the modding tools' data as a source hasn't been feasible for over a year. What's the alternative?

New solution, part #1

Utilizing the game files directly for generating a list for commander abilities would ensure the source data is always up to date. However this is a bit more complicated because most of the game's data isn't available in human-readable format. Instead the data is in binary format, RGD. There are no official tools for extracting information from this data format but luckily most video game binary formats have been cracked/reverse engineered. Examples of individuals who have released tools for the Relic's RGD format are Corsix (author of the Corsix's Mod Studio) and Copernicus (author of the Cope's RGD Tools).


I used Cope's RGD Tools to process CoH2 RGD files. Unfortunately CoH2 RGD format had changed since the release of Cope's RGD Tools a bit. Luckily Cope implemented their tools with .NET which allowed me to decompile them with ILSpy and add support for RGD data type 4 which is a value scoped to a specific mod's GUID. It seems that more recent CoH2 updates accidentally left in references to mods used to create community-powered patches to the game. Gaining access to source code (although machine-generated) of Cope's RGD Tools also allowed me to inject a generated hash dictionary based on the modding tools data.
RGD uses hashes to shorten text-like references into just 4 bytes of data.


Now that we have a better source of data for generating a list of commander abilities, we're ready to dive into the bigger problem that lies below the surface.

The bigger problem

While either source can be used to generate a list of commander abilities, the process doesn't end there. In order for the mod to unlock a certain commander ability for a player, the mod has to know which ability to grant to the player, and which upgrade to unlock when a player selects a commander ability. It's the upgrade that finally unlocks the ability, thus making is visible and accessible. This is because most CoH2 commander abilities are already present in the player's inventory at all times. The reason why the mod also has to grant the ability is because "...already present in the player's inventory" is true per faction.


How do we figure out which ability and upgrade belong to which commander abilities? Easy! Commander abilities have direct pointers to both entries. But these fields are not mandatory. Either one of them could be missing, but not both. There are 215 commander abilities. What are the stats regarding how complete the data is?


40/215 has both ability and upgrade
18/215 has ability, but no upgrade
157/215 has upgrade, but no ability
0/215 has both ability and upgrade missing

Oof. 40/215 complete, only 175 missing.

Original solution, part #2

My original approach of resolving the missing ability<upgrade> and upgrade<ability> links was to iterate over the data and make educated guesses. Some examples of these guesses:

  • If an ability has a listed requirement of the known upgrade, it's probably linked to the commander ability
  • If there are multiple linked abilities, use some clever criteria to figure out which is the most likely match. This includes comparing the ability's name, checking its location (is it in any faction's default inventory), and other even more complicated and risky tricks.

This approach lead to an incredibly complex program that grew over the years. It was able to figure out most of the links and it was something that I could trust with some level of certainty. The output of the original solution was also adapted to support other projects, namely the official Relic Entertainment's New Commanders preview mod.

New solution, part #2

Technically the original approach simulates the game to some degree in order to figure out the missing links. This process will never be perfect until the program is the game. This rather pessimistic realization lead to a tiny, seemingly silly idea. What if we use the actual game to do the simulation?


The mod is based on SCAR. It's an embedded version of the Lua programing language. The game provides a large API to manipulate the game itself. I personally host a copy of the officially released SCARDOC on https://scardoc.coh.fi/.


The SCAR API can be used to do almost anything, including checking if a certain player can cast a certain ability on a certain target. One of these methods is Player_CanCastAbilityOnPosition. As its name suggests, it allows us to check if a player can cast an ability on a position.


Here's the silly part of the idea. What if we iterate over all commander abilities, and based on missing data we make some tests:

  • If both ability and upgrade are set on the commander ability, we blindly trust it
  • If the ability is known, make a test with every possible upgrade to see which one makes it possible to cast the ability
  • If the upgrade is known, make a test with every possible ability to see which ability becomes available after unlocking the upgrade

In addition to the tests above, the lists of ability and upgrade candidates for both test cases are heavily filtered to prevent duplicate matches:

  • If we're trying to find a matching ability, the ability has to be in one of the factions' default inventory
  • If we're trying to find a matching upgrade, the upgrade owner type has to be "player" (upgrade owner type can be one of the following: "none", "player", "self", "entity_in_squad")

This approach could be considered to be brute-force.


With the logically filtered test data sets, we can start testing each ability. However not all abilities are casted on a position. Some abilities are very specific, like the British Forces' Early warning.



Early warning ability has to be clicked on a position that is on a friendly frontline sector.


Additionally some commander abilities have tech requirements - e.g. the player has to complete certain tasks before a powerful call-in ability becomes available. In order to cater to all the possible additional requirements of the commander abilities, the process is split into 2 parts: A setup step and per-ability tests.

Setup step

The commander abilities are resolved per faction because there's at least one commander ability that is explicitly limited to the British Forces only (the British M5 Halftrack call-in).

The setup involves creating an environment where all known additional requirements are met:

  • Fog of War is disabled. Some abilities like off-map airstrikes require line of sight to be casted
  • Player has infinite resources. Most commander abilities have a resource cost which is an implicit requirement for casting the ability
  • Player has 32 command points. Highest currently in-game command point requirement of a commander ability is 14 command points, shared across most super-heavy tank call-ins like ISU-152, Elefant, Jagdtiger, etc.
  • All already existing upgrades on the player are removed to prevent false matches
  • All necessary tech buildings and upgrades are spawned and completed for the player. This includes:
    • All Ostheer battle phase upgrades and tech buildings
    • All Soviet tech buildings
    • All Oberkommando West truck upgrades
    • All US Forces officer unlocks (Lieutenant, Captain, Major)
    • All British Forces tech buildings, tech upgrades, and Anvil & Hammer specialization upgrades
  • All territory points on the map captured by the player

Once all possible requirements are filled, a list of ability test cases is created. The possible ability cast tets are:

  • Player's starting position (HQ's position)
  • Map's center position (usually passes for abilities that cannot be casted on player's HQ sector)
  • Enemy vehicle (e.g. for the Soviet "Mark target" ability)
  • List of all territory sectors on the map (e.g. for the British "Early warning" ability which requires the target to be a friendly frontline sector)
  • Garrisoned ambient building (e.g. for the Ostheer and Soviet forward HQ abilities)

Per-ability tests

Having ensured that the player has all possible additional requirements met, we can iterate over all commander abilities and resolve the missing links. As described earlier, the test executed per ability is quite simple:

  • If both ability and upgrade are set on the commander ability, we blindly trust it
  • If the ability is known, make a test with every possible upgrade to see which one makes it possible to cast the ability
  • If the upgrade is known, make a test with every possible ability to see which ability becomes available after unlocking the upgrade

The per-ability test goes through all the ability cast tests. If one of them passes, the link is considered to be resolved and a match is formed.


How accurate is this process then? As of October 2nd, 2021, the process is 99.5% accurate. Only 1 of 215 commander abilities cannot be resolved with the methods described above. It's the British Forces' Commando Glider Insertion, which is a directional ability. I've decided to manually resolve this commander ability for now. Perhaps one day I'll look into fixing it, but 1/215 isn't too bad. Certainly not a repetitive task, by definition!

Conclusion

CheatCommands Mod II is finally up to date and any future updates to CoH2 should be easy to apply to the mod as well. Hopefully this article provided interesting inside information on how complicated properly maintaining a video game mod can be.


Commenting is based on https://disqus.com/ - a commonly used 3rd party commenting service for websites and blogs. It will most certainly add cookies to your browser and track you. It will not be loaded until the user clicks the button above.