Obsidian’s 'Karma Police': How to Architect and Balance RPG Skills

When players boot up a deep role-playing game, they are making an implicit contract with the developers. By selecting a character class, allocating attribute points, and choosing specific skills, they are defining how they want to experience the virtual world. If a player decides to spend their hard-earned points on non-combat skills like Barter, Science, or Lockpicking, they expect those choices to be respected by the game's systems. Unfortunately, many RPGs fail this contract, leaving non-combat builds feeling useless in a world where combat is the only viable path forward.

In a recent appearance on the Human Can Opener podcast, Obsidian Design Director Josh Sawyer shared an inside look at how the studio tackles this exact problem. Sawyer revealed that during the development of Fallout: New Vegas and other Obsidian titles, the team employed an informal internal review loop they jokingly referred to as the "Karma Police." These design auditors were tasked with going through scripts, dialogue files, and level designs to ensure that every single skill in the game has enough "cool stuff" to do, ensuring no player is ever left out to dry.

As game developers, analyzing the mechanics behind the "Karma Police" provides a masterclass in systems design, dialogue reactivity, and the prevention of the dreaded "fundamentally bad build."

The Philosophy of Build Integrity

A core pillar of Josh Sawyer’s design philosophy is the prevention of "bad builds." While RPGs should encourage players to think critically about their character design and make meaningful choices, there is a vast difference between a build that is sub-optimal and one that is fundamentally unplayable.

In many classic RPGs, a player who focuses entirely on passive, intellectual, or mercantile skills will eventually hit a brick wall. When a boss fight begins, or a dungeon requires high combat throughput, these characters are simply unable to progress. Sawyer’s approach shifts the responsibility back to the designer: if a skill exists in the game, it is the designer's job to support it.

This philosophy requires a systematic audit of the game's content. If a player invests heavily in a niche skill, they should be rewarded with unique quest solutions, specialized dialogue, and alternative paths that bypass traditional combat encounters. This is what we call Build Integrity—the guarantee that any path a player chooses is viable and supported by the game's core architecture.

Deconstructing the "Karma Police" Audit Loop

The "Karma Police" is not a software tool or an automated linter; it is a human-centric review pipeline. Designers audit script files and dialogue trees to map the frequency and impact of skill checks.

The audit workflow operates in a continuous loop throughout development, as shown in the diagram below:

During this audit, the team tracks several key metrics:

• Check Frequency: How often does a skill check occur throughout the game?
• Impact Level: Does passing the check provide a meaningful shortcut, a unique narrative reward, or a mechanical advantage? Or is it just flavor text?
• Alternative Paths: Are there enough non-combat solutions to major quests to allow a pacifist or stealth-focused character to finish the game?

If the audit reveals that a skill like Barter or Science has only a handful of checks in the first ten hours, the "Karma Police" intervene. Designers are sent back to the script to inject new checks, add unique merchant discounts, or design terminal-hacking paths that bypass guarded checkpoints.

Case Study: Systemic Changes in Fallout: New Vegas

To understand the practical impact of this design philosophy, we can look at the mechanical changes made between Fallout 3 and Fallout: New Vegas.

One of the most notable decisions was the removal of the "Big Guns" skill. In Fallout 3, Big Guns was a separate skill category. However, this created a balancing nightmare. Heavy weapons like miniguns and missile launchers are rare in the early game. Players who invested in Big Guns at character creation had virtually no weapons to use for the first several hours, effectively leaving them "out to dry."

Obsidian solved this by folding heavy weapons into three existing categories:

1. Guns (conventional firearms, including miniguns)

2. Energy Weapons (gatling lasers and plasma casters)

3. Explosives (missile launchers and fat mans)

By distributing these weapons across common skills, players could start using smaller firearms or explosives early on and naturally transition to heavier ordnance later, without their early-game points being wasted.

Additionally, Obsidian overhauled dialogue checks. In Fallout 3, dialogue checks were chance-based percentages tied to the Charisma attribute. In New Vegas, Sawyer implemented static skill thresholds (e.g., requiring exactly 50 Barter to pass a check). This made character progression predictable and rewarded players who explicitly invested in non-combat skills.

How to Implement a Skill Audit in Your Game

For indie developers looking to build rich, choice-driven RPGs, implementing your own "Karma Police" loop is highly achievable. Here are three steps to get started:

1. Build a Check Matrix

Create a simple spreadsheet listing all skills along the Y-axis and all game quests/areas along the X-axis. Every time a designer implements a skill check, they record it in the matrix. This gives you a clear visual overview of which skills are being neglected.

2. Standardize Check Rewards

Ensure that passing a non-combat check is just as satisfying as winning a fight. If a Lockpick check opens a chest, it should contain unique loot or a shortcut that saves resources. If a Science check hacks a robot, the robot should actively fight for the player, turning a hard encounter into a tactical victory.

3. Review Narrative Reactivity

Reactivity is the key to immersion. When players use a high Barter skill to negotiate a better reward, the NPC should react to their greed or intelligence. This makes the skill feel like a living part of the character's identity, rather than just a mechanical button to press.

Conclusion: Balancing for Player Agency

Game balance is often discussed in terms of statistics—dps, armor values, and item drop rates. But Josh Sawyer’s "Karma Police" philosophy reminds us that the most important balance in an RPG is the balance of player agency.

When every skill is treated as a first-class citizen, the game world feels alive, responsive, and infinitely replayable. By auditing our designs and ensuring that every build has its moment to shine, we respect the player's time and create truly memorable role-playing experiences.