Stop Letting the AI Do the Math: Scoring Players and Grounding the LLM in My Fantasy Football Assistant

In Part 1, I admitted that fantasy football isn't mostly luck, that the managers who know the players win most often, and that letting a chat assistant draft my team got me a confident pep talk and the worst draft grade in the league.

The fix I landed on was a simple rule: code does the math, the AI explains it. This post is about both halves of that rule. First, how the app turns a pile of rankings into one number I can actually compare. Second, what happened when I handed those numbers to an LLM, and the embarrassing waiver suggestion that taught me numbers alone weren't enough either.

Two Opinions, One Number

Every player in the app gets a Player Value Score (PVS). It's built from two outside ranking sites, because each one answers a different question:

• FantasyCalc says what a player is worth this season. When the app refreshes rankings, it requests values that match my league's actual scoring (PPR or not, superflex or not), so a quarterback in my one-QB league isn't priced like he would be in a superflex league.
• KeepTradeCut says what a player is worth long term, in dynasty terms. It's also the only source I have for player ages, since ESPN's API doesn't expose them.

Both are normalized to a 0 to 100 scale and stored per league. That gives every player two numbers: a redraft score and a dynasty score.

The two sites don't rank identical player pools, though. If a player only shows up on one of them, the app fills in the missing side from the one that did match, rather than treating it as zero:

estimated = (redraft is None) != (dynasty is None)
if estimated:
    # Exactly one side is missing, fill it from the other.
    redraft = dynasty if redraft is None else redraft
    dynasty = redraft if dynasty is None else dynasty

The reasoning was sound. A player who only appears on a dynasty list isn't worthless in redraft, he's just unranked there. Those scores get flagged as estimated and shown softened in the UI. Hold onto this snippet, though. It comes back later in this post, and not in a good way.

Age Matters (Especially for Running Backs)

A dynasty score alone doesn't tell you whether a player's value is about to rise or fall. Anyone who has played fantasy football for a while knows running backs fall off a cliff in their late twenties, while quarterbacks can play into their late thirties, and sometimes early forties.

So before the dynasty score gets used, it's multiplied by a position-specific age curve:

# (prime_start, prime_end, falloff_per_year_outside_prime)
AGE_CURVES = {
    "QB": (26, 33, 0.03),
    "WR": (24, 29, 0.03),
    "RB": (23, 27, 0.12),  # steep cliff
    "TE": (25, 30, 0.04),
}
DEFAULT_CURVE = (23, 28, 0.05)


def age_multiplier(position: str, age: float | None) -> float:
    if age is None:
        return 1.0
    prime_start, prime_end, falloff = AGE_CURVES.get(position, DEFAULT_CURVE)
    if prime_start <= age <= prime_end:
        return 1.0
    years_outside = (prime_start - age) if age < prime_start else (age - prime_end)
    return max(0.25, 1.0 - falloff * years_outside)

Inside a player's prime, nothing changes. Outside it, he loses a percentage of his dynasty value for every year past the edge of the window, in either direction (a 20-year-old rookie is discounted too, not just a veteran), with a floor of 25%.

The running back numbers do a lot of work here. A 32-year-old wide receiver is a little under four years past his prime and keeps about 88% of his dynasty value. A 32-year-old running back is almost six years past his prime and keeps only about 29%.

Is this a perfect model of aging? No. But it's consistent, it's visible (the app shows each player's multiplier), and it's a lot better than what I was doing before, which was not thinking about it at all.

Contender or Rebuilder?

Here's where it gets interesting. The age-adjusted dynasty score and the redraft score get blended into one PVS, but the blend depends on what my team is trying to do this season:

MODE_WEIGHTS = {
    "contender": {"redraft": 0.7, "dynasty": 0.3},
    "rebuilder": {"redraft": 0.3, "dynasty": 0.7},
    "redraft_only": {"redraft": 1.0, "dynasty": 0.0},
}

pvs = redraft_score * weights["redraft"] + dynasty_adjusted * weights["dynasty"]

A contender cares mostly about this season. A rebuilder cares mostly about the next few. A redraft league (no keepers at all) only cares about this season, so the dynasty side drops out entirely. I set the mode per team with a toggle. I play in two leagues this year, a keeper league and a redraft league, and in the keeper league I'm firmly a rebuilder.

To see why this matters, here are two running backs from my keeper league's current rankings:

To a contender, these two are close. Henry is still producing right now. To a rebuilder like me, Jeanty is worth almost twice as much.

That gap is exactly what makes some trades good for both sides. If a contender sends Jeanty to a rebuilder for Henry, each team gives up something it values less than the other team does. The app evaluates every trade by scoring each side's whole roster, before and after, under that team's own mode. A player you'd acquire is valued under your strategy, not his current owner's.

Before this, my trade evaluation process was "do I recognize this name?" Now it's a number on each side that I can argue with.

Code Finds the Moves, the AI Explains Them

With PVS in place, the app finds candidate moves itself, before an LLM is involved:

• Trade matching looks at every pair of teams for a specific shape: team A has a surplus at a position where team B is thin, and team B has a surplus at a different position where team A is thin. The most promising player combinations get run through the same evaluation the Trade Simulator uses, and anything lopsided gets filtered out.
• Waiver pairs assume my roster is full, so every pickup needs a drop. A pickup only qualifies if its PVS beats the player I'd drop, and the app precomputes which drops still leave a legal starting lineup.

Then all of that goes to the LLM as structured JSON, along with rules for what it may and may not recommend (I showed a piece of those rules in Part 1). The model's job is to rank the options and explain them in plain English.

I thought that was enough. It wasn't.

The Waiver Suggestion That Broke My Trust

Shortly after the season started, the Waiver Targets tab in my keeper league gave me this:

Add Jalen McMillan, drop Stefon Diggs

Rationale: Jalen McMillan has a higher pvs (32.5) and, being a younger player, fits well with a rebuilding strategy focused on youth and upside. Stefon Diggs, at 32.8 years old, sees his redraft and dynasty scores dwindling, making him a less valuable long-term asset despite past performance.

It reads well. Every claim in it is technically true. It follows my rules: higher PVS, fits my rebuilder mode, legal drop.

And it was a terrible suggestion. After week 2, McMillan was on injured reserve and wasn't expected back until November 22nd. He hadn't scored a single fantasy point all season.

Two things went wrong at the same time.

The numbers had a blind spot. By the last time both sites listed him, FantasyCalc had McMillan's redraft value down to 0.3 out of 100, and eventually it stopped ranking him at all. Remember that "fill the missing side" snippet? With his redraft score gone, the app filled it in from his dynasty score, and a player with essentially zero value this season suddenly looked like a solid all-around asset. A heuristic built for "this site doesn't happen to list him" did exactly the wrong thing for "this site stopped listing him because he's hurt."

(I've since fixed this. A player FantasyCalc used to rank and then dropped now gets a redraft score of 0 instead of borrowing his dynasty score, and so does any player listed on IR, out, or suspended. The fill-in still exists, but only for players a site has simply never ranked.)

The AI had no way to know. This is the part that really bothered me. The waiver prompt at the time actually told the model this:

ESPN's free-agent feed carries no age, so for the free agents themselves lean
on general knowledge of each player's situation

"Lean on general knowledge" sounds reasonable until you remember what an LLM's general knowledge is: whatever its training data said, a season or more ago. As far as the model knew, Jalen McMillan was a promising young receiver. It had no idea he was on IR. It was Plan A from Part 1 all over again, just hidden behind a nicer UI and a real number.

Grounding the AI in Real News

The fix had two parts: give the model current, dated facts, and forbid it from using anything else.

On the data side, the nightly sync now pulls ESPN's player news for every rostered player plus the top of the free-agent pool, and every player in the payload carries it alongside injury status and recent weekly points. The payload also carries an as_of block with today's date and the current NFL week, because without it the model has no idea when "now" is.

On the prompt side, I replaced "lean on general knowledge" with rules that say the opposite:

_GROUNDING_RULES = """\
Ground every claim about a player's situation in the payload only. Your own
knowledge of players, teams, depth charts, and injuries is out of date,
do not use it. `as_of` says today's date and NFL week. For each player,
rely on:
- recent_news: dated ESPN news, newest first. This is the most current
  information you have and overrides anything else you believe.
- injury_status, recent_points (last few weeks' fantasy points and season
  average), and pvs.
Rules:
- Never recommend adding a player whose recent_news is negative for his
  value (out long-term or for the season, demoted, released, suspended).
- Never recommend dropping a player whose recent_news says he is returning
  from injury or gaining a bigger role, unless you explain why the pickup
  still outweighs that.
- For each rationale, cite the specific news item (with its date) or the
  specific points/pvs numbers behind it. If a player's recent_news is empty,
  say the case rests on his value and points alone, do not invent an
  injury, role change, or storyline.
"""

A few details that turned out to matter:

• "Your knowledge is out of date" has to be said explicitly. Without it, the model happily blends payload facts with stale memories and doesn't tell you which is which.
• Citations make bad reasoning visible. Requiring a dated news item or a specific number in every rationale means I can check the claim in seconds. A rationale that can't cite anything has to say so, which is itself useful information.
• "No news" and "news wasn't checked" are different. The payload always includes the recent_news key when news was looked up, even if it's an empty list. That lets the model (and me) tell "nothing happened" apart from "we don't know."

With those rules in place, the McMillan suggestion can't happen. His injury status says IR, his recent news says he's out until late November, and his recent points are zero. The rules forbid recommending him, no matter what his PVS says.

The AI as a Starting Point, Not the Decision

Not every AI suggestion was bad. Here's one from my redraft league, where there are no keepers and every player is valued on this season alone:

C.J. Stroud (QB, HOU): With your only QB being Justin Herbert, having a backup like Stroud could be a strategic move. Stroud has surprised many with his adaptability and performance, suggesting he's ready to step up if your star QB is on a bye or faces a tough matchup. He's young, learning quickly.

Look at that rationale. "Surprised many with his adaptability." "Learning quickly." There isn't a single date or number in it. It's exactly the kind of generic, vibes-based reasoning the grounding rules are designed to eliminate.

But it pointed me at a player worth looking at, and when I actually looked, I made a different call than the AI did. It suggested Stroud as a backup. I looked at the waiver wire, saw several quarterbacks I'd take over Herbert, and dropped Herbert for Stroud outright.

Since that move, Stroud has scored 44.0 more fantasy points while Herbert has scored 19.8. I know that because the app tracked it, which is the subject of Part 3.

That's the relationship I want with the AI: it surfaces options and explains the numbers, and I make the decision. When it's grounded, it's a genuinely useful analyst. When it isn't, it's a very articulate guesser.

What I Learned

• Deterministic first. Anything that can be computed should be computed in code, where it's testable, consistent, and visible. PVS, trade matching, and lineup-legal drops are all plain Python with unit tests, not prompt instructions.
• Numbers can lie too. My own scoring had a blind spot that turned an injured player into a top waiver target. A single number never tells the whole story, which is why the news grounding matters even with good math underneath it.
• Tell the model what it doesn't know. An LLM will fill gaps with training data unless you explicitly tell it that data is stale and give it something better.
• Make it cite. If a recommendation can't point to a date or a number, it's an opinion, and I already learned on draft day what the AI's opinions are worth.

Next Up

So far I've covered why I built this and how it decides what to recommend. In Part 3, I'll get to my favorite feature: the Decision Log. It freezes a snapshot of every move I make, compares "then vs. now" in both value and real points, lets me grade myself A through F, and then has the AI tell me where my grades were wrong. It's the same "look back at the results" exercise that started this whole project, except now I get to run it on myself every week.