Discrepancy Resolution & Dispute Routing

A freight audit that stops at the variance flag has done half a job. The validation tier can prove an invoice overbilled by $312.40 on a mis-rated lane, but that number is worthless until it becomes a claim a carrier accepts and a credit that lands in the ledger. Between the flag and the recovered dollar sits a whole pipeline stage that most programs never build deterministically — and without it, flagged variance quietly evaporates. Disputes rot in a shared spreadsheet nobody owns, short-pays get deducted but never tracked against a remittance, and by the time an analyst circles back, the carrier has auto-rejected the claim because it arrived outside the contractual filing window. The money was found; it was never collected.

This page defines the discrepancy layer: the deterministic path a flagged line takes from variance verdict to settled recovery. It reads compliance verdicts from Rule-Based Rate Validation & Accessorial Auditing and the point-in-time tariff context from Freight Contract Architecture & Rate Mapping, then classifies each variance, decides whether it is worth disputing, routes it to the right carrier channel, submits the claim idempotently, records both the claim and its outcome in a double-entry ledger, and settles the result as a recovery or a tracked short-pay. Every hop is auditable and every dollar is accounted for on both sides of the book.

The audience is the same freight-audit and Python ETL engineers who own the upstream tiers, now responsible for the recovery mechanics: claim state machines, carrier API contracts, deduction codes, and the reconciliation math that ties a disputed invoice to the credit memo that eventually clears it. Get this layer right and found variance turns into collected cash with a defensible trail; get it wrong and the audit becomes a report of money you noticed leaving rather than money you brought back.

Pipeline architecture overview

The discrepancy layer is a directed flow, not a queue of tickets. A variance verdict enters, is classified into a specific dispute-worthy type, routed to the carrier channel that matches that type, submitted as a claim, and then tracked through reconciliation until the ledger balances. Failures at any hop — a rejected claim, a carrier API timeout, a variance too small to pursue — branch to explicit terminal states rather than silently stalling.

The flow is event-driven. Each variance verdict is published as a discrete event and consumed by a dispatch worker that owns one claim through its whole lifecycle. Because a claim can sit open for weeks while a carrier investigates, state lives in a durable store keyed by a stable dispute id, not in worker memory — a worker restart resumes claims exactly where they were. The canonical data flow below maps each stage to the record it reads and the record it writes.

Variance-type coverage

Not every flagged line is disputed the same way, and treating them uniformly is the fastest route to a low win rate. A duplicate-billing claim needs proof the same movement was invoiced twice; a fuel-surcharge overcharge needs the recomputed index value; a detention charge needs the gate-in/gate-out timestamps. The classifier assigns each verdict a variance type, and the type determines the evidence package, the carrier channel, and the filing deadline. The taxonomy below is the coverage contract for the discrepancy layer; the classification engine that produces it lives in Charge Variance Classification.

Weight and zone errors arrive pre-flagged from Weight & Zone Cross-Validation, and accessorial mismatches carry a penalty weight from Accessorial Charge Scoring; the discrepancy layer does not re-derive those signals, it acts on them. A variance whose type cannot be determined with sufficient confidence is not force-fit into a bucket — it routes to manual review, because a mis-typed claim submitted to the wrong channel is worse than one a human triages.

Dispute contract and configuration

A dispute is a long-lived financial object, so its schema has to encode everything a carrier and an auditor will ask for months later: which invoice, which line, which contract version, how much is claimed, on what basis, and to which channel it was sent. The configuration that governs routing — filing windows, minimum claim amounts, channel endpoints — is versioned alongside the carrier registry so that a claim is always evaluated against the rules in force when the shipment was billed, exactly as tariffs are resolved point-in-time upstream.

The DisputeCase schema below is the object that flows through every stage. It carries the classified type and confidence, the claimed amount as an exact Decimal, the resolved contract_version_id so the claim references the same tariff the audit used, and a dispute_key derived deterministically from the shipment identity — the key that makes submission idempotent later.

# models/dispute.py
from pydantic import BaseModel, Field, field_validator
from datetime import date
from decimal import Decimal
from enum import Enum
import hashlib


class VarianceType(str, Enum):
    LINEHAUL_OVERCHARGE = "LINEHAUL_OVERCHARGE"
    DUPLICATE_BILLING = "DUPLICATE_BILLING"
    ACCESSORIAL_MISMATCH = "ACCESSORIAL_MISMATCH"
    FUEL_SURCHARGE_DRIFT = "FUEL_SURCHARGE_DRIFT"
    WEIGHT_ZONE_ERROR = "WEIGHT_ZONE_ERROR"
    REWEIGH_RECLASS = "REWEIGH_RECLASS"


class DisputeState(str, Enum):
    CLASSIFIED = "CLASSIFIED"
    ROUTED = "ROUTED"
    SUBMITTED = "SUBMITTED"
    ACKNOWLEDGED = "ACKNOWLEDGED"
    RECOVERED = "RECOVERED"
    REJECTED = "REJECTED"
    WRITTEN_OFF = "WRITTEN_OFF"


class DisputeCase(BaseModel):
    carrier_scac: str
    invoice_number: str
    pro_number: str
    contract_version_id: str
    variance_type: VarianceType
    confidence: float
    claimed_amount: Decimal
    ship_date: date
    state: DisputeState = DisputeState.CLASSIFIED
    dispute_key: str = ""

    @field_validator("claimed_amount")
    @classmethod
    def positive_claim(cls, v: Decimal) -> Decimal:
        # A zero or negative claim means the classifier fired on noise;
        # never let it reach a carrier as a real dispute.
        if v <= 0:
            raise ValueError("claimed_amount must be strictly positive")
        return v

    def with_key(self) -> "DisputeCase":
        # Deterministic identity: same movement + same variance type always
        # produces the same key, so a redelivered verdict cannot open a
        # second dispute for the same overcharge.
        raw = "|".join([
            self.carrier_scac.strip().upper(),
            self.invoice_number.strip(),
            self.pro_number.strip(),
            self.variance_type.value,
        ])
        self.dispute_key = hashlib.sha256(raw.encode()).hexdigest()
        return self

Core routing and dispatch implementation

The heart of the discrepancy layer is the dispatch loop: it takes a stream of classified DisputeCase objects and, for each, decides whether to pursue it, on which channel, and against which deadline — then advances its state. The loop is deliberately a pure decision function wrapped in an I/O shell, so the routing rules are unit-testable without a live carrier API. The most common production defect here is routing on the variance amount alone while ignoring the filing window: a $4,000 overcharge that missed the carrier’s 180-day claim deadline is uncollectable, and submitting it anyway just burns the carrier relationship and pollutes the win-rate metric.

# routing/dispatch.py
import logging
from datetime import date, timedelta
from decimal import Decimal

logger = logging.getLogger("discrepancy.router")


class DispositionRules:
    """Point-in-time routing rules resolved from the channel registry."""

    def __init__(self, registry: dict):
        self.registry = registry

    def route(self, case) -> dict:
        rules = self.registry[case.carrier_scac]

        # 1. Threshold gate: below the carrier's minimum, disputing costs
        #    more in handling than the claim is worth — write it off.
        if case.claimed_amount < Decimal(str(rules["min_claim_amount"])):
            return {"decision": "WRITE_OFF", "reason": "below_min_claim"}

        # 2. Deadline gate: a claim filed after the contractual window is
        #    auto-rejected by the carrier, so never spend a submission on it.
        window_days = rules["filing_window_days"][case.variance_type.value]
        deadline = case.ship_date + timedelta(days=window_days)
        if date.today() > deadline:
            return {"decision": "WRITE_OFF", "reason": "past_filing_window"}

        # 3. Confidence gate: low-confidence classifications go to a human
        #    rather than to the wrong channel with the wrong evidence.
        if case.confidence < rules.get("min_confidence", 0.80):
            return {"decision": "MANUAL_REVIEW", "reason": "low_confidence"}

        # 4. Channel selection is keyed by variance type — duplicates are
        #    recovered by short-pay, most rate errors by the dispute API.
        channel = rules["channel_by_type"][case.variance_type.value]
        return {"decision": "ROUTE", "channel": channel, "deadline": deadline.isoformat()}


def dispatch(cases, rules: DispositionRules, submitter, ledger):
    """Advance each classified case to its next durable state."""
    for case in cases:
        outcome = rules.route(case)
        if outcome["decision"] == "WRITE_OFF":
            ledger.record_writeoff(case, reason=outcome["reason"])
            logger.info("write_off", extra={"dispute_key": case.dispute_key,
                                            "reason": outcome["reason"]})
            continue
        if outcome["decision"] == "MANUAL_REVIEW":
            ledger.record_manual(case, reason=outcome["reason"])
            continue
        # ROUTE: hand off to the idempotent submitter for this channel.
        receipt = submitter.submit(case, channel=outcome["channel"])
        ledger.record_submission(case, receipt)

The routing rules read the resolved rate context — the contracted amount, the contract version, the fuel index in force — from the same store the validation tier used, sourced through Freight Contract Architecture & Rate Mapping, so a claim never argues a number the audit itself did not stand behind. Channel selection then hands off to the carrier client, whose integration details (auth, payload shape, webhook callbacks) are owned by Carrier Dispute API Integration.

Idempotency for dispute submission

Submission is the hop where duplication does real financial damage. A dispatch worker can crash after a carrier accepts a claim but before it records the receipt; on restart it re-processes the same verdict and, without a guard, files the same claim twice. Now the carrier has two open disputes for one overcharge, the ledger double-counts the expected recovery, and reconciliation later shows a phantom shortfall when only one credit arrives. Idempotency is therefore enforced at the submission boundary, keyed by the deterministic dispute_key computed on the case.

# submission/idempotent.py
import logging

logger = logging.getLogger("discrepancy.submit")


class IdempotentSubmitter:
    def __init__(self, store, carrier_client):
        self.store = store            # durable claim-receipt store
        self.client = carrier_client

    def submit(self, case, channel: str) -> dict:
        # Claim the key atomically. If a receipt already exists, this is a
        # retry of an already-filed claim — return the stored receipt instead
        # of filing again. The carrier's own idempotency header is a second
        # line of defence, not a replacement for this one.
        existing = self.store.get_receipt(case.dispute_key)
        if existing is not None:
            logger.info("submit_skip_duplicate",
                        extra={"dispute_key": case.dispute_key})
            return existing

        receipt = self.client.file_claim(
            channel=channel,
            idempotency_key=case.dispute_key,   # carrier dedups on this too
            payload=case.model_dump(mode="json"),
        )
        # Persist the receipt BEFORE acknowledging success so a crash after
        # the carrier accepts can never lose the external claim id.
        self.store.put_receipt(case.dispute_key, receipt)
        return receipt

The guarantee is that filing the same variance any number of times yields exactly one carrier claim and one expected-recovery posting. The webhook side of this — carriers calling back with status changes, which can also arrive twice — is handled with the same key discipline in Handling Dispute Webhook Callbacks Idempotently. The duplicate-billing case, where the invoice itself is the duplicate rather than the submission, is a classification problem solved in Classifying Overcharge vs Duplicate Billing in Python.

Observability and the dispute state machine

A dispute that cannot say where it is in its lifecycle is an unrecoverable dollar in waiting. Every case moves through an explicit state machine, and every transition emits a structured log line carrying the dispute_key, the from/to states, the claimed amount, and the reason. Because claims are long-lived, the observability model is built around aging, not just throughput: the metric that matters is how many open claims are approaching their carrier response deadline, not how many were submitted today.

# observability/state_machine.py
import logging

logger = logging.getLogger("discrepancy.state")

# Only these transitions are legal; anything else is a bug, not a state.
ALLOWED = {
    "CLASSIFIED": {"ROUTED", "WRITTEN_OFF"},
    "ROUTED": {"SUBMITTED", "WRITTEN_OFF"},
    "SUBMITTED": {"ACKNOWLEDGED", "REJECTED"},
    "ACKNOWLEDGED": {"RECOVERED", "REJECTED"},
}


def transition(case, to_state: str) -> None:
    frm = case.state.value
    if to_state not in ALLOWED.get(frm, set()):
        # Illegal transitions route to a dead-letter for inspection rather
        # than corrupting the claim's audit trail with an impossible jump.
        raise ValueError(f"illegal transition {frm} -> {to_state}")
    logger.info("dispute_transition", extra={
        "dispute_key": case.dispute_key,
        "from": frm, "to": to_state,
        "claimed_amount": str(case.claimed_amount),
    })
    case.state = case.state.__class__(to_state)

Three terminal destinations cover every failure. A variance below threshold or past its window terminates as WRITTEN_OFF with the reason attached. A claim the carrier denies terminates as REJECTED, preserving the denial code so recurring rejection patterns surface a bad classification rule rather than being retried blindly. A transient carrier-API failure — a timeout, a 503 — routes to a dead-letter queue and retries with backoff, never abandoning a legitimate claim because the endpoint blipped. Aging alerts derive from the open-claim ledger position, and the alerting policy itself is tuned through Threshold Tuning & Alerting so a spike in rejections on one carrier pages the right owner.

Performance and scaling

Discrepancy volume tracks audit volume, which is bursty around month-end and carrier batch drops. The work splits into two very different profiles: submission is I/O-bound and rate-limited by carrier APIs, while classification and reconciliation are CPU-bound over Decimal arithmetic and set operations. The scaling model runs them as separate worker pools with independent concurrency caps, so a slow carrier endpoint cannot starve the classifier, and the large batch fan-out is delegated to Async Batch Processing Workflows rather than threaded inside the dispatch loop.

As a rough benchmark, a single classification worker types and confidence-scores several thousand verdicts per second against cached rate context; submission throughput is governed entirely by the carrier’s rate limit, which is why per-carrier concurrency caps and a dead-letter drain are load-bearing rather than optional. Reconciliation is cheapest when postings are batched into one transaction per carrier remittance rather than one per line.

Failure modes and troubleshooting

The recurring incidents in this layer have stable signatures. The table maps the symptom an operator sees to its root cause and resolution.

When a claim stalls, the first diagnostic is always to replay its state-machine log: the last legal transition tells you whether it died in routing, submission, or reconciliation, which separates a carrier-side denial from an internal posting bug before anyone touches the carrier.

Discrepancy modules

The discrepancy layer is implemented across four focused areas. Each owns one slice of the flow above and hands its output to the next.

• Charge Variance Classification — The taxonomy and engine that types each flagged line into a specific variance with a confidence score and decides dispute-worthiness before anything is routed.
• Carrier Dispute API Integration — The client, auth, payload contracts, and webhook handling that submit claims to carrier dispute endpoints and ingest their status callbacks without duplication.
• Reconciliation Ledger Schema — The double-entry Postgres model that records every claim, credit, and deduction so the book balances and a disputed invoice ties to the credit that clears it.
• Short-Pay & Deduction Management — The deduction-code logic and remittance matching that turn a variance into a tracked short-pay recovered against the carrier’s next payment.

Related

• Rule-Based Rate Validation & Accessorial Auditing — the validation tier whose variance verdicts feed this layer.
• Freight Contract Architecture & Rate Mapping — the versioned rate store that supplies the contract context every claim references.
• Accessorial Charge Scoring — the penalty weights that prioritize which accessorial variances are worth disputing.
• Threshold Tuning & Alerting — turns the dispute decision and aging stream into actionable alerts.
• Async Batch Processing Workflows — the batch fan-out that scales classification and reconciliation under month-end load.

---

Up: Freight Bill Auditing & Carrier Rate Contract Automation — the freight audit pipeline home.