Weight & Zone Cross-Validation: Implementation Guide for Freight Audit Pipelines

Weight & zone cross-validation is the deterministic stage that recomputes a shipment’s billable weight and shipping zone from first principles and reconciles the resulting base charge against the contracted rate table before any payment is approved. It consumes normalized shipment records, independently resolves the expected zone and weight bracket, queries the contract rate, and emits a PASS / FLAG verdict with an exact monetary variance. This stage sits inside Rule-Based Rate Validation & Accessorial Auditing, immediately after lane resolution and before accessorial scoring, and exists to guarantee that base freight charges are arithmetically sound — using contract math only, no probabilistic models and no per-invoice human judgment — so that every downstream module evaluates surcharges against a trusted baseline.

This guide covers the data contract the stage expects, the field map that drives reconciliation, a numbered deterministic engine for zone, weight, and rate, the test patterns that keep the math honest, its failure modes under partial carrier data, and how its PASS / FLAG verdict feeds accessorial scoring and dispute routing.

Prerequisites

Cross-validation is a stage with hard input expectations, not a standalone script. Each dependency below must be satisfied upstream or the stage quarantines the record rather than guessing.

If any dependency is absent, the correct behaviour is to route the record to an explicit status (CONTRACT_MISSING, ZONE_UNRESOLVED) — never to reconcile against defaults and emit a false PASS.

Architecture Detail & Stage Boundaries

Strict stage isolation keeps execution idempotent and the audit trail traceable. The engine begins only after a record has been normalized into the canonical schema, and it ends the moment a PASS / FLAG verdict and variance are attached. It does not parse documents, validate accessorial fees, or open dispute tickets — those concerns belong to neighbouring stages.

Inbound contract: a normalized shipment record with validated types, a bound contract_id, and resolved lane keys. Outbound contract: the same record enriched with resolved_zone, billable_weight, expected_charge, variance_abs, variance_pct, and a status of PASS, RATE_VARIANCE, ZONE_UNRESOLVED, or CONTRACT_MISSING.

The mapping between an inbound field and the rule it powers is fixed, so a reviewer can trace any variance back to its inputs:

Canonical Schema & Contract Table Preparation

The validation engine requires strictly typed, normalized shipment records. Schema validation runs at the pipeline boundary using Pydantic; records that violate type constraints, omit mandatory fields, or carry malformed ZIPs are rejected to a dead-letter queue before any math executes, so the engine only ever sees well-formed input.

from pydantic import BaseModel, Field, field_validator
from typing import Optional
import re

class CanonicalShipment(BaseModel):
    shipment_id: str
    carrier_scac: str
    origin_zip: str = Field(pattern=r"^\d{5}$")
    dest_zip: str = Field(pattern=r"^\d{5}$")
    billed_weight_lbs: float
    actual_weight_lbs: Optional[float] = None
    dim_length_in: Optional[float] = None
    dim_width_in: Optional[float] = None
    dim_height_in: Optional[float] = None
    service_level: str
    billed_zone: Optional[int] = None
    billed_freight_charge: float
    contract_id: str

    @field_validator("origin_zip", "dest_zip")
    @classmethod
    def validate_zip(cls, v: str) -> str:
        if not re.match(r"^\d{5}$", v):
            raise ValueError("ZIP code must be exactly 5 digits")
        return v

Contract rate tables must be pre-loaded into a columnar, query-optimized store. DuckDB or Parquet-backed data lakes give sub-second lookups across millions of rate combinations. Each table maps weight_bracket, zone, and service_level to a base rate, alongside fuel surcharge multipliers and minimum charge floors. Store every monetary value as DECIMAL(19,4) to prevent floating-point drift during aggregation, consistent with how the Fuel Surcharge Formula Implementation persists FSC multipliers upstream.

Step-by-Step Implementation

The engine evaluates each record in a fixed sequence — resolve zone, snap weight, reconcile rate. Missing data routes to an explicit status rather than raising mid-batch, which keeps the stage running through partial carrier outages.

Stage 1 — Resolve the expected zone

Carrier zone assignments are rarely static: parcel networks rely on annual zip-to-zone grid updates, while LTL carriers use distance-based matrices and freight-class routing. The pipeline must independently derive the expected zone before comparing it to the carrier’s billed value. Resolution begins with a direct lookup against the carrier’s published zip-pair table; when that fails, the engine falls back to the centroid and state-group logic owned by Lane Matching Algorithms. The resolved zone is then validated against service-level constraints (Ground capped at Zone 8, Express at Zone 10).

import duckdb
from dataclasses import dataclass
from typing import Optional

@dataclass
class ZoneResolutionResult:
    resolved_zone: int
    resolution_method: str  # "direct", "centroid_fallback", "state_group"
    is_valid_for_service: bool

class ZoneResolver:
    def __init__(self, duckdb_conn: duckdb.DuckDBPyConnection):
        self.conn = duckdb_conn

    def resolve(self, origin_zip: str, dest_zip: str, service_level: str) -> ZoneResolutionResult:
        # Direct lookup against the carrier's published zip-pair grid
        query = """
            SELECT zone FROM carrier_zone_grid
            WHERE origin_zip = ? AND dest_zip = ?
        """
        result = self.conn.execute(query, [origin_zip, dest_zip]).fetchone()

        if result:
            return self._validate_service(result[0], service_level, "direct")

        # Fallback to centroid/state logic (delegated to lane matching module)
        fallback_zone = self._compute_fallback_zone(origin_zip, dest_zip)
        return self._validate_service(fallback_zone, service_level, "centroid_fallback")

    def _validate_service(self, zone: int, service: str, method: str) -> ZoneResolutionResult:
        service_caps = {"GROUND": 8, "EXPRESS": 10, "FREIGHT": 12}
        cap = service_caps.get(service.upper(), 12)
        return ZoneResolutionResult(
            resolved_zone=zone,
            resolution_method=method,
            is_valid_for_service=zone <= cap,
        )

    def _compute_fallback_zone(self, origin: str, dest: str) -> int:
        # In production, this delegates to the lane matching pipeline.
        # Default safe zone for demonstration only.
        return 5

Common mistake: trusting the carrier’s billed_zone as the lookup key. Always resolve the zone from origin/destination independently, then compare it to billed_zone. A carrier that bills Zone 7 on a Zone 5 lane is exactly the overcharge this stage exists to catch — keying off their number hides it.

Stage 2 — Compute billable weight and snap to a bracket

Billable weight is rarely the raw scale weight. Carriers apply a dimensional weight formula — (L × W × H) / divisor — and snap the result up to a predefined weight bracket. The pipeline must replicate this deterministically: when dimensional data is present, the higher of dimensional and scale weight becomes billable, then it is rounded up to the nearest contracted bracket. Tolerance thresholds (typically ±1.0 lb or ±2%) absorb carrier scale calibration variance; the detailed handling of scale-versus-log discrepancies lives in Cross-Checking Billable Weight Against Actual Weight Logs.

from typing import Optional, Tuple

def calculate_billable_weight(
    actual: Optional[float],
    dims: Tuple[Optional[float], Optional[float], Optional[float]],
    dim_divisor: int = 166,
    bracket_step: int = 50,
) -> int:
    if actual is None:
        raise ValueError("Actual weight is required for billable weight calculation")

    dim_weight = 0.0
    if all(d is not None for d in dims):
        l, w, h = dims
        dim_weight = (l * w * h) / dim_divisor

    raw_billable = max(actual, dim_weight)

    # Snap to contracted bracket: round up to the next bracket boundary
    snapped = int(((raw_billable - 0.01) // bracket_step) * bracket_step + bracket_step)
    return snapped

Common mistake: hardcoding the DIM divisor. 166 is standard for ground shipments (cubic inches to pounds) and air freight typically uses 139, but the legal divisor is whatever the active contract version specifies. Source it from the same tariff snapshot the rate row comes from, or a renegotiated divisor silently produces a stale billable weight on every shipment.

Stage 3 — Reconcile against the contract rate

With the resolved zone and snapped weight bracket established, the engine queries the contract rate table to derive the expected base charge, applies the fuel multiplier and minimum charge floor, and compares the result to billed_freight_charge. This step validates base freight charges only; it does not parse, score, or validate accessorial fees — those are routed to Accessorial Charge Scoring to keep concerns separate.

from decimal import Decimal, ROUND_HALF_UP

def reconcile_rate(
    conn: duckdb.DuckDBPyConnection,
    weight_bracket: int,
    zone: int,
    service_level: str,
    contract_id: str,
    billed_charge: float,
) -> dict:
    query = """
        SELECT base_rate, fuel_surcharge_pct, min_charge
        FROM contract_rates
        WHERE contract_id = ?
          AND weight_bracket = ?
          AND zone = ?
          AND service_level = ?
    """
    row = conn.execute(query, [contract_id, weight_bracket, zone, service_level]).fetchone()

    if not row:
        raise LookupError(f"No contract rate found for {contract_id} | {weight_bracket} | {zone}")

    base, fuel_pct, min_charge = row
    expected = Decimal(str(base)) * (Decimal("1.0") + Decimal(str(fuel_pct)) / Decimal("100"))
    expected = max(expected, Decimal(str(min_charge)))
    expected = expected.quantize(Decimal("0.01"), rounding=ROUND_HALF_UP)

    billed = Decimal(str(billed_charge))
    variance_abs = abs(billed - expected)
    variance_pct = (variance_abs / expected * Decimal("100")).quantize(Decimal("0.01"))

    return {
        "expected_charge": float(expected),
        "variance_abs": float(variance_abs),
        "variance_pct": float(variance_pct),
        "status": "PASS" if variance_abs <= Decimal("0.50") else "FLAG",
    }

Common mistake: comparing billed against expected as binary floats. A 0.50 tolerance evaluated in float arithmetic flips on rounding noise, so two replays of the same shipment can disagree. Coerce every value through Decimal(str(...)) and pin ROUND_HALF_UP — reproducibility is what makes the variance defensible in a carrier dispute.

Validation & Testing

Because zone, weight, and rate are pure functions, every rule is unit-testable against fixed inputs with no mocks. Build fixtures from real carrier edge cases and assert on both the verdict and the computed value, so a regression in why a charge flagged is caught alongside a regression in the number.

import duckdb
import pytest
from decimal import Decimal


@pytest.fixture()
def rate_conn() -> duckdb.DuckDBPyConnection:
    conn = duckdb.connect(":memory:")
    conn.execute(
        "CREATE TABLE contract_rates("
        "contract_id VARCHAR, weight_bracket INTEGER, zone INTEGER, "
        "service_level VARCHAR, base_rate DECIMAL(19,4), "
        "fuel_surcharge_pct DECIMAL(19,4), min_charge DECIMAL(19,4))"
    )
    conn.execute(
        "INSERT INTO contract_rates VALUES "
        "('C-100', 100, 5, 'GROUND', 84.0000, 25.0000, 95.0000)"
    )
    return conn


def test_dim_weight_beats_scale_and_snaps_up():
    # 24x24x24 / 166 = 83.1 lb DIM > 40 lb scale; snaps to the 100 bracket
    assert calculate_billable_weight(40.0, (24.0, 24.0, 24.0), bracket_step=50) == 100


def test_scale_weight_used_when_dims_absent():
    assert calculate_billable_weight(51.0, (None, None, None), bracket_step=50) == 100


def test_min_charge_floor_applied(rate_conn):
    # base 84 * 1.25 = 105.00 > 95 floor; billed matches expected
    result = reconcile_rate(rate_conn, 100, 5, "GROUND", "C-100", 105.00)
    assert result["status"] == "PASS"
    assert result["expected_charge"] == 105.00


def test_overcharge_flags_with_exact_variance(rate_conn):
    result = reconcile_rate(rate_conn, 100, 5, "GROUND", "C-100", 130.00)
    assert result["status"] == "FLAG"
    assert result["variance_abs"] == 25.00


def test_missing_rate_row_raises_for_contract_routing(rate_conn):
    with pytest.raises(LookupError):
        reconcile_rate(rate_conn, 999, 5, "GROUND", "C-100", 100.00)

Fixture design that matters in this domain: a shipment whose DIM weight crosses a bracket boundary that scale weight would not, a service_level whose cap rejects the resolved zone, a record with actual_weight_lbs=None (must raise, never default to zero), and a charge that lands exactly on the 0.50 tolerance edge. The bracket-boundary and tolerance-edge tests are not optional — they are where carrier billing logic and contract math most often diverge by a single rounding step.

Performance & Tuning

The arithmetic is CPU-light and each record is independent, so throughput is governed by rate-table lookups, not by the math.

• Columnar lookups: keep contract_rates and carrier_zone_grid in a single in-process DuckDB connection per worker. A composite index (or sort) on (contract_id, weight_bracket, zone, service_level) turns each reconciliation into a point lookup; re-opening the connection per record is the most common throughput regression.
• Batch size: validate in batches of 1,000–5,000 records per worker pull. Smaller batches add queue overhead; larger batches inflate redelivery cost when a single poison record fails the boundary.
• Decimal cost: Decimal is roughly 10x slower than float, but the handful of operations per record keeps it negligible against I/O. Never trade it for float to chase microseconds — precision is the product here.
• Memory footprint: load each carrier’s zone grid once and pin it to its contract_version_id; never re-read the grid per shipment. Keep raw source payloads out of the hot path and rehydrate them only when a record routes to dispute.

Failure Modes

Five scenarios account for nearly all production incidents in this stage. Each has a deterministic root cause and a diagnostic you can run against a captured batch.

1. Missing contract or rate row. The (contract_id, bracket, zone, service) tuple has no matching row, so reconcile_rate raises LookupError. Tag the record CONTRACT_MISSING and route it to a contract reconciliation queue with no variance computed.

missing = [s for s in batch
           if conn.execute(
               "SELECT 1 FROM contract_rates WHERE contract_id = ? LIMIT 1",
               [s.contract_id]).fetchone() is None]
logger.warning("contract_missing", extra={"count": len(missing),
               "contracts": sorted({s.contract_id for s in missing})})

Resolution: re-sync the tariff snapshot from FTL Base Rate Extraction; do not approve on a default rate.

2. Unresolvable zone. Both the direct grid lookup and the centroid fallback fail. Tag ZONE_UNRESOLVED and log origin/dest metadata for manual review — never substitute the carrier’s billed_zone.

3. DIM divisor drift. A renegotiated contract changes the divisor but the pipeline still uses 166, inflating or deflating billable weight on every shipment. Diagnose by asserting the active divisor equals the contract’s divisor before the batch runs; a stage-wide shift in billable_weight for one contract_id points here.

4. Tolerance exceeded (true variance). Schema and contract checks pass but the recomputed charge diverges beyond 0.50. Tag RATE_VARIANCE and pass the record, with expected_charge and variance_abs, to the dispute layer — this is the stage doing its job, not an error.

5. Float contamination. A billed_freight_charge arrives as a JSON float and loses precision at the tolerance boundary. Diagnose by enforcing Decimal(str(...)) coercion at ingestion and rejecting non-string numeric input at the Pydantic boundary.

All failures are logged with structured JSON carrying shipment_id, carrier_scac, failure_code, and stack_trace. Retry logic is disabled for deterministic validation failures; only transient infrastructure errors (e.g. a DuckDB connection timeout) trigger exponential backoff. Silent defaults are prohibited — a missing rate goes to a queue, never to a guessed PASS.

Integration Points

The enriched record is the contract this stage owes the rest of the audit. Its status is consumed directly downstream:

The resolved_zone and billable_weight fields are load-bearing at the boundary: weight-driven and zone-driven accessorials in the next stage reuse them rather than re-deriving, and Threshold Tuning & Alerting reads the variance_pct distribution to decide whether a pattern of flags warrants a tightened tolerance. Emit counters for records_validated, status_distribution, and zone_fallback_invocations, and track reconciliation latency percentiles (p50, p95) — a rising p95 here almost always signals a cold rate-table cache, not a math regression.

Deep-Dive Guides

Step-by-step walkthroughs that build on this stage:

• Cross-Checking Billable Weight Against Actual Weight Logs — debugging and scaling the scale-weight-versus-log reconciliation that feeds Stage 2’s billable weight.

Related

• Lane Matching Algorithms — resolves origin/destination and the contract_id this stage’s zone lookup depends on.
• Accessorial Charge Scoring — scores surcharges against the clean base charge this stage certifies.
• Threshold Tuning & Alerting — turns the variance_pct distribution into alert thresholds and tolerance adjustments.
• FTL Base Rate Extraction — produces the versioned rate rows the reconciliation step queries.
• Automated Invoice Parsing & EDI/XML Ingestion — normalizes the shipment records this stage consumes.

---

Up: Rule-Based Rate Validation & Accessorial Auditing