FForge Outcome Delivery
SAR Multi-Crop Acreage Estimator

Auditor Objections

← Back to outcome

AUDITOR OBJECTIONS (pre-written)

Hostile-auditor objections, the evidence response, the remaining weakness, and a status. SUSTAINED = stands; ADDRESSED = evidence answers it; PARTIAL = answered with a residual weakness. Machine summary: proof/PROOF_SCORECARD.json.

O1. "You claim crop-acreage accuracy but never touched the real dataset."

  • Response: Correct and disclosed. No official MSE is claimed. All metrics are

labelled synthetic (verification-report.json#/syntheticBenchmark, proof/LIMITATIONS.md items 1, 3). Customer outcome = DISCLOSED_SEAM.

  • Remaining weakness: the headline deliverable (a leaderboard score) does not

exist yet.

  • Status: SUSTAINED (by design; status PROOF_INCOMPLETE).

O2. "The 16/16 'official' run is circular — you scored your own synthetic data."

  • Response: Agreed and flagged. Fixture is generated by

tools/synth-to-official.mjs; claim scoped to "the official path executes and emits a valid submission" (proof/CLAIM_EVIDENCE.json status DISCLOSED_SEAM; proof/LIMITATIONS.md item 4).

  • Remaining weakness: no independent dataset exercises the path.
  • Status: ADDRESSED (downgraded to path-execution only).

O3. "Synthetic skill (R² 0.59) is meaningless for the real task."

  • Response: Transfer is not claimed. Synthetic skill bounds *implementation*

correctness on area-weighted linear-power mixing under noise, not field accuracy (proof/VERIFY.md, proof/LIMITATIONS.md items 3, 5).

  • Remaining weakness: real signatures/noise may break the linear assumption.
  • Status: PARTIAL.

O4. "Numbers could be cherry-picked or non-reproducible."

  • Response: All stochastic steps are seeded (mulberry32). verify.mjs asserts

bit-stable MSE (check 4). proof/REPRODUCE.md reproduces the table. proof/EXECUTION_TRACE.json records commands/exit/stdout-sha256; proof/CHECKSUMS.json pins every file and self-verifies via tools/forge-proof-verify.mjs.

  • Remaining weakness: reproducibility is within this code+seed, not a validation

of modelling choices against reality.

  • Status: ADDRESSED.

O5. "Ridge regression is too simple for this problem."

  • Response: Physically motivated — backscatter mixes linearly in power space

weighted by cover area (README.md, src/sar.mjs power features). A residual random forest exists (useForest:true) but is disabled by default and not in the verified numbers (proof/LIMITATIONS.md item 7).

  • Remaining weakness: real non-linearities (terrain, moisture, dielectric) are

unmodelled; no real-data ablation.

  • Status: PARTIAL.

O6. "Stage A (raster→zonal) is hand-waved."

  • Response: Disclosed as not executed (proof/LIMITATIONS.md item 2). The

ingestion seam (src/ingest.mjs) is round-trip-checked (verify.mjs check 6); upstream raster math is documented, not run.

  • Remaining weakness: calibration/speckle/zonal paths untested on real GeoTIFFs.
  • Status: SUSTAINED (disclosed seam).

O7. "Marketing language inflates the result."

  • Response: Banned terms removed/downgraded (certification-report.md

§"Downgraded statements"). Status PROOF_INCOMPLETE. proof/PROOF_SCORECARD.json#/marketingScan reports the automated scan; #/unsupportedClaims targets 0.

  • Remaining weakness: wording can still over-read; numbers are the authority.
  • Status: ADDRESSED.

Net position

The implementation is reproducible and recovers crop areas on a disclosed synthetic benchmark. The customer-facing outcome (official leaderboard MSE) is not delivered and is a disclosed seam. Overall: PROOF_INCOMPLETE.