JevMade hello@JevMade.com
← Back to guides

JevMade field notes / Port and benchmark report

Laya-CoreML

A reproducible Apple Neural Engine port that separates one-question latency and energy estimates from full Snake-loop performance, with raw results and conversion-fidelity checks.

Original by mizorewwwEvaluationGitHub repositorySource reviewed

Before you dive in

What you’ll find in the original

  1. Measure prompt preparation, tokenization, synchronous inference, calibration, and formatting together for an end-to-end latency figure.
  2. Report single-question latency separately from game-frame timing and say when a target—here 10×—was not achieved.
  3. Check numerical fidelity against the source runtime and enforce the short ANE bundle's 96-token total limit.
Worth knowing

On an M3 Max, the author reports 4.98 ms p50, 5.31 ms p95, and a 2.78× whole-system energy-per-decision improvement for one FP16 question. The energy figure is an SMC-sensor estimate, and JevMade did not rerun it.