Create a model comparison

Compare several function versions on one dataset.

Scores a saved Golden Data Set against each entry's function version (each as an eval-score run) and reports per-entry accuracy, latency, and cost, plus lift of every entry against the baseline (the first entry). Entries may span different functions and different versions.

Runs asynchronously. The response carries a comparisonID; poll GET /v3/model-comparisons/{comparisonID} until status is complete.

curl -X POST "https://api.bem.ai/v3/model-comparisons" \  -H "Content-Type: application/json" \  -d '{    "name": "invoices v3 vs v2",    "datasetID": "gds_2a8f...",    "entries": [      {        "functionName": "invoices",        "functionVersionNum": 2,        "label": "v2"      },      {        "functionName": "invoices",        "functionVersionNum": 3,        "label": "v3"      }    ]  }'

{
  "comparisonID": "string",
  "status": "pending"
}

See also

• System overview — evaluating extraction quality