Compare Metrics Between Function Versions

Compare metrics between two function versions.

Computes aggregate and field-level lift/regression between any two versions of a function: accuracy, precision, recall, F1, and PR-AUC. Field-level changes are returned only for fields whose lift exceeds 1% in either direction.

Supported for every function type that produces labeled transformations: extract, transform, analyze, join. Pass isRegression: true to compare only the regression dataset (rows produced by POST /v3/functions/regression) — the canonical way to judge a candidate version before promoting it.

Defaults: baselineVersionNum = currentVersionNum - 1, comparisonVersionNum = currentVersionNum.

curl -X POST "https://api.bem.ai/v3/functions/compare" \  -H "Content-Type: application/json" \  -d '{    "functionName": "invoice-extractor",    "baselineVersionNum": 2,    "comparisonVersionNum": 3,    "isRegression": true  }'

{
  "functionName": "invoice-extractor",
  "baselineVersionNum": 2,
  "comparisonVersionNum": 3,
  "baselineTransformationCount": 100,
  "comparisonTransformationCount": 150,
  "aggregateComparison": {
    "accuracy": {
      "baselineValue": 0.85,
      "comparisonValue": 0.92,
      "difference": 0.07,
      "liftPercent": 8.24
    },
    "f1Score": {
      "baselineValue": 0.85,
      "comparisonValue": 0.92,
      "difference": 0.07,
      "liftPercent": 8.24
    }
  },
  "fieldMetricsChanges": []
}

{
  "message": "string",
  "code": 0,
  "details": {}
}

{
  "message": "string",
  "code": 0,
  "details": {}
}

{
  "message": "string",
  "code": 0,
  "details": {}
}

See also

• System overview — evaluating extraction quality