Function Review

Analyzes function performance and estimate human review requirements. Calculates sample sizes needed to achieve target accuracy with statistical confidence, finds optimal confidence thresholds within a configurable range, and estimates review effort. Supports custom threshold ranges for focused analysis (e.g., analyze only 0.7-0.9 range).

Authorization

x-api-key<token>

Authenticate using API Key in request header

In: header

Request Body

functionNamestring

Name of the function to analyze

functionVersionNum?integer

Optional function version number to analyze. If not provided, uses the latest/current version of the function.

Formatuint

Range1 <= value

evaluationVersion?string

Optional evaluation version to filter evaluations by. Must be one of the supported versions. If not provided, defaults to "0.1.0-gemini".

Default"0.1.0-gemini"

Value in"0.1.0-gemini"

marginOfError?number

Margin of error for statistical calculations

Default0.05

Formatfloat

Range0.001 <= value <= 0.1

thresholdMin?number

Minimum confidence threshold to analyze

Default0.5

Formatfloat

Range0 <= value <= 1

thresholdMax?number

Maximum confidence threshold to analyze

Default1

Formatfloat

Range0 <= value <= 1

thresholdStep?number

Step size for threshold analysis (smaller = more granular)

Default0.01

Formatfloat

Range0.001 <= value <= 0.1

confidenceLevels?array<integer>

Confidence levels for statistical analysis as integers representing percentages (e.g., [90, 95, 99] for 90%, 95%, 99%). IMPORTANT: Only integers are accepted, floats like 0.95 will be rejected.

Default[95]

confidenceMethod?string

Confidence interval calculation method (default "wald").

"wald": Normal approximation method (faster, standard)
"wilson": Wilson score interval (more robust for extreme rates)

Default"wald"

Value in"wald" | "wilson"

Response Body

curl -X POST "https://api.bem.ai/v2/functions/review" \
  -H "Content-Type: application/json" \
  -d '{
    "functionName": "invoice-transformer",
    "marginOfError": 0.05,
    "thresholdMin": 0.5,
    "thresholdMax": 1,
    "thresholdStep": 0.01
  }'

curl -X POST "https://api.bem.ai/v2/functions/review" \  -H "Content-Type: application/json" \  -d '{    "functionName": "invoice-transformer",    "marginOfError": 0.05,    "thresholdMin": 0.5,    "thresholdMax": 1,    "thresholdStep": 0.01  }'

{
  "status": "success",
  "functionName": "invoice-transformer",
  "functionVersionNum": 3,
  "estimate": {
    "totalTransformations": 1000,
    "labeledTransformations": 200,
    "unlabeledTransformations": 800,
    "missingEvaluations": 50,
    "confidenceDistribution": {
      "high": 500,
      "medium": 350,
      "low": 150
    },
    "thresholdMatrix": [
      {
        "threshold": 0.8,
        "tp": 85,
        "fp": 12,
        "fn": 15,
        "tn": 88,
        "accuracyAboveThreshold": {
          "95": {
            "ciLower": 0.8456,
            "mid": 0.875,
            "ciUpper": 0.9044,
            "currentSample": 120,
            "sampleNeeded": 30
          }
        }
      }
    ]
  },
  "metrics": {
    "fieldMetrics": [
      {
        "fieldPath": "/invoice/number",
        "metrics": {
          "accuracy": 0.95,
          "precision": 0.98,
          "recall": 0.92,
          "f1Score": 0.95,
          "tp": 92,
          "fp": 2,
          "tn": 0,
          "fn": 8
        }
      },
      {
        "fieldPath": "/invoice/amount",
        "metrics": {
          "accuracy": 0.88,
          "precision": 0.9,
          "recall": 0.85,
          "f1Score": 0.87,
          "tp": 85,
          "fp": 9,
          "tn": 0,
          "fn": 15
        }
      }
    ],
    "precisionRecallAuc": 0.8542,
    "aggregateMetrics": {
      "accuracy": 0.7407,
      "precision": 0.9524,
      "recall": 0.7692,
      "f1Score": 0.8511,
      "tp": 40,
      "fp": 2,
      "tn": 0,
      "fn": 12
    }
  }
}

{
  "message": "invalid request body: missing required field 'transformationIDs'",
  "code": 400,
  "details": {
    "field": "transformationIDs",
    "reason": "required field missing"
  }
}

{
  "message": "invalid request body: missing required field 'transformationIDs'",
  "code": 400,
  "details": {
    "field": "transformationIDs",
    "reason": "required field missing"
  }
}