Run Function Regression Testing

Initiates regression testing between function versions using historical transformation data

This endpoint creates function calls to test historical data with the latest function version, allowing you to measure performance improvements or regressions against ground truth data.

How It Works

1. Selects Historical Data: Retrieves transformations with user corrections (ground truth)
2. Creates Function Calls: Generates new function calls using the latest function version
3. Returns Call IDs: Provides immediate response with call tracking
4. Async Processing: Calls execute in background using standard infrastructure

Checking Results

Use standard call endpoints to monitor progress and retrieve results:

• GET /v2/calls/{callID} - Check individual call status

Best Practices

• Start with smaller sample sizes (10-50) for initial testing
• Use onlyCorrectedData: true to ensure quality ground truth comparisons
• Monitor call completion using webhooks or polling
• Compare results manually using transformation endpoints

Data Requirements

• Function must have historical transformations with correctedJSON (user corrections)
• Baseline version must exist and have associated transformation data
• Function must be currently active and callable

curl -X POST "https://api.bem.ai/v2/functions/regression" \  -H "Content-Type: application/json" \  -d '{    "functionName": "invoice-transformer",    "baselineVersionNum": 3,    "comparisonVersionNum": 5,    "sampleSize": 100,    "onlyCorrectedData": true  }'

{
  "functionName": "invoice-transformer",
  "result": {
    "functionName": "invoice-transformer",
    "totalSamples": 50,
    "calls": [
      {
        "originalReferenceID": "invoice-123",
        "callID": "wc_2N6gH8ZKCmvb6BnFcGqhKJ98VzP"
      },
      {
        "originalReferenceID": "invoice-456",
        "callID": "wc_3M7gH9ZLDnwc7CoGdHriLK99WzQ"
      },
      {
        "originalReferenceID": "invoice-789",
        "callID": "wc_4P8hI0ZMEoxy8DpHeSsjMLA0XzR"
      }
    ]
  }
}

{
  "message": "Cannot run regression test: schema incompatible between versions 1 and 2 of function 'invoice-transformer': 3 structural difference(s) found. Differences: [properties.age.type: value changed from number to string, properties.email: field removed in comparison schema, required: required field 'phone' added]"
}

{
  "message": "Function version 10 not found for function 'invoice-transformer'"
}