Enrich Functions

Enrich functions perform semantic search against collections to add contextual information to your data. They're useful for looking up reference data, adding context from knowledge bases, or matching against existing records.

When to Use

Use an Enrich function when you need to:

• Look up vendor information from a master list
• Match extracted data against existing records
• Add context from a knowledge base or documentation
• Find similar items in a collection
• Validate data against reference sources

Prerequisites

Before creating an Enrich function, you need:

1. A Collection with your reference data
2. Items added to the collection with searchable content

See the Collections API for creating and populating collections.

Configuration Fields

Required Fields

Optional Fields

Enrich Config

The config object holds a steps array. Each step reads a value from the input, searches a collection, and writes the result back onto the data.

Each entry in steps carries:

When sourceField uses array notation (e.g. lineItems[*].description), targetField must use the same notation (e.g. lineItems[*].matchedProduct) so each item carries its own match.

Search modes

searchMode selects how candidates are retrieved from the collection before results are returned. Every mode returns distinct items (duplicates are collapsed), best match first.

Notes:

• scoreThreshold applies to semantic and hybrid. It's a maximum dissimilarity on a 0–2 scale (lower keeps closer matches; default 0.6). For hybrid, the fusion score is mapped onto that same scale, so one threshold covers both modes. exact does keyword matching and ignores it. Note 0.6 is calibrated for cosine distance and is relatively strict for hybrid, so consider tuning it on hybrid steps.
• With includeScore, each match carries a score and a scoreType: cosineDistance for semantic or hybridScore for hybrid. Both are 0–2 dissimilarities where lower is better (hybrid's fusion score is mapped onto cosine distance's scale; 0.0 = top of both the semantic and keyword rankings).
• semantic and hybrid results are additionally re-ranked by an LLM; exact is not.

Identifying a match

Every match carries an id. What it holds depends on where the match came from, and the prefix tells you which:

Because a step has one source, every match within a given enriched field carries the same kind of id.

Use id to reference a match back to bem — notably when submitting a ground-truth re-ranking to POST /v3/events/{eventID}/enrich-feedback, which resolves candidates by id and rejects anything else. For collection matches the same value joins directly against your collection.

Two things worth knowing:

• A collection id is a durable handle: editing the item's data does not change it, so ground truth recorded against it keeps resolving. A content hash does not have this property — editing the underlying data changes the id, and labels against the old value stop resolving.
• Matches are de-duplicated by payload, so where the same data occupies several rows (the uniqueness constraint is per collection and embedding model, and a search can span collections) the oldest row is the representative.
• An endpoint step with no matchInstructions returns the raw fetched values, which carry no id at all and therefore cannot be re-ranked.

Example

Vendor Lookup

Enrich invoice data with vendor information from a master vendor list:

{
  "functionName": "vendor-enricher",
  "type": "enrich",
  "displayName": "Vendor Information Lookup",
  "config": {
    "steps": [
      {
        "sourceField": "vendorName",
        "collectionName": "vendor_master_list",
        "targetField": "vendor",
        "topK": 1,
        "searchMode": "semantic"
      }
    ]
  },
  "tags": ["vendor", "lookup"]
}

Product Matching

Match extracted product names against a product catalog:

{
  "functionName": "product-matcher",
  "type": "enrich",
  "displayName": "Product Catalog Matcher",
  "config": {
    "steps": [
      {
        "sourceField": "lineItems[*].description",
        "collectionName": "product_catalog",
        "targetField": "lineItems[*].matchedProduct",
        "topK": 1,
        "searchMode": "semantic"
      }
    ]
  },
  "tags": ["products", "matching"]
}

Workflow Pattern

Enrich functions typically follow Transform functions:

Document
    │
    ▼
┌───────────┐
│ Transform │  Extract raw data
└───────────┘
    │
    ▼
┌───────────┐
│  Enrich   │  Add context from collection
└───────────┘
    │
    ▼
Enriched Output

Example Flow

1. Transform extracts vendor name "Acme Corp" from invoice
2. Enrich searches vendor collection for "Acme Corp"
3. Output includes matched vendor ID, address, payment terms

Setting Up Collections

Before using Enrich functions, create a collection and add items to it. Each item carries a data field that can be either a string or an arbitrary JSON object — the data is embedded asynchronously, and POST /v3/collections/items returns immediately with status: "pending" and an eventID you can correlate with webhook notifications once embedding completes.

# Create the collection
curl -X POST https://api.bem.ai/v3/collections \
  -H "Content-Type: application/json" \
  -H "x-api-key: $BEM_API_KEY" \
  -d '{
    "collectionName": "vendor_master_list"
  }'

# Add items (mix of string and object payloads)
curl -X POST https://api.bem.ai/v3/collections/items \
  -H "Content-Type: application/json" \
  -H "x-api-key: $BEM_API_KEY" \
  -d '{
    "collectionName": "vendor_master_list",
    "items": [
      { "data": "Acme Corporation - primary supplier for office supplies" },
      {
        "data": {
          "vendorID": "V001",
          "name": "Acme Corporation",
          "paymentTerms": "Net 30"
        }
      }
    ]
  }'

Collection names must contain only letters, digits, underscores, and dots — each segment must start with a letter or underscore — and dot notation is meaningful: customers.premium.vip is a child of customers.premium. Texts above the embedding model's 8,192-token limit are rejected; check sizes with POST /v3/collections/token-count before bulk uploads.

Related