AI Optimization

AI optimization determines what content in your table your Knowledge Base can understand, search, and retrieve effectively. This configuration controls how each field is processed and how retrieval balances relevance against noise.

Use this guide to:

• Assign the right role to each field
• Understand how field roles affect search relevance
• Know when to trigger a re-sync after changing field configuration

---

1. Overview: Why AI Optimization Matters

Field roles directly impact:

• Search Relevance — AI retrieves better results when only meaningful fields are embedded
• System Efficiency — Embedding every field increases processing time for no benefit
• Content Explainability — Correctly scoped fields make AI responses easier to trace and validate

Avoid treating every field equally. The goal is to embed semantic content and use structured fields for filtering instead.

---

2. AI Setup & Configuration

Accessing the Configuration Panel

Open a table and click Configure AI Search to open the field configuration dialog.

Step-by-Step Configuration

1. Review Detected Fields All fields from your table are listed with sample values for context.

2. Assign a Role to Each Field Every field gets one of four roles (see below).

3. Save Save the configuration to apply it. Depending on table size, this can trigger an immediate or queued vectorization pass — connected tables also let you choose Incremental vs Full sync from the Data Sources & Sync panel if you need to reprocess everything.

---

3. The Four Field Roles

Unlike a simple "include or exclude" toggle, MergeBase gives every field one of four distinct roles:

Vectorize

Embedded for semantic (AI) search. Use for long-form text that carries meaning.

Examples: description, body, comments, notes

Filter

Used for exact-match structured filtering (not embedded). Use for fields you'd narrow results by.

Examples: status, category, tags, date, region

Display

Shown alongside search results but neither embedded nor filterable. Use for context fields that are useful to see but not to search or filter by.

Examples: owner, last_updated, record_id (when you still want it visible)

Ignore

Excluded entirely from AI processing.

Examples: internal timestamps, raw numeric values, system-generated IDs with no semantic meaning

---

4. Field Role Strategy

Fields to Vectorize

Focus on fields that provide semantic context or describe content users will search for.

Examples:

• Long-form text: description, body, comments
• Labels or categories with descriptive text: tags, themes
• Titles or names: product_name, report_title

Fields to mark Filter instead of Vectorize

Structured values that are better matched exactly than embedded semantically.

Examples:

• Status fields: status, stage
• Categorical tags: department, priority
• Dates: created_at, due_date

Fields to Ignore

Examples:

• System-generated identifiers: ID, UUID, record_ref
• Numeric-only values with no descriptive text: price, quantity

Best Practices

• Start with 3–5 core fields marked Vectorize
• Use the sample data preview to validate your choices
• Prioritize longer text fields for richer embeddings
• Mark exact-match fields as Filter rather than Vectorize — this improves both search relevance and filtering accuracy
• Revisit field roles after observing search behavior or user feedback

---

5. Field Selection Interface

Each field in the configuration dialog shows:

• Its current role (Vectorize / Filter / Display / Ignore)
• A sample value to show real content
• A counter for how many fields are currently set to Vectorize

Use this to spot:

• Fields that look irrelevant for search (e.g., raw IDs, "$0", "2022-01-01")
• Duplicate or alias fields that don't both need to be Vectorized
• Fields that should be Filter instead of Vectorize

---

6. How to Think About Field Roles

Use this mental model:

"If a user asks a question, can this field help answer it in words?" If yes, mark it Vectorize.

"Would I ever want to narrow results to an exact value of this field?" If yes, mark it Filter.

Example: A field like Approval Notes is useful for understanding why something was approved — mark it Vectorize. A field like Approval Status (Approved/Rejected/Pending) is better as Filter, so you can narrow to "only Approved" without polluting the semantic search.

Think of your table like a custom internal search engine: embed content-rich, explainable text, use structured fields for exact filtering, and ignore system noise.