diff --git a/apps/docs/content/docs/integrations/elasticsearch.mdx b/apps/docs/content/docs/integrations/elasticsearch.mdx index a0fb33108fb..fc520cd52a9 100644 --- a/apps/docs/content/docs/integrations/elasticsearch.mdx +++ b/apps/docs/content/docs/integrations/elasticsearch.mdx @@ -306,7 +306,7 @@ Retrieve index information including settings, mappings, and aliases. | Parameter | Type | Description | | --------- | ---- | ----------- | -| `index` | json | Index information including aliases, mappings, and settings | +| `indices` | json | Matched indices keyed by index name, each with its aliases, mappings, and settings | ### Elasticsearch Cluster Health @@ -324,7 +324,7 @@ Get the health status of the Elasticsearch cluster. | `username` | string | No | Username for basic auth | | `password` | string | No | Password for basic auth | | `waitForStatus` | string | No | Wait until cluster reaches this status: green, yellow, or red | -| `timeout` | string | No | Timeout for the wait operation \(e.g., 30s, 1m\) | +| `clusterTimeout` | string | No | How long Elasticsearch waits for the cluster to reach the requested status, as an Elasticsearch time value \(e.g., 30s, 1m\). Not named "timeout": that name is reserved by the tool transport as a client-side abort deadline in milliseconds. | #### Output @@ -377,12 +377,13 @@ List all indices in the Elasticsearch cluster with their health, status, and sta | `apiKey` | string | No | Elasticsearch API key | | `username` | string | No | Username for basic auth | | `password` | string | No | Password for basic auth | +| `includeSystemIndices` | boolean | No | Include Elasticsearch system indices \(names starting with "."\). Omitted by default. | #### Output | Parameter | Type | Description | | --------- | ---- | ----------- | | `message` | string | Summary message about the indices | -| `indices` | json | Array of index information objects | +| `indices` | json | Array of index information objects \(index, health, status, docsCount, storeSize, primaryShards, replicaShards\). System indices are omitted unless includeSystemIndices is set. | diff --git a/apps/docs/content/docs/integrations/file.mdx b/apps/docs/content/docs/integrations/file.mdx index 4222a602a7d..ec5bf7914cd 100644 --- a/apps/docs/content/docs/integrations/file.mdx +++ b/apps/docs/content/docs/integrations/file.mdx @@ -1,6 +1,6 @@ --- title: File -description: Read, get content, fetch, write, append, compress, decompress, and manage sharing for files +description: Read, search, get content, fetch, write, append, compress, decompress, and manage sharing for files --- import { BlockInfoCard } from "@/components/ui/block-info-card" @@ -11,23 +11,24 @@ import { BlockInfoCard } from "@/components/ui/block-info-card" /> {/* MANUAL-CONTENT-START:intro */} -The File block is a built-in Sim block for working with files stored in the workspace, or fetched from external URLs. It handles reading, writing, appending, compressing, decompressing, and sharing files as part of a workflow. +The File block is a built-in Sim block for working with files stored in the workspace, or fetched from external URLs. It handles reading, searching, writing, appending, compressing, decompressing, and sharing files as part of a workflow. With the File block, you can: - **Read and extract content**: Load workspace file objects and extract their text content +- **Search workspace content**: Find literal text across indexed active workspace files with bounded line-level results - **Fetch from URLs**: Retrieve and parse files from external URLs with custom headers - **Write and append**: Create new workspace files or append content to existing ones - **Compress and decompress**: Bundle files into a .zip archive or extract an archive into the workspace - **Manage sharing**: Enable or disable a public share link for a file, with public, password, email, or SSO access modes -In Sim, the File block allows your agents to read and extract text from workspace files, fetch and parse files from URLs, write or append content to files, bundle files into or out of .zip archives, and control public sharing access for a file—all programmatically as steps in a workflow. This makes it possible to move file content into and out of a workflow, package outputs for download or transfer, and expose files to external users through a managed share link. +In Sim, the File block allows your agents to search, read, and extract text from workspace files, fetch and parse files from URLs, write or append content to files, bundle files into or out of .zip archives, and control public sharing access for a file—all programmatically as steps in a workflow. This makes it possible to explore workspace content, move file content into and out of a workflow, package outputs for download or transfer, and expose files to external users through a managed share link. {/* MANUAL-CONTENT-END */} ## Usage Instructions -Read workspace file objects, extract the text content of files, fetch and parse files from URLs with optional headers, write new workspace files, append content to existing files, compress files into a .zip archive, extract a .zip archive into the workspace, or manage the public share link for a file. +Read workspace file objects, search indexed text across all active workspace files, extract the text content of files, fetch and parse files from URLs with optional headers, write new workspace files, append content to existing files, compress files into a .zip archive, extract a .zip archive into the workspace, or manage the public share link for a file. @@ -67,6 +68,35 @@ Extract the text content of one or more workspace files from selected file objec | --------- | ---- | ----------- | | `contents` | array | Array of file text contents, one entry per file in input order | +### File Search + +Search indexed text across active workspace files using literal smart-case substring matching. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `query` | string | Yes | Literal text to find \(3-512 characters\). Uppercase Unicode letters make matching case-sensitive. | +| `maxResults` | number | No | Hard result cap configured by the workflow builder \(1-200, default 50\). | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `results` | array | Matching logical lines with their workspace file ID and 1-based line number. | +| ↳ `fileId` | string | Canonical workspace file ID. | +| ↳ `lineNumber` | number | 1-based logical line number. | +| ↳ `text` | string | Matching line or bounded match-centered preview. | +| `count` | number | Number of returned matching lines. | +| `truncated` | boolean | Whether more matching lines exist beyond the configured hard cap. | +| `complete` | boolean | Whether indexing has no pending or failed current revisions; skipped and partial coverage is reported separately. | +| `indexStatus` | object | Current workspace search-index coverage by file status. | +| ↳ `readyFiles` | number | Files whose current revision is searchable. | +| ↳ `pendingFiles` | number | Files still waiting to be indexed. | +| ↳ `failedFiles` | number | Files whose current indexing attempt failed. | +| ↳ `skippedFiles` | number | Files intentionally excluded because they are unsupported or oversized. | +| ↳ `partialFiles` | number | Searchable files whose extracted text was truncated by the parser or cap. | + ### File Fetch Fetch and parse a file from a URL with optional custom headers. @@ -87,15 +117,17 @@ Fetch and parse a file from a URL with optional custom headers. ### File Write -Create a new workspace file. If a file with the same name already exists, a numeric suffix is added (e.g., "data (1).csv"). +Create a new workspace file, either from text content or from an existing file. If a file with the same name already exists, a numeric suffix is added (e.g., "data (1).csv") unless overwrite is enabled. #### Input | Parameter | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `fileName` | string | Yes | File name \(e.g., "data.csv"\). If a file with this name exists, a numeric suffix is added automatically. | -| `content` | string | Yes | The text content to write to the file. | -| `contentType` | string | No | MIME type for new files \(e.g., "text/plain"\). Auto-detected from file extension if omitted. | +| `fileName` | string | No | File name \(e.g., "data.csv"\). Required when writing text; optional when storing a file, which keeps its own name unless this overrides it. If the name already exists, a numeric suffix is added automatically unless overwrite is enabled. | +| `content` | string | No | The text content to write to the file. Provide exactly one of content or fileInput. | +| `fileInput` | file | No | An existing file to store in the workspace, such as one produced by an earlier tool. Use this for anything that is not text — PDFs, images, audio, archives. Provide exactly one of content or fileInput. | +| `contentType` | string | No | MIME type for new files \(e.g., "text/plain"\). Auto-detected from the file extension, or taken from the stored file, if omitted. | +| `overwrite` | boolean | No | Replace the contents of an existing file at the exact target path \(folder and name\) instead of creating a suffixed copy. Creates the file when that path does not exist yet. | #### Output diff --git a/apps/docs/content/docs/tables/index.mdx b/apps/docs/content/docs/tables/index.mdx index e44ca483464..41b821d995e 100644 --- a/apps/docs/content/docs/tables/index.mdx +++ b/apps/docs/content/docs/tables/index.mdx @@ -24,6 +24,7 @@ Every column has a type, which decides how its values are stored and validated. | **Currency** | An amount in a currency you pick per column | `$1,234.56` | | **Boolean** | `true` or `false` | `true` | | **Date** | A date | `2026-03-16` | +| **Expiration** | An absolute row expiration time, stored as Unix epoch seconds (seconds since January 1, 1970 UTC) | `1773671400` | | **JSON** | An object or array | `{ "tier": "pro" }` | | **Select** | One of a fixed set of options, or several | `Pro` | @@ -31,6 +32,8 @@ Types are enforced as you enter values, so a Number column only takes numbers. A Currency column stores a plain number and renders it in the currency you choose for that column, so filters, sorts, and exports all see the amount itself. Changing a column's currency relabels it — it does not convert the amounts. +A table can have one Expiration column. Adding it enables row expiration; rows with a non-empty expiration value become eligible for deletion after that time passes. Cleanup runs periodically, so actual row removal may happen after the expiration timestamp rather than exactly at it. Deleting the Expiration column disables expiration for the table. Expiration cells use the date editor, while APIs and workflows read and write integer Unix epoch seconds. + ## Editing a table Open the **Tables** section in the sidebar and click **New table** to create one. Add columns from the column header, type into a cell to edit it, and paste rows from a spreadsheet to bulk-load. Filter and sort from the toolbar without changing the underlying data. The editor has full keyboard support; see [keyboard shortcuts](/keyboard-shortcuts). diff --git a/apps/docs/content/docs/workflows/blocks/function.mdx b/apps/docs/content/docs/workflows/blocks/function.mdx index 2b130f2ad7b..b79425b698e 100644 --- a/apps/docs/content/docs/workflows/blocks/function.mdx +++ b/apps/docs/content/docs/workflows/blocks/function.mdx @@ -102,6 +102,49 @@ Sim supplies the rendered heredoc privately while preserving the quoted delimite | --- | --- | | `` | The value your code returns (object, array, string, number, …) | | `` | Anything printed with `console.log()` or `print()` | +| `` | Files your code wrote to `/tmp/sim/outputs`, ready to attach or upload | + +## Files + +**Reading.** Reference a file's `path` and it is mounted for you: + +```python +import pandas as pd + +frame = pd.read_csv() +frame.describe().to_csv('/tmp/sim/outputs/summary.csv') +``` + +`.path` resolves to the file's location on the sandbox filesystem, so any language +can open it — pandas, ffmpeg, a CLI. It is the counterpart to `.base64`, which +inlines the contents instead and works only in JavaScript. Both appear in the +reference dropdown next to `.name` and `.size`. + +**Writing.** Anything your code writes to `/tmp/sim/outputs` comes back as +``, a list of file objects any file-accepting block takes directly — +attach them to an email, upload them to storage, or save them to the workspace with +the File block. There is nothing to turn on. + +The one exception is a call that names an explicit `outputSandboxPath`. That asks +for particular paths to be exported and answers with that export's own result, so +the output directory is not harvested alongside it — choose one or the other rather +than expecting both in the same run. + + +Referencing `.path` runs the block in the remote sandbox, since the local +JavaScript VM has no filesystem — expect the slower start of a remote run even for +plain JavaScript. Referencing the file itself (``, `.name`, +`.url`) does not, and stays local. Up to 20 files come back per run, 50MB total, +nested no more than 11 directories deep; a run that exceeds any of these fails +rather than returning part of what your code wrote. + + + +Returned files live with the execution rather than in your workspace, and a text +file containing a resolved secret value is refused rather than returned — there is +nowhere on an execution file to record that it carries one. Write such a file to a +workspace path instead, or keep the secret out of the output. + ## Language @@ -401,8 +444,8 @@ The lazy `sim.files` and `sim.values` helpers are available only in JavaScript f { question: "What languages does the Function block support?", answer: "JavaScript, Python, and Shell. JavaScript is the default. Python remains a stable saved language choice; Shell and custom Sandbox controls appear when a remote sandbox provider is enabled. Python and Shell execution require that provider." }, { question: "When does code run locally vs. in a sandbox?", answer: "JavaScript without external imports runs in a local isolated sandbox for speed. JavaScript that uses import or require, Python, and Shell run in the configured remote sandbox." }, { question: "Does JavaScript still work without E2B or Daytona?", answer: "Yes. JavaScript without import or require runs in Sim's local isolated VM and does not require a remote provider. JavaScript with external imports, Python, Shell, and custom Sandboxes require E2B or Daytona and fail explicitly when it is unavailable." }, - { question: "How do I reference outputs from other blocks inside my code?", answer: "Use angle-bracket syntax directly, like or , with no quotes around the tag — Sim replaces it with the real value before execution. For environment variables, use double curly braces: {{API_KEY}}." }, - { question: "What does the Function block return?", answer: "Two outputs: result and stdout. Use return in JavaScript, assign __sim_result__ in Python, or print an __SIM_RESULT__= marker in Shell to set result. Ordinary console, print, and command output goes to stdout." }, + { question: "How do I reference outputs from other blocks inside my code?", answer: "Use angle-bracket syntax directly, like or , with no quotes around the tag — Sim replaces it with the real value before execution. For environment variables, use double curly braces: {{API_KEY}}. To read a file, reference its path — mounts it and resolves to a location any language can open." }, + { question: "What does the Function block return?", answer: "Three outputs: result, stdout, and files. Use return in JavaScript, assign __sim_result__ in Python, or print an __SIM_RESULT__= marker in Shell to set result. Ordinary console, print, and command output goes to stdout. Anything your code writes to /tmp/sim/outputs comes back in files as a file object later blocks can accept directly." }, { question: "Can I make HTTP requests from a Function block?", answer: "Yes. fetch() is available in JavaScript with async/await. In Python, use requests or httpx. In Shell, use curl or a CLI available on the selected sandbox." }, { question: "Is there a timeout for Function block execution?", answer: "Yes, a configurable execution timeout. If your code exceeds it, the run is terminated and the block reports an error. Keep this in mind for external calls or heavy processing." }, ]} /> diff --git a/apps/docs/openapi-v2-tables.json b/apps/docs/openapi-v2-tables.json index 359c8e1d4b9..253692f499d 100644 --- a/apps/docs/openapi-v2-tables.json +++ b/apps/docs/openapi-v2-tables.json @@ -4979,7 +4979,16 @@ }, "type": { "type": "string", - "enum": ["string", "number", "currency", "boolean", "date", "json", "select"], + "enum": [ + "string", + "number", + "currency", + "boolean", + "date", + "ttl", + "json", + "select" + ], "description": "Data type of values stored in the column." }, "required": { @@ -5257,7 +5266,16 @@ }, "type": { "type": "string", - "enum": ["string", "number", "currency", "boolean", "date", "json", "select"], + "enum": [ + "string", + "number", + "currency", + "boolean", + "date", + "ttl", + "json", + "select" + ], "description": "Column data type." }, "required": { @@ -5436,7 +5454,16 @@ }, "type": { "type": "string", - "enum": ["string", "number", "currency", "boolean", "date", "json", "select"], + "enum": [ + "string", + "number", + "currency", + "boolean", + "date", + "ttl", + "json", + "select" + ], "description": "Data type of values stored in the column." }, "required": { @@ -5536,7 +5563,16 @@ }, "type": { "type": "string", - "enum": ["string", "number", "currency", "boolean", "date", "json", "select"], + "enum": [ + "string", + "number", + "currency", + "boolean", + "date", + "ttl", + "json", + "select" + ], "description": "Column data type." }, "required": { @@ -5633,7 +5669,7 @@ "type": { "description": "Replacement column data type.", "type": "string", - "enum": ["string", "number", "currency", "boolean", "date", "json", "select"] + "enum": ["string", "number", "currency", "boolean", "date", "ttl", "json", "select"] }, "required": { "description": "Whether inserts must supply a value for this column.", @@ -7397,7 +7433,16 @@ }, "type": { "type": "string", - "enum": ["string", "number", "currency", "boolean", "date", "json", "select"], + "enum": [ + "string", + "number", + "currency", + "boolean", + "date", + "ttl", + "json", + "select" + ], "description": "Data type of values stored in the column." }, "required": { @@ -7597,7 +7642,16 @@ }, "type": { "type": "string", - "enum": ["string", "number", "currency", "boolean", "date", "json", "select"], + "enum": [ + "string", + "number", + "currency", + "boolean", + "date", + "ttl", + "json", + "select" + ], "description": "Output column data type." }, "required": { @@ -7738,7 +7792,16 @@ }, "type": { "type": "string", - "enum": ["string", "number", "currency", "boolean", "date", "json", "select"], + "enum": [ + "string", + "number", + "currency", + "boolean", + "date", + "ttl", + "json", + "select" + ], "description": "Output column data type." }, "required": { @@ -7856,7 +7919,16 @@ }, "type": { "type": "string", - "enum": ["string", "number", "currency", "boolean", "date", "json", "select"], + "enum": [ + "string", + "number", + "currency", + "boolean", + "date", + "ttl", + "json", + "select" + ], "description": "Data type of values stored in the column." }, "required": { diff --git a/apps/sim/.env.example b/apps/sim/.env.example index 443ff1d2da9..1e4c16df28c 100644 --- a/apps/sim/.env.example +++ b/apps/sim/.env.example @@ -201,6 +201,7 @@ CRON_SECRET=your_cron_secret # Use `openssl rand -hex 32` to generate. Authentic # DATA_DRAINS_ENABLED= / NEXT_PUBLIC_DATA_DRAINS_ENABLED= # Export streams # FORKING_ENABLED= # Workspace forks # CREDENTIAL_GROUPS= # Enterprise managed OAuth collections +# TABLE_ROW_TTL= # Table TTL columns and expired-row cleanup # ORGANIZATIONS_ENABLED= / NEXT_PUBLIC_ORGANIZATIONS_ENABLED= # Organizations only # Instance organization (Optional). Most enterprise features read their settings from the diff --git a/apps/sim/app/(landing)/integrations/data/seo-content.ts b/apps/sim/app/(landing)/integrations/data/seo-content.ts index e0b49d84bc3..4383ff13beb 100644 --- a/apps/sim/app/(landing)/integrations/data/seo-content.ts +++ b/apps/sim/app/(landing)/integrations/data/seo-content.ts @@ -58,15 +58,17 @@ export const INTEGRATION_SEO: Record = { 'slack workflow automation', 'slack integration', ], - h1: 'Slack Integrations for Workflow Automation', + h1: 'Slack Workflow Automation with Sim', tagline: 'Build Slack workflow automation in Sim. Send, update, delete, and read messages; manage channels, users, canvases, and modals; and trigger AI agents from mentions, messages, and reactions in real time.', overview: - 'Use Sim as your Slack integration for team communication and operations. Build Slack automation that routes requests, posts alerts, summarises threads, updates tickets, and keeps work moving. Sim supports messages, reactions, canvases, views, channel and user lookups, file downloads, and real-time Slack workflows in one workspace.', + 'Sim automates Slack workflows that depend on conversation context, including message routing, alerts, thread summaries, ticket updates, and incident response. Slack messages and events start agent workflows that interpret what was said and choose the next action in Slack or a connected tool, so routine coordination and time-sensitive operations keep moving without anyone relaying details by hand.', triggersIntro: - 'Connect the Slack Webhook trigger to Sim and run Slack workflow automation the moment a mention, message, or reaction happens, no polling, no delay.', + 'Sim supports one real-time Slack trigger. Select the Slack events you care about, such as mentions, messages, and reactions, and Sim starts the connected workflow the moment one arrives instead of waiting for a scheduled check. A monitoring alert posted in Slack can open an incident-response workflow, and a ticketing update posted in Slack can be summarised and passed to another connected tool.', templatesIntro: - 'Ready-to-use Slack automation templates for Q&A bots, sales alerts, incident response, standups, digests, and CRM updates. Click any template to launch a workflow faster.', + 'Pre-built agent templates turn common Slack workflows into editable starting points: routing templates classify messages and send them to the right channel or owner, summarisation templates condense long threads into updates that preserve decisions and action items, and ticket sync and incident response templates update connected records and coordinate follow-up. Every template is editable, so you can adapt its channels, routing rules, data sources, and approval requirements.', + toolsSubtitleSuffix: + ' across messaging, channels, threads, users, reactions and files, and canvases and views. Combine multiple Slack actions in one workflow to summarise a message, route it, update a ticket, and post the ticket update back in Slack', }, airtable: { title: 'Airtable Automation with Sim', diff --git a/apps/sim/app/api/cron/cleanup-table-row-ttl/route.test.ts b/apps/sim/app/api/cron/cleanup-table-row-ttl/route.test.ts new file mode 100644 index 00000000000..9e62e4eb1ce --- /dev/null +++ b/apps/sim/app/api/cron/cleanup-table-row-ttl/route.test.ts @@ -0,0 +1,132 @@ +/** + * @vitest-environment node + */ +import { createMockRequest } from '@sim/testing' +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' + +const { mockEnqueue, mockGetJobQueue, mockIsTableRowTtlEnabled, mockVerifyCronAuth } = vi.hoisted( + () => ({ + mockEnqueue: vi.fn(), + mockGetJobQueue: vi.fn(), + mockIsTableRowTtlEnabled: vi.fn(), + mockVerifyCronAuth: vi.fn(), + }) +) + +vi.mock('@/lib/auth/internal', () => ({ verifyCronAuth: mockVerifyCronAuth })) +vi.mock('@/lib/core/async-jobs', () => ({ getJobQueue: mockGetJobQueue })) +vi.mock('@/lib/table/ttl-availability', () => ({ + isTableRowTtlEnabled: mockIsTableRowTtlEnabled, +})) + +import { GET } from '@/app/api/cron/cleanup-table-row-ttl/route' + +describe('table row TTL cleanup route', () => { + beforeEach(() => { + vi.clearAllMocks() + vi.useFakeTimers() + vi.setSystemTime(new Date('2026-08-22T17:01:00Z')) + mockVerifyCronAuth.mockReturnValue(null) + mockIsTableRowTtlEnabled.mockResolvedValue(true) + mockEnqueue.mockResolvedValue('job-ttl-1') + mockGetJobQueue.mockResolvedValue({ enqueue: mockEnqueue }) + }) + + afterEach(() => { + vi.useRealTimers() + }) + + it('enqueues one serialized cleanup job', async () => { + const response = await GET( + createMockRequest( + 'GET', + undefined, + {}, + 'http://localhost:3000/api/cron/cleanup-table-row-ttl' + ) + ) + + expect(response.status).toBe(200) + await expect(response.json()).resolves.toEqual({ triggered: true, jobId: 'job-ttl-1' }) + expect(mockEnqueue).toHaveBeenCalledWith( + 'cleanup-table-row-ttl', + {}, + expect.objectContaining({ + maxAttempts: 1, + jobId: 'cleanup-table-row-ttl:1986020', + concurrencyKey: 'cleanup:table-row-ttl', + concurrencyLimit: 1, + runner: expect.any(Function), + }) + ) + }) + + it('deduplicates retries within the same fifteen-minute schedule window', async () => { + const request = () => + createMockRequest( + 'GET', + undefined, + {}, + 'http://localhost:3000/api/cron/cleanup-table-row-ttl' + ) + + await GET(request()) + vi.advanceTimersByTime(13 * 60 * 1000) + await GET(request()) + + expect(mockEnqueue.mock.calls[0]?.[2]?.jobId).toBe(mockEnqueue.mock.calls[1]?.[2]?.jobId) + }) + + it('uses a new id immediately after the next fifteen-minute window begins', async () => { + const request = () => + createMockRequest( + 'GET', + undefined, + {}, + 'http://localhost:3000/api/cron/cleanup-table-row-ttl' + ) + + vi.setSystemTime(new Date('2026-08-22T17:14:59.999Z')) + await GET(request()) + vi.setSystemTime(new Date('2026-08-22T17:15:00.000Z')) + await GET(request()) + + expect(mockEnqueue.mock.calls[0]?.[2]?.jobId).not.toBe(mockEnqueue.mock.calls[1]?.[2]?.jobId) + }) + + it('returns the cron auth refusal without touching the queue', async () => { + mockVerifyCronAuth.mockReturnValue(new Response(null, { status: 401 })) + + const response = await GET( + createMockRequest( + 'GET', + undefined, + {}, + 'http://localhost:3000/api/cron/cleanup-table-row-ttl' + ) + ) + + expect(response.status).toBe(401) + expect(mockGetJobQueue).not.toHaveBeenCalled() + }) + + it('does not enqueue cleanup while the feature is disabled', async () => { + mockIsTableRowTtlEnabled.mockResolvedValue(false) + + const response = await GET( + createMockRequest( + 'GET', + undefined, + {}, + 'http://localhost:3000/api/cron/cleanup-table-row-ttl' + ) + ) + + expect(response.status).toBe(200) + await expect(response.json()).resolves.toEqual({ + triggered: false, + reason: 'feature-disabled', + }) + expect(mockGetJobQueue).not.toHaveBeenCalled() + }) +}) diff --git a/apps/sim/app/api/cron/cleanup-table-row-ttl/route.ts b/apps/sim/app/api/cron/cleanup-table-row-ttl/route.ts new file mode 100644 index 00000000000..a7bdab822e5 --- /dev/null +++ b/apps/sim/app/api/cron/cleanup-table-row-ttl/route.ts @@ -0,0 +1,47 @@ +import { createLogger } from '@sim/logger' +import { type NextRequest, NextResponse } from 'next/server' +import { verifyCronAuth } from '@/lib/auth/internal' +import { getJobQueue } from '@/lib/core/async-jobs' +import { withRouteHandler } from '@/lib/core/utils/with-route-handler' +import { isTableRowTtlEnabled } from '@/lib/table/ttl-availability' + +export const dynamic = 'force-dynamic' + +const logger = createLogger('CleanupTableRowTtlApi') +const TTL_CLEANUP_INTERVAL_MS = 15 * 60 * 1000 + +export const GET = withRouteHandler(async (request: NextRequest) => { + try { + const authError = verifyCronAuth(request, 'table row TTL cleanup') + if (authError) return authError + + if (!(await isTableRowTtlEnabled())) { + logger.info('Table row TTL cleanup skipped because the feature is disabled') + return NextResponse.json({ triggered: false, reason: 'feature-disabled' }) + } + + const queue = await getJobQueue() + const scheduleWindow = Math.floor(Date.now() / TTL_CLEANUP_INTERVAL_MS) + const jobId = await queue.enqueue( + 'cleanup-table-row-ttl', + {}, + { + maxAttempts: 1, + jobId: `cleanup-table-row-ttl:${scheduleWindow}`, + name: 'Table row TTL cleanup', + concurrencyKey: 'cleanup:table-row-ttl', + concurrencyLimit: 1, + runner: async (_payload, signal) => { + const { runCleanupTableRowTtl } = await import('@/background/cleanup-table-row-ttl') + return runCleanupTableRowTtl(signal) + }, + } + ) + + logger.info('Table row TTL cleanup dispatched', { jobId }) + return NextResponse.json({ triggered: true, jobId }) + } catch (error) { + logger.error('Failed to dispatch table row TTL cleanup', { error }) + return NextResponse.json({ error: 'Failed to dispatch table row TTL cleanup' }, { status: 500 }) + } +}) diff --git a/apps/sim/app/api/cron/workspace-file-search-dispatch/route.test.ts b/apps/sim/app/api/cron/workspace-file-search-dispatch/route.test.ts new file mode 100644 index 00000000000..1940eec0d84 --- /dev/null +++ b/apps/sim/app/api/cron/workspace-file-search-dispatch/route.test.ts @@ -0,0 +1,68 @@ +/** + * @vitest-environment node + */ +import { createMockRequest } from '@sim/testing' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const mocks = vi.hoisted(() => ({ + enqueueDispatch: vi.fn(), + verifyCronAuth: vi.fn(), +})) + +vi.mock('@/lib/auth/internal', () => ({ verifyCronAuth: mocks.verifyCronAuth })) +vi.mock('@/lib/workspace-files/search/enqueue-dispatch', () => ({ + enqueueWorkspaceFileSearchDispatch: mocks.enqueueDispatch, +})) + +import { GET } from '@/app/api/cron/workspace-file-search-dispatch/route' + +function request() { + return createMockRequest( + 'GET', + undefined, + {}, + 'http://localhost:3000/api/cron/workspace-file-search-dispatch' + ) +} + +describe('workspace file search dispatch route', () => { + beforeEach(() => { + vi.clearAllMocks() + mocks.verifyCronAuth.mockReturnValue(null) + }) + + it('returns as soon as Trigger.dev accepts the dispatcher run', async () => { + mocks.enqueueDispatch.mockResolvedValue({ backend: 'trigger-dev', jobId: 'run-1' }) + + const response = await GET(request()) + + expect(response.status).toBe(202) + await expect(response.json()).resolves.toEqual({ + success: true, + triggered: true, + backend: 'trigger-dev', + jobId: 'run-1', + }) + }) + + it('returns the cron auth refusal without dispatching', async () => { + mocks.verifyCronAuth.mockReturnValue(new Response(null, { status: 401 })) + + const response = await GET(request()) + + expect(response.status).toBe(401) + expect(mocks.enqueueDispatch).not.toHaveBeenCalled() + }) + + it('fails closed when Trigger.dev does not accept the dispatcher run', async () => { + mocks.enqueueDispatch.mockRejectedValue(new Error('trigger unavailable')) + + const response = await GET(request()) + + expect(response.status).toBe(500) + await expect(response.json()).resolves.toEqual({ + success: false, + error: 'Dispatcher enqueue failed', + }) + }) +}) diff --git a/apps/sim/app/api/cron/workspace-file-search-dispatch/route.ts b/apps/sim/app/api/cron/workspace-file-search-dispatch/route.ts new file mode 100644 index 00000000000..2ab25d6ba2f --- /dev/null +++ b/apps/sim/app/api/cron/workspace-file-search-dispatch/route.ts @@ -0,0 +1,30 @@ +import { createLogger } from '@sim/logger' +import { toError } from '@sim/utils/errors' +import { type NextRequest, NextResponse } from 'next/server' +import { verifyCronAuth } from '@/lib/auth/internal' +import { withRouteHandler } from '@/lib/core/utils/with-route-handler' +import { enqueueWorkspaceFileSearchDispatch } from '@/lib/workspace-files/search/enqueue-dispatch' + +const logger = createLogger('WorkspaceFileSearchDispatchRoute') + +export const dynamic = 'force-dynamic' +export const maxDuration = 60 + +export const GET = withRouteHandler(async (request: NextRequest) => { + const authError = verifyCronAuth(request, 'Workspace file search dispatcher') + if (authError) return authError + + try { + const result = await enqueueWorkspaceFileSearchDispatch() + logger.info('Workspace file search dispatcher accepted', result) + return NextResponse.json({ success: true, triggered: true, ...result }, { status: 202 }) + } catch (error) { + logger.error('Workspace file search dispatcher enqueue failed', { + error: toError(error).message, + }) + return NextResponse.json( + { success: false, error: 'Dispatcher enqueue failed' }, + { status: 500 } + ) + } +}) diff --git a/apps/sim/app/credential-groups/enroll/[token]/page.tsx b/apps/sim/app/credential-groups/enroll/[token]/page.tsx index fcafbdb8af5..3336b84ad1b 100644 --- a/apps/sim/app/credential-groups/enroll/[token]/page.tsx +++ b/apps/sim/app/credential-groups/enroll/[token]/page.tsx @@ -134,8 +134,15 @@ export default async function CredentialGroupEnrollmentPage({ Connect your accounts

- {enrollment.inviterName}{' '} - invited you to connect accounts for{' '} + {enrollment.inviterName ? ( + <> + {enrollment.inviterName}{' '} + invited you + + ) : ( + 'You have been invited' + )}{' '} + to connect accounts for{' '} {enrollment.workspaceName}.

diff --git a/apps/sim/app/workspace/[workspaceId]/layout.tsx b/apps/sim/app/workspace/[workspaceId]/layout.tsx index 1cf2246b119..01d1c56062a 100644 --- a/apps/sim/app/workspace/[workspaceId]/layout.tsx +++ b/apps/sim/app/workspace/[workspaceId]/layout.tsx @@ -3,6 +3,7 @@ import { cookies } from 'next/headers' import { redirect } from 'next/navigation' import { getSession } from '@/lib/auth' import { getActiveOrganizationId } from '@/lib/auth/session-response' +import { isTableRowTtlEnabled } from '@/lib/table/ttl-availability' import { getQueryClient } from '@/app/_shell/providers/get-query-client' import { ImpersonationBanner } from '@/app/workspace/[workspaceId]/components/impersonation-banner' import { SessionExpired } from '@/app/workspace/[workspaceId]/components/session-expired' @@ -15,6 +16,7 @@ import { import { BlockVisibilityLoader } from '@/app/workspace/[workspaceId]/providers/block-visibility-loader' import { CustomBlocksLoader } from '@/app/workspace/[workspaceId]/providers/custom-blocks-loader' import { DesktopOAuthConnectListener } from '@/app/workspace/[workspaceId]/providers/desktop-oauth-connect-listener' +import { FeatureFlagsProvider } from '@/app/workspace/[workspaceId]/providers/feature-flags-provider' import { GlobalCommandsProvider } from '@/app/workspace/[workspaceId]/providers/global-commands-provider' import { ProviderModelsLoader } from '@/app/workspace/[workspaceId]/providers/provider-models-loader' import { SettingsLoader } from '@/app/workspace/[workspaceId]/providers/settings-loader' @@ -44,7 +46,7 @@ export default async function WorkspaceLayout({ } const activeOrganizationId = getActiveOrganizationId(session) - const [cookieStore, initialOrgSettings] = await Promise.all([ + const [cookieStore, initialOrgSettings, , tableRowTtlEnabled] = await Promise.all([ cookies(), hostContext.hostOrganizationId ? getOrgWhitelabelSettings(hostContext.hostOrganizationId) @@ -56,36 +58,39 @@ export default async function WorkspaceLayout({ hostContext, activeOrganizationId ), + isTableRowTtlEnabled(), ]) const initialSidebarCollapsed = cookieStore.get('sidebar_collapsed')?.value === '1' return ( - - - - - - - - -
- - - - - - {children} - - -
- - - + + + + + + + + + +
+ + + + + + {children} + + +
+ + + + ) } diff --git a/apps/sim/app/workspace/[workspaceId]/providers/feature-flags-provider.tsx b/apps/sim/app/workspace/[workspaceId]/providers/feature-flags-provider.tsx new file mode 100644 index 00000000000..631b6ed2094 --- /dev/null +++ b/apps/sim/app/workspace/[workspaceId]/providers/feature-flags-provider.tsx @@ -0,0 +1,26 @@ +'use client' + +import { createContext, type ReactNode, useContext } from 'react' + +export interface WorkspaceFeatureFlags { + 'table-row-ttl': boolean +} + +const FeatureFlagsContext = createContext(null) + +interface FeatureFlagsProviderProps { + children: ReactNode + flags: WorkspaceFeatureFlags +} + +/** Makes server-resolved runtime flags available to workspace client surfaces. */ +export function FeatureFlagsProvider({ children, flags }: FeatureFlagsProviderProps) { + return {children} +} + +/** Reads one server-resolved runtime flag without exposing AppConfig to the browser. */ +export function useFeatureFlag(name: keyof WorkspaceFeatureFlags): boolean { + const flags = useContext(FeatureFlagsContext) + if (!flags) throw new Error('useFeatureFlag must be used within FeatureFlagsProvider') + return flags[name] +} diff --git a/apps/sim/app/workspace/[workspaceId]/settings/components/general/general.tsx b/apps/sim/app/workspace/[workspaceId]/settings/components/general/general.tsx index bb6758aec4d..faa87c11096 100644 --- a/apps/sim/app/workspace/[workspaceId]/settings/components/general/general.tsx +++ b/apps/sim/app/workspace/[workspaceId]/settings/components/general/general.tsx @@ -33,6 +33,10 @@ import { generalViewParam, generalViewUrlKeys, } from '@/app/workspace/[workspaceId]/settings/components/general/search-params' +import { + getTimezonePickerPresentation, + timezonePreferenceFromPickerValue, +} from '@/app/workspace/[workspaceId]/settings/components/general/timezone-picker' import type { SettingsAction } from '@/app/workspace/[workspaceId]/settings/components/settings-header/settings-header' import { SettingsPanel } from '@/app/workspace/[workspaceId]/settings/components/settings-panel' import { SettingsSection } from '@/app/workspace/[workspaceId]/settings/components/settings-section/settings-section' @@ -221,7 +225,12 @@ export function General() { } const handleTimezoneChange = async (value: string) => { - await updateSetting.mutateAsync({ key: 'timezone', value }) + const timezone = timezonePreferenceFromPickerValue(value) + if (timezone === undefined) return + await updateSetting.mutateAsync({ + key: 'timezone', + value: timezone, + }) } const handleAutoConnectChange = async (checked: boolean) => { @@ -288,6 +297,14 @@ export function General() { return } + const browserTimezone = getBrowserTimezone() + const savedTimezone = settings?.timezone ?? null + const timezonePicker = getTimezonePickerPresentation( + savedTimezone, + browserTimezone, + TIMEZONE_OPTIONS + ) + return ( <> @@ -433,10 +450,10 @@ export function General() { dropdownWidth={240} searchable searchPlaceholder='Search timezones' - value={settings?.timezone ?? getBrowserTimezone()} + value={timezonePicker.value} onChange={handleTimezoneChange} placeholder='Select timezone' - options={TIMEZONE_OPTIONS} + options={timezonePicker.options} /> diff --git a/apps/sim/app/workspace/[workspaceId]/settings/components/general/timezone-picker.test.ts b/apps/sim/app/workspace/[workspaceId]/settings/components/general/timezone-picker.test.ts new file mode 100644 index 00000000000..5c07c49956e --- /dev/null +++ b/apps/sim/app/workspace/[workspaceId]/settings/components/general/timezone-picker.test.ts @@ -0,0 +1,63 @@ +/** + * @vitest-environment node + */ +import { describe, expect, it } from 'vitest' +import { + AUTO_TIMEZONE_OPTION_VALUE, + getTimezonePickerPresentation, + INVALID_TIMEZONE_OPTION_VALUE, + timezonePreferenceFromPickerValue, +} from '@/app/workspace/[workspaceId]/settings/components/general/timezone-picker' + +const timezoneOptions = [{ label: 'Los Angeles (GMT-07:00)', value: 'America/Los_Angeles' }] + +describe('getTimezonePickerPresentation', () => { + it('shows an unset preference as an explicit browser-managed option', () => { + expect(getTimezonePickerPresentation(null, 'America/Los_Angeles', timezoneOptions)).toEqual({ + value: AUTO_TIMEZONE_OPTION_VALUE, + options: [ + { label: 'Auto: Los Angeles (GMT-07:00)', value: AUTO_TIMEZONE_OPTION_VALUE }, + ...timezoneOptions, + ], + }) + }) + + it('keeps a valid saved timezone selected independently of Auto', () => { + expect( + getTimezonePickerPresentation('America/Los_Angeles', 'America/Los_Angeles', timezoneOptions) + .value + ).toBe('America/Los_Angeles') + }) + + it('adds a valid saved timezone that is absent from the curated options', () => { + expect(getTimezonePickerPresentation('Etc/GMT+5', 'UTC', timezoneOptions)).toEqual({ + value: 'Etc/GMT+5', + options: [ + { label: 'Auto: UTC', value: AUTO_TIMEZONE_OPTION_VALUE }, + { label: 'Etc/GMT+5', value: 'Etc/GMT+5' }, + ...timezoneOptions, + ], + }) + }) + + it('surfaces an invalid saved timezone without making it selectable', () => { + expect(getTimezonePickerPresentation('Mars/Olympus', 'UTC', timezoneOptions)).toEqual({ + value: INVALID_TIMEZONE_OPTION_VALUE, + options: [ + { label: 'Auto: UTC', value: AUTO_TIMEZONE_OPTION_VALUE }, + { + label: 'Invalid: Mars/Olympus', + value: INVALID_TIMEZONE_OPTION_VALUE, + disabled: true, + }, + ...timezoneOptions, + ], + }) + }) + + it('persists Auto as an unset preference', () => { + expect(timezonePreferenceFromPickerValue(AUTO_TIMEZONE_OPTION_VALUE)).toBeNull() + expect(timezonePreferenceFromPickerValue('Asia/Tokyo')).toBe('Asia/Tokyo') + expect(timezonePreferenceFromPickerValue(INVALID_TIMEZONE_OPTION_VALUE)).toBeUndefined() + }) +}) diff --git a/apps/sim/app/workspace/[workspaceId]/settings/components/general/timezone-picker.ts b/apps/sim/app/workspace/[workspaceId]/settings/components/general/timezone-picker.ts new file mode 100644 index 00000000000..98ba10acb6f --- /dev/null +++ b/apps/sim/app/workspace/[workspaceId]/settings/components/general/timezone-picker.ts @@ -0,0 +1,62 @@ +import type { ComboboxOption } from '@sim/emcn' +import { isValidTimezone, sanitizeTimezoneForDisplay } from '@/lib/core/utils/timezone' + +export const AUTO_TIMEZONE_OPTION_VALUE = '__auto_timezone__' +export const INVALID_TIMEZONE_OPTION_VALUE = '__invalid_timezone__' + +interface TimezonePickerPresentation { + value: string + options: ComboboxOption[] +} + +/** Builds the picker state without making an unset browser fallback look persisted. */ +export function getTimezonePickerPresentation( + savedTimezone: string | null, + browserTimezone: string, + timezoneOptions: readonly ComboboxOption[] +): TimezonePickerPresentation { + const hasInvalidTimezone = savedTimezone !== null && !isValidTimezone(savedTimezone) + const unlistedTimezone = + savedTimezone !== null && + !hasInvalidTimezone && + !timezoneOptions.some((option) => option.value === savedTimezone) + ? savedTimezone + : null + const safeInvalidTimezone = + savedTimezone === null ? '' : sanitizeTimezoneForDisplay(savedTimezone) + const browserTimezoneLabel = + timezoneOptions.find((option) => option.value === browserTimezone)?.label ?? + sanitizeTimezoneForDisplay(browserTimezone) + + return { + value: hasInvalidTimezone + ? INVALID_TIMEZONE_OPTION_VALUE + : (savedTimezone ?? AUTO_TIMEZONE_OPTION_VALUE), + options: [ + { label: `Auto: ${browserTimezoneLabel}`, value: AUTO_TIMEZONE_OPTION_VALUE }, + ...(hasInvalidTimezone + ? [ + { + label: `Invalid: ${safeInvalidTimezone || '(empty)'}`, + value: INVALID_TIMEZONE_OPTION_VALUE, + disabled: true, + }, + ] + : []), + ...(unlistedTimezone + ? [ + { + label: sanitizeTimezoneForDisplay(unlistedTimezone), + value: unlistedTimezone, + }, + ] + : []), + ...timezoneOptions, + ], + } +} + +export function timezonePreferenceFromPickerValue(value: string): string | null | undefined { + if (value === INVALID_TIMEZONE_OPTION_VALUE) return undefined + return value === AUTO_TIMEZONE_OPTION_VALUE ? null : value +} diff --git a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/column-config-sidebar/column-config-sidebar.tsx b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/column-config-sidebar/column-config-sidebar.tsx index ffcef8ce465..303742fdd2b 100644 --- a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/column-config-sidebar/column-config-sidebar.tsx +++ b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/column-config-sidebar/column-config-sidebar.tsx @@ -17,7 +17,7 @@ import { } from '@/app/workspace/[workspaceId]/tables/[tableId]/components/sidebar-fields' import { useAddTableColumn, useUpdateColumn } from '@/hooks/queries/tables' import { SelectOptionsEditor } from '../select-field' -import { PLAIN_COLUMN_TYPE_OPTIONS } from './column-types' +import { columnTypeOptionsForTable } from './column-types' /** Whether a column type carries an option set. */ function isSelectType(type: ColumnDefinition['type']): boolean { @@ -52,6 +52,8 @@ interface ColumnConfigSidebarProps { onClose: () => void /** Existing column record for `mode: 'edit'`; ignored otherwise. */ existingColumn: ColumnDefinition | null + allColumns: readonly ColumnDefinition[] + tableRowTtlEnabled: boolean workspaceId: string tableId: string /** Notify parent of a rename so it can rewrite local `columnOrder` / @@ -102,6 +104,8 @@ function ColumnConfigBody({ config, onClose, existingColumn, + allColumns, + tableRowTtlEnabled, workspaceId, tableId, onColumnRename, @@ -274,11 +278,16 @@ function ColumnConfigBody({
Type ({ - label: o.label, - value: o.type, - icon: o.icon, - }))} + options={columnTypeOptionsForTable(allColumns, existingColumn, { + tableRowTtlEnabled, + }) + .filter((option) => option.type !== 'workflow') + .map((option) => ({ + label: option.label, + value: option.type, + icon: option.icon, + disabled: option.disabledReason !== undefined, + }))} value={typeInput} onChange={(v) => setTypeInput(v as ColumnDefinition['type'])} placeholder='Select type' diff --git a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/column-config-sidebar/column-type-limits.test.ts b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/column-config-sidebar/column-type-limits.test.ts new file mode 100644 index 00000000000..f4325de8ae2 --- /dev/null +++ b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/column-config-sidebar/column-type-limits.test.ts @@ -0,0 +1,56 @@ +/** + * @vitest-environment node + */ +import { afterEach, describe, expect, it } from 'vitest' +import { COLUMN_TYPE_REGISTRY } from '@/lib/table/column-types' +import { + COLUMN_TYPE_OPTIONS, + columnTypeOptionsForTable, +} from '@/app/workspace/[workspaceId]/tables/[tableId]/components/column-config-sidebar/column-types' + +const option = COLUMN_TYPE_OPTIONS.find((candidate) => candidate.type === 'string') +if (!option) throw new Error('String column type option is missing') +const originalMaxPerTable = option.maxPerTable +const definition = COLUMN_TYPE_REGISTRY.string +const originalDefinitionMaxPerTable = definition.maxPerTable + +afterEach(() => { + if (originalMaxPerTable === undefined) { + Reflect.deleteProperty(option, 'maxPerTable') + } else { + option.maxPerTable = originalMaxPerTable + } + + if (originalDefinitionMaxPerTable === undefined) { + Reflect.deleteProperty(definition, 'maxPerTable') + } else { + Object.assign(definition, { maxPerTable: originalDefinitionMaxPerTable }) + } +}) + +describe('column type picker limits', () => { + it('keeps a limited type visible but disables it once the limit is reached', () => { + option.maxPerTable = 1 + Object.assign(definition, { maxPerTable: 1 }) + + const result = columnTypeOptionsForTable([{ name: 'first', type: 'string' }], undefined, { + tableRowTtlEnabled: true, + }) + const stringOption = result.find((candidate) => candidate.type === 'string') + + expect(stringOption?.disabledReason).toBe('Only one Text column allowed per table') + }) + + it('keeps the current type selectable while editing its existing column', () => { + option.maxPerTable = 1 + Object.assign(definition, { maxPerTable: 1 }) + const currentColumn = { name: 'first', type: 'string' } as const + + const result = columnTypeOptionsForTable([currentColumn], currentColumn, { + tableRowTtlEnabled: true, + }) + const stringOption = result.find((candidate) => candidate.type === 'string') + + expect(stringOption?.disabledReason).toBeUndefined() + }) +}) diff --git a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/column-config-sidebar/column-types.test.ts b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/column-config-sidebar/column-types.test.ts new file mode 100644 index 00000000000..c59d36132b1 --- /dev/null +++ b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/column-config-sidebar/column-types.test.ts @@ -0,0 +1,43 @@ +/** + * @vitest-environment node + */ +import { describe, expect, it } from 'vitest' +import type { ColumnDefinition } from '@/lib/table' +import { columnTypeOptionsForTable } from '@/app/workspace/[workspaceId]/tables/[tableId]/components/column-config-sidebar/column-types' + +describe('columnTypeOptionsForTable', () => { + const ttlColumn: ColumnDefinition = { name: 'expires_at', type: 'ttl' } + + it('disables TTL with an explanation when the table already has one', () => { + const availableTtl = columnTypeOptionsForTable([{ name: 'name', type: 'string' }], undefined, { + tableRowTtlEnabled: true, + }).find((option) => option.type === 'ttl') + const unavailableTtl = columnTypeOptionsForTable([ttlColumn], undefined, { + tableRowTtlEnabled: true, + }).find((option) => option.type === 'ttl') + + expect(availableTtl?.disabledReason).toBeUndefined() + expect(unavailableTtl?.disabledReason).toBe('Only one Expiration column allowed per table') + }) + + it('keeps TTL enabled while editing the existing TTL column', () => { + const ttlOption = columnTypeOptionsForTable([ttlColumn], ttlColumn, { + tableRowTtlEnabled: true, + }).find((option) => option.type === 'ttl') + + expect(ttlOption?.disabledReason).toBeUndefined() + }) + + it('hides TTL while disabled unless editing an existing TTL column', () => { + expect( + columnTypeOptionsForTable([], undefined, { tableRowTtlEnabled: false }).some( + (option) => option.type === 'ttl' + ) + ).toBe(false) + expect( + columnTypeOptionsForTable([ttlColumn], ttlColumn, { tableRowTtlEnabled: false }).some( + (option) => option.type === 'ttl' + ) + ).toBe(true) + }) +}) diff --git a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/column-config-sidebar/column-types.ts b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/column-config-sidebar/column-types.ts index a6ea0ba2ac1..85ef1ad1045 100644 --- a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/column-config-sidebar/column-types.ts +++ b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/column-config-sidebar/column-types.ts @@ -1,19 +1,25 @@ import type React from 'react' import { PlayOutline } from '@sim/emcn/icons' import type { ColumnDefinition } from '@/lib/table' -import { ALL_COLUMN_TYPES } from '@/lib/table/column-types' +import { ALL_COLUMN_TYPES, wouldExceedColumnTypeLimit } from '@/lib/table/column-types' /** * UI-only column type. `'workflow'` is the virtual entry users pick from the * "+ New column" dropdown to spawn a workflow group; the resulting columns are * stored as scalar types under the hood (none carry `'workflow'`). */ -type SidebarColumnType = ColumnDefinition['type'] | 'workflow' +export type SidebarColumnType = ColumnDefinition['type'] | 'workflow' -interface ColumnTypeOption { +export interface ColumnTypeOption { type: SidebarColumnType label: string icon: React.ComponentType<{ className?: string }> + maxPerTable?: number + disabledReason?: string +} + +interface ColumnTypeAvailability { + tableRowTtlEnabled: boolean } /** @@ -26,9 +32,39 @@ export const COLUMN_TYPE_OPTIONS: ColumnTypeOption[] = [ type: definition.id, label: definition.label, icon: definition.icon, + maxPerTable: definition.maxPerTable, })), { type: 'workflow', label: 'Workflow', icon: PlayOutline }, ] -/** Plain column types (no workflow). Used by ``'s type combobox in edit mode. */ -export const PLAIN_COLUMN_TYPE_OPTIONS = COLUMN_TYPE_OPTIONS.filter((o) => o.type !== 'workflow') +/** Plain column types (no workflow). Used by the column type combobox in edit mode. */ +export const PLAIN_COLUMN_TYPE_OPTIONS = COLUMN_TYPE_OPTIONS.filter( + (option) => option.type !== 'workflow' +) + +function columnTypeLimitMessage(label: string, maxPerTable: number): string { + return maxPerTable === 1 + ? `Only one ${label} column allowed per table` + : `Only ${maxPerTable} ${label} columns allowed per table` +} + +/** Picker entries with unavailable cardinality-limited types marked as disabled. */ +export function columnTypeOptionsForTable( + columns: readonly ColumnDefinition[], + currentColumn: ColumnDefinition | null | undefined, + availability: ColumnTypeAvailability +): ColumnTypeOption[] { + return COLUMN_TYPE_OPTIONS.filter( + (option) => + option.type !== 'ttl' || availability.tableRowTtlEnabled || currentColumn?.type === 'ttl' + ).map((option) => { + if (option.type === 'workflow') return option + if (currentColumn?.type === option.type) return option + if (option.maxPerTable === undefined) return option + if (!wouldExceedColumnTypeLimit(columns, option.type, 1)) return option + return { + ...option, + disabledReason: columnTypeLimitMessage(option.label, option.maxPerTable), + } + }) +} diff --git a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/column-config-sidebar/index.ts b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/column-config-sidebar/index.ts index 0308447977f..f5d7d9a197d 100644 --- a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/column-config-sidebar/index.ts +++ b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/column-config-sidebar/index.ts @@ -1,3 +1,9 @@ export type { ColumnConfig } from './column-config-sidebar' export { ColumnConfigSidebar } from './column-config-sidebar' -export { COLUMN_TYPE_OPTIONS, PLAIN_COLUMN_TYPE_OPTIONS } from './column-types' +export { + COLUMN_TYPE_OPTIONS, + type ColumnTypeOption, + columnTypeOptionsForTable, + PLAIN_COLUMN_TYPE_OPTIONS, + type SidebarColumnType, +} from './column-types' diff --git a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/column-dropdown/column-dropdown.test.tsx b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/column-dropdown/column-dropdown.test.tsx index 6409eb7f513..e4321a7eb59 100644 --- a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/column-dropdown/column-dropdown.test.tsx +++ b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/column-dropdown/column-dropdown.test.tsx @@ -29,6 +29,8 @@ describe('ColumnDropdown', () => { act(() => { root.render( ` trigger. Same dropdown content either way. */ trigger: 'header' | 'inline-header' @@ -36,12 +40,48 @@ interface ColumnDropdownProps { onBlocked: () => void } +interface ColumnTypeMenuItemProps { + option: ColumnTypeOption + onSelect: () => void +} + +function ColumnTypeMenuItem({ option, onSelect }: ColumnTypeMenuItemProps) { + const Icon = option.icon + const item = ( + { + if (option.disabledReason) { + event.preventDefault() + return + } + onSelect() + }} + > + + {option.label} + + ) + + if (!option.disabledReason) return item + + return ( + + {item} + {option.disabledReason} + + ) +} + /** * "+ New column" dropdown — the single entry point for creating a column. * Lists every column type plus "Workflow" and "Enrichments"; picking a type * opens the right sidebar pre-seeded. */ export function ColumnDropdown({ + columns, + tableRowTtlEnabled, trigger, disabled, onPickType, @@ -86,18 +126,12 @@ export function ColumnDropdown({ {triggerButton} - {COLUMN_TYPE_OPTIONS.map((option) => { - const Icon = option.icon + {columnTypeOptionsForTable(columns, undefined, { tableRowTtlEnabled }).map((option) => { const onSelect = option.type === 'workflow' ? onPickWorkflow : () => onPickType(option.type as ColumnDefinition['type']) - return ( - - - {option.label} - - ) + return })} diff --git a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/row-modal/row-modal.test.tsx b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/row-modal/row-modal.test.tsx new file mode 100644 index 00000000000..1c0a37a21c2 --- /dev/null +++ b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/row-modal/row-modal.test.tsx @@ -0,0 +1,300 @@ +/** + * @vitest-environment jsdom + */ +import { act, createElement, type ReactNode } from 'react' +import { createRoot } from 'react-dom/client' +import { beforeEach, describe, expect, it, vi } from 'vitest' +import type { TableInfo, TableRow } from '@/lib/table' +import { RowModal } from '@/app/workspace/[workspaceId]/tables/[tableId]/components/row-modal/row-modal' + +const { mockToastError, mockUseTimezoneState, mockUpdateRow, mockDeleteRow, mockDeleteRows } = + vi.hoisted(() => ({ + mockToastError: vi.fn(), + mockUseTimezoneState: vi.fn(), + mockUpdateRow: vi.fn(), + mockDeleteRow: vi.fn(), + mockDeleteRows: vi.fn(), + })) + +vi.mock('next/navigation', () => ({ + useParams: () => ({ workspaceId: 'workspace-1' }), +})) +vi.mock('@/hooks/queries/general-settings', () => ({ + useTimezoneState: mockUseTimezoneState, +})) +vi.mock('@/hooks/queries/tables', () => ({ + useUpdateTableRow: () => ({ mutateAsync: mockUpdateRow, isPending: false }), + useDeleteTableRow: () => ({ mutateAsync: mockDeleteRow, isPending: false }), + useDeleteTableRows: () => ({ mutateAsync: mockDeleteRows, isPending: false }), +})) +vi.mock('@sim/emcn', () => { + const passthrough = ({ children }: { children?: ReactNode }) => children ?? null + return { + Checkbox: () => null, + Chip: ({ children, ...props }: { children?: ReactNode }) => + createElement('button', { type: 'button', ...props }, children), + ChipConfirmModal: passthrough, + ChipDatePicker: ({ value, onChange }: { value?: string; onChange: (value: string) => void }) => + createElement( + 'button', + { type: 'button', 'data-testid': 'date', onClick: () => onChange(value ?? '2026-11-01') }, + value + ), + ChipModal: passthrough, + ChipModalBody: passthrough, + ChipModalError: passthrough, + ChipModalField: ({ + type, + value, + onChange, + children, + }: { + type?: string + value?: string + onChange?: (value: string) => void + children?: ReactNode | ((aria: Record) => ReactNode) + }) => + type === 'input' + ? createElement('input', { + 'data-testid': 'modal-input', + value: value ?? '', + onChange: (event: { currentTarget: { value: string } }) => + onChange?.(event.currentTarget.value), + }) + : typeof children === 'function' + ? children({ 'aria-describedby': 'field-hint' }) + : (children ?? null), + ChipModalFooter: ({ + primaryAction, + }: { + primaryAction: { disabled?: boolean; onClick?: () => void } + }) => + createElement( + 'button', + { + type: 'button', + 'data-testid': 'submit', + disabled: primaryAction.disabled, + onClick: primaryAction.onClick, + }, + 'Update Row' + ), + ChipModalHeader: passthrough, + ChipTimePicker: ({ value, onChange }: { value?: string; onChange: (value: string) => void }) => + createElement('input', { + 'data-testid': 'time', + value: value ?? '', + onChange: (event: { currentTarget: { value: string } }) => + onChange(event.currentTarget.value), + }), + Label: passthrough, + toast: { error: mockToastError }, + } +}) + +const table: TableInfo = { + id: 'table-1', + name: 'Expiring rows', + schema: { columns: [{ name: 'expires_at', type: 'ttl' }] }, +} + +const row: TableRow = { + id: 'row-1', + data: { expires_at: Date.parse('2026-11-01T08:00:00Z') / 1000 }, + executions: {}, + position: 0, + createdAt: '2026-01-01T00:00:00Z', + updatedAt: '2026-01-01T00:00:00Z', +} + +function changeInput(input: HTMLInputElement, value: string) { + const setter = Object.getOwnPropertyDescriptor(HTMLInputElement.prototype, 'value')?.set + setter?.call(input, value) + input.dispatchEvent(new Event('input', { bubbles: true })) +} + +describe('RowModal expiration editing', () => { + beforeEach(() => { + vi.clearAllMocks() + mockUpdateRow.mockResolvedValue(undefined) + }) + + it('waits for the saved timezone, freezes it, and chooses the later repeated hour', async () => { + mockUseTimezoneState.mockReturnValue({ timezone: 'Asia/Tokyo', status: 'loading' }) + const container = document.createElement('div') + document.body.appendChild(container) + const root = createRoot(container) + const props = { + mode: 'edit' as const, + isOpen: true, + onClose: vi.fn(), + table, + row, + onSuccess: vi.fn(), + } + + ;(globalThis as { IS_REACT_ACT_ENVIRONMENT?: boolean }).IS_REACT_ACT_ENVIRONMENT = true + act(() => root.render(createElement(RowModal, props))) + + expect(container.querySelector('[aria-label="Edit expires_at"]')?.textContent).toBe( + 'Loading timezone…' + ) + expect(container.querySelector('[data-testid="time"]')).toBeNull() + expect(container.querySelector('[data-testid="submit"]')?.disabled).toBe( + true + ) + + mockUseTimezoneState.mockReturnValue({ + timezone: 'America/Los_Angeles', + status: 'ready', + }) + act(() => root.render(createElement(RowModal, props))) + + mockUseTimezoneState.mockReturnValue({ + timezone: 'America/New_York', + status: 'ready', + }) + act(() => root.render(createElement(RowModal, props))) + + const timeInput = container.querySelector('[data-testid="time"]') + expect(timeInput?.value).toBe('01:00') + act(() => changeInput(timeInput as HTMLInputElement, '01:30')) + + const submit = container.querySelector('[data-testid="submit"]') + await act(async () => submit?.click()) + + expect(mockUpdateRow).toHaveBeenCalledWith({ + rowId: 'row-1', + data: { expires_at: Date.parse('2026-11-01T09:30:00Z') / 1000 }, + }) + expect(props.onSuccess).toHaveBeenCalledTimes(1) + + act(() => root.unmount()) + container.remove() + }) + + it('also waits for timezone settings on an ordinary Date column', () => { + mockUseTimezoneState.mockReturnValue({ timezone: 'Asia/Tokyo', status: 'loading' }) + const container = document.createElement('div') + document.body.appendChild(container) + const root = createRoot(container) + const props = { + mode: 'edit' as const, + isOpen: true, + onClose: vi.fn(), + table: { + id: 'table-2', + name: 'Dates', + schema: { columns: [{ name: 'starts_at', type: 'date' as const }] }, + }, + row: { ...row, data: { starts_at: '2026-06-15T09:00:00+09:00' } }, + onSuccess: vi.fn(), + } + + act(() => root.render(createElement(RowModal, props))) + + expect(container.querySelector('[aria-label="Edit starts_at"]')?.textContent).toBe( + 'Loading timezone…' + ) + expect(container.querySelector('[data-testid="time"]')).toBeNull() + + mockUseTimezoneState.mockReturnValue({ + timezone: 'America/Los_Angeles', + status: 'ready', + }) + act(() => root.render(createElement(RowModal, props))) + + expect(container.querySelector('[data-testid="time"]')).not.toBeNull() + act(() => root.unmount()) + container.remove() + }) + + it('blocks an invalid saved timezone with the plain-text guidance', () => { + mockUseTimezoneState.mockReturnValue({ + timezone: 'America/Los_Angeles', + savedTimezone: 'Mars/Olympus', + status: 'invalid', + }) + const container = document.createElement('div') + document.body.appendChild(container) + const root = createRoot(container) + const props = { + mode: 'edit' as const, + isOpen: true, + onClose: vi.fn(), + table, + row, + onSuccess: vi.fn(), + } + + act(() => root.render(createElement(RowModal, props))) + + const blockedField = container.querySelector( + '[aria-label="Edit expires_at"]' + ) + expect(blockedField?.textContent).toBe(String(row.data.expires_at)) + expect(container.querySelector('[data-testid="submit"]')?.disabled).toBe( + true + ) + expect(mockToastError).not.toHaveBeenCalled() + act(() => blockedField?.click()) + expect(mockToastError).toHaveBeenCalledWith( + 'Your saved timezone “Mars/Olympus” is invalid. Update it in Settings → General before editing Date or Expiration cells.' + ) + act(() => root.unmount()) + container.remove() + }) + + it('keeps unrelated fields editable and omits blocked date values from the update', async () => { + mockUseTimezoneState.mockReturnValue({ + timezone: 'America/Los_Angeles', + savedTimezone: 'Mars/Olympus', + status: 'invalid', + }) + const container = document.createElement('div') + document.body.appendChild(container) + const root = createRoot(container) + const mixedTable: TableInfo = { + ...table, + schema: { + columns: [ + { name: 'name', type: 'string' }, + { name: 'expires_at', type: 'ttl' }, + ], + }, + } + const mixedRow = { ...row, data: { name: 'Ada', expires_at: row.data.expires_at } } + const props = { + mode: 'edit' as const, + isOpen: true, + onClose: vi.fn(), + table: mixedTable, + row: mixedRow, + onSuccess: vi.fn(), + } + + act(() => root.render(createElement(RowModal, props))) + + const nameInput = container.querySelector('[data-testid="modal-input"]') + const blockedField = container.querySelector( + '[aria-label="Edit expires_at"]' + ) + const submit = container.querySelector('[data-testid="submit"]') + expect(nameInput?.value).toBe('Ada') + expect(blockedField?.textContent).toBe(String(row.data.expires_at)) + expect(submit?.disabled).toBe(false) + + act(() => changeInput(nameInput as HTMLInputElement, 'Grace')) + await act(async () => submit?.click()) + + expect(mockUpdateRow).toHaveBeenCalledWith({ + rowId: 'row-1', + data: { name: 'Grace' }, + }) + expect(props.onSuccess).toHaveBeenCalledTimes(1) + expect(mockToastError).not.toHaveBeenCalled() + + act(() => root.unmount()) + container.remove() + }) +}) diff --git a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/row-modal/row-modal.tsx b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/row-modal/row-modal.tsx index d46e2193330..bbab5f353f2 100644 --- a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/row-modal/row-modal.tsx +++ b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/row-modal/row-modal.tsx @@ -1,8 +1,9 @@ 'use client' -import { useId, useState } from 'react' +import { useId, useRef, useState } from 'react' import { Checkbox, + Chip, ChipConfirmModal, ChipDatePicker, ChipModal, @@ -13,6 +14,7 @@ import { ChipModalHeader, ChipTimePicker, Label, + toast, } from '@sim/emcn' import { createLogger } from '@sim/logger' import { getErrorMessage } from '@sim/utils/errors' @@ -20,7 +22,8 @@ import { useParams } from 'next/navigation' import type { ColumnDefinition, TableInfo, TableRow } from '@/lib/table' import { columnTypeOf } from '@/lib/table/column-types' import { resolveCurrencyCode } from '@/lib/table/currency' -import { useTimezone } from '@/hooks/queries/general-settings' +import { getTimezoneEditBlockedMessage } from '@/app/workspace/[workspaceId]/tables/[tableId]/components/timezone-editing' +import { type TimezoneState, useTimezoneState } from '@/hooks/queries/general-settings' import { useDeleteTableRow, useDeleteTableRows, useUpdateTableRow } from '@/hooks/queries/tables' import { cleanCellValue, @@ -46,12 +49,16 @@ export interface RowModalProps { function cleanRowData( columns: ColumnDefinition[], rowData: Record, - timeZone: string + timeZone: string, + dateEditorsReady: boolean ): Record { const cleanData: Record = {} columns.forEach((col) => { const value = rowData[col.name] + if (columnTypeOf(col).editor === 'date' && !dateEditorsReady) { + return + } try { cleanData[col.name] = cleanCellValue(value, col, timeZone) } catch { @@ -78,7 +85,13 @@ export function RowModal({ mode, isOpen, onClose, table, row, rowIds, onSuccess const schema = table?.schema const columns = schema?.columns || [] - const timeZone = useTimezone() + const timezoneState = useTimezoneState() + const editTimeZoneRef = useRef(null) + if (timezoneState.status === 'ready' && editTimeZoneRef.current === null) { + editTimeZoneRef.current = timezoneState.timezone + } + const dateEditorsReady = editTimeZoneRef.current !== null + const timeZone = editTimeZoneRef.current ?? timezoneState.timezone const [rowData, setRowData] = useState>(() => mode === 'edit' && row ? row.data : {} ) @@ -89,12 +102,18 @@ export function RowModal({ mode, isOpen, onClose, table, row, rowIds, onSuccess const isSubmitting = updateRowMutation.isPending || deleteRowMutation.isPending || deleteRowsMutation.isPending + const timezoneBlockedMessage = getTimezoneEditBlockedMessage(timezoneState) + const hasEditableColumn = columns.some( + (column) => columnTypeOf(column).editor !== 'date' || dateEditorsReady + ) + const handleFormSubmit = async (e?: React.FormEvent) => { e?.preventDefault() setError(null) + if (!hasEditableColumn) return try { - const cleanData = cleanRowData(columns, rowData, timeZone) + const cleanData = cleanRowData(columns, rowData, timeZone, dateEditorsReady) if (row) { await updateRowMutation.mutateAsync({ rowId: row.id, data: cleanData }) @@ -169,15 +188,28 @@ export function RowModal({ mode, isOpen, onClose, table, row, rowIds, onSuccess Update values for {table?.name ?? 'table'}

-
- - - Adds {missingInputColumnNames.join(', ')} to the workflow's Start block - - - )}
{workflowState.isLoading ? ( @@ -896,12 +752,16 @@ export function WorkflowSidebarBody({
Workflow ({ label: wf.name, value: wf.id })) ?? []} + options={ + workflows + ?.filter((workflow) => workflow.isDeployed) + .map((workflow) => ({ label: workflow.name, value: workflow.id })) ?? [] + } value={selectedWorkflowId} onChange={(v) => setSelectedWorkflowId(v)} placeholder='Select a workflow' disabled={!workflows || workflows.length === 0 || isEditOutputMode || isEnrichment} - emptyMessage='No manual triggers configured' + emptyMessage='No deployed workflows available' maxHeight={260} searchable searchPlaceholder='Search workflows...' @@ -993,25 +853,8 @@ export function WorkflowSidebarBody({
{showAdvanced && ( <> - {!isEnrichment && ( - <> -
- - - setDeploymentMode(v === 'deployed' ? 'deployed' : 'live') - } - > - Live - Deployed - -
- - - )} getColumnId(c) === columnConfig.columnName) ?? null) diff --git a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/utils.test.ts b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/utils.test.ts index 146fb11cc23..c77ce7256e3 100644 --- a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/utils.test.ts +++ b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/utils.test.ts @@ -194,4 +194,31 @@ describe('formatValueForInput', () => { ) expect(formatValueForInput('2026-07-06', 'date')).toBe('2026-07-06') }) + + it('renders TTL instants in the editor timezone without changing the instant', () => { + expect(formatValueForInput(1_700_000_000, 'ttl', 'America/New_York')).toBe( + '2023-11-14T17:13:20-05:00' + ) + expect( + cleanCellValue('2023-11-14 17:13:20', { name: 'expires_at', type: 'ttl' }, 'America/New_York') + ).toBe(1_700_000_000) + expect( + cleanCellValue('2023-11-14', { name: 'expires_at', type: 'ttl' }, 'America/New_York') + ).toBe(1_699_938_000) + }) + + it('uses the latest effective timezone for each TTL edit', () => { + const column = { name: 'expires_at', type: 'ttl' } as const + const input = '2026-06-15 09:00:30' + + expect(cleanCellValue(input, column, 'America/New_York')).toBe( + Date.parse('2026-06-15T13:00:30Z') / 1000 + ) + expect(cleanCellValue(input, column, 'Asia/Kathmandu')).toBe( + Date.parse('2026-06-15T03:15:30Z') / 1000 + ) + expect(cleanCellValue(input, column, 'America/New_York')).toBe( + Date.parse('2026-06-15T13:00:30Z') / 1000 + ) + }) }) diff --git a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/utils.ts b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/utils.ts index 69f7722d11d..c9892f8466e 100644 --- a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/utils.ts +++ b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/utils.ts @@ -1,7 +1,8 @@ +import { getWallClockParts } from '@/lib/core/utils/timezone' import type { ColumnDefinition, JsonValue } from '@/lib/table' import type { ColumnType } from '@/lib/table/column-types' import { columnTypeById, columnTypeOf } from '@/lib/table/column-types' -import { formatDateCellDisplay, getWallClockParts, normalizeDateCellValue } from '@/lib/table/dates' +import { formatDateCellDisplay, normalizeDateCellValue } from '@/lib/table/dates' /** * Pick a fresh "untitled[_N]" name not already taken by `columns`. Used by @@ -55,7 +56,7 @@ export function cleanCellValue( // Everything else runs the SAME coercion the server will run, so the // optimistic cache holds exactly the value that gets persisted. const columnType = columnTypeOf(column) - const coerced = columnType.coerce(value as JsonValue, column) + const coerced = columnType.coerce(value as JsonValue, column, { timezone: timeZone }) if (coerced.ok) return coerced.value const salvaged = columnType.salvage?.(value as JsonValue, column) return salvaged?.ok ? salvaged.value : null @@ -68,7 +69,7 @@ export function cleanCellValue( * row data already has the new mapping's value) would otherwise render * `[object Object]` via `String(value)`. */ -export function formatValueForInput(value: unknown, type: string): string { +export function formatValueForInput(value: unknown, type: string, timeZone?: string): string { if (value === null || value === undefined) return '' const definition = columnTypeById(type) // Shape-drift guard, kept ahead of the registry: a column whose declared type @@ -78,7 +79,11 @@ export function formatValueForInput(value: unknown, type: string): string { if (typeof value === 'object' && !definition.storesOpaqueIds && type !== 'json') { return JSON.stringify(value) } - return definition.formatForInput(value, { name: '', type: type as ColumnType }) + return definition.formatForInput( + value, + { name: '', type: type as ColumnType }, + { timezone: timeZone } + ) } /** A canonical date-cell value split into its wall-clock editing parts. */ @@ -142,46 +147,12 @@ export function storageToDisplay(stored: string, options?: { seconds?: boolean } */ export function displayToStorage(display: string, timeZone?: string): string | null { const trimmed = display.trim() - const withTime = trimmed.match( - /^(\d{1,2})\/(\d{1,2})\/(\d{4})[ ,]+(\d{1,2}):(\d{2})(?::(\d{2}))?(?:\s*(AM|PM))?$/i - ) - if (withTime) { - const [, m, d, y, h, min, sec, meridiem] = withTime - let hours = Number(h) - if (meridiem) { - if (hours < 1 || hours > 12) return null - hours = (hours % 12) + (meridiem.toUpperCase() === 'PM' ? 12 : 0) - } else if (hours > 23) { - return null - } - if (Number(min) > 59 || Number(sec ?? 0) > 59) return null - if (!isValidCalendarDay(Number(y), Number(m), Number(d))) return null - const pad = (n: string) => n.padStart(2, '0') - // Route through the shared normalizer so the wall time resolves in the - // effective zone. - return normalizeDateCellValue( - `${y}-${pad(m)}-${pad(d)}T${String(hours).padStart(2, '0')}:${min}:${sec ?? '00'}`, - { timezone: timeZone } - ) - } - const full = trimmed.match(/^(\d{1,2})\/(\d{1,2})\/(\d{4})$/) - if (full) { - if (!isValidCalendarDay(Number(full[3]), Number(full[1]), Number(full[2]))) return null - return `${full[3]}-${full[1].padStart(2, '0')}-${full[2].padStart(2, '0')}` - } const partial = trimmed.match(/^(\d{1,2})\/(\d{1,2})$/) if (partial) { const year = Number(todayLocalCalendarDate(timeZone).slice(0, 4)) - if (!isValidCalendarDay(year, Number(partial[1]), Number(partial[2]))) return null - return `${year}-${partial[1].padStart(2, '0')}-${partial[2].padStart(2, '0')}` + return normalizeDateCellValue( + `${year}-${partial[1].padStart(2, '0')}-${partial[2].padStart(2, '0')}` + ) } return normalizeDateCellValue(trimmed, { timezone: timeZone }) } - -/** True when Y/M/D is a real calendar day — `Date` rolls impossible days over - * (02/30 → 03/02) instead of rejecting them, so compare the round-trip. */ -function isValidCalendarDay(year: number, month: number, day: number): boolean { - if (month < 1 || month > 12 || day < 1 || day > 31) return false - const check = new Date(year, month - 1, day) - return check.getMonth() === month - 1 && check.getDate() === day -} diff --git a/apps/sim/background/cleanup-table-row-ttl.test.ts b/apps/sim/background/cleanup-table-row-ttl.test.ts new file mode 100644 index 00000000000..b73e0c4748d --- /dev/null +++ b/apps/sim/background/cleanup-table-row-ttl.test.ts @@ -0,0 +1,295 @@ +/** + * @vitest-environment node + */ +import type { SQL } from 'drizzle-orm' +import { PgDialect } from 'drizzle-orm/pg-core' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +vi.unmock('@sim/db/schema') +vi.unmock('drizzle-orm') + +const { + mockDeleteExecute, + mockListExecute, + mockIsTableRowTtlEnabled, + mockSignalTableRowsChanged, + mockTask, + mockWithLockedTable, + mockFireTableTrigger, +} = vi.hoisted(() => ({ + mockDeleteExecute: vi.fn(), + mockListExecute: vi.fn(), + mockIsTableRowTtlEnabled: vi.fn(), + mockSignalTableRowsChanged: vi.fn(), + mockTask: vi.fn((config: unknown) => config), + mockWithLockedTable: vi.fn(), + mockFireTableTrigger: vi.fn(), +})) + +vi.mock('@sim/db', () => ({ + dbFor: vi.fn(() => ({ execute: mockListExecute })), +})) + +vi.mock('@trigger.dev/sdk', () => ({ task: mockTask })) +vi.mock('@/lib/table/events', () => ({ signalTableRowsChanged: mockSignalTableRowsChanged })) +vi.mock('@/lib/table/constants', () => ({ + getDeleteSnapshotBatchSize: () => 500, + TABLE_LIMITS: { DELETE_SNAPSHOT_BATCH_MAX_BYTES: 32 * 1024 * 1024 }, +})) +vi.mock('@/lib/table/service', () => ({ withLockedTable: mockWithLockedTable })) +vi.mock('@/lib/table/ttl-availability', () => ({ + isTableRowTtlEnabled: mockIsTableRowTtlEnabled, +})) +vi.mock('@/lib/table/trigger', () => ({ fireTableTrigger: mockFireTableTrigger })) + +import { cleanupTableRowTtlTask, runCleanupTableRowTtl } from '@/background/cleanup-table-row-ttl' + +const dialect = new PgDialect() + +const table = { + id: 'table-1', + name: 'Expiring rows', + workspaceId: 'workspace-1', + schema: { columns: [{ id: 'col-ttl', name: 'expires_at', type: 'ttl' }] }, + locks: { insertLocked: false, updateLocked: false, deleteLocked: false, schemaLocked: false }, +} + +function deletedRows(count: number, start = 1) { + return Array.from({ length: count }, (_, index) => { + const number = start + index + return { id: `row-${number}`, data: { value: number } } + }) +} + +function returnedRows(count: number, start = 1, createdAt = '2026-01-01T00:00:00.000000') { + return deletedRows(count, start).map((row) => ({ + ...row, + createdAt, + snapshotBytes: 20, + })) +} + +describe('table row TTL cleanup', () => { + beforeEach(() => { + vi.clearAllMocks() + mockIsTableRowTtlEnabled.mockResolvedValue(true) + mockListExecute.mockResolvedValue([{ id: table.id, workspaceId: table.workspaceId }]) + mockWithLockedTable.mockImplementation( + async ( + _tableId: string, + mutate: ( + fresh: typeof table, + trx: { execute: typeof mockDeleteExecute } + ) => Promise + ) => mutate(table, { execute: mockDeleteExecute }) + ) + }) + + it('deletes expired rows in locked, created-at keyset batches and signals the table', async () => { + mockDeleteExecute + .mockResolvedValueOnce([ + ...returnedRows(499, 1, '2026-01-01T00:00:00.123455'), + ...returnedRows(1, 500, '2026-01-01T00:00:00.123456'), + ]) + .mockResolvedValueOnce(returnedRows(12, 501)) + .mockResolvedValueOnce([]) + + await expect(runCleanupTableRowTtl()).resolves.toEqual({ + batches: 3, + deleted: 512, + limitReached: false, + }) + expect(mockWithLockedTable).toHaveBeenCalledTimes(3) + expect(mockDeleteExecute).toHaveBeenCalledTimes(3) + const secondQuery = dialect.sqlToQuery(mockDeleteExecute.mock.calls[1][0] as SQL) + expect(secondQuery.sql.replace(/\$\d+/g, '?').replace(/\s+/g, ' ')).toContain( + 'AND (table_row.created_at, table_row.id) > (?::timestamp, ?)' + ) + expect(secondQuery.params).toEqual( + expect.arrayContaining(['2026-01-01T00:00:00.123456', 'row-500']) + ) + expect(mockSignalTableRowsChanged).toHaveBeenCalledWith(table.id) + expect(mockFireTableTrigger).toHaveBeenCalledTimes(2) + expect(mockFireTableTrigger).toHaveBeenNthCalledWith( + 1, + table.id, + table.workspaceId, + table.name, + 'delete', + deletedRows(500), + null, + table.schema, + 'ttl-cleanup' + ) + }) + + it('compares TTL values with whole Date.now epoch seconds', async () => { + const nowEpochMilliseconds = 1_700_000_000_999 + const nowEpochSeconds = 1_700_000_000 + const nowSpy = vi.spyOn(Date, 'now').mockReturnValue(nowEpochMilliseconds) + mockDeleteExecute.mockResolvedValue([]) + + try { + await runCleanupTableRowTtl() + } finally { + nowSpy.mockRestore() + } + + expect(dialect.sqlToQuery(mockListExecute.mock.calls[0][0] as SQL).params).toContain( + nowEpochSeconds + ) + expect(dialect.sqlToQuery(mockDeleteExecute.mock.calls[0][0] as SQL).params).toContain( + nowEpochSeconds + ) + }) + + it('checks the oldest expired rows first without using creation time as an expiry rule', async () => { + mockDeleteExecute.mockResolvedValue([]) + + await runCleanupTableRowTtl() + + const query = dialect + .sqlToQuery(mockDeleteExecute.mock.calls[0][0] as SQL) + .sql.replace(/\s+/g, ' ') + .replace(/\$\d+/g, '?') + .trim() + expect(query).toContain('AND (table_row.data->>?)::numeric <= ?') + expect(query).toContain('ORDER BY table_row.created_at, table_row.id') + expect(query).toContain('octet_length(table_row.data::text) AS snapshot_bytes') + expect(query).toContain('cumulative_snapshot_bytes <= ?') + expect(query).toContain('OR snapshot_order = 1') + expect(query).toContain( + `to_char(table_row.created_at, 'YYYY-MM-DD"T"HH24:MI:SS.US') AS "createdAt"` + ) + expect(query).toContain('candidates.snapshot_bytes AS "snapshotBytes"') + expect(query).not.toContain('table_row.created_by') + }) + + it('rejects a batch without a creation-time cursor', async () => { + mockDeleteExecute.mockResolvedValue([{ id: 'row-1', data: { value: 1 } }]) + + await expect(runCleanupTableRowTtl()).rejects.toThrow( + 'Table row TTL cleanup did not return a creation-time cursor' + ) + }) + + it('does no work when already aborted', async () => { + const controller = new AbortController() + controller.abort() + + await expect(runCleanupTableRowTtl(controller.signal)).resolves.toEqual({ + batches: 0, + deleted: 0, + limitReached: false, + }) + expect(mockListExecute).not.toHaveBeenCalled() + }) + + it('does no work when the feature is disabled', async () => { + mockIsTableRowTtlEnabled.mockResolvedValue(false) + + await expect(runCleanupTableRowTtl()).resolves.toEqual({ + batches: 0, + deleted: 0, + limitReached: false, + }) + expect(mockListExecute).not.toHaveBeenCalled() + expect(mockWithLockedTable).not.toHaveBeenCalled() + }) + + it('honors a delete lock re-read inside the table advisory lock', async () => { + mockWithLockedTable.mockImplementationOnce(async (_tableId, mutate) => + mutate( + { ...table, locks: { ...table.locks, deleteLocked: true } }, + { execute: mockDeleteExecute } + ) + ) + + await expect(runCleanupTableRowTtl()).resolves.toEqual({ + batches: 0, + deleted: 0, + limitReached: false, + }) + expect(mockDeleteExecute).not.toHaveBeenCalled() + expect(mockSignalTableRowsChanged).not.toHaveBeenCalled() + }) + + it('stops after one hundred full batches', async () => { + mockDeleteExecute.mockResolvedValue(returnedRows(500)) + + await expect(runCleanupTableRowTtl()).resolves.toEqual({ + batches: 100, + deleted: 50_000, + limitReached: true, + }) + expect(mockDeleteExecute).toHaveBeenCalledTimes(100) + expect(mockSignalTableRowsChanged).toHaveBeenCalledTimes(1) + }) + + it('gives each table one batch before returning to a backlogged table', async () => { + const secondTable = { + ...table, + id: 'table-2', + } + const attemptedTableIds: string[] = [] + const tableAttempts = new Map() + mockListExecute.mockResolvedValue([ + { id: table.id, workspaceId: table.workspaceId }, + { id: secondTable.id, workspaceId: secondTable.workspaceId }, + ]) + mockWithLockedTable.mockImplementation(async (tableId, mutate) => { + const freshTable = tableId === secondTable.id ? secondTable : table + return mutate(freshTable, { + execute: vi.fn(async () => { + attemptedTableIds.push(tableId) + const attempt = (tableAttempts.get(tableId) ?? 0) + 1 + tableAttempts.set(tableId, attempt) + if (tableId === table.id && attempt === 1) { + return returnedRows(500) + } + if (tableId === secondTable.id && attempt === 1) { + return returnedRows(1) + } + return [] + }), + }) + }) + + await expect(runCleanupTableRowTtl()).resolves.toEqual({ + batches: 4, + deleted: 501, + limitReached: false, + }) + expect(attemptedTableIds).toEqual([table.id, secondTable.id, table.id, secondTable.id]) + expect(mockSignalTableRowsChanged).toHaveBeenCalledWith(table.id) + expect(mockSignalTableRowsChanged).toHaveBeenCalledWith(secondTable.id) + }) + + it('signals tables changed before a later table cleanup failure propagates', async () => { + const secondTable = { + ...table, + id: 'table-2', + } + mockListExecute.mockResolvedValue([ + { id: table.id, workspaceId: table.workspaceId }, + { id: secondTable.id, workspaceId: secondTable.workspaceId }, + ]) + mockWithLockedTable.mockImplementation(async (tableId, mutate) => { + if (tableId === secondTable.id) throw new Error('second table cleanup failed') + return mutate(table, { execute: vi.fn().mockResolvedValue(returnedRows(1)) }) + }) + + await expect(runCleanupTableRowTtl()).rejects.toThrow('second table cleanup failed') + expect(mockSignalTableRowsChanged).toHaveBeenCalledTimes(1) + expect(mockSignalTableRowsChanged).toHaveBeenCalledWith(table.id) + }) + + it('registers one serialized Trigger.dev task', () => { + expect(cleanupTableRowTtlTask).toEqual( + expect.objectContaining({ + id: 'cleanup-table-row-ttl', + queue: { concurrencyLimit: 1 }, + }) + ) + }) +}) diff --git a/apps/sim/background/cleanup-table-row-ttl.ts b/apps/sim/background/cleanup-table-row-ttl.ts new file mode 100644 index 00000000000..9b05e26e702 --- /dev/null +++ b/apps/sim/background/cleanup-table-row-ttl.ts @@ -0,0 +1,329 @@ +import { dbFor } from '@sim/db' +import { userTableDefinitions, userTableRows } from '@sim/db/schema' +import { createLogger } from '@sim/logger' +import { task } from '@trigger.dev/sdk' +import { sql } from 'drizzle-orm' +import { asOrchestrationError } from '@/lib/core/orchestration/types' +import { getColumnId } from '@/lib/table/column-keys' +import { getDeleteSnapshotBatchSize, TABLE_LIMITS } from '@/lib/table/constants' +import { signalTableRowsChanged } from '@/lib/table/events' +import { assertRowDelete, TableLockedError } from '@/lib/table/mutation-locks' +import type { DbTransaction } from '@/lib/table/planner' +import type { DeletedTableRow } from '@/lib/table/rows/ordering' +import { withLockedTable } from '@/lib/table/service' +import { fireTableTrigger } from '@/lib/table/trigger' +import { isTableRowTtlEnabled } from '@/lib/table/ttl-availability' +import type { RowData, TableSchema } from '@/lib/table/types' + +const logger = createLogger('CleanupTableRowTtl') +const cleanupDb = dbFor('cleanup') + +const TTL_CLEANUP_MAX_BATCHES = 100 + +interface ExpiredTtlTableRef { + [key: string]: unknown + id: string + workspaceId: string +} + +interface DeletedTtlRows { + deleted: number + cursor: TtlCleanupCursor | null + rows: DeletedTableRow[] +} + +type DeletedTtlBatch = + | { attempted: false; deleted: 0; cursor: null } + | (DeletedTtlRows & { + attempted: true + tableName: string + schema: TableSchema + }) + +interface TtlCleanupCursor { + createdAt: string + id: string +} + +interface TtlTableCleanupState { + ref: ExpiredTtlTableRef + after?: TtlCleanupCursor + deleted: number + complete: boolean +} + +export interface TableRowTtlCleanupResult { + batches: number + deleted: number + limitReached: boolean +} + +async function listExpiredTtlTables(nowEpochSeconds: number): Promise { + const rows = await cleanupDb.execute(sql` + SELECT + ${userTableDefinitions.id} AS id, + ${userTableDefinitions.workspaceId} AS "workspaceId" + FROM ${userTableDefinitions} + WHERE ${userTableDefinitions.archivedAt} IS NULL + AND ${userTableDefinitions.deleteLocked} = false + AND EXISTS ( + SELECT 1 + FROM jsonb_array_elements( + COALESCE(${userTableDefinitions.schema}->'columns', '[]'::jsonb) + ) AS ttl_column(column_definition) + JOIN ${userTableRows} AS table_row + ON table_row.table_id = ${userTableDefinitions.id} + AND table_row.workspace_id = ${userTableDefinitions.workspaceId} + WHERE ttl_column.column_definition->>'type' = 'ttl' + AND jsonb_typeof( + table_row.data->COALESCE( + ttl_column.column_definition->>'id', + ttl_column.column_definition->>'name' + ) + ) = 'number' + AND ( + table_row.data->>COALESCE( + ttl_column.column_definition->>'id', + ttl_column.column_definition->>'name' + ) + )::numeric <= ${nowEpochSeconds} + ) + ORDER BY + md5(${userTableDefinitions.id} || ${nowEpochSeconds}::text), + ${userTableDefinitions.id} + LIMIT ${TTL_CLEANUP_MAX_BATCHES} + `) + return Array.isArray(rows) ? rows : [] +} + +function parseDeletedBatch(rows: unknown, batchSize: number): DeletedTtlRows { + if (!Array.isArray(rows)) { + throw new Error('Table row TTL cleanup did not return deleted rows') + } + const deletedRows = rows as Array<{ + id?: unknown + data?: unknown + createdAt?: unknown + snapshotBytes?: unknown + }> + if (deletedRows.length > batchSize) { + throw new Error('Table row TTL cleanup returned an invalid deleted count') + } + const parsed = deletedRows.map((row) => { + if (typeof row.id !== 'string') { + throw new Error('Table row TTL cleanup did not return a row cursor') + } + if (typeof row.createdAt !== 'string') { + throw new Error('Table row TTL cleanup did not return a creation-time cursor') + } + const snapshotBytes = Number(row.snapshotBytes) + if (!Number.isFinite(snapshotBytes) || snapshotBytes < 0) { + throw new Error('Table row TTL cleanup did not return a valid snapshot size') + } + if (snapshotBytes > TABLE_LIMITS.DELETE_SNAPSHOT_BATCH_MAX_BYTES) { + logger.warn('Deleting oversized legacy TTL row in an isolated snapshot batch', { + rowId: row.id, + snapshotBytes, + maxBytes: TABLE_LIMITS.DELETE_SNAPSHOT_BATCH_MAX_BYTES, + }) + } + return { + cursor: { createdAt: row.createdAt, id: row.id }, + row: { id: row.id, data: row.data as RowData }, + } + }) + return { + deleted: parsed.length, + cursor: parsed[parsed.length - 1]?.cursor ?? null, + rows: parsed.map(({ row }) => row), + } +} + +async function deleteExpiredTableRowBatch( + trx: DbTransaction, + tableId: string, + workspaceId: string, + columnKey: string, + nowEpochSeconds: number, + batchSize: number, + after?: TtlCleanupCursor +): Promise { + const rows = await trx.execute<{ + id: string + data: RowData + createdAt: string + snapshotBytes: number + }>(sql` + WITH locked_rows AS MATERIALIZED ( + SELECT table_row.id, table_row.created_at, octet_length(table_row.data::text) AS snapshot_bytes + FROM ${userTableRows} AS table_row + WHERE table_row.table_id = ${tableId} + AND table_row.workspace_id = ${workspaceId} + ${ + after + ? sql`AND (table_row.created_at, table_row.id) > (${after.createdAt}::timestamp, ${after.id})` + : sql`` + } + AND jsonb_typeof(table_row.data->${columnKey}) = 'number' + AND (table_row.data->>${columnKey})::numeric <= ${nowEpochSeconds} + ORDER BY table_row.created_at, table_row.id + LIMIT ${batchSize} + FOR UPDATE OF table_row SKIP LOCKED + ), ranked_rows AS ( + SELECT + id, + created_at, + snapshot_bytes, + row_number() OVER (ORDER BY created_at, id) AS snapshot_order, + sum(snapshot_bytes) OVER (ORDER BY created_at, id) AS cumulative_snapshot_bytes + FROM locked_rows + ), candidates AS ( + SELECT id, snapshot_bytes + FROM ranked_rows + WHERE cumulative_snapshot_bytes <= ${TABLE_LIMITS.DELETE_SNAPSHOT_BATCH_MAX_BYTES} + OR snapshot_order = 1 + ), deleted AS ( + DELETE FROM ${userTableRows} AS table_row + USING candidates + WHERE table_row.id = candidates.id + RETURNING + table_row.id, + table_row.data, + to_char(table_row.created_at, 'YYYY-MM-DD"T"HH24:MI:SS.US') AS "createdAt", + candidates.snapshot_bytes AS "snapshotBytes" + ) + SELECT id, data, "createdAt", "snapshotBytes" + FROM deleted + ORDER BY "createdAt", id + `) + return parseDeletedBatch(rows, batchSize) +} + +async function deleteExpiredRowsForTable( + ref: ExpiredTtlTableRef, + nowEpochSeconds: number, + batchSize: number, + after?: TtlCleanupCursor +): Promise { + try { + const batch = await withLockedTable( + ref.id, + async (table, trx): Promise => { + try { + assertRowDelete(table) + } catch (error) { + if (error instanceof TableLockedError) { + return { attempted: false, deleted: 0, cursor: null } + } + throw error + } + + const ttlColumn = table.schema.columns.find((column) => column.type === 'ttl') + if (!ttlColumn) return { attempted: false, deleted: 0, cursor: null } + + const batch = await deleteExpiredTableRowBatch( + trx, + table.id, + table.workspaceId, + getColumnId(ttlColumn), + nowEpochSeconds, + batchSize, + after + ) + return { + attempted: true, + ...batch, + tableName: table.name, + schema: table.schema, + } satisfies DeletedTtlBatch + }, + { expectedWorkspaceId: ref.workspaceId } + ) + if (batch.attempted && batch.rows.length > 0) { + await fireTableTrigger( + ref.id, + ref.workspaceId, + batch.tableName, + 'delete', + batch.rows, + null, + batch.schema, + 'ttl-cleanup' + ) + } + return batch + } catch (error) { + if (asOrchestrationError(error)?.code === 'not_found') { + return { attempted: false, deleted: 0, cursor: null } + } + throw error + } +} + +/** Deletes rows whose table TTL cell is at or before the current Unix epoch second. */ +export async function runCleanupTableRowTtl( + signal?: AbortSignal +): Promise { + if (signal?.aborted) return { batches: 0, deleted: 0, limitReached: false } + if (!(await isTableRowTtlEnabled())) { + logger.info('Table row TTL cleanup skipped because the feature is disabled') + return { batches: 0, deleted: 0, limitReached: false } + } + + const nowEpochSeconds = Math.floor(Date.now() / 1000) + const batchSize = getDeleteSnapshotBatchSize() + const tableRefs = await listExpiredTtlTables(nowEpochSeconds) + const tableStates: TtlTableCleanupState[] = tableRefs.map((ref) => ({ + ref, + deleted: 0, + complete: false, + })) + let deleted = 0 + let batches = 0 + + try { + while ( + batches < TTL_CLEANUP_MAX_BATCHES && + !signal?.aborted && + tableStates.some((state) => !state.complete) + ) { + for (const state of tableStates) { + if (state.complete) continue + if (batches === TTL_CLEANUP_MAX_BATCHES || signal?.aborted) break + + const batch = await deleteExpiredRowsForTable( + state.ref, + nowEpochSeconds, + batchSize, + state.after + ) + if (!batch.attempted) { + state.complete = true + continue + } + + batches++ + deleted += batch.deleted + state.deleted += batch.deleted + state.after = batch.cursor ?? undefined + if (batch.deleted === 0) state.complete = true + } + } + } finally { + for (const state of tableStates) { + if (state.deleted > 0) signalTableRowsChanged(state.ref.id) + } + } + + const limitReached = + batches === TTL_CLEANUP_MAX_BATCHES && + (tableStates.some((state) => !state.complete) || tableRefs.length === TTL_CLEANUP_MAX_BATCHES) + logger.info('Table row TTL cleanup completed', { batches, deleted, limitReached }) + return { batches, deleted, limitReached } +} + +export const cleanupTableRowTtlTask = task({ + id: 'cleanup-table-row-ttl', + queue: { concurrencyLimit: 1 }, + run: () => runCleanupTableRowTtl(), +}) diff --git a/apps/sim/background/sandbox-image-build.ts b/apps/sim/background/sandbox-image-build.ts index d88a0d9888f..58c235a46b6 100644 --- a/apps/sim/background/sandbox-image-build.ts +++ b/apps/sim/background/sandbox-image-build.ts @@ -1,7 +1,5 @@ import { task } from '@trigger.dev/sdk' import { - LEGACY_SANDBOX_IMAGE_BUILD_TASK_ID, - PREVIOUS_SANDBOX_IMAGE_BUILD_TASK_ID, runSandboxImageBuild, SANDBOX_IMAGE_BUILD_TASK_ID, type SandboxImageBuildPayload, @@ -29,38 +27,3 @@ export const sandboxImageBuildTask = task({ await runSandboxImageBuild(payload) }, }) - -/** - * Rollout bridge for web replicas deployed before revisioned task routing. - * Remove only after every old web release that emits this ID has drained. - */ -export const legacySandboxImageBuildTask = task({ - id: LEGACY_SANDBOX_IMAGE_BUILD_TASK_ID, - machine: 'small-1x', - maxDuration: 1200, - retry: { maxAttempts: 1 }, - queue: { - name: 'sandbox-image-build', - concurrencyLimit: 5, - }, - run: async (payload: SandboxImageBuildPayload) => { - await runSandboxImageBuild(payload) - }, -}) - -/** One-revision bridge for a worker-first materializer rollout. */ -export const previousSandboxImageBuildTask = PREVIOUS_SANDBOX_IMAGE_BUILD_TASK_ID - ? task({ - id: PREVIOUS_SANDBOX_IMAGE_BUILD_TASK_ID, - machine: 'small-1x', - maxDuration: 1200, - retry: { maxAttempts: 1 }, - queue: { - name: 'sandbox-image-build', - concurrencyLimit: 5, - }, - run: async (payload: SandboxImageBuildPayload) => { - await runSandboxImageBuild(payload) - }, - }) - : undefined diff --git a/apps/sim/background/workflow-column-execution.test.ts b/apps/sim/background/workflow-column-execution.test.ts index 3cb88624c7d..306e925133f 100644 --- a/apps/sim/background/workflow-column-execution.test.ts +++ b/apps/sim/background/workflow-column-execution.test.ts @@ -7,6 +7,7 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' import { createTimeoutAbortController, getExecutionDeadlineAt } from '@/lib/core/execution-limits' import { abortManualExecution } from '@/lib/execution/manual-cancellation' import { + assertWorkflowGroupMatchesLatestDeployment, buildTableAbortState, buildTableUsageLimitClear, createWorkflowGroupAttemptTimeoutController, @@ -14,9 +15,15 @@ import { terminalizeAbortedQueuedCarrierMarker, } from '@/background/workflow-column-execution' -const { appendTableEventMock } = vi.hoisted(() => ({ appendTableEventMock: vi.fn() })) +const { appendTableEventMock, flattenWorkflowOutputsMock } = vi.hoisted(() => ({ + appendTableEventMock: vi.fn(), + flattenWorkflowOutputsMock: vi.fn(), +})) vi.mock('@/lib/table/events', () => ({ appendTableEvent: appendTableEventMock })) +vi.mock('@/lib/workflows/blocks/flatten-outputs', () => ({ + flattenWorkflowOutputs: flattenWorkflowOutputsMock, +})) beforeEach(() => { vi.clearAllMocks() @@ -50,6 +57,106 @@ const QUEUED_PAYLOAD = { }, } +function latestDeployment( + startType = 'start_trigger' +): Parameters[1] { + return { + blocks: { + start: { + id: 'start', + type: startType, + subBlocks: { + inputFormat: { value: [{ name: 'company', type: 'string' }] }, + }, + }, + agent: { + id: 'agent', + type: 'agent', + subBlocks: {}, + }, + }, + edges: [{ id: 'start-agent', source: 'start', target: 'agent' }], + loops: {}, + parallels: {}, + variables: {}, + isFromNormalizedTables: false, + deploymentVersionId: 'deployment-version-latest', + } as Parameters[1] +} + +describe('latest table workflow deployment mappings', () => { + beforeEach(() => { + flattenWorkflowOutputsMock.mockReturnValue([ + { + blockId: 'agent', + blockName: 'Agent', + blockType: 'agent', + path: 'content', + leafType: 'string', + }, + ]) + }) + + it('accepts saved mappings that the latest active deployment still supports', () => { + expect(() => + assertWorkflowGroupMatchesLatestDeployment( + { + id: 'group-1', + workflowId: 'workflow-1', + outputs: [{ blockId: 'agent', path: 'content', columnName: 'column-output' }], + inputMappings: [{ inputName: 'company', columnName: 'column-company' }], + }, + latestDeployment() + ) + ).not.toThrow() + }) + + it('accepts a canonical split manual start block', () => { + expect(() => + assertWorkflowGroupMatchesLatestDeployment( + { + id: 'group-1', + workflowId: 'workflow-1', + outputs: [{ blockId: 'agent', path: 'content', columnName: 'column-output' }], + inputMappings: [{ inputName: 'company', columnName: 'column-company' }], + }, + latestDeployment('manual_trigger') + ) + ).not.toThrow() + }) + + it('rejects an output mapping removed by the latest active deployment', () => { + expect(() => + assertWorkflowGroupMatchesLatestDeployment( + { + id: 'group-1', + workflowId: 'workflow-1', + outputs: [{ blockId: 'agent', path: 'score', columnName: 'column-output' }], + }, + latestDeployment() + ) + ).toThrow( + 'Workflow group group-1 output agent::score is not available in the latest active deployment' + ) + }) + + it('rejects an input mapping removed by the latest active deployment', () => { + expect(() => + assertWorkflowGroupMatchesLatestDeployment( + { + id: 'group-1', + workflowId: 'workflow-1', + outputs: [{ blockId: 'agent', path: 'content', columnName: 'column-output' }], + inputMappings: [{ inputName: 'website', columnName: 'column-website' }], + }, + latestDeployment() + ) + ).toThrow( + 'Workflow group group-1 input website is not available in the latest active deployment' + ) + }) +}) + describe('table workflow carrier deadline', () => { it('preserves one absolute deadline when a later cascade group creates its controller', () => { vi.useFakeTimers() diff --git a/apps/sim/background/workflow-column-execution.ts b/apps/sim/background/workflow-column-execution.ts index 28264abd776..33c21174d70 100644 --- a/apps/sim/background/workflow-column-execution.ts +++ b/apps/sim/background/workflow-column-execution.ts @@ -54,12 +54,58 @@ import { type QueuedWorkflowGroupCellPayload, type WorkflowGroupCellPayload, } from '@/lib/table/workflow-columns' +import { flattenWorkflowOutputs } from '@/lib/workflows/blocks/flatten-outputs' +import { normalizeInputFormatValue } from '@/lib/workflows/input-format' +import type { DeployedWorkflowData } from '@/lib/workflows/persistence/utils' +import { TriggerUtils } from '@/lib/workflows/triggers/triggers' import { ResolvedSecretTraceRegistry } from '@/executor/utils/resolved-secret-trace-registry' export type { WorkflowGroupCellPayload } const logger = createLogger('TriggerWorkflowGroupCell') +/** + * Fails before workflow execution when saved table mappings are incompatible + * with the latest active deployment loaded for this cell run. + */ +export function assertWorkflowGroupMatchesLatestDeployment( + group: WorkflowGroup, + deployment: DeployedWorkflowData +): void { + const validOutputs = new Set( + flattenWorkflowOutputs(Object.values(deployment.blocks), deployment.edges).map( + (output) => `${output.blockId}::${output.path}` + ) + ) + const invalidOutput = group.outputs.find( + (output) => !validOutputs.has(`${output.blockId}::${output.path}`) + ) + if (invalidOutput) { + throw new Error( + `Workflow group ${group.id} output ${invalidOutput.blockId}::${invalidOutput.path} is not available in the latest active deployment` + ) + } + + const startCandidate = TriggerUtils.findStartBlock(deployment.blocks, 'manual') + if (!startCandidate) { + throw new Error('Workflow is missing a Start trigger') + } + + const validInputNames = new Set( + normalizeInputFormatValue(startCandidate.block.subBlocks?.inputFormat?.value).map( + (input) => input.name + ) + ) + const invalidInput = (group.inputMappings ?? []).find( + (mapping) => !validInputNames.has(mapping.inputName) + ) + if (invalidInput) { + throw new Error( + `Workflow group ${group.id} input ${invalidInput.inputName} is not available in the latest active deployment` + ) + } +} + function requirePayloadBillingAttribution( payload: WorkflowGroupCellPayload ): BillingAttributionSnapshot { @@ -389,19 +435,13 @@ async function runWorkflowAndWriteTerminal( const billingAttribution = requirePayloadBillingAttribution(payload) const timeoutController = createWorkflowGroupAttemptTimeoutController(payload, signal) const attemptSignal = timeoutController.signal - // Read from the live `group`, not the payload: in a cascade the payload is the - // first group's snapshot, so a downstream group with a different version must - // use its own setting (same reason `workflowId` is re-derived per iteration). - const deploymentMode = group.deploymentMode const requestId = `wfgrp-${executionId}` try { return await runWithRequestContext({ requestId }, async () => { const { getRowById } = await import('@/lib/table/rows/service') const { executeWorkflow } = await import('@/lib/workflows/executor/execute-workflow') - const { loadWorkflowFromNormalizedTables, loadDeployedWorkflowState } = await import( - '@/lib/workflows/persistence/utils' - ) + const { loadDeployedWorkflowState } = await import('@/lib/workflows/persistence/utils') const { buildCancelledExecution, createWorkflowCellProgressWriter, @@ -693,32 +733,22 @@ async function runWorkflowAndWriteTerminal( return 'error' } - // `deployed` groups run the workflow's latest active deployment; `live` - // (default) runs the editable draft. A `deployed` group whose workflow - // has never been deployed fails the cell — no silent fallback to draft. - let normalizedData: Awaited> - if (deploymentMode === 'deployed') { - try { - normalizedData = await loadDeployedWorkflowState(workflowId, workspaceId) - } catch (err) { - // Surface the real reason (missing deployment vs. transient DB/migration - // failure) rather than always claiming the workflow isn't deployed. - await writeState({ - status: 'error', - executionId, - jobId: null, - workflowId, - error: toError(err).message, - }) - return 'error' - } - } else { - normalizedData = await loadWorkflowFromNormalizedTables(workflowId) + let normalizedData: Awaited> + try { + normalizedData = await loadDeployedWorkflowState(workflowId, workspaceId) + assertWorkflowGroupMatchesLatestDeployment(group, normalizedData) + } catch (err) { + await writeState({ + status: 'error', + executionId, + jobId: null, + workflowId, + error: toError(err).message, + }) + return 'error' } - const startBlock = normalizedData - ? Object.values(normalizedData.blocks).find((b) => b?.type === 'start_trigger') - : undefined - if (!startBlock) { + const startCandidate = TriggerUtils.findStartBlock(normalizedData.blocks, 'manual') + if (!startCandidate) { await writeState({ status: 'error', executionId, @@ -1006,11 +1036,9 @@ async function runWorkflowAndWriteTerminal( }, executionMode: 'sync', workflowTriggerType: 'table', - triggerBlockId: startBlock.id, - // `deployed` groups execute the latest active deployment; everything - // else runs the editable draft (the table default). Matches the - // state loaded above for start-block / output-block resolution. - useDraftState: deploymentMode !== 'deployed', + triggerBlockId: startCandidate.blockId, + useDraftState: false, + workflowStateOverride: normalizedData, abortSignal: attemptSignal, onBlockStart: progressWriter.onBlockStart, onBlockComplete: progressWriter.onBlockComplete, diff --git a/apps/sim/background/workspace-file-search-dispatch.test.ts b/apps/sim/background/workspace-file-search-dispatch.test.ts new file mode 100644 index 00000000000..ef2960f0483 --- /dev/null +++ b/apps/sim/background/workspace-file-search-dispatch.test.ts @@ -0,0 +1,45 @@ +/** + * @vitest-environment node + */ +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const mocks = vi.hoisted(() => ({ + dispatch: vi.fn(), + task: vi.fn((config: unknown) => config), +})) + +vi.mock('@trigger.dev/sdk', () => ({ task: mocks.task })) +vi.mock('@/lib/workspace-files/search/dispatcher', () => ({ + dispatchWorkspaceFileSearchIndexJobs: mocks.dispatch, +})) + +import { FILE_SEARCH_DISPATCH_MAX_DURATION_SECONDS } from '@/lib/workspace-files/search/constants' +import { workspaceFileSearchDispatchTask } from '@/background/workspace-file-search-dispatch' + +describe('workspace file search dispatch task', () => { + beforeEach(() => { + vi.clearAllMocks() + }) + + it('serializes bounded dispatcher runs outside the cron request', async () => { + expect(workspaceFileSearchDispatchTask).toMatchObject({ + id: 'workspace-file-search-dispatch', + machine: 'small-1x', + maxDuration: FILE_SEARCH_DISPATCH_MAX_DURATION_SECONDS, + retry: { maxAttempts: 3 }, + queue: { + name: 'workspace-file-search-dispatch', + concurrencyLimit: 1, + }, + }) + + mocks.dispatch.mockResolvedValue({ + dispatchedFiles: 2, + backfilledFiles: 1000, + reapedClaims: 0, + lockAcquired: true, + }) + await workspaceFileSearchDispatchTask.run() + expect(mocks.dispatch).toHaveBeenCalledOnce() + }) +}) diff --git a/apps/sim/background/workspace-file-search-dispatch.ts b/apps/sim/background/workspace-file-search-dispatch.ts new file mode 100644 index 00000000000..d233486066a --- /dev/null +++ b/apps/sim/background/workspace-file-search-dispatch.ts @@ -0,0 +1,19 @@ +import { task } from '@trigger.dev/sdk' +import { FILE_SEARCH_DISPATCH_MAX_DURATION_SECONDS } from '@/lib/workspace-files/search/constants' +import { dispatchWorkspaceFileSearchIndexJobs } from '@/lib/workspace-files/search/dispatcher' + +/** + * Runs the bounded search-index control plane outside the cron request. Per-file parsing remains + * isolated in `workspace-file-search-index`; this task only backfills, claims, and enqueues work. + */ +export const workspaceFileSearchDispatchTask = task({ + id: 'workspace-file-search-dispatch', + machine: 'small-1x', + maxDuration: FILE_SEARCH_DISPATCH_MAX_DURATION_SECONDS, + retry: { maxAttempts: 3 }, + queue: { + name: 'workspace-file-search-dispatch', + concurrencyLimit: 1, + }, + run: () => dispatchWorkspaceFileSearchIndexJobs(), +}) diff --git a/apps/sim/background/workspace-file-search-index.test.ts b/apps/sim/background/workspace-file-search-index.test.ts new file mode 100644 index 00000000000..84b71627813 --- /dev/null +++ b/apps/sim/background/workspace-file-search-index.test.ts @@ -0,0 +1,68 @@ +/** + * @vitest-environment node + */ +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const mocks = vi.hoisted(() => ({ + indexWorkspaceFile: vi.fn(), + markFailed: vi.fn(), + task: vi.fn((config: unknown) => config), +})) + +vi.mock('@trigger.dev/sdk', () => ({ task: mocks.task })) +vi.mock('@/lib/workspace-files/search/indexing', () => ({ + indexWorkspaceFileForSearch: mocks.indexWorkspaceFile, + markWorkspaceFileSearchIndexFailed: mocks.markFailed, +})) + +import { + FILE_SEARCH_INDEX_GLOBAL_CONCURRENCY, + FILE_SEARCH_INDEX_MAX_DURATION_SECONDS, +} from '@/lib/workspace-files/search/constants' +import { workspaceFileSearchIndexTask } from '@/background/workspace-file-search-index' + +const payload = { + workspaceId: 'workspace-1', + fileId: 'file-1', + sourceContentUpdatedAt: '2026-08-29T12:00:00.000Z', +} + +describe('workspace file search index task', () => { + beforeEach(() => { + vi.clearAllMocks() + }) + + it('uses isolated medium workers with a hard global concurrency and duration cap', () => { + expect(workspaceFileSearchIndexTask).toMatchObject({ + id: 'workspace-file-search-index', + machine: 'medium-1x', + maxDuration: FILE_SEARCH_INDEX_MAX_DURATION_SECONDS, + retry: { maxAttempts: 3 }, + queue: { + name: 'workspace-file-search-index', + concurrencyLimit: FILE_SEARCH_INDEX_GLOBAL_CONCURRENCY, + }, + }) + }) + + it('passes the Trigger.dev abort signal to the single-revision indexer', async () => { + const signal = new AbortController().signal + mocks.indexWorkspaceFile.mockResolvedValue(undefined) + + await workspaceFileSearchIndexTask.run(payload, { signal }) + + expect(mocks.indexWorkspaceFile).toHaveBeenCalledWith(payload, signal) + }) + + it('marks the revision failed only from the terminal onFailure hook', async () => { + mocks.indexWorkspaceFile.mockRejectedValue(new Error('retryable parser failure')) + + await expect( + workspaceFileSearchIndexTask.run(payload, { signal: new AbortController().signal }) + ).rejects.toThrow('retryable parser failure') + expect(mocks.markFailed).not.toHaveBeenCalled() + + await workspaceFileSearchIndexTask.onFailure({ payload }) + expect(mocks.markFailed).toHaveBeenCalledWith(payload) + }) +}) diff --git a/apps/sim/background/workspace-file-search-index.ts b/apps/sim/background/workspace-file-search-index.ts new file mode 100644 index 00000000000..fbba50476ca --- /dev/null +++ b/apps/sim/background/workspace-file-search-index.ts @@ -0,0 +1,30 @@ +import { task } from '@trigger.dev/sdk' +import { + FILE_SEARCH_INDEX_GLOBAL_CONCURRENCY, + FILE_SEARCH_INDEX_MAX_DURATION_SECONDS, +} from '@/lib/workspace-files/search/constants' +import { + indexWorkspaceFileForSearch, + markWorkspaceFileSearchIndexFailed, + type WorkspaceFileSearchIndexPayload, +} from '@/lib/workspace-files/search/indexing' + +/** + * Builds one immutable workspace-file search revision. PostgreSQL owns the durable state; this + * task only supplies isolated compute, retries, and a hard global execution cap. + */ +export const workspaceFileSearchIndexTask = task({ + id: 'workspace-file-search-index', + machine: 'medium-1x', + maxDuration: FILE_SEARCH_INDEX_MAX_DURATION_SECONDS, + retry: { maxAttempts: 3 }, + queue: { + name: 'workspace-file-search-index', + concurrencyLimit: FILE_SEARCH_INDEX_GLOBAL_CONCURRENCY, + }, + run: (payload: WorkspaceFileSearchIndexPayload, { signal }) => + indexWorkspaceFileForSearch(payload, signal), + onFailure: async ({ payload }) => { + await markWorkspaceFileSearchIndexFailed(payload) + }, +}) diff --git a/apps/sim/blocks/blocks.test.ts b/apps/sim/blocks/blocks.test.ts index 1ab2265f55e..935f70365ea 100644 --- a/apps/sim/blocks/blocks.test.ts +++ b/apps/sim/blocks/blocks.test.ts @@ -170,6 +170,7 @@ describe.concurrent('Blocks Module', () => { expect(block?.subBlocks[0].options?.map((option) => option.id)).toEqual([ 'file_read', 'file_get_content', + 'file_search', 'file_fetch', 'file_write', 'file_append', diff --git a/apps/sim/blocks/blocks/elasticsearch.ts b/apps/sim/blocks/blocks/elasticsearch.ts index 24c1acdba5f..5777db503b2 100644 --- a/apps/sim/blocks/blocks/elasticsearch.ts +++ b/apps/sim/blocks/blocks/elasticsearch.ts @@ -483,15 +483,32 @@ Return ONLY valid JSON - no explanations, no markdown code blocks.`, condition: { field: 'operation', value: 'elasticsearch_cluster_health' }, }, - // Cluster health timeout + // Cluster health timeout. The subBlock id stays `timeout` so saved workflow + // state keeps resolving; `tools.config.params` remaps it to `clusterTimeout` + // and clears the transport's reserved `timeout` key. { id: 'timeout', - title: 'Timeout (seconds)', + title: 'Timeout', type: 'short-input', - placeholder: '30', + placeholder: '30s', + mode: 'advanced', condition: { field: 'operation', value: 'elasticsearch_cluster_health' }, }, + // Include system indices + { + id: 'includeSystemIndices', + title: 'Include System Indices', + type: 'dropdown', + options: [ + { label: 'No', id: '' }, + { label: 'Yes', id: 'true' }, + ], + value: () => '', + mode: 'advanced', + condition: { field: 'operation', value: 'elasticsearch_list_indices' }, + }, + // Retry on conflict { id: 'retryOnConflict', @@ -528,9 +545,15 @@ Return ONLY valid JSON - no explanations, no markdown code blocks.`, if (params.size) result.size = Number(params.size) if (params.from) result.from = Number(params.from) if (params.retryOnConflict) result.retryOnConflict = Number(params.retryOnConflict) - if (params.timeout && typeof params.timeout === 'string') { - result.timeout = params.timeout.endsWith('s') ? params.timeout : `${params.timeout}s` + + if (params.includeSystemIndices === 'true') result.includeSystemIndices = true + + const rawTimeout = typeof params.timeout === 'string' ? params.timeout.trim() : '' + if (rawTimeout) { + result.clusterTimeout = /^\d+$/.test(rawTimeout) ? `${rawTimeout}s` : rawTimeout } + result.timeout = undefined + return result }, }, @@ -559,7 +582,14 @@ Return ONLY valid JSON - no explanations, no markdown code blocks.`, mappings: { type: 'string', description: 'Index mappings as JSON' }, refresh: { type: 'string', description: 'Refresh policy' }, waitForStatus: { type: 'string', description: 'Wait for cluster status' }, - timeout: { type: 'string', description: 'Timeout for wait operations' }, + timeout: { + type: 'string', + description: 'How long Elasticsearch waits for the cluster to reach the requested status', + }, + includeSystemIndices: { + type: 'string', + description: 'Include Elasticsearch system indices when listing', + }, retryOnConflict: { type: 'number', description: 'Retry attempts on conflict' }, }, @@ -581,8 +611,14 @@ Return ONLY valid JSON - no explanations, no markdown code blocks.`, items: { type: 'json', description: 'Bulk operation results' }, // Count outputs count: { type: 'number', description: 'Document count' }, + _shards: { + type: 'json', + description: 'Shard statistics (total, successful, skipped, failed)', + }, // Index outputs acknowledged: { type: 'boolean', description: 'Whether operation was acknowledged' }, + // List indices outputs + message: { type: 'string', description: 'Summary message about the indices listed' }, // Cluster outputs cluster_name: { type: 'string', description: 'Cluster name' }, status: { type: 'string', description: 'Cluster health status' }, diff --git a/apps/sim/blocks/blocks/file.test.ts b/apps/sim/blocks/blocks/file.test.ts index a5f3b46c4a3..41c1e96d95d 100644 --- a/apps/sim/blocks/blocks/file.test.ts +++ b/apps/sim/blocks/blocks/file.test.ts @@ -62,11 +62,54 @@ describe('FileV5Block', () => { expect(FileV5Block.tools.config.tool({ operation: 'file_fetch' })).toBe('file_fetch') expect(FileV5Block.tools.config.tool({ operation: 'file_write' })).toBe('file_write') expect(FileV5Block.tools.config.tool({ operation: 'file_append' })).toBe('file_append') + expect(FileV5Block.tools.config.tool({ operation: 'file_search' })).toBe('file_search') }) + it('keeps the builder-configured search limit as a fixed hard cap', () => { + expect( + buildParams({ + operation: 'file_search', + query: '', + maxResults: '25', + }) + ).toEqual({ query: '', maxResults: 25 }) + + const query = FileV5Block.subBlocks.find((subBlock) => subBlock.id === 'query') + const maxResults = FileV5Block.subBlocks.find((subBlock) => subBlock.id === 'maxResults') + expect(query?.paramVisibility).toBe('user-or-llm') + expect(maxResults?.paramVisibility).toBe('user-only') + expect(query?.canonicalParamId).toBeUndefined() + expect(maxResults?.canonicalParamId).toBeUndefined() + expect(maxResults?.value?.()).toBe('50') + }) + + it('uses the default search cap when the builder field is cleared', () => { + expect( + buildParams({ + operation: 'file_search', + query: 'needle', + maxResults: '', + }) + ).toEqual({ query: 'needle', maxResults: 50 }) + }) + + it.each(['10.5', '10results', '0', '201'])( + 'rejects invalid builder-configured search cap %s', + (maxResults) => { + expect(() => + buildParams({ + operation: 'file_search', + query: 'needle', + maxResults, + }) + ).toThrow('Maximum Results must be an integer between 1 and 200') + } + ) + it('read returns only the files output (no redundant file)', () => { expect(FileV5Block.outputs.files).toBeDefined() expect(FileV5Block.outputs.contents).toBeDefined() + expect(FileV5Block.outputs.results).toBeDefined() expect(FileV5Block.outputs.file).toBeUndefined() }) diff --git a/apps/sim/blocks/blocks/file.ts b/apps/sim/blocks/blocks/file.ts index e566a746dbd..6328cdc7e20 100644 --- a/apps/sim/blocks/blocks/file.ts +++ b/apps/sim/blocks/blocks/file.ts @@ -80,6 +80,9 @@ const APPEND_FILE_FIELD = ['appendFile', 'appendFileName'] as const const COMPRESS_FILE_FIELD = ['compressFile', 'compressFileId'] as const const DECOMPRESS_FILE_FIELD = ['decompressFile', 'decompressFileId'] as const const SHARE_FILE_FIELD = ['shareFile', 'shareFileId'] as const +/* Text and file are mutually exclusive sources, so the clause names whichever + one the card actually carries. */ +const WRITE_CONTENT_FIELD = ['content', 'writeFile', 'writeFileId'] as const export const FileBlock: BlockConfig = { type: 'file', @@ -896,17 +899,19 @@ export const FileV5Block: BlockConfig = { type: 'file_v5', name: 'File', description: - 'Read, get content, fetch, write, append, compress, decompress, and manage sharing for files', + 'Read, search, get content, fetch, write, append, compress, decompress, and manage sharing for files', longDescription: - 'Read workspace file objects, extract the text content of files, fetch and parse files from URLs with optional headers, write new workspace files, append content to existing files, compress files into a .zip archive, extract a .zip archive into the workspace, or manage the public share link for a file.', + 'Read workspace file objects, search indexed text across all active workspace files, extract the text content of files, fetch and parse files from URLs with optional headers, write new workspace files, append content to existing files, compress files into a .zip archive, extract a .zip archive into the workspace, or manage the public share link for a file.', hideFromToolbar: false, bestPractices: ` - Read returns workspace file objects in the "files" output and does NOT include their text. Use it to pick files or pass file references downstream (e.g. as attachments). - Get Content is how you read file text. It accepts file objects or canonical file IDs and returns a "contents" array with one extracted text string per file (PDF, DOCX, CSV, etc. are parsed automatically). - To read the text of files produced by another block, chain into Get Content: set its file input to the upstream file output, e.g. , , or . Never assume Read (or any file-object output) already contains the text. - Get Content's "contents" can be large; it is persisted through the execution large-value system automatically, so prefer it over inlining file text any other way. + - Search finds literal text across all active workspace files and returns structured results with fileId, lineNumber, and text. Lowercase queries are case-insensitive; adding any uppercase letter makes the search case-sensitive. + - Search is eventually consistent. Check "complete" and "indexStatus" when pending, failed, skipped, or partially indexed files matter to the task. - Use Fetch for external file URLs. Add headers for authenticated downloads, for example Slack private file URLs require an Authorization Bearer token. - - Use Write to create a new workspace file and Append to add content to an existing one. + - Use Write to create a new workspace file and Append to add content to an existing one. Write adds a numeric suffix when the name is taken; turn on "Overwrite Existing File" to replace the contents of the file at that exact path (folder and name) instead — a same-named file in another folder is left alone. - Use Compress to bundle one or more files into a single .zip archive stored in the workspace. The new archive is returned in the "files" output. - Use Decompress to extract a .zip archive back into the workspace; the extracted files are returned in the "files" output, ready to chain into Get Content or downstream blocks. `, @@ -918,10 +923,11 @@ export const FileV5Block: BlockConfig = { file_get_content: [ { text: 'Extract text from', field: GET_CONTENT_FILE_FIELD, core: true }, ], + file_search: [{ text: 'Search workspace files for', field: 'query', core: true }], file_fetch: [{ text: 'Fetch and parse', field: 'fileUrl', core: true }], file_write: [ { text: 'Create', field: 'fileName', core: true }, - { text: 'containing', field: 'content' }, + { text: 'containing', field: WRITE_CONTENT_FIELD }, ], file_append: [ { text: 'Append', field: 'appendContent', core: true }, @@ -947,6 +953,7 @@ export const FileV5Block: BlockConfig = { options: [ { label: 'Read', id: 'file_read' }, { label: 'Get Content', id: 'file_get_content' }, + { label: 'Search', id: 'file_search' }, { label: 'Fetch', id: 'file_fetch' }, { label: 'Write', id: 'file_write' }, { label: 'Append', id: 'file_append' }, @@ -1000,6 +1007,28 @@ export const FileV5Block: BlockConfig = { condition: { field: 'operation', value: 'file_get_content' }, required: { field: 'operation', value: 'file_get_content' }, }, + { + id: 'query', + title: 'Query', + type: 'short-input' as SubBlockType, + placeholder: 'Text to find across workspace files', + description: 'Literal search text, 3-512 characters. Leave blank for the agent to supply.', + condition: { field: 'operation', value: 'file_search' }, + required: { field: 'operation', value: 'file_search' }, + paramVisibility: 'user-or-llm', + }, + { + id: 'maxResults', + title: 'Maximum Results', + type: 'short-input' as SubBlockType, + placeholder: '50', + description: 'Hard cap for results returned to the agent (1-200).', + value: () => '50', + condition: { field: 'operation', value: 'file_search' }, + required: { field: 'operation', value: 'file_search' }, + mode: 'advanced', + paramVisibility: 'user-only', + }, { id: 'fileUrl', title: 'File URL', @@ -1031,7 +1060,25 @@ export const FileV5Block: BlockConfig = { type: 'long-input' as SubBlockType, placeholder: 'File content to write...', condition: { field: 'operation', value: 'file_write' }, - required: { field: 'operation', value: 'file_write' }, + }, + { + id: 'writeFile', + title: 'File', + type: 'file-upload' as SubBlockType, + canonicalParamId: 'writeFileInput', + acceptedTypes: '*', + placeholder: 'Store an existing file', + mode: 'basic', + condition: { field: 'operation', value: 'file_write' }, + }, + { + id: 'writeFileId', + title: 'File', + type: 'short-input' as SubBlockType, + canonicalParamId: 'writeFileInput', + placeholder: 'File from an earlier block', + mode: 'advanced', + condition: { field: 'operation', value: 'file_write' }, }, { id: 'contentType', @@ -1041,6 +1088,12 @@ export const FileV5Block: BlockConfig = { condition: { field: 'operation', value: 'file_write' }, mode: 'advanced', }, + { + id: 'overwrite', + title: 'Overwrite Existing File', + type: 'switch' as SubBlockType, + condition: { field: 'operation', value: 'file_write' }, + }, { id: 'appendFile', title: 'File', @@ -1193,6 +1246,7 @@ export const FileV5Block: BlockConfig = { access: [ 'file_read', 'file_get_content', + 'file_search', 'file_fetch', 'file_write', 'file_append', @@ -1205,11 +1259,38 @@ export const FileV5Block: BlockConfig = { params: (params) => { const operation = params.operation || 'file_read' + if (operation === 'file_search') { + const maxResultsInput = + params.maxResults == null || params.maxResults === '' ? 50 : params.maxResults + const maxResults = Number(maxResultsInput) + if (!Number.isInteger(maxResults) || maxResults < 1 || maxResults > 200) { + throw new Error('Maximum Results must be an integer between 1 and 200') + } + return { + query: params.query, + maxResults, + } + } + if (operation === 'file_write') { + // Writing stores one file, so the single form. + const fileInput = normalizeFileInput(params.writeFileInput, { single: true }) + // The contract counts any defined `content` as "text was provided", and + // an untouched Content box serializes as an empty string — so sending it + // unconditionally would make every file write collide with its own empty + // text box. The selected file is what disambiguates: with one present, + // an empty Content box means "not used" and is dropped, while a + // non-empty one is still forwarded so the contract can report that both + // were filled. With no file, `content` always goes through, which keeps + // writing a deliberately empty text file possible. + const contentText = typeof params.content === 'string' ? params.content : undefined + const omitContent = Boolean(fileInput) && !contentText return { fileName: params.fileName, - content: params.content, + ...(omitContent ? {} : { content: params.content }), + ...(fileInput ? { fileInput } : {}), contentType: params.contentType, + overwrite: params.overwrite === true || params.overwrite === 'true', workspaceId: params._context?.workspaceId, } } @@ -1417,8 +1498,10 @@ export const FileV5Block: BlockConfig = { inputs: { operation: { type: 'string', - description: 'Operation to perform (read, get content, fetch, write, or append)', + description: 'Operation to perform (read, search, get content, fetch, write, or append)', }, + query: { type: 'string', description: 'Literal workspace file search query' }, + maxResults: { type: 'number', description: 'Hard maximum search results (1-200)' }, readFileInput: { type: 'json', description: 'Selected workspace file or canonical file ID for read', @@ -1432,7 +1515,15 @@ export const FileV5Block: BlockConfig = { fileType: { type: 'string', description: 'File type for fetch' }, fileName: { type: 'string', description: 'Name for a new file (write)' }, content: { type: 'string', description: 'File content to write' }, + writeFileInput: { + type: 'json', + description: 'An existing file to store in the workspace, instead of text content', + }, contentType: { type: 'string', description: 'MIME content type for write' }, + overwrite: { + type: 'boolean', + description: 'Replace an existing file with the same name instead of creating a copy (write)', + }, appendFileInput: { type: 'json', description: 'File to append to' }, appendContent: { type: 'string', description: 'Content to append to file' }, compressInput: { @@ -1468,6 +1559,24 @@ export const FileV5Block: BlockConfig = { type: 'array', description: 'Array of file text contents, one entry per file (get content)', }, + results: { + type: 'array', + description: 'Matching lines as objects with fileId, lineNumber, and text fields (search)', + }, + count: { type: 'number', description: 'Returned matching line count (search)' }, + truncated: { + type: 'boolean', + description: 'Whether more search matches exist beyond the configured cap', + }, + complete: { + type: 'boolean', + description: 'Whether all current workspace file revisions are indexed without failures', + }, + indexStatus: { + type: 'json', + description: + 'Workspace search-index coverage counts: readyFiles, pendingFiles, failedFiles, skippedFiles, and partialFiles', + }, combinedContent: { type: 'string', description: 'All fetched file contents merged into a single text string (fetch)', diff --git a/apps/sim/blocks/blocks/function.test.ts b/apps/sim/blocks/blocks/function.test.ts new file mode 100644 index 00000000000..96cc313d258 --- /dev/null +++ b/apps/sim/blocks/blocks/function.test.ts @@ -0,0 +1,33 @@ +/** + * @vitest-environment node + */ +import { describe, expect, it } from 'vitest' +import { USER_FILE_ACCESSIBLE_PROPERTIES } from '@/lib/workflows/types' +import { FunctionBlock } from '@/blocks/blocks/function' + +describe('Function block file surface', () => { + it('has no file configuration fields', () => { + // Files reach the sandbox by being referenced in code as + // — the same way every other block output is referenced. + // A dedicated field would be a second way to say the same thing, and would + // need a home in the panel that the reference syntax does not. + const ids = FunctionBlock.subBlocks.map((subBlock) => subBlock.id) + + expect(ids).not.toContain('files') + expect(ids).not.toContain('uploadedFiles') + expect(ids).not.toContain('collectOutputFiles') + expect(FunctionBlock.inputs).not.toHaveProperty('files') + expect(FunctionBlock.inputs).not.toHaveProperty('collectOutputFiles') + }) + + it('returns harvested files so downstream blocks can consume them', () => { + expect(FunctionBlock.outputs.files).toMatchObject({ type: 'file[]' }) + }) + + it('offers path alongside base64 as a referenceable file property', () => { + // This is what puts `.path` in the tag dropdown: block-outputs.ts maps the + // list into `${path}.${prop}` suggestions. + expect(USER_FILE_ACCESSIBLE_PROPERTIES).toContain('path') + expect(USER_FILE_ACCESSIBLE_PROPERTIES).toContain('base64') + }) +}) diff --git a/apps/sim/blocks/blocks/function.ts b/apps/sim/blocks/blocks/function.ts index 930d7c233dd..ebf685b687c 100644 --- a/apps/sim/blocks/blocks/function.ts +++ b/apps/sim/blocks/blocks/function.ts @@ -1,6 +1,7 @@ import { CodeIcon } from '@/components/icons' import { isSandboxesEnabled } from '@/lib/core/config/env-flags' import { CodeLanguage, getLanguageDisplayName } from '@/lib/execution/languages' +import { SANDBOX_OUTPUT_DIR } from '@/lib/execution/remote-sandbox/sandbox-paths' import type { BlockConfig } from '@/blocks/types' import type { CodeExecutionOutput } from '@/tools/function/types' @@ -17,6 +18,9 @@ export const FunctionBlock: BlockConfig = { - Shell code runs CLI commands in a remote sandbox. - To import third-party packages or add curated CLI tools, create a sandbox in Settings > Sandboxes and select it under the block's advanced options. Without one, only the default image's packages and commands are available. - Can reference workflow variables using syntax as usual within code. Avoid XML/HTML tags. + - To read a file from an earlier block, reference its path: mounts the file and resolves to its location on the sandbox filesystem, which any language can open. Use instead when you only want the contents inline in JavaScript. + - Anything the code writes to ${SANDBOX_OUTPUT_DIR} is returned as \`files\`, ready to attach to an email or upload without any extra step. + - Referencing a file path runs the block in the remote sandbox, so it is slower to start than a plain local JavaScript run. `, docsLink: 'https://docs.sim.ai/workflows/blocks/function', category: 'blocks', @@ -174,5 +178,9 @@ try { type: 'string', description: 'Console log output and debug messages from function execution', }, + files: { + type: 'file[]', + description: `Files the code wrote to ${SANDBOX_OUTPUT_DIR}, ready to attach or upload downstream`, + }, }, } diff --git a/apps/sim/blocks/blocks/github.ts b/apps/sim/blocks/blocks/github.ts index b38c15f26c5..71547b70d64 100644 --- a/apps/sim/blocks/blocks/github.ts +++ b/apps/sim/blocks/blocks/github.ts @@ -9,6 +9,44 @@ import { getTrigger } from '@/triggers' /** Reviewers can be named individually or by team slug; either identifies the request. */ const REVIEWER_FIELD = ['reviewers', 'team_reviewers'] as const +/** + * Block subBlock ids that differ from the tool param they feed, each scoped to + * the operations whose tool declares that target. `sort` has two sources and + * `title`/`description`/`state` share their names with fields on other + * operations, so the scoping is what keeps them from colliding. + * + * `toBoolean` marks a dropdown feeding a boolean tool param: a dropdown stores + * its option id, so the value arrives as the string 'true'/'false' and the + * generic handler only JSON-parses `json`/`array` inputs. + */ +const GITHUB_PARAM_ALIASES: ReadonlyArray<{ + from: string + to: string + operations: readonly string[] + toBoolean?: true +}> = [ + { + from: 'reaction_content', + to: 'content', + operations: ['github_create_issue_reaction', 'github_create_comment_reaction'], + }, + { + from: 'milestone_title', + to: 'title', + operations: ['github_create_milestone', 'github_update_milestone'], + }, + { + from: 'milestone_description', + to: 'description', + operations: ['github_create_milestone', 'github_update_milestone'], + }, + { from: 'milestone_state', to: 'state', operations: ['github_list_milestones'] }, + { from: 'milestone_sort', to: 'sort', operations: ['github_list_milestones'] }, + { from: 'fork_name', to: 'name', operations: ['github_fork_repo'] }, + { from: 'fork_sort', to: 'sort', operations: ['github_list_forks'] }, + { from: 'gist_public', to: 'public', operations: ['github_create_gist'], toBoolean: true }, +] + export const GitHubBlock: BlockConfig = { type: 'github', name: 'GitHub (Legacy)', @@ -2279,6 +2317,58 @@ Return ONLY the timestamp string - no explanations, no quotes, no extra text.`, return 'github_repo_info' } }, + /** + * Bridges the subBlock ids that do not match their tool's param name. + * + * A tool param is populated only when a subBlock's `id` equals it — the + * serializer keys values by subBlock id, and nothing else renames them. + * Each aliased field below renders, accepts input, and then arrives under + * a name its tool never reads. + * + * Every alias is scoped to the operations whose tool actually declares + * the target param, and that scoping is load-bearing. Seven of these + * sources are `mode: 'advanced'`, and `shouldSerializeSubBlock` + * (`serializer/index.ts:91-93`) serializes a non-empty advanced field + * WITHOUT evaluating its condition. So a `milestone_title` left over from + * an earlier operation is still in `params` after the user switches to, + * say, Update PR — and an unscoped alias would rewrite it to `title` and + * clobber the PR's own title with stale milestone data. + * + * Presence is tested rather than truthiness so that a deliberate `false` + * or `'false'` is not mistaken for an unset field; only nullish and empty + * defer to the tool's own default. + * + * `generic-handler.ts` merges `{ ...inputs, ...params(inputs) }` and + * `providers/utils.ts` installs this as the provider `paramsTransform`, + * spreading over the model's tool-call arguments — so emitting a key the + * block did not supply would clobber a model-supplied value on the agent + * path. + * + * On the agent tool-calling path `operation` is not part of the params + * this receives: `providers/utils.ts` spreads it in for the tool-selection + * call (`:736-739`) but builds the transform's input from `block.params` + * alone (`:776`). Every alias therefore skips there, which is the same + * behaviour as before this mapper existed - the agent path already works + * because a model supplies `content`/`title`/`sort` by their real names. + * That gap is shared by every block whose mapper branches on + * `params.operation`, so closing it belongs in the provider layer rather + * than here. + */ + params: (params) => { + const result: Record = {} + const operation = typeof params.operation === 'string' ? params.operation : '' + + const isSet = (value: unknown) => value !== undefined && value !== null && value !== '' + + for (const alias of GITHUB_PARAM_ALIASES) { + if (!alias.operations.includes(operation)) continue + const value = params[alias.from] + if (!isSet(value)) continue + result[alias.to] = alias.toBoolean ? value === true || value === 'true' : value + } + + return result + }, }, }, inputs: { diff --git a/apps/sim/components/emails/credential-groups/credential-group-invitation-email.tsx b/apps/sim/components/emails/credential-groups/credential-group-invitation-email.tsx index 1e2383a9579..0b0e4f04650 100644 --- a/apps/sim/components/emails/credential-groups/credential-group-invitation-email.tsx +++ b/apps/sim/components/emails/credential-groups/credential-group-invitation-email.tsx @@ -5,7 +5,8 @@ import { getBrandConfig } from '@/ee/whitelabeling' interface CredentialGroupInvitationEmailProps { recipientEmail: string - inviterName: string + /** Absent when a workflow issued the invitation: there is no person to name. */ + inviterName?: string workspaceName: string credentialGroupName: string invitationLink: string @@ -22,14 +23,26 @@ export function CredentialGroupInvitationEmail({ return ( Hello, - {inviterName} invited {recipientEmail} to connect accounts - for {credentialGroupName} in the {workspaceName} workspace - on {brand.name}. + {inviterName ? ( + <> + {inviterName} invited {recipientEmail} + + ) : ( + <> + {recipientEmail} has been invited + + )}{' '} + to connect accounts for {credentialGroupName} in the{' '} + {workspaceName} workspace on {brand.name}. diff --git a/apps/sim/components/emails/credential-groups/render.ts b/apps/sim/components/emails/credential-groups/render.ts index 63779541237..f5bc8188fb0 100644 --- a/apps/sim/components/emails/credential-groups/render.ts +++ b/apps/sim/components/emails/credential-groups/render.ts @@ -3,7 +3,7 @@ import { CredentialGroupInvitationEmail } from '@/components/emails/credential-g export async function renderCredentialGroupInvitationEmail(params: { recipientEmail: string - inviterName: string + inviterName?: string workspaceName: string credentialGroupName: string invitationLink: string diff --git a/apps/sim/components/emails/subjects.ts b/apps/sim/components/emails/subjects.ts index e2d5f71fa95..c803df0eaab 100644 --- a/apps/sim/components/emails/subjects.ts +++ b/apps/sim/components/emails/subjects.ts @@ -110,10 +110,16 @@ export function getOtpSubject(resourceLabel: string): string { return `Verification code for ${resourceLabel}` } -/** Names both the inviter and workspace so an external recipient can identify the request. */ +/** + * Names the workspace so an external recipient can identify the request, and the + * inviter when there is one — a workflow-issued invitation has no person to name. + */ export function getCredentialGroupInvitationSubject( - inviterName: string, + inviterName: string | undefined, workspaceName: string ): string { - return `${inviterName} invited you to connect accounts for ${workspaceName} on ${getBrandConfig().name}` + const brandName = getBrandConfig().name + return inviterName + ? `${inviterName} invited you to connect accounts for ${workspaceName} on ${brandName}` + : `You have been invited to connect accounts for ${workspaceName} on ${brandName}` } diff --git a/apps/sim/content/library/ai-agents-for-marketing-automation/index.mdx b/apps/sim/content/library/ai-agents-for-marketing-automation/index.mdx new file mode 100644 index 00000000000..1ba3e8eee7d --- /dev/null +++ b/apps/sim/content/library/ai-agents-for-marketing-automation/index.mdx @@ -0,0 +1,127 @@ +--- +slug: ai-agents-for-marketing-automation +title: 'AI Agents for Marketing Automation: Building Agentic Workflows Beyond HubSpot and Zapier' +description: 'How to build agentic marketing workflows that go beyond HubSpot, ActiveCampaign, Zapier, Make, and n8n—where AI reasoning belongs, where fixed rules belong, and four workflows worth building.' +date: 2026-08-29 +updated: 2026-08-29 +authors: + - andrew +readingTime: 10 +tags: [Marketing Automation, AI Agents, Agentic Workflows, Sim] +ogImage: /library/ai-agents-for-marketing-automation/cover.jpg +canonical: https://www.sim.ai/library/ai-agents-for-marketing-automation +draft: false +faq: + - q: "Do I need to replace HubSpot to use AI agents?" + a: "An AI agent can add reasoning and cross-tool actions while HubSpot remains your CRM. With Sim, you can use CRM data to classify or draft content, then send the result to a connected tool. You keep HubSpot's contact records and lifecycle management while adding custom workflows around them." + - q: "Can Zapier or Make build a real AI agent?" + a: "An AI agent interprets context and chooses bounded actions, and both Zapier and Make can run agents within their automation products. Sim places agent reasoning and deterministic steps in one workflow graph, while Zapier places agents beside Zaps and Make adds AI Agent blocks inside scenarios. This distinction helps you choose between connector-focused automation and a graph that treats reasoning and fixed controls as peers." + - q: "What's the difference between an AI agent and workflow automation?" + a: "Workflow automation executes predefined rules, while an AI agent interprets context and makes bounded decisions. In Sim, an Agent block can classify lead intent before a deterministic branch assigns the lead. Combining both approaches provides contextual judgment and predictable execution in the same workflow." + - q: "Do agentic workflows require engineering resources?" + a: "Agentic workflows require you to define inputs and decision boundaries. In Sim, we provide a visual builder and Chat for creating workflows without requiring you to write every step as code. You can still add technical control when a workflow needs custom logic or integrations." + - q: "Can an agent post directly to Slack or social media?" + a: "An agent can draft content and pass it to connected Slack or social actions. With a Sim workflow, you can add an approval step before a deterministic action schedules or publishes the post. You can require review for sensitive channels and automate routine publishing elsewhere." +--- + +## TL;DR + +- [HubSpot](https://www.hubspot.com/products/marketing) and [ActiveCampaign](https://www.activecampaign.com/marketing-automation) work well for CRM and email automation, but their AI primarily operates within each platform's data model. +- [Zapier](https://zapier.com/agents), [Make](https://www.make.com/en/ai-agents), and [n8n](https://docs.n8n.io/advanced-ai/) connect more tools, but their workflow engines often use AI as another step rather than as the reasoning layer. +- Build agentic workflows when work spans tools and requires judgment, such as when drafting content or routing leads. In Sim, we combine agent reasoning with fixed workflow logic in one graph. +- Keep point tools for ready-made CRM and lifecycle features, and add agentic workflows when decisions require context from multiple systems. + +## Why bolted-on AI hits a ceiling in marketing ops + +Marketing ops teams hit the limits of bolted-on AI when campaign work crosses platform boundaries. [HubSpot Breeze can draft emails and summarize records](https://www.hubspot.com/products/artificial-intelligence). However, [HubSpot workflows follow configured triggers, actions, and branches](https://knowledge.hubspot.com/workflows/create-workflows) rather than making judgment-based decisions during execution. Breeze also works mainly at the individual-record level, which limits its ability to detect patterns across related CRM records ([workflow analysis](https://cotera.co/articles/hubspot-ai-crm-automation)). + +Breeze is designed around HubSpot's platform rather than as a shared runtime inside other products. A campaign that involves LinkedIn Campaign Manager or a WordPress site therefore uses [separate integrations and workflow logic](https://ecosystem.hubspot.com/marketplace/apps). HubSpot can accelerate tasks within its own data model, but it does not provide one reasoning layer across the marketing stack. + +ActiveCampaign follows a similar pattern. Its AI features, including [Predictive Sending](https://help.activecampaign.com/hc/en-us/articles/360001958940-Predictive-sending) and [Win Probability scoring](https://help.activecampaign.com/hc/en-us/articles/360001426799-Win-Probability), support predictive sending and lead scoring within ActiveCampaign. As work spreads across external reporting systems and multiple business units, the platform offers limited multi-object analysis and cross-system governance. Automation Strategists rates ActiveCampaign's auditability as low to medium and describes its agent execution capabilities as limited ([marketing automation comparison](https://automationstrategists.com/blog/ai-in-marketing-automation-tools/)). + +Task-level AI performs a defined job inside one product. An [agentic workflow](https://www.sim.ai/library/what-is-an-agentic-workflow) evaluates context from connected tools and chooses an action. Controlled workflow steps then execute that action. For example, an agent could compare CRM activity with campaign engagement and explain a lead classification. A fixed branch could then route the lead. A shared reasoning layer gives you control over decisions that draw on data no single marketing platform can access. + +## Can AI agents replace marketing automation tools like HubSpot? + +AI agents and CRM platforms solve different layers of marketing operations. [HubSpot](https://www.hubspot.com/products/crm) and [ActiveCampaign](https://www.activecampaign.com/platform) already manage contact records and campaign delivery. For most marketing teams, that makes agents a complement to HubSpot or ActiveCampaign rather than a replacement. + +Agentic workflows serve a different role. An agent drafts content, classifies information, or chooses a route based on context. Deterministic steps then carry out predictable actions, such as updating a CRM field or sending an approved message. For example, an agent can assess a lead using form responses and enrichment data, then explain its classification. A fixed branch can assign the lead to the correct sales queue. + +Adding an agent does not resolve inconsistent CRM data or unclear ownership across systems. Define which platform owns each field, which actions require approval, and what the agent may read or change. HubSpot or ActiveCampaign can enforce lifecycle and record rules, while the agent handles decisions that require context from other tools. Our guide to [AI agents for sales and CRM automation](https://www.sim.ai/library/best-ai-agents-sales-crm-automation) explores this hybrid approach further. + +With [Sim](https://sim.ai), you can add a reasoning and action layer beside those platforms. HubSpot or ActiveCampaign can remain the system of record, while a Sim agent interprets context across connected tools. Fixed workflow blocks can then update the CRM or send content to Slack for approval. You keep the marketing infrastructure and add custom reasoning where native workflows rely on predefined rules. + +## Why Zapier, Make, and n8n remain automation-first + +Zapier, Make, and n8n come closer to agentic marketing automation than CRM-centered tools because they connect work across applications. All three products still start with automation. [Zapier places Agents beside Zaps](https://zapier.com/agents), while [Make adds AI Agent modules to visual scenarios](https://www.make.com/en/ai-agents). [n8n inserts AI nodes into its node-based execution engine](https://docs.n8n.io/advanced-ai/). Each product can support model calls and tool use, but users still construct the surrounding workflow through traditional workflow controls. + +If your main need is conventional cross-app automation, compare these platforms based on connector breadth, workflow design, and hosting requirements. Choose Zapier when [broad app coverage](https://zapier.com/apps) and no-code trigger-action automation are priorities. Choose Make when you want a [visual canvas for branching and repeated operations](https://help.make.com/get-started-with-make). Choose n8n when [self-hosting](https://docs.n8n.io/hosting/) and [code extensions](https://docs.n8n.io/code/) are primary requirements. See our comparison of the [best Zapier alternatives](https://www.sim.ai/library/best-zapier-alternatives) for a broader look at these tradeoffs. + +Before choosing a platform, determine whether the platform lets an agent use context to select the next action within the graph while fixed blocks enforce approvals and business rules. If the workflow only feeds prepared input to an AI step, reasoning does not control the sequence. + +In [Sim](https://sim.ai), we treat agent reasoning and deterministic logic as peers in one graph. For example, an Agent block can classify a lead and explain the route it chose. Branches can then enforce territory rules and update the CRM. The workflow can notify the assigned rep after routing the lead. In Zapier, Make, and n8n, you build the same pattern by placing agent features inside or beside their existing automation structures. + +## Four agentic marketing workflows worth building + +Each workflow uses an Agent block for interpretation or judgment and fixed steps for approved actions. You can build these workflows with existing marketing tools rather than replacing the underlying CRM or publishing platforms. + +### Content repurposing: one asset, many channels + +A webinar transcript can supply several channel-specific drafts without forcing one model to control publication. A Sim Agent block reads the transcript, identifies its main claims, and drafts copy for each requested destination. For example, LinkedIn may receive a concise post with a professional tone, while an email newsletter receives a longer summary tied to the original asset. + +The Agent block returns structured fields for the channel, format, and draft. Branches then route each draft to the correct scheduling or publishing step. Because fixed branches control the destination, the model cannot decide where content gets posted or substitute an unapproved channel. + +You can place a human approval step between drafting and publication. An editor reviews the copy in Slack or another connected tool, then approves, rejects, or requests changes. Approved drafts continue to the scheduled posting step, while rejected drafts return for revision or stop the workflow. + +### Campaign reporting that explains itself + +Campaign reports become more useful when a workflow interprets changes across data sources instead of merely displaying metrics. On a schedule, a Sim workflow can retrieve advertising spend, email engagement, and CRM conversion data from connected tools. An Agent block can compare periods and summarize which campaigns or channels account for the largest changes. + +[HubSpot](https://www.hubspot.com/products/reporting-dashboards) and [ActiveCampaign](https://www.activecampaign.com/marketing-analytics) reports primarily organize metrics stored within their own products. They may not explain a cross-tool pattern, such as why paid clicks increased while qualified pipeline declined. The agent can examine records from both systems and write a plain-language summary to a Sim Table. A deterministic branch can also post the summary to Slack when a metric crosses a defined threshold. + +Sim's block-level traces keep each explanation inspectable. You can review the source data passed into the Agent block and compare it with the generated summary. That record helps you separate observations supported by the data from causal claims that require further investigation. Learn more about what to inspect in our guide to [AI agent observability](https://www.sim.ai/library/ai-agent-observability). + +### Lead scoring and routing beyond static rules + +A numerical score gives sales reps little context for deciding what to do next. HubSpot and ActiveCampaign can [predict lead quality or close probability](https://help.activecampaign.com/hc/en-us/articles/360001426799-Win-Probability), but the output may not explain which signals influenced the prediction. Reps must inspect each record or rely on fixed score thresholds. + +Reasoning-based classification can consider intent and fit while returning a written justification. In one [50-deal comparison](https://cotera.co/articles/hubspot-ai-crm-automation), an external agent predicted 38 outcomes correctly, compared with 31 for HubSpot Breeze. One test cannot establish a general accuracy advantage, but it shows the potential value of evaluating richer context rather than returning an unexplained number. + +In [Sim](https://sim.ai), an Agent block can classify a lead as sales-ready, nurture, partner, or disqualified. The classification can include supporting signals such as company profile, campaign engagement, and stated need. A deterministic branch then assigns the correct rep or queue and updates the relevant CRM field. You can inspect the agent's justification while keeping routing rules predictable. + +### Social posting as an agentic last mile + +A new webinar recording can trigger a social workflow as soon as the transcript arrives. In [Sim](https://sim.ai), an Agent block can identify the main argument and draft separate posts for LinkedIn and X. The prompt can apply each channel's length, tone, formatting requirements, and account voice without forcing one generic caption into every destination. + +The workflow can send each draft to Slack with the source excerpt for review. An approval action then triggers fixed scheduling and publishing steps, while a rejection returns the post with feedback for revision. If your publishing policy permits direct posting, the workflow can skip approval and send the finished copy to the connected social channel. + +Drafting remains agentic because the Agent block interprets source material and adapts its output. Scheduling, approval status, and posting remain deterministic because each action follows an explicit rule. + +## Sim vs HubSpot/ActiveCampaign vs Zapier/Make/n8n for agentic marketing workflows + +Compare these products by examining where reasoning runs and what context it can access. Then consider how thoroughly you can inspect the workflow and where you can deploy it. Builder style and model choice also affect who can maintain it. + +| Product | Builder model | Agent depth | Deterministic control | Context layer | Action layer | Model flexibility | Deployment surfaces | Observability | +| --- | --- | --- | --- | --- | --- | --- | --- | --- | +| Sim | Natural language or visual graph | Built-in agent runtime | Branches, loops, functions, and approvals | Native Tables, Files, and Knowledge Bases | Integrations, APIs, code, MCP, and skills | Hosted models, BYOK, and Enterprise local models | API, hosted chat, and MCP server | Block traces, logs, errors, and run cost | +| [HubSpot](https://www.hubspot.com/products/marketing) | [Visual workflows and feature configuration](https://knowledge.hubspot.com/workflows/create-workflows) | [AI features inside a CRM suite](https://www.hubspot.com/products/artificial-intelligence) | [Rule-based workflows, branches, and approvals](https://knowledge.hubspot.com/workflows/create-workflows) | [Native CRM records and marketing data](https://www.hubspot.com/products/crm) | [HubSpot tools, channels, and marketplace apps](https://ecosystem.hubspot.com/marketplace/apps) | [HubSpot-managed AI](https://www.hubspot.com/products/artificial-intelligence) | [HubSpot workspace and connected channels](https://www.hubspot.com/products/marketing) | [Workflow history and CRM reporting](https://knowledge.hubspot.com/workflows/view-your-workflow-history) | +| [ActiveCampaign](https://www.activecampaign.com/platform) | [Visual automation builder](https://www.activecampaign.com/marketing-automation) | [AI features inside marketing automation](https://www.activecampaign.com/ai) | [Triggers, conditions, branches, and goals](https://www.activecampaign.com/marketing-automation) | [Native contacts, deals, and campaign data](https://www.activecampaign.com/platform) | [Marketing channels and connected apps](https://www.activecampaign.com/apps) | [ActiveCampaign-managed AI](https://www.activecampaign.com/ai) | [ActiveCampaign workspace and connected channels](https://www.activecampaign.com/platform) | [Automation reports and activity history](https://help.activecampaign.com/hc/en-us/articles/360000063804-Automations-Performance-report) | +| [Zapier](https://zapier.com/) | [No-code trigger-action builder](https://zapier.com/how-it-works) | [Agents sit beside Zaps](https://zapier.com/agents) | [Paths, filters, loops, and approvals](https://zapier.com/features) | [Zapier Tables and connected apps](https://zapier.com/tables) | [Broad app catalog, APIs, and MCP](https://zapier.com/apps) | [Models through Zapier AI products and apps](https://zapier.com/agents) | [Zaps, Agents, Chatbots, and MCP](https://zapier.com/products) | [Task and activity history](https://help.zapier.com/hc/en-us/articles/8496248813453-View-and-manage-your-Zap-history) | +| [Make](https://www.make.com/) | [Visual scenario canvas](https://help.make.com/get-started-with-make) | [AI Agent modules inside scenarios](https://www.make.com/en/ai-agents) | [Routers, iterators, filters, and code](https://help.make.com/flow-control) | [Scenario data and connected apps](https://www.make.com/en/integrations) | [Broad app catalog, APIs, code, and MCP](https://www.make.com/en/integrations) | [Models through agent settings and apps](https://www.make.com/en/ai-agents) | [Cloud scenarios and MCP](https://www.make.com/en/ai-agents) | [Scenario execution history](https://help.make.com/scenario-history) | +| [n8n](https://n8n.io/) | [Node-based low-code builder](https://docs.n8n.io/workflows/) | [AI nodes inside an automation engine](https://docs.n8n.io/advanced-ai/) | [Branches, loops, code, and approvals](https://docs.n8n.io/flow-logic/) | [Workflow data and connected sources](https://docs.n8n.io/workflows/) | [Large node ecosystem, APIs, and code](https://n8n.io/integrations/) | [Multiple model providers and credentials](https://docs.n8n.io/integrations/builtin/cluster-nodes/root-nodes/n8n-nodes-langchain.agent/) | [Cloud or self-hosted workflows and webhooks](https://docs.n8n.io/hosting/) | [Execution logs and node-level data](https://docs.n8n.io/workflows/executions/) | + +## When to build agentic workflows vs when to stick with point tools + +Choose [HubSpot](https://www.hubspot.com/products/crm) or [ActiveCampaign](https://www.activecampaign.com/platform) when you need a ready-made CRM data model and standard campaign templates. These products manage contact data and email delivery inside a familiar interface. They also fit your needs when you lack engineering resources and can express most automation through fixed rules. + +Build an agentic workflow when a decision requires context from several tools. An agent can interpret unstructured content or assess intent before a fixed branch takes action. With Sim, you can connect that reasoning to APIs and approval steps within one graph. + +Choose an agentic workflow when reviewers need to understand why an action occurred. In Sim, we record block-level traces and run details such as errors and cost. You can inspect the inputs and outputs behind a classification or routing decision. When an automation log records only rule execution, it cannot show the model inputs and outputs behind an agent's decision. + +A hybrid setup often preserves the most useful parts of each category. HubSpot or ActiveCampaign can remain the contact database and campaign delivery tool, while you use Sim to handle cross-tool reasoning and write the result back. Start with one workflow whose manual decisions create delays or inconsistency. Keep using the point tool if fixed rules handle the workflow reliably, and use an agent when the decision depends on changing context. + +## Getting started with agentic marketing workflows in Sim + +Start with one workflow whose output you can review quickly. For content repurposing, ask an Agent block to turn a blog post into channel-specific drafts. Add an approval step before publishing. For lead routing, let the Agent block classify intent, then use fixed branches to update your CRM or notify the right person in Slack. + +With [Sim's Free plan](https://sim.ai), you can test the workflow using hosted model access. You can ask Sim in Chat to create a starting workflow from a plain-language description, or assemble each block directly in the visual builder. Run several real inputs through the workflow and inspect the outputs before connecting any step that publishes content or changes customer records. diff --git a/apps/sim/content/library/best-ai-agents-for-lead-enrichment-2026/index.mdx b/apps/sim/content/library/best-ai-agents-for-lead-enrichment-2026/index.mdx new file mode 100644 index 00000000000..e16759256d2 --- /dev/null +++ b/apps/sim/content/library/best-ai-agents-for-lead-enrichment-2026/index.mdx @@ -0,0 +1,207 @@ +--- +slug: best-ai-agents-for-lead-enrichment-2026 +title: 'Best AI Agents for Lead Enrichment in 2026' +description: 'A ranked comparison of the best lead enrichment tools in 2026, covering provider fallback, conflict resolution, CRM write-back, and pricing.' +date: 2026-08-31 +updated: 2026-08-31 +authors: + - andrew +readingTime: 14 +tags: [AI Agents, Lead Enrichment, Sales Automation, Sim] +ogImage: /library/best-ai-agents-for-lead-enrichment-2026/cover.jpg +canonical: https://www.sim.ai/library/best-ai-agents-for-lead-enrichment-2026 +draft: false +faq: + - q: "How often should leads be re-enriched?" + a: "Lead re-enrichment is the process of refreshing records on a schedule or when a person's company details change; a practical baseline is to recheck records older than 90 days because B2B contact data decays by roughly 22 percent annually. With Sim, you can send those records through the same enrichment loop used for new leads. Regular checks keep titles and contact details current for routing and outreach." + - q: "What does human-in-the-loop escalation mean in practice?" + a: "Human-in-the-loop escalation sends low-confidence or unresolved matches to a person for review. With Sim, you can pause the workflow before uncertain data reaches Salesforce or HubSpot. A reviewer decides whether to approve the proposed update and can correct it first." + - q: "Does agent-based enrichment replace existing CRM data?" + a: "Agent-based enrichment adds to or updates existing CRM records rather than replacing the CRM itself. With Sim, the workflow reads current fields, checks external sources, removes duplicates, and writes approved values back. This keeps the CRM as the system of record while automating research and controlled updates." +--- + +## TL;DR + +Agent-based enrichment is the strongest approach when records require conditional source selection, conflict resolution, human review, and CRM write-back. Based on those criteria, Sim ranks first, while the other tools fit narrower technical, template-based, database, or sales-engagement needs. + +- **1. Sim** works best for agent-based enrichment that applies conditional logic and writes results back to your CRM across several deployment options. +- **2. n8n** suits technical users who want full control over node-based enrichment workflows. +- **3. Zapier** fits lightweight enrichment automations built around its broad app catalog. +- **4. Gumloop** helps GTM users launch packaged enrichment templates quickly. +- **5. Clay** suits no-code users who want to coordinate several enrichment providers. +- **6. Apollo** combines contact data with sales engagement tools in one product. +- **7. ZoomInfo** serves enterprise buyers seeking a proprietary contact database with CRM sync and intent features. + +[Explore Sim](https://sim.ai) to see how an agent-based enrichment workflow can evaluate data and update CRM records according to conditional rules. + +## Lead enrichment and the fields that matter + +Lead enrichment adds missing information from internal and external sources to a lead record that may contain only a name and email address. The fields that support qualification, routing, and outreach fall into [five practical categories](https://www.commonroom.io/blog/data-enrichment/). + +Firmographic data describes the company, including its industry, revenue, headcount, and funding stage. Technographic data identifies tools the company uses, such as its CRM, cloud provider, or analytics platform. Demographic data describes the person's role, seniority, function, and department. + +Behavioral and intent data records signals such as website visits, product activity, content engagement, and community participation. Contact-level data provides verified work emails and mobile numbers. Match the fields to your sales motion. Account scoring relies heavily on firmographic and technographic fields, while routing and personalized outreach require accurate role and contact details. + +You should cleanse records before enriching them. Standardize account names and other fields, then remove duplicate records before an automated lead enrichment workflow appends new fields. + +Enrichment also needs to run continuously because B2B contact data decays by [roughly 22 percent each year](https://www.unifygtm.com/explore/waterfall-enrichment-b2b-data). Changes to a person's job or company can make a previously complete record unreliable. Recheck older records and leads associated with updated accounts. With Sim, you can route unresolved records through additional providers or web research before sending uncertain matches for review. + +## Why static waterfall enrichment leaves gaps + +Waterfall enrichment leaves gaps because each provider has incomplete coverage, and later queries recover progressively fewer records. A waterfall queries providers in a fixed priority order. When the first provider returns no result or a low-confidence match, the workflow tries the next provider. + +[Match-rate benchmarks published by Unify](https://www.unifygtm.com/explore/waterfall-enrichment-b2b-data) show how fallback improves coverage but does not eliminate missing records. + +| Enrichment architecture | Typical match rate | +| --- | --- | +| Single source | 55 to 70% | +| Sequential waterfall | 80 to 92% | +| Parallel multi-source | 75 to 90% | + +A second provider typically recovers 15 to 25% of the first provider's misses. Later providers add less coverage. The third recovers another 8 to 12%, and the fourth adds only 3 to 5%. Remaining records often lack enough reliable identifiers for any connected source to match them, so added queries produce diminishing returns. + +Vendor design determines how you handle those misses. A [documented vendor comparison](https://www.devcommx.com/blogs/waterfall-enrichment-clay-vs-zoominfo-vs-apollo) reports that Clay users may spend one or two weeks configuring their first waterfall and must set priority rules for conflicting values. ZoomInfo and Apollo rely on proprietary databases without built-in fallback. Breeze Intelligence follows fixed enrichment rules and matches about [70 to 75% of records](https://www.squad4.io/blog/breeze-intelligence-vs-zoominfo). + +Verification is also important after a provider returns a match. Unify reports that single-source enrichment correlates with outbound bounce rates of 8 to 15%, showing why workflows should validate contact details before outreach rather than treating every match as current and deliverable. + +## How an AI agent handles enrichment differently + +An AI agent treats lead enrichment as a conditional reasoning loop. The agent chooses its next action based on available data, confidence, and the CRM record instead of sending every lead through a fixed sequence. + +1. **Pull the lead and select a source.** The agent reviews the existing record and queries an appropriate provider for missing fields. When the provider returns no match, the agent can use another API, web search, or scraping rather than leaving the field blank. +2. **Reconcile conflicting results.** The agent normalizes provider responses into the CRM schema and compares them with existing records. Rules help the agent choose between conflicting values by weighing source reliability and recency. Deduplication prevents a new response from creating a second record for the same person or company. +3. **Verify and escalate uncertain matches.** The agent checks whether identifiers such as the company domain and profile URL are consistent with the lead's stated person and company. A configured confidence threshold can route uncertain matches to a person for approval instead of overwriting CRM data. Missing identifiers require extra scrutiny because their absence can prevent reliable automated matching. One vendor describes a workflow in which [20 percent of records failed to match](https://cotera.co/articles/ai-lead-enrichment-automation) when records lacked LinkedIn URLs, though the example comes from that vendor's own marketing. +4. **Write the approved result back.** The agent records the source and applies scoring logic before updating the CRM or table. With Sim, you can apply this pattern using native Salesforce and HubSpot read and write actions, plus built-in Tables and Knowledge Bases. + +Assembly effort distinguishes an integrated agent workflow from a point solution that relies on an external automation stack. With n8n, you connect model nodes and parse their outputs. You also map CRM fields and manage errors across separate steps. Zapier may require stacked Zaps when connector actions omit needed fields. Gumloop templates start faster, but unsupported branches or providers require changes beyond the template's fixed shape. With Sim, you can keep conditional logic, enrichment, scoring, and CRM updates in [one workflow](https://www.sim.ai/library/best-ai-agents-sales-crm-automation). This is also the core distinction between an [agentic workflow](https://www.sim.ai/library/what-is-an-agentic-workflow) and a fixed automation sequence. + +## Enrichment approaches compared: static vendor vs. agent-based + +Static waterfalls query providers in a preset order, while agents can choose sources and actions according to each record. + +| Criterion | Static waterfall vendor | Agent-based enrichment with Sim | +| --- | --- | --- | +| Source breadth | Queries a fixed provider sequence. Three or four providers usually capture most marginal gains. [Later providers add less coverage](https://www.unifygtm.com/explore/waterfall-enrichment-b2b-data). | Selects connected providers or web sources based on missing fields. [Fallback logic runs within one workflow](https://www.sim.ai/library/best-ai-agents-sales-crm-automation). | +| Conflict handling | Provider priority determines which value wins. Parallel queries require separate resolution rules. | Normalizes returned values and applies workflow rules before accepting a match. | +| CRM write-back | Depends on vendor connectors and field mapping. Some connectors require deduplication and sync configuration. [ZoomInfo illustrates this maintenance](https://www.squad4.io/blog/breeze-intelligence-vs-zoominfo). | Reads and updates CRM records inside the enrichment workflow after verification. | +| Human escalation | Vendor rules may send failed records for manual review or exclude them. | Conditional logic can route low-confidence matches to a person before CRM write-back. | + +## What to look for in a lead enrichment tool + +The architectural differences above translate into five practical evaluation criteria. + +**Provider breadth and fallback logic.** Choose a tool that can query multiple providers according to rules suited to each record. It should retry fields that remain missing or have low confidence. Provider strengths vary by field type and geography, so one fallback sequence may not suit every field. + +**Conflict resolution.** Check how the tool handles contradictory employment and contact details. Effective lead enrichment compares source freshness and confidence rather than accepting the first available value. + +**CRM write-back depth.** Confirm that the tool can remove duplicates and map custom fields before updating existing CRM entries. Basic connectors may require extra workflows for conditional updates. + +**Human-in-the-loop escalation.** Look for configurable confidence thresholds that send uncertain matches to a reviewer before the tool changes a CRM record or starts outreach. + +**Pricing model transparency.** Calculate the full cost of data retrieval and workflow execution, including model usage and refreshes. Per-lookup pricing can discourage regular validation, even though B2B contact data [decays by roughly 22 percent each year](https://www.commonroom.io/blog/data-enrichment/). + +## Best AI agents and tools for lead enrichment + +### Sim + +**Best for:** Sim ranks first for teams that want a multi-provider enrichment agent with native CRM write-back and flexible model choice. + +**What it is:** Sim builds lead enrichment as a reasoning loop rather than a fixed sequence of provider calls. The agent can query a data provider, use web search or scraping when the provider returns no record, normalize the results, remove duplicates, score the account, and write the enriched record back. Conditional logic lets the workflow choose its next action based on the data it finds. + +Sim provides [native read and write actions for Salesforce and HubSpot](https://www.sim.ai/library/best-ai-agents-sales-crm-automation). Built-in Tables can hold enriched records, while Knowledge Bases can supply company-specific context for scoring and classification. You can keep source selection, schema rules, deduplication, scoring, and CRM updates inside one workflow instead of mapping them across separate tools. A Human in the Loop block can pause low-confidence records for review before write-back. + +Sim also supports several deployment formats, including cloud workflows, API access, chat, and embedded experiences. Bring-your-own-key support covers more than 15 model providers, so you can choose models according to cost, latency, or task requirements. Switching providers does not require rebuilding the surrounding enrichment logic. These capabilities make Sim one of the [AI agent platforms for connecting existing tools](https://www.sim.ai/library/best-ai-agent-platforms-for-connecting-your-existing-tools) rather than a proprietary contact database. + +**Pros:** The agent can select fallback sources and reconcile data within the same workflow that handles scoring and CRM updates. Native Salesforce and HubSpot actions reduce the manual field mapping required by general automation tools. Tables, Knowledge Bases, conditional logic, human review, and broad model support give you control over how the agent evaluates each lead. Conditional agent logic handles provider fallback and CRM updates while adapting to conflicts and schema changes without separate automation systems. + +**Cons:** Building the enrichment loop requires setting provider priorities and defining how to handle confidence and conflicts, so setup takes more thought than launching a packaged template. Teams that require mandatory human review before CRM updates must configure that workflow branch and approval behavior. Sim also requires you to define source selection, confidence rules, and the target CRM schema rather than purchasing a finished proprietary contact database. + +**Pricing:** Sim uses [usage-based pricing](https://sim.ai/pricing) and supports your own model-provider API keys. Your total cost depends on workflow executions, model calls, and any external enrichment providers the agent queries. + +Sim ranks first because it combines the capabilities used throughout this comparison: conditional source selection, conflict reconciliation, human review, and native CRM write-back in one workflow. Its Salesforce and HubSpot actions, built-in Tables and Knowledge Bases, deployment options, and bring-your-own-key support reduce the need to divide enrichment logic across separate data and automation systems. + +[Explore Sim](https://sim.ai) to learn how to build an agent-based enrichment workflow. + +### n8n + +**What it is.** n8n uses a node-based visual builder to connect data sources with CRM actions. [AI model nodes](https://docs.n8n.io/integrations/builtin/cluster-nodes/sub-nodes/n8n-nodes-langchain.lmchatopenai/) can process the data between those steps. You can build an enrichment loop that branches when a provider returns no match. The next nodes transform the result before updating the relevant CRM record. n8n supports [cloud and self-hosted deployment](https://docs.n8n.io/hosting/). + +**Best for.** Choose n8n when you want full control over how each enrichment step runs. + +**Pros.** n8n lets you choose each provider and define conditional logic while controlling where workflow context lives. Self-hosting also gives you more control over deployment and data handling. + +**Cons.** n8n has no built-in enrichment logic. Every provider call, field mapping, and error path is something you wire yourself in the node editor, and a broken step fails silently unless you build a dedicated error-handling branch for it. That [assembly effort](https://www.sim.ai/library/best-ai-agents-sales-crm-automation) means a lead-enrichment workflow that took a vendor a week to configure can take longer to build from scratch in n8n, since you're building both the logic and the plumbing. + +**Pricing.** [n8n uses execution-based pricing tiers](https://n8n.io/pricing/), starting at €20/month for 2.5K workflow executions. Your cost depends on how often workflows run rather than how many individual tasks each run performs. + +### Zapier + +**What it is.** Zapier builds [linear workflows called Zaps](https://help.zapier.com/hc/en-us/articles/8496180298253-Create-Zaps). A new lead can trigger an enrichment request, after which Zapier maps the returned data and updates the connected CRM. + +**Best for.** Choose Zapier when you already use its app catalog and need lightweight, trigger-based lead enrichment. + +**Pros.** [Zapier's broad app catalog](https://zapier.com/apps) makes it practical when your CRM and enrichment provider already have supported connectors. The trigger-action model also suits straightforward workflows with predictable inputs and updates. + +**Cons.** Zapier's connectors expose the fields supported by each integration, so conflict resolution or unsupported custom field updates can need a stacked second or third Zap to finish the job a single agent workflow would handle in one pass. You configure deduplication with steps such as Filters or Paths. According to [Sim's platform comparison](https://www.sim.ai/library/best-ai-agents-sales-crm-automation), model processing and CRM integration run as separate steps rather than one reasoning loop, which means every added condition is another Zap to maintain and another task consumed. + +**Pricing.** [Zapier uses task-based pricing tiers](https://zapier.com/pricing), starting at $19.99/month for 750 tasks on the Professional plan. Each successful action can count as a task, so multi-step enrichment workflows consume more tasks per lead. + +### Gumloop + +**What it is.** Gumloop provides a node-based visual builder with [packaged templates](https://www.gumloop.com/templates) for lead enrichment. The templates can include scraping and sales outreach. Each template keeps its working context within the template, and Gumloop runs in the cloud. + +**Best for.** Choose Gumloop when you want a packaged enrichment template without building every step yourself. + +**Pros.** Packaged flows reduce the initial assembly work and give you a structure that you can adjust through visual nodes. + +**Cons.** A Gumloop template locks in a fixed workflow shape at the moment you launch it. Add a data provider the template didn't ship with, or a scoring rule it didn't anticipate, and you're not configuring a setting, you're rebuilding that section of the flow by hand. + +**Pricing.** [Gumloop's Pro plan starts at $37 per month](https://www.gumloop.com/pricing) with 20,000 included credits. Estimate costs using expected credit consumption because larger enrichment runs and additional processing steps consume more credits. + +### Clay + +**What it is.** Clay connects [many data sources](https://www.clay.com/integrations) in a spreadsheet-style workspace. You arrange providers in priority order, and Clay queries the next provider when an earlier one cannot fill a field. [Claygent](https://www.clay.com/claygent) can also research public web sources for less structured data. + +**Best for.** Choose Clay when you want no-code waterfall enrichment across multiple data providers without building an agent. + +**Pros.** Clay gives you broad provider choice and code-optional workflow controls. Its [waterfall model](https://www.clay.com/university/lesson/enrichments-waterfalls) can reduce unnecessary spending by moving through a provider sequence until it finds a result. + +**Cons.** DevCommX puts first-waterfall setup time at [one to two weeks](https://www.devcommx.com/blogs/waterfall-enrichment-clay-vs-zoominfo-vs-apollo), and adding a provider or reworking priority order requires revisiting the waterfall. You set the winning-source rules by hand. Connected providers may bill separately on top of Clay's own fee, and failed match attempts can still consume resources depending on the provider and configuration. + +**Pricing.** Clay publishes its current plans on its [pricing page](https://www.clay.com/pricing). Third-party data-provider fees may be additional, so total spending depends on usage and the services connected. + +### Apollo + +**What it is.** Apollo combines a [contact and company database with sales engagement](https://www.apollo.io/product/sales-engagement) and data maintenance tools. You can enrich prospect records and run outreach within the same platform. + +**Best for.** Choose Apollo when you want a contact database and sales engagement suite in one subscription. + +**Pros.** Apollo reduces the need to connect a separate contact provider to an engagement tool. Its free plan also gives you a low-cost way to test the database and workflow. + +**Cons.** Apollo's enrichment depends on its own data rather than a configurable multi-provider waterfall. [Coverage can vary by market](https://www.devcommx.com/blogs/waterfall-enrichment-clay-vs-zoominfo-vs-apollo), and DevCommX rates its mobile-number data weaker than ZoomInfo's. The product is built around sequencing as well as enrichment, so teams using it purely as an enrichment layer should assess whether they need its sales-engagement features. + +**Pricing.** [Apollo offers a free plan and paid tiers](https://www.apollo.io/pricing). Review its current per-seat pricing and credit allowances when comparing plans. + +### ZoomInfo + +**What it is.** ZoomInfo [enriches B2B contact and company records](https://www.zoominfo.com/products/enrich) from its proprietary database. The platform offers CRM enrichment alongside [buyer intent data](https://www.zoominfo.com/products/intent). + +**Best for.** Choose ZoomInfo when you need a large proprietary contact database with CRM sync and intent data. + +**Pros.** ZoomInfo provides B2B coverage and [connects enrichment to CRM systems](https://www.zoominfo.com/products/enrich). Its intent data can help you prioritize accounts showing signs of active research. + +**Cons.** ZoomInfo relies on its own database rather than querying a configurable fallback provider. CRM sync requires field mapping and deduplication rules, and that maintenance belongs in the integration setup. + +**Pricing.** ZoomInfo directs buyers to [request pricing](https://www.zoominfo.com/pricing). Review contract and cancellation terms before comparing its total cost with usage-based or monthly alternatives. + +## Choosing the right enrichment approach + +Choose an enrichment tool that matches how you build, review, and maintain workflows. The broader market for [AI automation tools](https://www.sim.ai/library/best-ai-automation-tools-2026) includes both agent-first systems and general-purpose workflow builders. + +- **Choose Sim** for one agent-based workflow that selects sources, evaluates confidence, and writes approved records to Salesforce or HubSpot. +- **Choose n8n** for granular technical control over node wiring, CRM mapping, deployment, and error handling. +- **Choose Zapier** for lightweight trigger-based enrichment using supported apps and predictable update paths. +- **Choose Gumloop** for a packaged template that launches quickly and requires limited custom branching. +- **Choose Clay** for a configurable, no-code waterfall across multiple data providers. +- **Choose Apollo** for contact data and sales engagement in one subscription. +- **Choose ZoomInfo** for a proprietary B2B database with intent data and native CRM sync. diff --git a/apps/sim/ee/workspace-forking/lib/create-fork.test.ts b/apps/sim/ee/workspace-forking/lib/create-fork.test.ts index d92b211cdfb..f7317395f5a 100644 --- a/apps/sim/ee/workspace-forking/lib/create-fork.test.ts +++ b/apps/sim/ee/workspace-forking/lib/create-fork.test.ts @@ -1,7 +1,8 @@ /** * @vitest-environment node */ -import { dbChainMockFns, resetDbChainMock } from '@sim/testing' +import { workspace } from '@sim/db/schema' +import { dbChainMockFns, queueTableRows, resetDbChainMock } from '@sim/testing' import { beforeEach, describe, expect, it, vi } from 'vitest' const { @@ -125,6 +126,12 @@ describe('createFork storage headroom gate', () => { beforeEach(() => { vi.clearAllMocks() resetDbChainMock() + /** + * The fork transaction re-reads the parent's organization under the lock to + * confirm it has not moved since `assertCanFork` captured the policy. + * Matches POLICY.organizationId, so the fork proceeds. + */ + queueTableRows(workspace, [{ organizationId: null }]) mockSumForkCopyBytes.mockResolvedValue(0) mockAssertForkStorageHeadroom.mockResolvedValue(undefined) mockLoadSourceDeployedStates.mockResolvedValue({ @@ -186,6 +193,24 @@ describe('createFork storage headroom gate', () => { expect(mockStartBackgroundWork).not.toHaveBeenCalled() }) + it('refuses when the parent changed organizations after the policy was captured', async () => { + resetDbChainMock() + /** + * `assertCanFork` captures `policy.organizationId` before this transaction, + * so an admin workspace move committing in between would otherwise leave + * the fork locking the organization the parent has already left and + * inserting the child there — the cross-organization edge the lock exists + * to prevent. The parent is re-read under the lock to catch exactly this. + */ + queueTableRows(workspace, [{ organizationId: 'org-moved-away' }]) + mockSumForkCopyBytes.mockResolvedValue(0) + + await expect(createFork(forkParams())).rejects.toThrow( + 'changed organizations while this fork was being created' + ) + expect(dbChainMockFns.insert).not.toHaveBeenCalled() + }) + it('proceeds under quota, summing exactly the selected files + knowledge bases', async () => { mockSumForkCopyBytes.mockResolvedValue(500) diff --git a/apps/sim/ee/workspace-forking/lib/create-fork.ts b/apps/sim/ee/workspace-forking/lib/create-fork.ts index c7dbfc3d036..31ccf6e12bf 100644 --- a/apps/sim/ee/workspace-forking/lib/create-fork.ts +++ b/apps/sim/ee/workspace-forking/lib/create-fork.ts @@ -39,6 +39,7 @@ import { } from '@/ee/workspace-forking/lib/copy/storage-quota' import { buildForkWorkflowIdMap } from '@/ee/workspace-forking/lib/copy/workflow-id-map' import { copyForkWorkflowMcpAttachments } from '@/ee/workspace-forking/lib/copy/workflow-mcp-attachments' +import { ForkError } from '@/ee/workspace-forking/lib/lineage/authz' import { setForkLockTimeout } from '@/ee/workspace-forking/lib/lineage/lineage' import { type ForkBlockPair, @@ -161,6 +162,61 @@ export async function createFork(params: CreateForkParams): Promise { await setForkLockTimeout(tx) + /** + * The lock alone is not enough: `policy.organizationId` was captured by + * `assertCanFork` BEFORE this transaction, so a re-home that commits in + * between leaves us locking the organization the parent has already left + * and inserting the child there, which is the exact cross-organization + * edge the lock was added to prevent. Re-read the parent under the lock + * and refuse if it moved; the caller can retry against the new + * organization. + */ + const [currentSource] = await tx + .select({ organizationId: workspace.organizationId }) + .from(workspace) + .where(eq(workspace.id, source.id)) + /** + * The row lock IS the serialization, and deliberately the only one. + * + * A fork parent and child must always share an organization. Every writer + * that can re-home the parent takes `FOR NO KEY UPDATE` on its row: the + * admin workspace move, and `lockWorkspaceRowsForPayerChanges` on the + * organization-attach path. Locking it here makes those wait, and the + * comparison below then sees their committed result. + * + * Scope, stated plainly. This closes the ordering the admin move + * introduces: a re-home that commits first can no longer be forked + * against a stale policy. It does NOT close the reverse ordering, where + * a fork commits while a bulk attach or detach is already waiting on + * this row with a workspace list snapshotted before the child existed. + * That batch would then re-home the parent alone. It is a pre-existing + * gap in `attachOwnedWorkspacesToOrganizationTx` and + * `detachOrganizationWorkspacesTx`, not one the move creates, and + * closing it needs the descendant closure, the disclosure set, and the + * advisory-lock plan to move together. Tracked separately rather than + * half-fixed here, because a partial repair at commit time is strictly + * worse than a documented gap. + * + * An organization mutation lock was tried here and removed: it bought + * nothing the row lock does not already provide, could not cover a null + * policy organization at all, and cost three real problems. A lock-order + * inversion against invitation acceptance (which takes the workspace row + * before the organization lock), a 5s timeout overwriting this + * transaction's 10s one, and an organization-wide lock held across the + * whole content copy. + */ + .for('no key update') + .limit(1) + if (!currentSource) { + throw new ForkError('Source workspace no longer exists', 404) + } + if ((currentSource.organizationId ?? null) !== (policy.organizationId ?? null)) { + throw new ForkError( + 'The source workspace changed organizations while this fork was being created. Try again.', + 409 + ) + } + const now = new Date() await tx.insert(workspace).values({ diff --git a/apps/sim/executor/execution/block-executor.retry.test.ts b/apps/sim/executor/execution/block-executor.retry.test.ts index 54811c71f8f..e22c93104eb 100644 --- a/apps/sim/executor/execution/block-executor.retry.test.ts +++ b/apps/sim/executor/execution/block-executor.retry.test.ts @@ -5,11 +5,13 @@ * client has already seen and cannot re-run the deterministic post-processing. */ import { beforeEach, describe, expect, it, vi } from 'vitest' +import { buildTraceSpans } from '@/lib/logs/execution/trace-spans/trace-spans' import { BlockType, EDGE } from '@/executor/constants' import type { DAGNode } from '@/executor/dag/builder' import { BlockExecutor } from '@/executor/execution/block-executor' import { ExecutionState } from '@/executor/execution/state' import type { BlockHandler, ExecutionContext } from '@/executor/types' +import { attachTrustedExecutionCost } from '@/executor/utils/errors' import { VariableResolver } from '@/executor/variables/resolver' import type { SerializedBlock, SerializedWorkflow } from '@/serializer/types' @@ -137,6 +139,68 @@ describe('BlockExecutor retry', () => { expect(ctx.blockLogs[0]?.tries).toBe(2) }) + it('adds the trusted cost of failed Function tries to the successful result', async () => { + const block = createBlock(enabled) + const firstFailure = new Error('first attempt failed') + attachTrustedExecutionCost(firstFailure, { input: 0, output: 0, total: 0.125 }) + const successfulOutput = { + result: 'done', + cost: { input: 0, output: 0, total: 0.25 }, + } + attachTrustedExecutionCost(successfulOutput, successfulOutput.cost) + const execute = vi + .fn() + .mockRejectedValueOnce(firstFailure) + .mockResolvedValueOnce(successfulOutput) + const state = new ExecutionState() + const ctx = createContext(state) + const executor = buildExecutor(block, { canHandle: () => true, execute }, state) + + const output = await executor.execute(ctx, createNode(block), block) + + expect(execute).toHaveBeenCalledTimes(2) + expect(output.cost).toEqual({ input: 0, output: 0, total: 0.375 }) + expect(ctx.blockLogs[0]?.output?.cost).toEqual(output.cost) + }) + + it('keeps earlier trusted Function costs when the final try is an infrastructure error', async () => { + const block = createBlock(enabled) + const firstFailure = new Error('first Function attempt failed') + const secondFailure = new Error('second Function attempt failed') + const finalFailure = new Error('provider unavailable') + attachTrustedExecutionCost(firstFailure, { input: 0, output: 0, total: 0.125 }) + attachTrustedExecutionCost(secondFailure, { input: 0, output: 0, total: 0.25 }) + const execute = vi + .fn() + .mockRejectedValueOnce(firstFailure) + .mockRejectedValueOnce(secondFailure) + .mockRejectedValueOnce(finalFailure) + const state = new ExecutionState() + const ctx = createContext(state) + const executor = buildExecutor(block, { canHandle: () => true, execute }, state) + + await expect(executor.execute(ctx, createNode(block), block)).rejects.toThrow( + 'provider unavailable' + ) + + expect(execute).toHaveBeenCalledTimes(3) + expect(ctx.blockLogs[0]?.output).toEqual({ + error: 'provider unavailable', + cost: { input: 0, output: 0, total: 0.375 }, + }) + + const { traceSpans } = buildTraceSpans({ + success: false, + output: { error: 'provider unavailable' }, + error: 'provider unavailable', + logs: ctx.blockLogs, + }) + expect(traceSpans[0]).toMatchObject({ + status: 'error', + cost: { input: 0, output: 0, total: 0.375 }, + }) + }) + it('stops at maxTries and rethrows the final error unchanged', async () => { const block = createBlock(enabled) const failure = new Error('still failing') diff --git a/apps/sim/executor/execution/block-executor.ts b/apps/sim/executor/execution/block-executor.ts index bc4241208fd..1695cde05b4 100644 --- a/apps/sim/executor/execution/block-executor.ts +++ b/apps/sim/executor/execution/block-executor.ts @@ -48,7 +48,13 @@ import { type StreamingExecution, } from '@/executor/types' import { streamingResponseFormatProcessor } from '@/executor/utils' -import { buildBlockExecutionError, normalizeError } from '@/executor/utils/errors' +import { + attachTrustedExecutionCost, + buildBlockExecutionError, + normalizeError, + readTrustedExecutionCost, + type TrustedExecutionCost, +} from '@/executor/utils/errors' import { buildUnifiedParentIterations, getIterationContext, @@ -76,6 +82,20 @@ import { SYSTEM_SUBBLOCK_IDS } from '@/triggers/constants' const logger = createLogger('BlockExecutor') +function addTrustedExecutionCosts( + accumulated: TrustedExecutionCost | undefined, + current: TrustedExecutionCost | undefined +): TrustedExecutionCost | undefined { + if (!accumulated) return current + if (!current) return accumulated + + return { + input: accumulated.input + current.input, + output: accumulated.output + current.output, + total: accumulated.total + current.total, + } +} + export class BlockExecutor { private execLogger: Logger @@ -229,6 +249,17 @@ export class BlockExecutor { cleanupSelfReference?.() let streamingPartialOutput: Record | undefined + /** + * Cost of a handler that already finished, kept for the catch below. + * + * A Function block's sandbox is paid for the moment it completes, but the + * steps after the handler returns — base64 hydration, and large-value + * redaction that deliberately throws rather than emit unredacted data — can + * still fail the block. The error those raise carries no cost of its own, so + * without holding it here the completed sandbox would go unbilled. Hoisted + * for the same reason `streamingPartialOutput` above is. + */ + let completedHandlerCost: TrustedExecutionCost | undefined try { /** * Only the handler call is retried. A streaming handler returns before any @@ -241,6 +272,8 @@ export class BlockExecutor { : handler.execute(blockCtx, block, resolvedInputs, nodeMetadata) ) + completedHandlerCost = readTrustedExecutionCost(output) + const isStreamingExecution = output && typeof output === 'object' && 'stream' in output && 'execution' in output @@ -416,7 +449,8 @@ export class BlockExecutor { inputDisplayRegistry, isSentinel, 'execution', - streamingPartialOutput + streamingPartialOutput, + completedHandlerCost ) } finally { commitBlockRegistry() @@ -506,15 +540,40 @@ export class BlockExecutor { const policy = resolveBlockRetryPolicy(block) if (!policy) return invoke() + const shouldAccumulateFunctionCost = block.metadata?.id === BlockType.FUNCTION + let accumulatedFunctionCost: TrustedExecutionCost | undefined let tries = 0 try { for (;;) { tries++ try { - return await invoke() + const output = await invoke() + if (!shouldAccumulateFunctionCost || !accumulatedFunctionCost || !isRecordLike(output)) { + return output + } + + const totalCost = addTrustedExecutionCosts( + accumulatedFunctionCost, + readTrustedExecutionCost(output) + ) + if (!totalCost) return output + + const outputWithCost = { ...output, cost: totalCost } + attachTrustedExecutionCost(outputWithCost, totalCost) + return outputWithCost as T } catch (error) { + if (shouldAccumulateFunctionCost) { + accumulatedFunctionCost = addTrustedExecutionCosts( + accumulatedFunctionCost, + readTrustedExecutionCost(error) + ) + } + const isFinalTry = tries >= policy.maxTries - if (isFinalTry || ctx.abortSignal?.aborted || !isRetryableBlockError(error)) throw error + if (isFinalTry || ctx.abortSignal?.aborted || !isRetryableBlockError(error)) { + attachTrustedExecutionCost(error, accumulatedFunctionCost) + throw error + } this.execLogger.warn('Block failed; retrying', { blockId: block.id, @@ -528,7 +587,10 @@ export class BlockExecutor { if (policy.waitBetweenTriesMs > 0) await sleep(policy.waitBetweenTriesMs) /** `sleep` is not abort-aware, so a run stopped mid-wait must not start another try. */ - if (ctx.abortSignal?.aborted) throw error + if (ctx.abortSignal?.aborted) { + attachTrustedExecutionCost(error, accumulatedFunctionCost) + throw error + } } } } finally { @@ -548,7 +610,8 @@ export class BlockExecutor { inputDisplayRegistry: ResolvedSecretTraceRegistry | undefined, isSentinel: boolean, phase: 'input_resolution' | 'execution', - streamingPartialOutput?: Record + streamingPartialOutput?: Record, + completedHandlerCost?: TrustedExecutionCost ): Promise { const endedAt = new Date().toISOString() const duration = performance.now() - startTime @@ -620,8 +683,10 @@ export class BlockExecutor { return softOutput } + const trustedExecutionCost = readTrustedExecutionCost(error) ?? completedHandlerCost const errorOutput: NormalizedBlockOutput = { error: errorMessage, + ...(trustedExecutionCost ? { cost: trustedExecutionCost } : {}), } // Keep any answer text already drained before timeout/failure so logs match diff --git a/apps/sim/executor/execution/engine.test.ts b/apps/sim/executor/execution/engine.test.ts index 2677a9f89a6..49163f6b3ad 100644 --- a/apps/sim/executor/execution/engine.test.ts +++ b/apps/sim/executor/execution/engine.test.ts @@ -28,7 +28,7 @@ import { EDGE } from '@/executor/constants' import type { DAG, DAGNode } from '@/executor/dag/builder' import type { EdgeManager } from '@/executor/execution/edge-manager' import type { NodeExecutionOrchestrator } from '@/executor/orchestrators/node' -import type { ExecutionContext } from '@/executor/types' +import type { ExecutionContext, ExecutionResult } from '@/executor/types' import { ResolvedSecretTraceRegistry } from '@/executor/utils/resolved-secret-trace-registry' import type { SerializedBlock } from '@/serializer/types' import { ExecutionEngine } from './engine' @@ -275,6 +275,39 @@ describe('ExecutionEngine', () => { expect(provenance?.entries).toEqual([{ name: 'TOKEN', encryptedValue: 'ciphertext' }]) }) + /** + * The crossing at the copilot boundary reads the absence of an attached result as "no block + * ran", so the attach has to be total. A block failure is normalized on the way in, so only + * a non-Error raised by `run`'s own work — here the cancellation subscribe it awaits before + * the queue — reaches the catch untouched and exercises the guarantee. + */ + it('attaches the execution result to a non-Error thrown by its own work', async () => { + const node = createMockNode('function-1', 'function') + const registry = new ResolvedSecretTraceRegistry([ + { name: 'TOKEN', plaintext: 'secret-value-1234', encryptedValue: 'ciphertext' }, + ]) + registry.recordResolved('TOKEN', 'secret-value-1234') + const context = createMockContext({ + decisions: { router: new Map(), condition: new Map() }, + resolvedSecretTraceRegistry: registry, + }) + mockIsExecutionCancelled.mockRejectedValueOnce('cancellation lookup exploded') + + const engine = new ExecutionEngine( + context, + createMockDAG([node]), + createMockEdgeManager(), + createMockNodeOrchestrator() + ) + + const thrown = await engine.run(node.id).catch((error: unknown) => error) + + expect(thrown).toBeInstanceOf(Error) + const attached = (thrown as Error & { executionResult?: ExecutionResult }).executionResult + expect(attached).toBeDefined() + expect(attached?.executionState?.resolvedSecretTraceProvenance).toBeDefined() + }) + /** Deriving must not weaken the guarantee: a latched registry still exports incomplete. */ it('keeps the final output envelope incomplete when the registry latched', async () => { const node = createMockNode('loop-1', 'loop') diff --git a/apps/sim/executor/execution/engine.ts b/apps/sim/executor/execution/engine.ts index acfa7cfa42b..b6091f395e7 100644 --- a/apps/sim/executor/execution/engine.ts +++ b/apps/sim/executor/execution/engine.ts @@ -185,10 +185,17 @@ export class ExecutionEngine { metadata: this.context.metadata, } - if (error instanceof Error) { - attachExecutionResult(error, executionResult) - } - throw error + /** + * Normalized first so the attach is total rather than conditional on the throw already + * being an `Error`. A block failure is normalized on the way in, so the old guard held in + * practice; what it did not give was a guarantee. The copilot crossing reads a missing + * result as proof that no block ran, and that inference has to hold for every throw out of + * here, including a non-`Error` raised by this file's own synchronous work. `toError` + * returns an `Error` unchanged, so ordinary failures keep their identity and their type. + */ + const thrown = toError(error) + attachExecutionResult(thrown, executionResult) + throw thrown } finally { this.cleanup() } diff --git a/apps/sim/executor/execution/types.ts b/apps/sim/executor/execution/types.ts index 35cf038381f..ba7e7db6a8a 100644 --- a/apps/sim/executor/execution/types.ts +++ b/apps/sim/executor/execution/types.ts @@ -56,6 +56,7 @@ export interface ExecutionMetadata { edges: Edge[] loops?: Record parallels?: Record + variables?: Record deploymentVersionId?: string } largeValueExecutionIds?: string[] diff --git a/apps/sim/executor/handlers/function/function-handler.test.ts b/apps/sim/executor/handlers/function/function-handler.test.ts index c914d2f05f9..72fe2ff4262 100644 --- a/apps/sim/executor/handlers/function/function-handler.test.ts +++ b/apps/sim/executor/handlers/function/function-handler.test.ts @@ -1,9 +1,11 @@ import { beforeEach, describe, expect, it, type Mock, vi } from 'vitest' import { createTimeoutAbortController } from '@/lib/core/execution-limits' import { DEFAULT_EXECUTION_TIMEOUT_MS } from '@/lib/execution/constants' +import { NonRetryableExecutionError } from '@/lib/execution/non-retryable-error' import { BlockType } from '@/executor/constants' import { FunctionBlockHandler } from '@/executor/handlers/function/function-handler' import type { ExecutionContext } from '@/executor/types' +import { readTrustedExecutionCost } from '@/executor/utils/errors' import { FUNCTION_BLOCK_CONTEXT_VARS_KEY, FUNCTION_BLOCK_DISPLAY_CODE_KEY, @@ -254,6 +256,45 @@ describe('FunctionBlockHandler', () => { expect(mockExecuteTool).toHaveBeenCalled() }) + it.each([ + { retryable: true, nonRetryable: false }, + { retryable: false, nonRetryable: true }, + ])( + 'attaches trusted cost to a failed execution when retryable is $retryable', + async ({ retryable, nonRetryable }) => { + const cost = { input: 0, output: 0, total: 0.125 } + mockExecuteTool.mockResolvedValue({ + success: false, + error: 'Remote Function failed', + retryable, + output: { result: null, stdout: '', cost }, + }) + + let thrown: unknown + try { + await handler.execute(mockContext, mockBlock, { code: 'throw new Error("failed")' }) + } catch (error) { + thrown = error + } + + expect(thrown).toBeInstanceOf(Error) + expect(thrown instanceof NonRetryableExecutionError).toBe(nonRetryable) + expect(readTrustedExecutionCost(thrown)).toEqual(cost) + } + ) + + it('attaches trusted cost to a successful execution for retry aggregation', async () => { + const cost = { input: 0, output: 0, total: 0.25 } + mockExecuteTool.mockResolvedValue({ + success: true, + output: { result: 42, stdout: '', cost }, + }) + + const output = await handler.execute(mockContext, mockBlock, { code: 'return 42' }) + + expect(readTrustedExecutionCost(output)).toEqual(cost) + }) + it('should pass runtime context variables to function_execute', async () => { const contextVariables = { __blockRef_0: { result: 'from-block' } } diff --git a/apps/sim/executor/handlers/function/function-handler.ts b/apps/sim/executor/handlers/function/function-handler.ts index aefb5ab39d4..22d0b3b9938 100644 --- a/apps/sim/executor/handlers/function/function-handler.ts +++ b/apps/sim/executor/handlers/function/function-handler.ts @@ -12,6 +12,7 @@ import { mergeFileKeys, mergeLargeValueKeys } from '@/lib/execution/payloads/acc import { BlockType } from '@/executor/constants' import type { BlockHandler, ExecutionContext } from '@/executor/types' import { collectBlockData } from '@/executor/utils/block-data' +import { attachTrustedExecutionCost } from '@/executor/utils/errors' import { FUNCTION_BLOCK_CONTEXT_VARS_KEY, FUNCTION_BLOCK_DISPLAY_CODE_KEY, @@ -111,15 +112,18 @@ export class FunctionBlockHandler implements BlockHandler { const result = await executeTool('function_execute', toolParams, { executionContext: ctx }) if (!result.success) { - if (result.retryable === false) { - throw new NonRetryableExecutionError(result.error || 'Function execution is indeterminate') - } - throw new Error(result.error || 'Function execution failed') + const error = + result.retryable === false + ? new NonRetryableExecutionError(result.error || 'Function execution is indeterminate') + : new Error(result.error || 'Function execution failed') + attachTrustedExecutionCost(error, result.output?.cost) + throw error } mergeLargeValueKeys(ctx, result.largeValueKeys ?? []) mergeFileKeys(ctx, result.fileKeys ?? []) + attachTrustedExecutionCost(result.output, result.output?.cost) return result.output } } diff --git a/apps/sim/executor/handlers/pi/cloud/authoring/backend.ts b/apps/sim/executor/handlers/pi/cloud/authoring/backend.ts index d874c6ab477..40d19e23385 100644 --- a/apps/sim/executor/handlers/pi/cloud/authoring/backend.ts +++ b/apps/sim/executor/handlers/pi/cloud/authoring/backend.ts @@ -435,7 +435,10 @@ async function runCloudAuthoringPi( const lifetimeMs = resolvePiRunLifetimeMs(context.signal) const piTimeoutMs = resolvePiTimeoutMs(lifetimeMs) - const authored = await withPiSandbox({ lifetimeMs }, async (runner) => { + // Bound to a local so the call stays on one line: inlining the second option + // reflows this whole callback body and buries the change in re-indentation. + const sandboxOptions = { lifetimeMs, cost: context.sandboxCost } + const authored = await withPiSandbox(sandboxOptions, async (runner) => { try { const clone = await raceAbort( runner.run(params.mode === 'cloud' ? CREATE_PR_CLONE_SCRIPT : UPDATE_BRANCH_CLONE_SCRIPT, { diff --git a/apps/sim/executor/handlers/pi/cloud/babysit/backend.ts b/apps/sim/executor/handlers/pi/cloud/babysit/backend.ts index 6860dee6a4e..08d4af65ac5 100644 --- a/apps/sim/executor/handlers/pi/cloud/babysit/backend.ts +++ b/apps/sim/executor/handlers/pi/cloud/babysit/backend.ts @@ -784,7 +784,7 @@ export async function runBabysitPiWithOptions( const lifetimeMs = resolvePiRunLifetimeMs(context.signal) const piTimeoutMs = resolvePiTimeoutMs(lifetimeMs) - return await withPiSandbox({ lifetimeMs }, async (runner) => { + return await withPiSandbox({ lifetimeMs, cost: context.sandboxCost }, async (runner) => { const clone = await raceAbort( runner.run(BABYSIT_CLONE_SCRIPT, { envs: { diff --git a/apps/sim/executor/handlers/pi/cloud/plan/backend.ts b/apps/sim/executor/handlers/pi/cloud/plan/backend.ts index 27444a4493b..3997fac8eee 100644 --- a/apps/sim/executor/handlers/pi/cloud/plan/backend.ts +++ b/apps/sim/executor/handlers/pi/cloud/plan/backend.ts @@ -80,7 +80,7 @@ export const runCloudPlanPi: PiBackendRun = async (params, const thinking = mapThinkingLevel(params.thinkingLevel) ?? 'medium' const lifetimeMs = resolvePiRunLifetimeMs(context.signal) - return withPiSandbox({ lifetimeMs }, async (runner) => { + return withPiSandbox({ lifetimeMs, cost: context.sandboxCost }, async (runner) => { try { const clone = await raceAbort( runner.run(PLAN_CLONE_SCRIPT, { diff --git a/apps/sim/executor/handlers/pi/cloud/review/backend.ts b/apps/sim/executor/handlers/pi/cloud/review/backend.ts index cfd25533952..289416ce96b 100644 --- a/apps/sim/executor/handlers/pi/cloud/review/backend.ts +++ b/apps/sim/executor/handlers/pi/cloud/review/backend.ts @@ -218,7 +218,7 @@ export const runCloudReviewPi: PiBackendRun = async (par const lifetimeMs = resolvePiRunLifetimeMs(context.signal) try { - return await withPiSandbox({ lifetimeMs }, async (runner) => { + return await withPiSandbox({ lifetimeMs, cost: context.sandboxCost }, async (runner) => { await runner.writeFile(GIT_ASKPASS_PATH, GIT_ASKPASS_SCRIPT) const fetched = await raceAbort( runner.run(FETCH_PR_SCRIPT, { diff --git a/apps/sim/executor/handlers/pi/core/backend.ts b/apps/sim/executor/handlers/pi/core/backend.ts index 4fb3b3ee3d2..488a0d50286 100644 --- a/apps/sim/executor/handlers/pi/core/backend.ts +++ b/apps/sim/executor/handlers/pi/core/backend.ts @@ -9,6 +9,7 @@ */ import type { TSchema } from 'typebox' +import type { SandboxCostSink } from '@/lib/execution/remote-sandbox/types' import type { SSHConnectionConfig } from '@/lib/internal/ssh/client' import type { Message } from '@/executor/handlers/agent/types' import type { PiEvent, PiRunTotals } from '@/executor/handlers/pi/core/events' @@ -172,6 +173,20 @@ export type PiRunParams = export interface PiRunContext { onEvent: (event: PiEvent) => void signal?: AbortSignal + /** + * Where a backend reports the cost of Sim-provisioned compute it used. + * + * Both modes can fill it, from different sources. Cloud modes run the agent in + * a Sim-paid sandbox and report that session. Local mode drives the caller's + * own machine over SSH, so the agent itself costs Sim nothing — but the Sim + * tools it calls still run here, and a `function_execute` among them bills its + * own remote sandbox into the same total. + * + * The handler folds whatever lands here into the block's `toolCost`, which is + * what keeps a BYOK Pi run — model unbilled by definition — from reporting no + * cost at all for compute Sim actually paid for. + */ + sandboxCost?: SandboxCostSink } /** Final result of a Pi run. */ diff --git a/apps/sim/executor/handlers/pi/local/sim-tools.test.ts b/apps/sim/executor/handlers/pi/local/sim-tools.test.ts index 60b99bb95ac..6427fc3cbf7 100644 --- a/apps/sim/executor/handlers/pi/local/sim-tools.test.ts +++ b/apps/sim/executor/handlers/pi/local/sim-tools.test.ts @@ -225,6 +225,55 @@ describe('buildSimToolSpecs', () => { }) }) + it('accumulates cost from canonical Function results while preserving failures', async () => { + mockTransformBlockTool + .mockResolvedValueOnce({ + id: 'function_execute', + name: 'Function Execute', + description: 'Execute code', + params: {}, + parameters: { type: 'object', properties: {} }, + }) + .mockResolvedValueOnce({ + id: 'exa_search', + name: 'Exa Search', + description: 'Search the web', + params: {}, + parameters: { type: 'object', properties: {} }, + }) + const functionToolCost = { total: 0 } + const [functionSpec, searchSpec] = await buildSimToolSpecs( + executionContext(undefined), + [ + { type: 'function', operation: 'execute', usageControl: 'auto' }, + { type: 'exa', operation: 'exa_search', usageControl: 'auto' }, + ], + functionToolCost + ) + + mockExecuteTool + .mockResolvedValueOnce({ + success: true, + output: { result: 'ok', cost: { total: 0.125 } }, + }) + .mockResolvedValueOnce({ + success: true, + output: { result: 'search result', cost: { total: 4 } }, + }) + .mockResolvedValueOnce({ + success: false, + output: { cost: { total: 8 } }, + error: 'execution failed', + }) + + await functionSpec.execute({}) + await searchSpec.execute({}) + const failedResult = await functionSpec.execute({}) + + expect(functionToolCost.total).toBe(8.125) + expect(failedResult).toEqual({ text: 'execution failed', isError: true }) + }) + it('projects named provenance in successful Sim tool output', async () => { mockToolAdapter({ apiKey: 'secret-value' }) encryptionMockFns.mockDecryptSecret.mockResolvedValue({ decrypted: 'secret-value' }) diff --git a/apps/sim/executor/handlers/pi/local/sim-tools.ts b/apps/sim/executor/handlers/pi/local/sim-tools.ts index 483eb87f0a6..876bb7e99ee 100644 --- a/apps/sim/executor/handlers/pi/local/sim-tools.ts +++ b/apps/sim/executor/handlers/pi/local/sim-tools.ts @@ -9,6 +9,7 @@ import { createLogger } from '@sim/logger' import { getErrorMessage } from '@sim/utils/errors' +import type { SandboxCostSink } from '@/lib/execution/remote-sandbox/types' import { readWorkflowInputFieldsForTool, readWorkflowMetadataForTool, @@ -107,7 +108,8 @@ function buildSimToolSpec( ctx: ExecutionContext, inputTools: ToolInput[], provider: ProviderToolConfig, - toolIndex: number + toolIndex: number, + sandboxCost?: SandboxCostSink ): PiToolSpec { const toolId = provider.canonicalId ?? provider.id const preseededParams = provider.params || {} @@ -170,6 +172,20 @@ function buildSimToolSpec( resolvedSecretTraceRegistry: toolCallRegistry, } ) + const resultCost = result.output?.cost + const resultCostTotal = + resultCost && typeof resultCost === 'object' + ? (resultCost as Record).total + : undefined + if ( + toolId === 'function_execute' && + sandboxCost && + typeof resultCostTotal === 'number' && + Number.isFinite(resultCostTotal) && + resultCostTotal > 0 + ) { + sandboxCost.total += resultCostTotal + } const projection = projectToolResult(result, toolCallRegistry?.forkForPropagatedEntries()) if (projection.safe && registry && toolCallRegistry?.isComplete()) { registry.mergeToolCallRegistry(toolCallRegistry) @@ -199,7 +215,8 @@ function buildSimToolSpec( */ export async function buildSimToolSpecs( ctx: ExecutionContext, - inputTools: unknown + inputTools: unknown, + sandboxCost?: SandboxCostSink ): Promise { if (!Array.isArray(inputTools)) return [] @@ -243,6 +260,6 @@ export async function buildSimToolSpecs( await annotateDuplicateToolBindings(ctx, providers) assignProviderToolIdentities(providers) return configuredTools.map(({ provider, toolIndex }) => - buildSimToolSpec(ctx, inputTools, provider, toolIndex) + buildSimToolSpec(ctx, inputTools, provider, toolIndex, sandboxCost) ) } diff --git a/apps/sim/executor/handlers/pi/pi-handler.test.ts b/apps/sim/executor/handlers/pi/pi-handler.test.ts index 063b319c6ff..d6dcc62fb41 100644 --- a/apps/sim/executor/handlers/pi/pi-handler.test.ts +++ b/apps/sim/executor/handlers/pi/pi-handler.test.ts @@ -20,6 +20,7 @@ const { mockResolveSearchKey, mockBuildSearchTool, mockAssertPermissionsAllowed, + mockBuildSimToolSpecs, MockToolNotAllowedError, } = vi.hoisted(() => ({ mockRunLocal: vi.fn(), @@ -38,6 +39,7 @@ const { mockResolveSearchKey: vi.fn(), mockBuildSearchTool: vi.fn(), mockAssertPermissionsAllowed: vi.fn(), + mockBuildSimToolSpecs: vi.fn(), MockToolNotAllowedError: class ToolNotAllowedError extends Error {}, })) @@ -64,7 +66,7 @@ vi.mock('@/executor/handlers/pi/core/context', () => ({ appendPiMemory: mockAppendMemory, })) vi.mock('@/executor/handlers/pi/local/sim-tools', () => ({ - buildSimToolSpecs: vi.fn().mockResolvedValue([]), + buildSimToolSpecs: mockBuildSimToolSpecs, })) vi.mock('@/executor/handlers/pi/local/backend', () => ({ runLocalPi: mockRunLocal })) vi.mock('@/executor/handlers/pi/cloud/authoring/backend', () => ({ @@ -109,8 +111,10 @@ vi.mock('@/blocks/utils', () => ({ }, })) +import type { PiRunContext } from '@/executor/handlers/pi/core/backend' import { PiBlockHandler, parsePiReviewMentions } from '@/executor/handlers/pi/pi-handler' import type { ExecutionContext, StreamingExecution } from '@/executor/types' +import { readTrustedExecutionCost } from '@/executor/utils/errors' import { ResolvedSecretTraceRegistry } from '@/executor/utils/resolved-secret-trace-registry' import type { SerializedBlock } from '@/serializer/types' @@ -153,6 +157,7 @@ describe('PiBlockHandler', () => { mockResolveSearchKey.mockReturnValue('search-key') mockBuildSearchTool.mockReturnValue({ name: 'web_search' }) mockAssertPermissionsAllowed.mockResolvedValue(undefined) + mockBuildSimToolSpecs.mockResolvedValue([]) mockResolveSkills.mockResolvedValue([]) mockLoadMemory.mockResolvedValue([]) mockAppendMemory.mockResolvedValue(undefined) @@ -275,6 +280,83 @@ describe('PiBlockHandler', () => { expect((output as Record).content).toBe('hi') }) + it('adds successful Function tool cost once to a non-streaming Local Dev result', async () => { + mockBuildSimToolSpecs.mockImplementation( + async (_ctx: unknown, _tools: unknown, functionToolCost: { total: number }) => { + functionToolCost.total += 0.125 + return [] + } + ) + + const output = (await handler.execute(ctx(), block, localInputs())) as { cost: unknown } + + expect(output.cost).toEqual({ input: 0, output: 0, toolCost: 0.125, total: 0.125 }) + }) + + it('bills the cloud sandbox a Pi session ran in, even when the model is BYOK', async () => { + // The regression this guards: the agent's own sandbox runs on Sim's provider + // account, so a BYOK run whose model cost is zero by definition would + // otherwise report no cost at all for tens of minutes of paid compute. + mockRunCloud.mockImplementation(async (_params: unknown, context: PiRunContext) => { + if (context.sandboxCost) context.sandboxCost.total += 0.0842 + return { totals: { finalText: 'done', inputTokens: 0, outputTokens: 0 } } + }) + + const output = (await handler.execute(ctx(), block, { + mode: 'cloud', + task: 'do it', + model: 'claude', + owner: 'o', + repo: 'r', + githubToken: 'ghp', + })) as { cost: unknown } + + expect(output.cost).toEqual({ input: 0, output: 0, toolCost: 0.0842, total: 0.0842 }) + }) + + it('keeps the sandbox charge on a cloud session whose agent reported an error', async () => { + // The backend returned, so the sandbox was billed and the sink holds the + // charge — but this path throws instead of reaching buildOutput, which is + // what would otherwise have published it. + mockRunCloud.mockImplementation(async (_params: unknown, context: PiRunContext) => { + if (context.sandboxCost) context.sandboxCost.total += 0.0631 + return { + totals: { finalText: '', inputTokens: 0, outputTokens: 0, errorMessage: 'agent gave up' }, + } + }) + + const error = await handler + .execute(ctx(), block, { + mode: 'cloud', + task: 'do it', + model: 'claude', + owner: 'o', + repo: 'r', + githubToken: 'ghp', + }) + .catch((thrown: unknown) => thrown) + + expect(error).toBeInstanceOf(Error) + // `toolCost` is absent by design: the trusted envelope validates exactly the + // three numeric fields it will let cross the handler boundary. `total` is + // what the ledger bills on, and it carries the sandbox charge intact. + expect(readTrustedExecutionCost(error)).toEqual({ input: 0, output: 0, total: 0.0631 }) + }) + + it('leaves a cloud run that provisioned no sandbox uncharged', async () => { + const output = (await handler.execute(ctx(), block, { + mode: 'cloud', + task: 'do it', + model: 'claude', + owner: 'o', + repo: 'r', + githubToken: 'ghp', + })) as { cost: Record } + + expect(output.cost.toolCost).toBeUndefined() + expect(output.cost.total).toBe(0) + }) + it('routes Create PR to the cloud backend and surfaces PR output', async () => { const output = (await handler.execute(ctx(), block, { mode: 'cloud', @@ -942,6 +1024,12 @@ describe('PiBlockHandler', () => { }) it('streams text when the block is selected for streaming output', async () => { + mockBuildSimToolSpecs.mockImplementation( + async (_ctx: unknown, _tools: unknown, functionToolCost: { total: number }) => { + functionToolCost.total += 0.25 + return [] + } + ) mockRunLocal.mockImplementation(async (_params, runCtx) => { runCtx.onEvent({ type: 'text', text: 'streamed' }) return { totals: { finalText: 'streamed', inputTokens: 0, outputTokens: 0, toolCalls: [] } } @@ -965,6 +1053,12 @@ describe('PiBlockHandler', () => { } expect(text).toContain('streamed') expect(result.execution.output.content).toBe('streamed') + expect(result.execution.output.cost).toEqual({ + input: 0, + output: 0, + toolCost: 0.25, + total: 0.25, + }) }) it('streams only the canonical final document for Plan mode', async () => { diff --git a/apps/sim/executor/handlers/pi/pi-handler.ts b/apps/sim/executor/handlers/pi/pi-handler.ts index 2f42719b71c..97f674ed1c8 100644 --- a/apps/sim/executor/handlers/pi/pi-handler.ts +++ b/apps/sim/executor/handlers/pi/pi-handler.ts @@ -8,6 +8,7 @@ import { createLogger } from '@sim/logger' import { projectResolvedModelInput } from '@/lib/execution/model-input-provenance' +import type { SandboxCostSink } from '@/lib/execution/remote-sandbox/types' import type { BlockOutput } from '@/blocks/types' import { parseOptionalNumberInput } from '@/blocks/utils' import { @@ -36,6 +37,7 @@ import { type PiMemoryConfig, resolvePiSkills, } from '@/executor/handlers/pi/core/context' +import type { PiRunTotals } from '@/executor/handlers/pi/core/events' import { streamTextForEvent } from '@/executor/handlers/pi/core/events' import { computePiCost, @@ -53,7 +55,9 @@ import type { NormalizedBlockOutput, StreamingExecution, } from '@/executor/types' +import { attachTrustedExecutionCost } from '@/executor/utils/errors' import { refuseResolvedSecretProjection } from '@/executor/utils/resolved-secret-projection-refusal' +import type { ModelCost } from '@/providers/cost-policy' import { isPiSupportedProvider, resolvePiModelId } from '@/providers/pi-providers' import { getProviderFromModel } from '@/providers/utils' import type { SerializedBlock } from '@/serializer/types' @@ -154,6 +158,34 @@ export function parsePiReviewMentions(value: unknown): string[] { return mentions } +/** + * What a Pi block charges: its model tokens plus the Sim-paid sandbox compute. + * + * Sandbox cost rides in `toolCost` so it survives a BYOK run — the model side is + * zero by definition there, and the ledger bills a model row on `total > 0`. + * Folding it in is what makes a BYOK Pi session bill for the provider time it + * actually consumed instead of nothing at all. + * + * Shared with the failure path deliberately: an agent that ran and then reported + * an error consumed exactly the same tokens and sandbox seconds as one that + * succeeded, so both have to arrive at the same number. + */ +function buildPiCost( + model: string, + isBYOK: boolean, + totals: PiRunTotals, + sandboxCost: number +): ModelCost { + const modelCost = computePiCost(model, totals.inputTokens, totals.outputTokens, isBYOK) + if (sandboxCost <= 0) return modelCost + + return { + ...modelCost, + toolCost: sandboxCost, + total: modelCost.total + sandboxCost, + } +} + export class PiBlockHandler implements BlockHandler { canHandle(block: SerializedBlock): boolean { return block.metadata?.id === BlockType.PI @@ -267,7 +299,8 @@ export class PiBlockHandler implements BlockHandler { } const usePrivateKey = inputs.authMethod === 'privateKey' const port = parseOptionalNumberInput(inputs.port, 'port', { integer: true, min: 1 }) ?? 22 - const tools = await buildSimToolSpecs(ctx, inputs.tools) + const sandboxCost: SandboxCostSink = { total: 0 } + const tools = await buildSimToolSpecs(ctx, inputs.tools, sandboxCost) const params: PiLocalRunParams = { ...contextualBase, mode: 'local', @@ -282,7 +315,7 @@ export class PiBlockHandler implements BlockHandler { passphrase: usePrivateKey ? asRawString(inputs.passphrase) : undefined, }, } - return this.runPi(ctx, block, runLocalPi, params, memoryConfig) + return this.runPi(ctx, block, runLocalPi, params, memoryConfig, sandboxCost) } const owner = asOptString(inputs.owner) @@ -473,10 +506,12 @@ export class PiBlockHandler implements BlockHandler { model: string, isBYOK: boolean, startTime: number, - startTimeISO: string + startTimeISO: string, + sandboxCost = 0 ): NormalizedBlockOutput { const { totals } = result const endTime = Date.now() + const cost = buildPiCost(model, isBYOK, totals, sandboxCost) return { content: totals.finalText, model, @@ -505,7 +540,7 @@ export class PiBlockHandler implements BlockHandler { output: totals.outputTokens, total: totals.inputTokens + totals.outputTokens, }, - cost: computePiCost(model, totals.inputTokens, totals.outputTokens, isBYOK), + cost, providerTiming: { startTime: startTimeISO, endTime: new Date(endTime).toISOString(), @@ -519,7 +554,15 @@ export class PiBlockHandler implements BlockHandler { block: SerializedBlock, backend: PiBackendRun

, params: P, - memoryConfig?: PiMemoryConfig + memoryConfig?: PiMemoryConfig, + /** + * One sink for every Sim-paid sandbox this block touches. Local mode fills it + * from the Function tools it runs host-side; cloud modes fill it from the + * sandbox the agent itself runs in. They are mutually exclusive in practice, + * and sharing one total means neither can be forgotten at the point the cost + * is folded into the block's output. + */ + sandboxCost: SandboxCostSink = { total: 0 } ): Promise { const startTime = Date.now() const startTimeISO = new Date(startTime).toISOString() @@ -545,9 +588,15 @@ export class PiBlockHandler implements BlockHandler { if (text) controller.enqueue(encoder.encode(text)) }, signal: ctx.abortSignal, + sandboxCost, }) if (result.totals.errorMessage) { - controller.error(new Error(result.totals.errorMessage)) + const error = new Error(result.totals.errorMessage) + attachTrustedExecutionCost( + error, + buildPiCost(params.model, params.isBYOK, result.totals, sandboxCost.total) + ) + controller.error(error) return } if (params.mode === 'cloud_plan' && result.totals.finalText) { @@ -561,7 +610,8 @@ export class PiBlockHandler implements BlockHandler { params.model, params.isBYOK, startTime, - startTimeISO + startTimeISO, + sandboxCost.total ) ) if (memoryConfig) { @@ -592,9 +642,25 @@ export class PiBlockHandler implements BlockHandler { } } - const result = await backend(params, { onEvent: () => {}, signal: ctx.abortSignal }) + const result = await backend(params, { + onEvent: () => {}, + signal: ctx.abortSignal, + sandboxCost, + }) if (result.totals.errorMessage) { - throw new Error(result.totals.errorMessage) + /* + * The backend returned, so the sandbox was billed and the sink holds the + * charge — but this throw skips `buildOutput`, which is what would have + * published it. Carrying the cost on the error is what keeps a session + * whose agent reported a failure from being run for free, the same way the + * Function handler carries its tool cost onto the error it raises. + */ + const error = new Error(result.totals.errorMessage) + attachTrustedExecutionCost( + error, + buildPiCost(params.model, params.isBYOK, result.totals, sandboxCost.total) + ) + throw error } if (memoryConfig) { await appendPiMemory( @@ -610,7 +676,8 @@ export class PiBlockHandler implements BlockHandler { params.model, params.isBYOK, startTime, - startTimeISO + startTimeISO, + sandboxCost.total ) } } diff --git a/apps/sim/executor/types.ts b/apps/sim/executor/types.ts index 649398ed616..8aa9e7cd6ce 100644 --- a/apps/sim/executor/types.ts +++ b/apps/sim/executor/types.ts @@ -403,6 +403,21 @@ export interface ExecutionContext { */ toolBindingLabelCache?: Map + /** + * Files produced during this execution, indexed by {@link UserFile.id}, so a + * model can name one by id in a tool argument and the runtime can hydrate it + * into the full object. + * + * Needed because a file an agent has just seen — a Gmail attachment fetched + * moments ago in the same turn — lives only in that turn's tool results, not + * in any block state or workspace row, so nothing else can resolve it. The + * index only *selects*; every read is still authorized on its own. + * + * A Map for the same reason as {@link toolBindingLabelCache}: `blockCtx` is a + * shallow clone per block execution, so only a shared reference survives. + */ + executionFilesById?: Map + blockStates: ReadonlyMap executedBlocks: ReadonlySet diff --git a/apps/sim/executor/utils/errors.ts b/apps/sim/executor/utils/errors.ts index deb1306e6ac..e3f0a9ee9b8 100644 --- a/apps/sim/executor/utils/errors.ts +++ b/apps/sim/executor/utils/errors.ts @@ -49,6 +49,22 @@ export function attachExecutionResult(error: Error, executionResult: ExecutionRe */ const attemptedExecutionIds = new WeakMap() +/** Cost emitted by a trusted execution boundary and safe to project into a block trace. */ +export interface TrustedExecutionCost { + readonly input: number + readonly output: number + readonly total: number +} + +/** + * Trusted execution costs, keyed by the value crossing the handler boundary. + * + * Cost stays in a side table until the executor deliberately copies it into block output. This + * prevents arbitrary properties on provider errors (or user-thrown values) from becoming billed + * trace data while still allowing a handler to preserve cost when it throws. + */ +const trustedExecutionCosts = new WeakMap() + /** * Names the run a failure belongs to once dispatch has been attempted. * @@ -72,6 +88,46 @@ export function readAttemptedExecutionId(error: unknown): string | undefined { return isRecordedThrown(error) ? attemptedExecutionIds.get(error) : undefined } +/** Attaches a validated, Sim-produced execution cost to an object crossing the handler boundary. */ +export function attachTrustedExecutionCost(subject: unknown, cost: unknown): void { + if (!isRecordedThrown(subject)) return + + const normalizedCost = normalizeTrustedExecutionCost(cost) + if (!normalizedCost) return + + trustedExecutionCosts.set(subject, normalizedCost) +} + +/** Reads execution cost only when a trusted caller previously attached it. */ +export function readTrustedExecutionCost(subject: unknown): TrustedExecutionCost | undefined { + return isRecordedThrown(subject) ? trustedExecutionCosts.get(subject) : undefined +} + +function normalizeTrustedExecutionCost(cost: unknown): TrustedExecutionCost | undefined { + if (!cost || typeof cost !== 'object' || Array.isArray(cost)) return undefined + + const candidate = cost as Record + if ( + typeof candidate.input !== 'number' || + !Number.isFinite(candidate.input) || + candidate.input < 0 || + typeof candidate.output !== 'number' || + !Number.isFinite(candidate.output) || + candidate.output < 0 || + typeof candidate.total !== 'number' || + !Number.isFinite(candidate.total) || + candidate.total < 0 + ) { + return undefined + } + + return { + input: candidate.input, + output: candidate.output, + total: candidate.total, + } +} + /** * Any non-null object, not only an `Error`. * diff --git a/apps/sim/executor/utils/resolved-secret-trace-registry.ts b/apps/sim/executor/utils/resolved-secret-trace-registry.ts index 05e8d51cc71..432e0178a85 100644 --- a/apps/sim/executor/utils/resolved-secret-trace-registry.ts +++ b/apps/sim/executor/utils/resolved-secret-trace-registry.ts @@ -226,6 +226,18 @@ export interface ResolvedSecretIncompletenessDiagnostics { export const ANONYMOUS_SECRET_TRACE_REPLACEMENT = OPAQUE_RESOLVED_SECRET_REPLACEMENT export const RESOLVED_SECRET_TRACE_CHECKPOINT_VERSION = 1 +/** + * The envelope for content no secret ever reached: vouched for, naming nothing. + * + * Distinct from an incomplete envelope, which says the opposite — that something may be carried + * and cannot be named. A boundary that knows nothing was resolved should say so with this rather + * than latch, since latching is the claim that redaction is impossible. Returned fresh so no + * caller shares a value it may serialize or extend. + */ +export function emptyResolvedSecretTraceProvenance(): ResolvedSecretTraceProvenanceV1 { + return { version: 1, complete: true, entries: [] } +} + const MAX_PROVENANCE_ENTRIES = PROVENANCE_MAX_ENTRIES const MAX_SERIALIZED_PROVENANCE_BYTES = PROVENANCE_MAX_SERIALIZED_BYTES const MAX_TRACE_CATALOG_ENTRIES = PROVENANCE_MAX_ENTRIES diff --git a/apps/sim/executor/variables/resolver.ts b/apps/sim/executor/variables/resolver.ts index f5e50e2d50a..94c0bee918e 100644 --- a/apps/sim/executor/variables/resolver.ts +++ b/apps/sim/executor/variables/resolver.ts @@ -10,10 +10,14 @@ import { isLargeValueRef, type LargeValueRef, } from '@/lib/execution/payloads/large-value-ref' +import { + createSandboxFileMountRef, + isSandboxFileMountRef, +} from '@/lib/execution/payloads/sandbox-file-mount-ref' import { isLikelyReferenceSegment } from '@/lib/workflows/sanitization/references' import { BlockType, parseReferencePath, REFERENCE } from '@/executor/constants' import type { ExecutionState, LoopScope } from '@/executor/execution/state' -import type { ExecutionContext } from '@/executor/types' +import type { ExecutionContext, UserFile } from '@/executor/types' import { createEnvVarPattern, createReferencePattern } from '@/executor/utils/reference-validation' import { BlockResolver } from '@/executor/variables/resolvers/block' import { EnvResolver } from '@/executor/variables/resolvers/env' @@ -453,6 +457,19 @@ export class VariableResolver { displayCursor = index + match.length try { + const sandboxFilePath = await this.resolveSandboxFilePathReference( + match, + resolutionContext, + language, + template, + index, + contextVarAccumulator + ) + if (sandboxFilePath) { + displayResult += sandboxFilePath.display + return sandboxFilePath.replacement + } + const lazyBase64 = await this.resolveLazyFileBase64Reference( match, resolutionContext, @@ -648,6 +665,107 @@ export class VariableResolver { return { resolvedCode: result, displayCode: displayResult } } + /** + * Resolves `` to the file's location on the sandbox filesystem. + * + * The counterpart to the `base64` reference above, and deliberately unlike it in + * two ways. It is not gated on the JavaScript runtime helpers, because a path is + * just a string and Python and Shell need it more than JavaScript does. And it + * stores a mount marker rather than the path itself: the sandbox does not exist + * yet at resolution time, and paths are assigned only once the whole mount set is + * known, since they are sanitized and de-duplicated together. + */ + private async resolveSandboxFilePathReference( + reference: string, + context: ResolutionContext, + language: string | undefined, + template: string, + matchIndex: number, + contextVarAccumulator: Record + ): Promise<{ replacement: string; display: string } | null> { + const parts = parseReferencePath(reference) + if (parts.length < 3 || parts.at(-1) !== 'path') { + return null + } + + const fileReference = `${REFERENCE.START}${parts.slice(0, -1).join(REFERENCE.PATH_DELIMITER)}${REFERENCE.END}` + const file = await this.resolveReference(fileReference, context) + if (!isUserFileWithMetadata(file) || !file.key) { + return null + } + + // Reuse the marker already standing for this file so a path referenced twice + // costs one context variable rather than two. What keeps it to one mount is + // `planUserFileMounts`, which collapses by storage key across every source — + // this only keeps the duplicate out of the request body. + const existing = Object.entries(contextVarAccumulator).find( + ([, value]) => isSandboxFileMountRef(value) && value.file.key === file.key + ) + const varName = existing?.[0] ?? `__blockRef_${Object.keys(contextVarAccumulator).length}` + if (!existing) { + // The bytes are fetched into the sandbox, so the inline copy would be dead + // weight in the request body. + const { base64: _base64, ...fileMetadata } = file + contextVarAccumulator[varName] = createSandboxFileMountRef(fileMetadata as UserFile) + } + + return { + replacement: this.formatContextVariablePathReference(varName, language, template, matchIndex), + display: reference, + } + } + + /** + * Formats a mount-path reference for splicing into code. + * + * Unlike {@link formatContextVariableReference}, a path inside a quoted string is + * spliced raw rather than JSON-encoded. The general formatter is right to encode + * an arbitrary value — the author of `""` wants its JSON form — but + * a path is always a plain string, so encoding it would put literal quote + * characters inside the string the code then opens, turning `open('')` + * into a lookup for a filename that begins with `"`. + * + * Splicing raw is safe precisely here: mount paths are built segment by segment + * through `buildStorageKeySegment`, which reduces anything outside + * `[A-Za-z0-9.-]` to `_`, so the value cannot carry a quote, backslash, backtick, + * or `$` that would escape the surrounding literal. + * + * Shell is delegated unchanged — its formatter already closes and reopens a + * single-quoted context around a double-quoted expansion, which expands + * correctly and needs no path-specific case. + */ + private formatContextVariablePathReference( + varName: string, + language: string | undefined, + template: string, + matchIndex: number + ): string { + if (language === 'shell') { + return this.formatShellContextVariableReference(varName, template, matchIndex, '') + } + + const quoteContext = this.getCodeStringQuoteContext(template, matchIndex, language) + + if (language === 'python') { + const expression = `globals()[${JSON.stringify(varName)}]` + if (this.isPythonStringQuoteContext(quoteContext)) { + const quote = this.getCodeStringQuoteToken(quoteContext) + return `${quote} + ${expression} + ${quote}` + } + return expression + } + + const expression = `globalThis[${JSON.stringify(varName)}]` + if (quoteContext === 'template') { + return `\${${expression}}` + } + if (quoteContext === 'single' || quoteContext === 'double') { + const quote = this.getCodeStringQuoteToken(quoteContext) + return `${quote} + ${expression} + ${quote}` + } + return expression + } + private async resolveLazyFileBase64Reference( reference: string, context: ResolutionContext, diff --git a/apps/sim/hooks/queries/general-settings.test.ts b/apps/sim/hooks/queries/general-settings.test.ts new file mode 100644 index 00000000000..bba242a5390 --- /dev/null +++ b/apps/sim/hooks/queries/general-settings.test.ts @@ -0,0 +1,96 @@ +/** + * @vitest-environment node + */ +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const { mockGetBrowserTimezone, mockIsValidTimezone, mockUseQuery } = vi.hoisted(() => ({ + mockGetBrowserTimezone: vi.fn(), + mockIsValidTimezone: vi.fn(), + mockUseQuery: vi.fn(), +})) + +vi.mock('@tanstack/react-query', () => ({ + useMutation: vi.fn(), + useQuery: mockUseQuery, + useQueryClient: vi.fn(), +})) +vi.mock('@/lib/core/utils/timezone', () => ({ + getBrowserTimezone: mockGetBrowserTimezone, + isValidTimezone: mockIsValidTimezone, +})) + +import { useTimezone, useTimezoneState } from '@/hooks/queries/general-settings' + +describe('useTimezone', () => { + beforeEach(() => { + vi.clearAllMocks() + mockGetBrowserTimezone.mockReturnValue('America/Los_Angeles') + mockIsValidTimezone.mockReturnValue(true) + }) + + it('uses the browser timezone while no preference is saved', () => { + mockUseQuery.mockReturnValue({ data: { timezone: null } }) + + expect(useTimezone()).toBe('America/Los_Angeles') + expect(useTimezoneState()).toEqual({ + timezone: 'America/Los_Angeles', + savedTimezone: null, + status: 'ready', + }) + }) + + it('uses a saved timezone instead of the browser fallback', () => { + mockUseQuery.mockReturnValue({ data: { timezone: 'Asia/Kathmandu' } }) + + expect(useTimezone()).toBe('Asia/Kathmandu') + expect(useTimezoneState()).toEqual({ + timezone: 'Asia/Kathmandu', + savedTimezone: 'Asia/Kathmandu', + status: 'ready', + }) + expect(mockGetBrowserTimezone).not.toHaveBeenCalled() + }) + + it('uses the browser timezone for display while preserving an invalid preference', () => { + mockUseQuery.mockReturnValue({ data: { timezone: 'Not/AZone' } }) + mockIsValidTimezone.mockReturnValue(false) + + expect(useTimezoneState()).toEqual({ + timezone: 'America/Los_Angeles', + savedTimezone: 'Not/AZone', + status: 'invalid', + }) + expect(useTimezone()).toBe('America/Los_Angeles') + }) + + it('reads the current setting again after it changes', () => { + let timezone: string | null = 'America/New_York' + mockUseQuery.mockImplementation(() => ({ data: { timezone } })) + + expect(useTimezone()).toBe('America/New_York') + timezone = 'Asia/Tokyo' + expect(useTimezone()).toBe('Asia/Tokyo') + timezone = null + expect(useTimezone()).toBe('America/Los_Angeles') + }) + + it('distinguishes an unresolved preference from an explicit browser fallback', () => { + mockUseQuery.mockReturnValue({ data: undefined, isError: false }) + + expect(useTimezoneState()).toEqual({ + timezone: 'America/Los_Angeles', + savedTimezone: null, + status: 'loading', + }) + }) + + it('reports an unavailable preference instead of treating it as resolved', () => { + mockUseQuery.mockReturnValue({ data: undefined, isError: true }) + + expect(useTimezoneState()).toEqual({ + timezone: 'America/Los_Angeles', + savedTimezone: null, + status: 'error', + }) + }) +}) diff --git a/apps/sim/hooks/queries/general-settings.ts b/apps/sim/hooks/queries/general-settings.ts index 2c3efa310ad..57dfa16780e 100644 --- a/apps/sim/hooks/queries/general-settings.ts +++ b/apps/sim/hooks/queries/general-settings.ts @@ -9,7 +9,7 @@ import { updateUserSettingsContract, } from '@/lib/api/contracts/user' import { syncThemeToNextThemes } from '@/lib/core/utils/theme' -import { getBrowserTimezone } from '@/lib/core/utils/timezone' +import { getBrowserTimezone, isValidTimezone } from '@/lib/core/utils/timezone' const logger = createLogger('GeneralSettingsQuery') @@ -144,13 +144,53 @@ export function useBillingUsageNotifications(): boolean { } /** - * The user's effective scheduling timezone: their saved preference, or the - * browser-detected zone when unset. Use this wherever a task's timezone is - * captured so scheduling honors the account preference rather than the device. + * The user's effective timezone: a valid saved preference, otherwise the browser zone. + * Callers that must distinguish Auto from invalid or unavailable settings use + * {@link useTimezoneState} instead. */ export function useTimezone(): string { - const { data } = useGeneralSettings() - return data?.timezone ?? getBrowserTimezone() + return useTimezoneState().timezone +} + +export interface TimezoneState { + /** Effective, always-valid timezone used by read-only consumers. */ + timezone: string + /** Raw saved preference, or `null` when the browser timezone is intentional. */ + savedTimezone: string | null + status: 'loading' | 'ready' | 'invalid' | 'error' +} + +/** + * The effective timezone together with the raw preference's validity. Time-based + * editors use the status so only an intentional Auto preference may write with the + * browser fallback; loading, invalid, and unavailable preferences remain read-only. + */ +export function useTimezoneState(): TimezoneState { + const { data, isError } = useGeneralSettings() + if (!data) { + return { + timezone: getBrowserTimezone(), + savedTimezone: null, + status: isError ? 'error' : 'loading', + } + } + + const savedTimezone = data.timezone + if (savedTimezone === null) { + return { + timezone: getBrowserTimezone(), + savedTimezone: null, + status: 'ready', + } + } + if (isValidTimezone(savedTimezone)) { + return { timezone: savedTimezone, savedTimezone, status: 'ready' } + } + return { + timezone: getBrowserTimezone(), + savedTimezone, + status: 'invalid', + } } /** diff --git a/apps/sim/lib/api/contracts/hotspots.ts b/apps/sim/lib/api/contracts/hotspots.ts index e030d94f9b0..95a40b983b9 100644 --- a/apps/sim/lib/api/contracts/hotspots.ts +++ b/apps/sim/lib/api/contracts/hotspots.ts @@ -4,10 +4,12 @@ import { privateSecretProvenanceBundleSchema, stringRecordSchema, unknownRecordSchema, + userFileSchema, } from '@/lib/api/contracts/primitives' import { defineRouteContract } from '@/lib/api/contracts/types' import { DEFAULT_CODE_LANGUAGE } from '@/lib/execution/languages' import { PRIVATE_SECRET_PROVENANCE_FIELD } from '@/lib/execution/private-tool-metadata' +import { MAX_BLOCK_MOUNTED_FILES } from '@/lib/execution/remote-sandbox/sandbox-paths' import { MAX_PII_VALIDATION_DETECTED_ENTITIES, MAX_PII_VALIDATION_TEXT_CHARACTERS, @@ -183,6 +185,19 @@ export const functionExecuteBodySchema = z }) .strict() .optional(), + /** + * Platform file objects mounted into the sandbox before the code runs. + * Distinct from `inputs.files`, which names workspace VFS paths: these are + * the same objects tools exchange, so an upstream block's output can be + * mounted without first being written to the workspace. + */ + files: z + .array(userFileSchema) + .max( + MAX_BLOCK_MOUNTED_FILES, + `At most ${MAX_BLOCK_MOUNTED_FILES} files can be mounted into the sandbox` + ) + .optional(), outputs: z .object({ files: z.array(functionOutputFileSchema).optional(), diff --git a/apps/sim/lib/api/contracts/tools/file.ts b/apps/sim/lib/api/contracts/tools/file.ts index e38343dc0d9..889704113f1 100644 --- a/apps/sim/lib/api/contracts/tools/file.ts +++ b/apps/sim/lib/api/contracts/tools/file.ts @@ -10,14 +10,42 @@ export const fileManageQuerySchema = z.object({ workspaceId: z.string().min(1).nullable().optional(), }) -export const fileManageWriteBodySchema = z.object({ - operation: z.literal('write'), - workspaceId: z.string().min(1).optional(), - fileName: z.string({ error: 'fileName is required for write operation' }).min(1), - content: z.string({ error: 'content is required for write operation' }), - contentType: z.string().optional(), - [PRIVATE_SECRET_PROVENANCE_FIELD]: privateSecretProvenanceBundleSchema.optional(), -}) +export const fileManageWriteBodySchema = z + .object({ + operation: z.literal('write'), + workspaceId: z.string().min(1).optional(), + fileName: z.string().min(1).optional(), + content: z.string().optional(), + /** + * An existing file object to store as-is, for content that is not text — + * a rendered PDF, a transcoded video, an image from an earlier tool. + */ + fileInput: z.unknown().optional(), + contentType: z.string().optional(), + overwrite: z.boolean().optional(), + [PRIVATE_SECRET_PROVENANCE_FIELD]: privateSecretProvenanceBundleSchema.optional(), + }) + .superRefine((body, context) => { + const hasContent = body.content !== undefined + const hasFileInput = body.fileInput !== undefined && body.fileInput !== null + if (hasContent === hasFileInput) { + context.addIssue({ + code: z.ZodIssueCode.custom, + path: ['content'], + message: + 'Provide exactly one of content (text to write) or fileInput (an existing file to store).', + }) + } + // A file object carries its own name, so fileName is the optional override + // there but the only source of a name when writing text. + if (hasContent && !body.fileName?.trim()) { + context.addIssue({ + code: z.ZodIssueCode.custom, + path: ['fileName'], + message: 'fileName is required when writing text content.', + }) + } + }) export const fileManageAppendBodySchema = z.object({ operation: z.literal('append'), diff --git a/apps/sim/lib/api/contracts/v1/admin/dashboard-workspaces.ts b/apps/sim/lib/api/contracts/v1/admin/dashboard-workspaces.ts index d5acae1f06c..3f557bc58ab 100644 --- a/apps/sim/lib/api/contracts/v1/admin/dashboard-workspaces.ts +++ b/apps/sim/lib/api/contracts/v1/admin/dashboard-workspaces.ts @@ -59,13 +59,114 @@ const adminDashboardWorkspaceCandidateSchema = z.object({ ownerEmail: z.string(), workspaceMode: z.string(), organizationId: z.string().nullable(), + /** Name of the organization that currently owns the workspace, if any. */ + organizationName: z.string().nullable(), billedAccountUserId: z.string(), /** Archived workspaces are movable; the flag lets admin UIs label them. */ archived: z.boolean(), + /** + * Non-null when the workspace cannot be moved. Ineligible rows are returned + * rather than filtered out so the admin learns the workspace exists and why + * it is stuck, instead of an empty result they cannot act on. + */ + ineligibleReason: z.string().nullable().optional(), +}) + +/** Usage split so the UI can separate what leaves from what breaks behind. */ +const adminDashboardCustomBlockUsageSchema = z.object({ + live: z.number().int().min(0), + deployed: z.number().int().min(0), +}) + +const adminDashboardWorkspaceSourceImpactSchema = z.object({ + unpublishedCustomBlocks: z + .array( + z.object({ + id: z.string(), + type: z.string(), + name: z.string(), + movingWorkspaceUsage: adminDashboardCustomBlockUsageSchema, + sourceOrgElsewhereUsage: adminDashboardCustomBlockUsageSchema, + }) + ) + .max(500), + /** Non-empty means the move is blocked until the fork is disconnected. */ + blockingForkEdges: z + .array( + z.object({ + workspaceId: z.string(), + name: z.string(), + organizationId: z.string().nullable(), + direction: z.enum(['parent', 'child']), + }) + ) + .max(500), + detachedPermissionGroups: z + .array(z.object({ permissionGroupId: z.string(), name: z.string() })) + .max(500), + strippedRetentionRules: z.object({ + piiRedactionRules: z.number().int().min(0), + retentionOverrides: z.number().int().min(0), + }), + retainedCollaboratorCaps: z + .array( + z.object({ + userId: z.string(), + email: z.string(), + sourceOrgLimitDollars: z.number().nullable(), + }) + ) + .max(1000), + brandingChanges: z.boolean(), + /** + * Rows omitted to keep the response inside the array bounds above. Non-null + * means the lists are incomplete and the notice says so. + */ + truncated: z + .object({ + customBlocks: z.number().int().min(0), + permissionGroups: z.number().int().min(0), + collaboratorCaps: z.number().int().min(0), + forkEdges: z.number().int().min(0), + credentials: z.number().int().min(0), + environmentVariableKeys: z.number().int().min(0), + }) + .nullable(), +}) + +/** Secrets that travel with the workspace. Never carries secret material. */ +const adminDashboardWorkspaceCredentialsSchema = z.object({ + items: z + .array( + z.object({ + id: z.string(), + displayName: z.string(), + type: z.string(), + backedBySourceOrgMember: z.boolean(), + }) + ) + .max(1000), + credentialGroupCount: z.number().int().min(0), + /** Variable names only — values are never sent. */ + environmentVariableKeys: z.array(z.string()).max(1000), + byokKeyCount: z.number().int().min(0), + /** Rows omitted to stay within the bounds above. */ + truncatedCredentials: z.number().int().min(0), + truncatedEnvironmentVariableKeys: z.number().int().min(0), }) const adminDashboardWorkspacePreflightSchema = z.object({ workspace: adminDashboardWorkspaceCandidateSchema, + /** `null` for a personal or grandfathered source. */ + sourceOrganization: z + .object({ + id: z.string(), + name: z.string(), + ownerId: z.string().nullable(), + ownerName: z.string().nullable(), + ownerEmail: z.string().nullable(), + }) + .nullable(), destinationOrganization: z.object({ id: z.string(), name: z.string(), @@ -80,6 +181,8 @@ const adminDashboardWorkspacePreflightSchema = z.object({ email: z.string(), permission: z.enum(['admin', 'write', 'read']), organizationMember: z.boolean(), + /** Retains access after the move, as an external collaborator. */ + sourceOrganizationMember: z.boolean(), }) ), invitations: z.array( @@ -91,6 +194,17 @@ const adminDashboardWorkspacePreflightSchema = z.object({ workspaceGrantCount: z.number().int().min(1), }) ), + sourceOrganizationImpact: adminDashboardWorkspaceSourceImpactSchema, + credentials: adminDashboardWorkspaceCredentialsSchema, + entitlements: z.object({ + sourceIsEnterprise: z.boolean(), + destinationIsEnterprise: z.boolean(), + capabilitiesLost: z.array(z.string()).max(50), + }), + /** Non-empty means the move will throw; the UI must not offer a confirm. */ + blockers: z.array(z.string()).max(20), + /** Advisory consequences worth reading, which never block. */ + notices: z.array(z.string()).max(20), warning: z.string().nullable(), }) diff --git a/apps/sim/lib/billing/core/subscription.ts b/apps/sim/lib/billing/core/subscription.ts index a6162a0d221..1140e654dbe 100644 --- a/apps/sim/lib/billing/core/subscription.ts +++ b/apps/sim/lib/billing/core/subscription.ts @@ -431,6 +431,21 @@ export async function isEnterpriseOrgAdminOrOwner(userId: string): Promise { try { if (!isBillingEnabled) { diff --git a/apps/sim/lib/billing/sandbox-pricing.test.ts b/apps/sim/lib/billing/sandbox-pricing.test.ts new file mode 100644 index 00000000000..e523120bc32 --- /dev/null +++ b/apps/sim/lib/billing/sandbox-pricing.test.ts @@ -0,0 +1,36 @@ +import { describe, expect, it } from 'vitest' +import { createSandboxPricing, priceSandboxUsage } from '@/lib/billing/sandbox-pricing' + +describe('sandbox pricing', () => { + it.each([ + ['e2b', 0.1656], + ['daytona', 0.16668], + ] as const)('prices one hour of the Function profile on %s', (provider, expected) => { + const pricing = createSandboxPricing(provider, 1) + + expect(priceSandboxUsage(pricing, 3_600_000, 3_600_000).rawCost).toBeCloseTo(expected, 8) + }) + + it('applies the multiplier once and rounds the final cost to eight decimals', () => { + const pricing = createSandboxPricing('e2b', 1.75) + + expect(priceSandboxUsage(pricing, 1234, 10_000).billedCost).toBe(0.00009934) + }) + + it('caps duration at the provider lifetime', () => { + const pricing = createSandboxPricing('daytona', 1) + + expect(priceSandboxUsage(pricing, 90_000, 60_000).durationMs).toBe(60_000) + }) + + it('allows a zero multiplier and rejects invalid multipliers', () => { + const freePricing = createSandboxPricing('e2b', 0) + + expect(priceSandboxUsage(freePricing, 1000, 1000).billedCost).toBe(0) + expect(() => createSandboxPricing('e2b', -1)).toThrow('finite nonnegative') + expect(() => createSandboxPricing('e2b', Number.NaN)).toThrow('finite nonnegative') + expect(() => createSandboxPricing('e2b', Number.POSITIVE_INFINITY)).toThrow( + 'finite nonnegative' + ) + }) +}) diff --git a/apps/sim/lib/billing/sandbox-pricing.ts b/apps/sim/lib/billing/sandbox-pricing.ts new file mode 100644 index 00000000000..828fb83204a --- /dev/null +++ b/apps/sim/lib/billing/sandbox-pricing.ts @@ -0,0 +1,103 @@ +import { getCostMultiplier } from '@/lib/core/config/env-flags' +import { + FUNCTION_DAYTONA_DISK_GB, + FUNCTION_SANDBOX_CPU_COUNT, + FUNCTION_SANDBOX_MEMORY_GB, +} from '@/lib/execution/remote-sandbox/function-resources' +import type { SandboxProviderId } from '@/lib/execution/remote-sandbox/types' + +const E2B_CPU_USD_PER_VCPU_SECOND = 0.000014 +const E2B_MEMORY_USD_PER_GIB_SECOND = 0.0000045 +const DAYTONA_CPU_USD_PER_VCPU_SECOND = 0.0504 / 3600 +const DAYTONA_MEMORY_USD_PER_GIB_SECOND = 0.0162 / 3600 +/** + * Sim prices the full provisioned disk at the marginal list rate; provider free allowances, + * credits, and discounts are intentionally not subtracted. + */ +const DAYTONA_DISK_USD_PER_GIB_SECOND = 0.000108 / 3600 + +export interface SandboxPricing { + provider: SandboxProviderId + multiplier: number + resources: { + vcpu: number + memoryGiB: number + diskGiB: number + } + rates: { + cpuUsdPerVcpuSecond: number + memoryUsdPerGiBSecond: number + diskUsdPerGiBSecond: number + } +} + +export interface PricedSandboxUsage { + durationMs: number + rawCost: number + billedCost: number +} + +const PRICING_BY_PROVIDER: Record< + SandboxProviderId, + Pick +> = { + e2b: { + resources: { + vcpu: FUNCTION_SANDBOX_CPU_COUNT, + memoryGiB: FUNCTION_SANDBOX_MEMORY_GB, + diskGiB: 0, + }, + rates: { + cpuUsdPerVcpuSecond: E2B_CPU_USD_PER_VCPU_SECOND, + memoryUsdPerGiBSecond: E2B_MEMORY_USD_PER_GIB_SECOND, + diskUsdPerGiBSecond: 0, + }, + }, + daytona: { + resources: { + vcpu: FUNCTION_SANDBOX_CPU_COUNT, + memoryGiB: FUNCTION_SANDBOX_MEMORY_GB, + diskGiB: FUNCTION_DAYTONA_DISK_GB, + }, + rates: { + cpuUsdPerVcpuSecond: DAYTONA_CPU_USD_PER_VCPU_SECOND, + memoryUsdPerGiBSecond: DAYTONA_MEMORY_USD_PER_GIB_SECOND, + diskUsdPerGiBSecond: DAYTONA_DISK_USD_PER_GIB_SECOND, + }, + }, +} + +export function createSandboxPricing( + provider: SandboxProviderId, + multiplier = getCostMultiplier() +): SandboxPricing { + if (!Number.isFinite(multiplier) || multiplier < 0) { + throw new Error('Sandbox pricing multiplier must be a finite nonnegative number') + } + const pricing = PRICING_BY_PROVIDER[provider] + return { + provider, + multiplier, + resources: { ...pricing.resources }, + rates: { ...pricing.rates }, + } +} + +export function priceSandboxUsage( + pricing: SandboxPricing, + observedDurationMs: number, + providerLifetimeMs: number +): PricedSandboxUsage { + const durationMs = Math.max(0, Math.min(observedDurationMs, providerLifetimeMs)) + const seconds = durationMs / 1000 + const rawCost = + seconds * pricing.resources.vcpu * pricing.rates.cpuUsdPerVcpuSecond + + seconds * pricing.resources.memoryGiB * pricing.rates.memoryUsdPerGiBSecond + + seconds * pricing.resources.diskGiB * pricing.rates.diskUsdPerGiBSecond + + return { + durationMs, + rawCost, + billedCost: Number.parseFloat((rawCost * pricing.multiplier).toFixed(8)), + } +} diff --git a/apps/sim/lib/charts/spec.test.ts b/apps/sim/lib/charts/spec.test.ts index c0bf9cd2046..1d708ede5e1 100644 --- a/apps/sim/lib/charts/spec.test.ts +++ b/apps/sim/lib/charts/spec.test.ts @@ -95,6 +95,7 @@ describe('shapeTableRows', () => { }) const XSS_FORMATTER = '' +const XSS_LINK = 'javascript:alert(document.domain)' describe('parseChartSpec option confinement', () => { it('forces the tooltip off the innerHTML path, keeping the formatter template', () => { @@ -175,6 +176,23 @@ describe('parseChartSpec option confinement', () => { expect(media[0].option.toolbox).toBeUndefined() }) + it('drops every navigation sink — title link/sublink and treemap/sunburst item links', () => { + const option = parse({ + schema_version: 1, + option: { + title: { text: 'click me', link: XSS_LINK, sublink: XSS_LINK, target: 'self' }, + series: [ + { type: 'treemap', data: [{ name: 'a', value: 1, link: XSS_LINK }] }, + { type: 'sunburst', data: [{ name: 'b', value: 1, link: XSS_LINK }] }, + ], + baseOption: { title: { link: XSS_LINK } }, + media: [{ query: { minWidth: 100 }, option: { title: { link: XSS_LINK } } }], + }, + }) + expect(JSON.stringify(option)).not.toContain('javascript:') + expect(option.title).toEqual({ text: 'click me', target: 'self' }) + }) + it('adds no tooltip to a document that declares none', () => { const option = parse({ schema_version: 1, option: { series: [{ type: 'bar', data: [1] }] } }) expect('tooltip' in option).toBe(false) @@ -191,7 +209,7 @@ describe('parseChartSpec option confinement', () => { }) it('leaves dataset rows alone — they hold data, not components', () => { - const rows = [{ tooltip: 'ok', toolbox: 'ok' }] + const rows = [{ tooltip: 'ok', toolbox: 'ok', link: 'ok' }] const option = parse({ schema_version: 1, option: { dataset: { source: rows } } }) expect((option.dataset as Record).source).toEqual(rows) }) @@ -268,6 +286,24 @@ describe('chart option confinement against echarts', () => { ) expect(model.getComponent('toolbox')).toBeUndefined() }) + + it('leaves the title component no link to hand to windowOpen', () => { + const model = renderModel( + parse({ + schema_version: 1, + option: { + xAxis: {}, + yAxis: {}, + series: [{ type: 'bar', data: [1] }], + title: { text: 'click me', link: XSS_LINK, sublink: XSS_LINK }, + }, + }) + ) + const title = model.getComponent('title') + expect(title?.get('text')).toBe('click me') + expect(title?.get('link')).toBeUndefined() + expect(title?.get('sublink')).toBeUndefined() + }) }) describe('parseChartSpec table-shaping validation', () => { diff --git a/apps/sim/lib/charts/spec.ts b/apps/sim/lib/charts/spec.ts index 5d1a6479c86..37bc55f0aad 100644 --- a/apps/sim/lib/charts/spec.ts +++ b/apps/sim/lib/charts/spec.ts @@ -52,6 +52,9 @@ export interface ChartSpec { /** ECharts' tooltip render mode that draws into the chart canvas instead of the DOM. */ const CANVAS_TOOLTIP_RENDER_MODE = 'richText' +/** Option keys stripped at every level: the `toolbox` DOM sink and the `link`/`sublink` navigation sinks. */ +const DROPPED_KEYS = ['toolbox', 'link', 'sublink'] as const + /** * Closes the paths by which an ECharts option reaches the DOM, so a `.chart` * document cannot inject markup into the page that renders it. A document is @@ -63,7 +66,14 @@ const CANVAS_TOOLTIP_RENDER_MODE = 'richText' * string `formatter` is used as that content's template verbatim — only the * values substituted into it are escaped. A `toolbox` assigns `dataView.lang` * entries to `innerHTML` and fills a `saveAsImage` popup with `document.write`. - * Forcing the render mode and dropping the toolbox leaves the document no DOM + * Forcing the render mode and dropping the toolbox leaves it no DOM sink. + * + * ECharts also navigates: `title.link`, `title.sublink`, and a `link` on a + * treemap or sunburst data item each reach `windowOpen`, which assigns the URL + * to `location.href` — so a `javascript:` URL runs on this origin on a single + * click. A chart has no reason to navigate its viewer, so the keys are dropped + * everywhere rather than scheme-checked, which would still leave an open + * redirect on an authenticated origin. Between them the document is left no * sink at all, which holds whatever any individual option value contains. * * The walk is deep because `tooltip` is not only a top-level component: @@ -79,8 +89,9 @@ function confineOptionToCanvas(node: unknown): void { } if (node === null || typeof node !== 'object') return const record = node as Record - // biome-ignore lint/performance/noDelete: the key must be absent, not undefined-valued - if ('toolbox' in record) delete record.toolbox + for (const key of DROPPED_KEYS) { + if (key in record) delete record[key] + } for (const key of Object.keys(record)) { if (key === 'dataset') continue const value = record[key] diff --git a/apps/sim/lib/copilot/generated/tool-catalog-v1.ts b/apps/sim/lib/copilot/generated/tool-catalog-v1.ts index b50898eb45a..5be407c6da4 100644 --- a/apps/sim/lib/copilot/generated/tool-catalog-v1.ts +++ b/apps/sim/lib/copilot/generated/tool-catalog-v1.ts @@ -4191,7 +4191,7 @@ export const QueryUserTable: ToolCatalogEntry = { filter: { type: 'object', description: - 'Predicate filter object for query_rows. A predicate is a tree: {"all":[...]} (AND) or {"any":[...]} (OR); members are leaves {field, op, value} or nested groups. Ops: eq, ne, gt, gte, lt, lte, in, nin, like, ilike (use * as the wildcard), nlike, nilike, contains, ncontains, startsWith, endsWith, isNull, isNotNull, isEmpty, isNotEmpty. in/nin take a non-empty array value. Examples: {"all":[{"field":"status","op":"eq","value":"active"}]}; {"any":[{"field":"status","op":"eq","value":"active"},{"field":"status","op":"eq","value":"pending"}]}; {"all":[{"field":"name","op":"ilike","value":"*jo*"}]}.', + 'Predicate filter object for query_rows. A predicate is a tree: {"all":[...]} (AND) or {"any":[...]} (OR); members are leaves {field, op, value} or nested groups. Ops: eq, ne, gt, gte, lt, lte, in, nin, like, ilike (use * as the wildcard), nlike, nilike, contains, ncontains, startsWith, endsWith, isNull, isNotNull, isEmpty, isNotEmpty. in/nin take a non-empty array value. TTL filter values are absolute whole Unix epoch seconds, never milliseconds. Examples: {"all":[{"field":"status","op":"eq","value":"active"}]}; {"any":[{"field":"status","op":"eq","value":"active"},{"field":"status","op":"eq","value":"pending"}]}; {"all":[{"field":"name","op":"ilike","value":"*jo*"}]}.', }, limit: { type: 'number', @@ -5312,12 +5312,6 @@ export const TableAutomations: ToolCatalogEntry = { }, }, }, - deploymentMode: { - type: 'string', - description: - 'Which workflow version rows execute: "live" (default, editable draft — edits take effect immediately) or "deployed" (latest active deployment; fails if the workflow was never deployed).', - enum: ['live', 'deployed'], - }, groupId: { type: 'string', description: @@ -5453,7 +5447,7 @@ export const TableColumns: ToolCatalogEntry = { column: { type: 'object', description: - 'Column definition for add_column: { name, type, unique?, position? }; select (enum) columns also take { options: [names], multiple?: true } — options is required for select.', + 'Column definition for add_column: { name, type, unique?, position? }; type may be string, number, boolean, date, json, select, or ttl. Select (enum) columns also take { options: [names], multiple?: true } — options is required for select. A table may have at most one ttl column; adding it enables row expiration, with cell values stored as absolute whole Unix epoch seconds rather than milliseconds.', }, columnName: { type: 'string', @@ -5474,7 +5468,7 @@ export const TableColumns: ToolCatalogEntry = { newType: { type: 'string', description: - 'New column type for update_column: string, number, boolean, date, json, select. Converting to select also requires options; conversion fails if an existing cell value matches no option. A multiple select round-trips through text as a comma-separated cell.', + 'New column type for update_column: string, number, boolean, date, json, select, ttl. Converting to select also requires options; conversion fails if an existing cell value matches no option. A multiple select round-trips through text as a comma-separated cell. Converting to ttl enables row expiration and fails if the table already has another ttl column; TTL values are absolute whole Unix epoch seconds, not milliseconds.', }, options: { type: 'array', @@ -5653,7 +5647,7 @@ export const TableManage: ToolCatalogEntry = { schema: { type: 'object', description: - 'Table schema with a columns array (required for create). Each column: { name, type, unique? }; a select (enum) column also requires options (display names) and takes multiple?.', + 'Table schema with a columns array (required for create). Each column: { name, type, unique? }; types are string, number, boolean, date, json, select, and ttl. A select (enum) column also requires options (display names) and takes multiple?. A table may have at most one ttl column; adding it enables row expiration, with cell values stored as absolute whole Unix epoch seconds rather than milliseconds.', }, tableId: { type: 'string', @@ -5699,12 +5693,12 @@ export const TableRows: ToolCatalogEntry = { data: { type: 'object', description: - 'Row data as column → value pairs (required for insert_row, update_row; the patch object for update_rows_by_filter). Select (enum) cells take the option NAME.', + 'Row data as column → value pairs (required for insert_row, update_row; the patch object for update_rows_by_filter). Select (enum) cells take the option NAME. TTL cells take an absolute whole Unix epoch timestamp in seconds, never JavaScript milliseconds; missing or null TTL means no expiration.', }, filter: { type: 'object', description: - 'Predicate filter for update_rows_by_filter / delete_rows_by_filter: {"all":[...]} (AND) or {"any":[...]} (OR) of {field, op, value} leaves or nested groups. Ops: eq, ne, gt, gte, lt, lte, in, nin, like, ilike (* wildcard), nlike, nilike, contains, ncontains, startsWith, endsWith, isNull, isNotNull, isEmpty, isNotEmpty. in/nin take a non-empty array. Single-select columns match by eq/ne/in/nin; multiple-select by contains/ncontains — values are option NAMES.', + 'Predicate filter for update_rows_by_filter / delete_rows_by_filter: {"all":[...]} (AND) or {"any":[...]} (OR) of {field, op, value} leaves or nested groups. Ops: eq, ne, gt, gte, lt, lte, in, nin, like, ilike (* wildcard), nlike, nilike, contains, ncontains, startsWith, endsWith, isNull, isNotNull, isEmpty, isNotEmpty. in/nin take a non-empty array. Single-select columns match by eq/ne/in/nin; multiple-select by contains/ncontains — values are option NAMES. TTL filter values are absolute whole Unix epoch seconds.', }, limit: { type: 'number', @@ -5729,7 +5723,8 @@ export const TableRows: ToolCatalogEntry = { }, rows: { type: 'array', - description: 'Array of row data objects (required for batch_insert_rows)', + description: + 'Array of row data objects (required for batch_insert_rows). TTL cells take absolute whole Unix epoch timestamps in seconds, never JavaScript milliseconds.', }, tableId: { type: 'string', description: 'Table ID (required for every operation)' }, updates: { @@ -6058,7 +6053,7 @@ export const UserTable: ToolCatalogEntry = { column: { type: 'object', description: - 'Column definition for add_column: { name, type, unique?, position? }. For a select (enum) column also pass { options: ["Open", "Closed"], multiple?: true } — options is a list of display names and is required for select.', + 'Column definition for add_column: { name, type, unique?, position? }. Type may be string, number, boolean, date, json, select, or ttl. For a select (enum) column also pass { options: ["Open", "Closed"], multiple?: true } — options is a list of display names and is required for select. A table may have at most one ttl column; adding it enables row expiration, with cell values stored as absolute whole Unix epoch seconds rather than milliseconds.', }, columnName: { type: 'string', @@ -6077,7 +6072,8 @@ export const UserTable: ToolCatalogEntry = { }, data: { type: 'object', - description: 'Row data as key-value pairs (required for insert_row, update_row)', + description: + 'Row data as key-value pairs (required for insert_row, update_row). TTL cells take an absolute whole Unix epoch timestamp in seconds, never JavaScript milliseconds; missing or null TTL means no expiration.', }, dependencies: { type: 'object', @@ -6092,12 +6088,6 @@ export const UserTable: ToolCatalogEntry = { }, }, }, - deploymentMode: { - type: 'string', - description: - "Which version of the backing workflow this group's per-row runs execute, for add_workflow_group and update_workflow_group. 'live' (default) runs the editable draft, so later edits take effect immediately. 'deployed' runs the workflow's latest active deployment, pinning rows to a published version — if that workflow has never been deployed the cell fails rather than falling back to the draft. Only meaningful for workflow groups; enrichment groups have no backing workflow.", - enum: ['live', 'deployed'], - }, description: { type: 'string', description: "Table description (optional for 'create')" }, enrichmentId: { type: 'string', @@ -6112,7 +6102,7 @@ export const UserTable: ToolCatalogEntry = { filter: { type: 'object', description: - 'Predicate filter object for query_rows, update_rows_by_filter, delete_rows_by_filter. A predicate is a tree: {"all":[...]} (AND) or {"any":[...]} (OR); members are leaves {field, op, value} or nested groups. Ops: eq, ne, gt, gte, lt, lte, in, nin, like, ilike (use * as the wildcard), nlike, nilike, contains, ncontains, startsWith, endsWith, isNull, isNotNull, isEmpty, isNotEmpty. in/nin take a non-empty array value. Examples: {"all":[{"field":"status","op":"eq","value":"active"}]}; {"all":[{"field":"wins","op":"gte","value":18},{"field":"status","op":"eq","value":"pending"}]}; {"any":[{"field":"status","op":"eq","value":"active"},{"field":"status","op":"eq","value":"pending"}]}; {"all":[{"field":"name","op":"ilike","value":"*jo*"}]}; {"all":[{"field":"slack_user_id","op":"in","value":["U1","U2"]}]}.', + 'Predicate filter object for query_rows, update_rows_by_filter, delete_rows_by_filter. A predicate is a tree: {"all":[...]} (AND) or {"any":[...]} (OR); members are leaves {field, op, value} or nested groups. Ops: eq, ne, gt, gte, lt, lte, in, nin, like, ilike (use * as the wildcard), nlike, nilike, contains, ncontains, startsWith, endsWith, isNull, isNotNull, isEmpty, isNotEmpty. in/nin take a non-empty array value. TTL filter values are absolute whole Unix epoch seconds, never milliseconds. Examples: {"all":[{"field":"status","op":"eq","value":"active"}]}; {"all":[{"field":"wins","op":"gte","value":18},{"field":"status","op":"eq","value":"pending"}]}; {"any":[{"field":"status","op":"eq","value":"active"},{"field":"status","op":"eq","value":"pending"}]}; {"all":[{"field":"name","op":"ilike","value":"*jo*"}]}; {"all":[{"field":"slack_user_id","op":"in","value":["U1","U2"]}]}.', }, groupId: { type: 'string', @@ -6201,7 +6191,7 @@ export const UserTable: ToolCatalogEntry = { newType: { type: 'string', description: - 'New column type (optional for update_column). Types: string, number, boolean, date, json, select. Converting a column to select also requires options; the conversion fails if any existing cell value doesn\'t match one of them. Converting to a multiple: true select also accepts a comma-separated cell ("Open, Urgent"), which is the form a multi column converts to text as — so multiselect → text → multiselect round-trips.', + 'New column type (optional for update_column). Types: string, number, boolean, date, json, select, ttl. Converting a column to select also requires options; the conversion fails if any existing cell value doesn\'t match one of them. Converting to a multiple: true select also accepts a comma-separated cell ("Open, Urgent"), which is the form a multi column converts to text as — so multiselect → text → multiselect round-trips. Converting to ttl enables row expiration and fails if the table already has another ttl column; TTL values are absolute whole Unix epoch seconds, not milliseconds.', }, options: { type: 'array', @@ -6279,7 +6269,8 @@ export const UserTable: ToolCatalogEntry = { }, rows: { type: 'array', - description: 'Array of row data objects (required for batch_insert_rows)', + description: + 'Array of row data objects (required for batch_insert_rows). TTL cells take absolute whole Unix epoch timestamps in seconds, never JavaScript milliseconds.', }, runMode: { type: 'string', @@ -6290,7 +6281,7 @@ export const UserTable: ToolCatalogEntry = { schema: { type: 'object', description: - 'Table schema with columns array (required for \'create\'). Each column: { name, type, unique? }. A select (enum) column also takes { options: ["Open", "Closed"], multiple?: true } — options is a list of display names and is required for select.', + 'Table schema with columns array (required for \'create\'). Each column: { name, type, unique? }. Types are string, number, boolean, date, json, select, and ttl. A select (enum) column also takes { options: ["Open", "Closed"], multiple?: true } — options is a list of display names and is required for select. A table may have at most one ttl column; adding it enables row expiration, with cell values stored as absolute whole Unix epoch seconds rather than milliseconds.', }, scope: { type: 'string', diff --git a/apps/sim/lib/copilot/generated/tool-schemas-v1.ts b/apps/sim/lib/copilot/generated/tool-schemas-v1.ts index 7625f38c296..a4b5c3be8a3 100644 --- a/apps/sim/lib/copilot/generated/tool-schemas-v1.ts +++ b/apps/sim/lib/copilot/generated/tool-schemas-v1.ts @@ -4080,7 +4080,7 @@ export const TOOL_RUNTIME_SCHEMAS: Record = { filter: { type: 'object', description: - 'Predicate filter object for query_rows. A predicate is a tree: {"all":[...]} (AND) or {"any":[...]} (OR); members are leaves {field, op, value} or nested groups. Ops: eq, ne, gt, gte, lt, lte, in, nin, like, ilike (use * as the wildcard), nlike, nilike, contains, ncontains, startsWith, endsWith, isNull, isNotNull, isEmpty, isNotEmpty. in/nin take a non-empty array value. Examples: {"all":[{"field":"status","op":"eq","value":"active"}]}; {"any":[{"field":"status","op":"eq","value":"active"},{"field":"status","op":"eq","value":"pending"}]}; {"all":[{"field":"name","op":"ilike","value":"*jo*"}]}.', + 'Predicate filter object for query_rows. A predicate is a tree: {"all":[...]} (AND) or {"any":[...]} (OR); members are leaves {field, op, value} or nested groups. Ops: eq, ne, gt, gte, lt, lte, in, nin, like, ilike (use * as the wildcard), nlike, nilike, contains, ncontains, startsWith, endsWith, isNull, isNotNull, isEmpty, isNotEmpty. in/nin take a non-empty array value. TTL filter values are absolute whole Unix epoch seconds, never milliseconds. Examples: {"all":[{"field":"status","op":"eq","value":"active"}]}; {"any":[{"field":"status","op":"eq","value":"active"},{"field":"status","op":"eq","value":"pending"}]}; {"all":[{"field":"name","op":"ilike","value":"*jo*"}]}.', }, limit: { type: 'number', @@ -5173,12 +5173,6 @@ export const TOOL_RUNTIME_SCHEMAS: Record = { }, }, }, - deploymentMode: { - type: 'string', - description: - 'Which workflow version rows execute: "live" (default, editable draft — edits take effect immediately) or "deployed" (latest active deployment; fails if the workflow was never deployed).', - enum: ['live', 'deployed'], - }, groupId: { type: 'string', description: @@ -5339,7 +5333,7 @@ export const TOOL_RUNTIME_SCHEMAS: Record = { column: { type: 'object', description: - 'Column definition for add_column: { name, type, unique?, position? }; select (enum) columns also take { options: [names], multiple?: true } — options is required for select.', + 'Column definition for add_column: { name, type, unique?, position? }; type may be string, number, boolean, date, json, select, or ttl. Select (enum) columns also take { options: [names], multiple?: true } — options is required for select. A table may have at most one ttl column; adding it enables row expiration, with cell values stored as absolute whole Unix epoch seconds rather than milliseconds.', }, columnName: { type: 'string', @@ -5363,7 +5357,7 @@ export const TOOL_RUNTIME_SCHEMAS: Record = { newType: { type: 'string', description: - 'New column type for update_column: string, number, boolean, date, json, select. Converting to select also requires options; conversion fails if an existing cell value matches no option. A multiple select round-trips through text as a comma-separated cell.', + 'New column type for update_column: string, number, boolean, date, json, select, ttl. Converting to select also requires options; conversion fails if an existing cell value matches no option. A multiple select round-trips through text as a comma-separated cell. Converting to ttl enables row expiration and fails if the table already has another ttl column; TTL values are absolute whole Unix epoch seconds, not milliseconds.', }, options: { type: 'array', @@ -5569,7 +5563,7 @@ export const TOOL_RUNTIME_SCHEMAS: Record = { schema: { type: 'object', description: - 'Table schema with a columns array (required for create). Each column: { name, type, unique? }; a select (enum) column also requires options (display names) and takes multiple?.', + 'Table schema with a columns array (required for create). Each column: { name, type, unique? }; types are string, number, boolean, date, json, select, and ttl. A select (enum) column also requires options (display names) and takes multiple?. A table may have at most one ttl column; adding it enables row expiration, with cell values stored as absolute whole Unix epoch seconds rather than milliseconds.', }, tableId: { type: 'string', @@ -5619,12 +5613,12 @@ export const TOOL_RUNTIME_SCHEMAS: Record = { data: { type: 'object', description: - 'Row data as column → value pairs (required for insert_row, update_row; the patch object for update_rows_by_filter). Select (enum) cells take the option NAME.', + 'Row data as column → value pairs (required for insert_row, update_row; the patch object for update_rows_by_filter). Select (enum) cells take the option NAME. TTL cells take an absolute whole Unix epoch timestamp in seconds, never JavaScript milliseconds; missing or null TTL means no expiration.', }, filter: { type: 'object', description: - 'Predicate filter for update_rows_by_filter / delete_rows_by_filter: {"all":[...]} (AND) or {"any":[...]} (OR) of {field, op, value} leaves or nested groups. Ops: eq, ne, gt, gte, lt, lte, in, nin, like, ilike (* wildcard), nlike, nilike, contains, ncontains, startsWith, endsWith, isNull, isNotNull, isEmpty, isNotEmpty. in/nin take a non-empty array. Single-select columns match by eq/ne/in/nin; multiple-select by contains/ncontains — values are option NAMES.', + 'Predicate filter for update_rows_by_filter / delete_rows_by_filter: {"all":[...]} (AND) or {"any":[...]} (OR) of {field, op, value} leaves or nested groups. Ops: eq, ne, gt, gte, lt, lte, in, nin, like, ilike (* wildcard), nlike, nilike, contains, ncontains, startsWith, endsWith, isNull, isNotNull, isEmpty, isNotEmpty. in/nin take a non-empty array. Single-select columns match by eq/ne/in/nin; multiple-select by contains/ncontains — values are option NAMES. TTL filter values are absolute whole Unix epoch seconds.', }, limit: { type: 'number', @@ -5654,7 +5648,8 @@ export const TOOL_RUNTIME_SCHEMAS: Record = { }, rows: { type: 'array', - description: 'Array of row data objects (required for batch_insert_rows)', + description: + 'Array of row data objects (required for batch_insert_rows). TTL cells take absolute whole Unix epoch timestamps in seconds, never JavaScript milliseconds.', }, tableId: { type: 'string', @@ -5998,7 +5993,7 @@ export const TOOL_RUNTIME_SCHEMAS: Record = { column: { type: 'object', description: - 'Column definition for add_column: { name, type, unique?, position? }. For a select (enum) column also pass { options: ["Open", "Closed"], multiple?: true } — options is a list of display names and is required for select.', + 'Column definition for add_column: { name, type, unique?, position? }. Type may be string, number, boolean, date, json, select, or ttl. For a select (enum) column also pass { options: ["Open", "Closed"], multiple?: true } — options is a list of display names and is required for select. A table may have at most one ttl column; adding it enables row expiration, with cell values stored as absolute whole Unix epoch seconds rather than milliseconds.', }, columnName: { type: 'string', @@ -6017,7 +6012,8 @@ export const TOOL_RUNTIME_SCHEMAS: Record = { }, data: { type: 'object', - description: 'Row data as key-value pairs (required for insert_row, update_row)', + description: + 'Row data as key-value pairs (required for insert_row, update_row). TTL cells take an absolute whole Unix epoch timestamp in seconds, never JavaScript milliseconds; missing or null TTL means no expiration.', }, dependencies: { type: 'object', @@ -6034,12 +6030,6 @@ export const TOOL_RUNTIME_SCHEMAS: Record = { }, }, }, - deploymentMode: { - type: 'string', - description: - "Which version of the backing workflow this group's per-row runs execute, for add_workflow_group and update_workflow_group. 'live' (default) runs the editable draft, so later edits take effect immediately. 'deployed' runs the workflow's latest active deployment, pinning rows to a published version — if that workflow has never been deployed the cell fails rather than falling back to the draft. Only meaningful for workflow groups; enrichment groups have no backing workflow.", - enum: ['live', 'deployed'], - }, description: { type: 'string', description: "Table description (optional for 'create')", @@ -6057,7 +6047,7 @@ export const TOOL_RUNTIME_SCHEMAS: Record = { filter: { type: 'object', description: - 'Predicate filter object for query_rows, update_rows_by_filter, delete_rows_by_filter. A predicate is a tree: {"all":[...]} (AND) or {"any":[...]} (OR); members are leaves {field, op, value} or nested groups. Ops: eq, ne, gt, gte, lt, lte, in, nin, like, ilike (use * as the wildcard), nlike, nilike, contains, ncontains, startsWith, endsWith, isNull, isNotNull, isEmpty, isNotEmpty. in/nin take a non-empty array value. Examples: {"all":[{"field":"status","op":"eq","value":"active"}]}; {"all":[{"field":"wins","op":"gte","value":18},{"field":"status","op":"eq","value":"pending"}]}; {"any":[{"field":"status","op":"eq","value":"active"},{"field":"status","op":"eq","value":"pending"}]}; {"all":[{"field":"name","op":"ilike","value":"*jo*"}]}; {"all":[{"field":"slack_user_id","op":"in","value":["U1","U2"]}]}.', + 'Predicate filter object for query_rows, update_rows_by_filter, delete_rows_by_filter. A predicate is a tree: {"all":[...]} (AND) or {"any":[...]} (OR); members are leaves {field, op, value} or nested groups. Ops: eq, ne, gt, gte, lt, lte, in, nin, like, ilike (use * as the wildcard), nlike, nilike, contains, ncontains, startsWith, endsWith, isNull, isNotNull, isEmpty, isNotEmpty. in/nin take a non-empty array value. TTL filter values are absolute whole Unix epoch seconds, never milliseconds. Examples: {"all":[{"field":"status","op":"eq","value":"active"}]}; {"all":[{"field":"wins","op":"gte","value":18},{"field":"status","op":"eq","value":"pending"}]}; {"any":[{"field":"status","op":"eq","value":"active"},{"field":"status","op":"eq","value":"pending"}]}; {"all":[{"field":"name","op":"ilike","value":"*jo*"}]}; {"all":[{"field":"slack_user_id","op":"in","value":["U1","U2"]}]}.', }, groupId: { type: 'string', @@ -6154,7 +6144,7 @@ export const TOOL_RUNTIME_SCHEMAS: Record = { newType: { type: 'string', description: - 'New column type (optional for update_column). Types: string, number, boolean, date, json, select. Converting a column to select also requires options; the conversion fails if any existing cell value doesn\'t match one of them. Converting to a multiple: true select also accepts a comma-separated cell ("Open, Urgent"), which is the form a multi column converts to text as — so multiselect → text → multiselect round-trips.', + 'New column type (optional for update_column). Types: string, number, boolean, date, json, select, ttl. Converting a column to select also requires options; the conversion fails if any existing cell value doesn\'t match one of them. Converting to a multiple: true select also accepts a comma-separated cell ("Open, Urgent"), which is the form a multi column converts to text as — so multiselect → text → multiselect round-trips. Converting to ttl enables row expiration and fails if the table already has another ttl column; TTL values are absolute whole Unix epoch seconds, not milliseconds.', }, options: { type: 'array', @@ -6242,7 +6232,8 @@ export const TOOL_RUNTIME_SCHEMAS: Record = { }, rows: { type: 'array', - description: 'Array of row data objects (required for batch_insert_rows)', + description: + 'Array of row data objects (required for batch_insert_rows). TTL cells take absolute whole Unix epoch timestamps in seconds, never JavaScript milliseconds.', }, runMode: { type: 'string', @@ -6253,7 +6244,7 @@ export const TOOL_RUNTIME_SCHEMAS: Record = { schema: { type: 'object', description: - 'Table schema with columns array (required for \'create\'). Each column: { name, type, unique? }. A select (enum) column also takes { options: ["Open", "Closed"], multiple?: true } — options is a list of display names and is required for select.', + 'Table schema with columns array (required for \'create\'). Each column: { name, type, unique? }. Types are string, number, boolean, date, json, select, and ttl. A select (enum) column also takes { options: ["Open", "Closed"], multiple?: true } — options is a list of display names and is required for select. A table may have at most one ttl column; adding it enables row expiration, with cell values stored as absolute whole Unix epoch seconds rather than milliseconds.', }, scope: { type: 'string', diff --git a/apps/sim/lib/copilot/tools/handlers/function-execute.test.ts b/apps/sim/lib/copilot/tools/handlers/function-execute.test.ts index c9893ad2b58..694e5f44f38 100644 --- a/apps/sim/lib/copilot/tools/handlers/function-execute.test.ts +++ b/apps/sim/lib/copilot/tools/handlers/function-execute.test.ts @@ -119,6 +119,7 @@ vi.mock('@/lib/execution/remote-sandbox/workspace-sandboxes', () => ({ import { projectToolResultForCopilot } from '@/lib/copilot/request/tools/resolved-secret-result' import { executeFunctionExecute } from '@/lib/copilot/tools/handlers/function-execute' import { executeRunCode } from '@/lib/copilot/tools/handlers/run-code' +import { SNAPSHOT_MAX_BYTES } from '@/lib/table/snapshot-cache' import { ResolvedSecretTraceRegistry } from '@/executor/utils/resolved-secret-trace-registry' const table = { @@ -595,6 +596,8 @@ describe('executeFunctionExecute table mounts', () => { type: 'url', path: '/home/user/tables/tbl_1.csv', url: 'https://s3.example/presigned?sig=abc', + // The snapshot's own ceiling, enforced on the bytes the sandbox pulls. + maxBytes: SNAPSHOT_MAX_BYTES, }) }) @@ -764,6 +767,9 @@ describe('executeFunctionExecute file mounts', () => { type: 'url', path: '/home/user/files/data.csv', url: 'https://s3.example/file?sig=abc', + // Copilot's URL mounts share the transport, so each is granted exactly + // the size it was charged against the aggregate. + maxBytes: 100, }) }) @@ -1025,6 +1031,9 @@ describe('executeFunctionExecute file mounts', () => { type: 'url', path: '/home/user/files/Reports/q1.csv', url: 'https://s3.example/file?sig=abc', + // Copilot's URL mounts share the transport, so each is granted exactly + // the size it was charged against the aggregate. + maxBytes: 100, }) }) diff --git a/apps/sim/lib/copilot/tools/handlers/function-execute.ts b/apps/sim/lib/copilot/tools/handlers/function-execute.ts index 9f619b0677e..31f5cb917dd 100644 --- a/apps/sim/lib/copilot/tools/handlers/function-execute.ts +++ b/apps/sim/lib/copilot/tools/handlers/function-execute.ts @@ -17,7 +17,17 @@ import { MOUNTED_WORKSPACE_FILES_PROVENANCE_KEY, PRIVATE_SECRET_PROVENANCE_FIELD, } from '@/lib/execution/private-tool-metadata' +import type { SandboxFile } from '@/lib/execution/remote-sandbox/types' import { MAX_PLAN_REQUIRED } from '@/lib/execution/remote-sandbox/workspace-sandboxes' +import { + createSandboxMountBudget, + MAX_INLINE_MOUNT_FILE_BYTES, + MAX_INLINE_MOUNT_TOTAL_BYTES, + MAX_TOTAL_URL_BYTES, + MOUNT_URL_TTL_SECONDS, + pushSandboxFileMount, + type SandboxMountBudget, +} from '@/lib/function-execution/sandbox-mounts' import { recordSecretUsage } from '@/lib/secrets/usage/record' import { getTableSnapshotModelMountSafety } from '@/lib/table/rows/secret-provenance' import { getTableById, listTables } from '@/lib/table/service' @@ -49,47 +59,10 @@ import { executeTool as executeAppTool } from '@/tools' const logger = createLogger('CopilotFunctionExecute') -const MAX_FILE_SIZE = 10 * 1024 * 1024 -const MAX_TOTAL_SIZE = 50 * 1024 * 1024 +const MAX_FILE_SIZE = MAX_INLINE_MOUNT_FILE_BYTES +const MAX_TOTAL_SIZE = MAX_INLINE_MOUNT_TOTAL_BYTES const MAX_MOUNTED_FILES = 500 -/** - * Lifetime of a presigned URL handed to the sandbox to fetch a mounted object (table snapshot or - * workspace file). Long enough to download a large file at sandbox startup; the URL grants read to - * only that one object. - */ -const MOUNT_URL_TTL_SECONDS = 600 - -/** - * Per-file ceiling for URL-mounted workspace files. The bytes never transit the web process — the - * sandbox curls them straight from storage — so the bound is sandbox disk, not web heap (unlike the - * inline MAX_FILE_SIZE path). - */ -const MOUNT_URL_MAX_BYTES = 500 * 1024 * 1024 - -/** - * Aggregate ceiling across all URL-mounted files in one request. URL mounts bypass the web heap (so - * they don't count against MAX_TOTAL_SIZE), but the sandbox still curls every byte onto its disk — - * this rejects an oversized request up front instead of filling the sandbox disk one slow curl at a - * time. Generous vs MAX_TOTAL_SIZE since the bytes never transit web memory. - */ -const MAX_TOTAL_URL_BYTES = 2 * 1024 * 1024 * 1024 - -type SandboxFile = - | { type?: 'content'; path: string; content: string; encoding?: 'base64' } - | { type: 'url'; path: string; url: string } - -/** - * Running byte totals for one resolveInputFiles call. `buffered` bytes pass through the web process - * (capped by MAX_TOTAL_SIZE); `url` bytes are curled straight into the sandbox (capped by - * MAX_TOTAL_URL_BYTES). Tracked separately because the two ceilings protect different resources — - * web heap vs sandbox disk. - */ -interface MountedBytes { - buffered: number - url: number -} - async function importMountedWorkspaceFileProvenance(args: { workspaceId: string record: WorkspaceFileRecord @@ -118,17 +91,18 @@ async function importMountedWorkspaceFileProvenance(args: { } /** - * Mounts a stored workspace file into the sandbox and records its bytes against the running totals. - * With cloud storage the sandbox fetches the bytes itself from a presigned URL (no web-heap transit, - * per-file ceiling MOUNT_URL_MAX_BYTES, aggregate ceiling MAX_TOTAL_URL_BYTES); with local storage a - * presigned URL is an app-internal serve path a remote sandbox can't reach, so we buffer the bytes - * through the web process under the inline MAX_FILE_SIZE / MAX_TOTAL_SIZE guards. + * Mounts a stored workspace file into the sandbox. The transport choice, the byte + * ceilings, and the budget accounting live in {@link pushSandboxFileMount}, which + * the Function block shares; what stays here is workspace-specific — reloading the + * record through its application operation, importing its secret provenance, and + * reading generated documents through the servable reader rather than presigning + * their generator source. */ async function pushWorkspaceFileMount( sandboxFiles: SandboxFile[], record: WorkspaceFileRecord, mountPath: string, - mounted: MountedBytes, + mounted: SandboxMountBudget, workspaceId: string, principal: Principal, registry?: ResolvedSecretTraceRegistry @@ -148,78 +122,51 @@ async function pushWorkspaceFileMount( // through the web process rather than presigning is affordable. const rendersFromSource = isGeneratedDocumentSourceType(record.type) - if (hasCloudStorage() && !rendersFromSource) { - if (record.size > MOUNT_URL_MAX_BYTES) { - throw new Error( - `Input file "${mountPath}" is ${Math.round(record.size / 1024 / 1024)}MB, over the ${MOUNT_URL_MAX_BYTES / 1024 / 1024}MB per-file mount limit.` - ) - } - if (mounted.url + record.size > MAX_TOTAL_URL_BYTES) { - throw new Error( - `Mounting "${mountPath}" would exceed the ${MAX_TOTAL_URL_BYTES / 1024 / 1024 / 1024}GB total mount limit. Mount fewer or smaller files.` - ) - } - const url = await generatePresignedDownloadUrl( - record.key, - record.storageContext ?? 'workspace', - MOUNT_URL_TTL_SECONDS - ) - sandboxFiles.push({ type: 'url', path: mountPath, url }) - mounted.url += record.size - return - } - - const remainingBudget = Math.max(0, MAX_TOTAL_SIZE - mounted.buffered) - - // A source-backed document declares the size of its generator, not of the document, - // so these pre-checks say nothing about what is about to be mounted. Its read is - // capped instead, and the real length is checked once it is known. - if (!rendersFromSource) { - if (record.size > MAX_FILE_SIZE) { - throw new Error( - `Input file "${mountPath}" is ${Math.round(record.size / 1024 / 1024)}MB, over the ${MAX_FILE_SIZE / 1024 / 1024}MB per-file mount limit.` - ) - } - if (record.size > remainingBudget) { - throw new Error( - `Mounting "${mountPath}" would exceed the ${MAX_TOTAL_SIZE / 1024 / 1024}MB total mount limit. Mount fewer or smaller files.` - ) - } - } - - const { buffer, contentType } = rendersFromSource - ? await fetchAuthorizedServableWorkspaceFileBuffer(record, principal, { - maxBytes: Math.min(MAX_FILE_SIZE, remainingBudget), - }).catch((error) => { - if (!isPayloadSizeLimitError(error)) throw error - throw new Error( - `Input file "${mountPath}" renders to more than the ${MAX_FILE_SIZE / 1024 / 1024}MB per-file mount limit, or than the mount budget left. Mount fewer or smaller files.` + await pushSandboxFileMount( + sandboxFiles, + { + mountPath, + key: record.key, + storageContext: record.storageContext ?? 'workspace', + declaredSize: record.size, + rendersFromSource, + readInline: async (maxBytes) => { + const { buffer, contentType } = rendersFromSource + ? await fetchAuthorizedServableWorkspaceFileBuffer(record, principal, { + maxBytes, + }).catch((error) => { + if (!isPayloadSizeLimitError(error)) throw error + throw new Error( + `Input file "${mountPath}" renders to more than the ${MAX_FILE_SIZE / 1024 / 1024}MB per-file mount limit, or than the mount budget left. Mount fewer or smaller files.` + ) + }) + : { + buffer: ( + await readWorkspaceFileContent.execute({ + principal, + input: { + fileId: record.id, + assertedWorkspaceId: workspaceId, + maxBytes, + }, + }) + ).content, + contentType: record.type, + } + // Keyed off the resolved type: a rendered document's source MIME is `text/x-…`, and + // decoding the binary as UTF-8 would corrupt it just as surely as shipping the source. + const isText = /^text\/|application\/json|application\/xml|application\/csv/.test( + contentType || '' ) - }) - : { - buffer: ( - await readWorkspaceFileContent.execute({ - principal, - input: { - fileId: record.id, - assertedWorkspaceId: workspaceId, - maxBytes: Math.min(MAX_FILE_SIZE, remainingBudget), - }, - }) - ).content, - contentType: record.type, - } - // Keyed off the resolved type: a rendered document's source MIME is `text/x-…`, and - // decoding the binary as UTF-8 would corrupt it just as surely as shipping the source. - const isText = /^text\/|application\/json|application\/xml|application\/csv/.test( - contentType || '' + return { + content: isText ? buffer.toString('utf-8') : buffer.toString('base64'), + ...(isText ? {} : { encoding: 'base64' as const }), + byteLength: buffer.length, + } + }, + }, + mounted ) - sandboxFiles.push({ - path: mountPath, - content: isText ? buffer.toString('utf-8') : buffer.toString('base64'), - encoding: isText ? undefined : 'base64', - }) - mounted.buffered += buffer.length } /** @@ -306,7 +253,7 @@ export async function resolveInputFiles( filePrincipal?: Principal ): Promise { const sandboxFiles: SandboxFile[] = [] - const mounted: MountedBytes = { buffered: 0, url: 0 } + const mounted = createSandboxMountBudget() if (inputFiles?.length && workspaceId) { if (!filePrincipal) { @@ -512,7 +459,7 @@ export async function resolveInputFiles( 'execution', MOUNT_URL_TTL_SECONDS ) - sandboxFiles.push({ type: 'url', path: mountPath, url }) + sandboxFiles.push({ type: 'url', path: mountPath, url, maxBytes: SNAPSHOT_MAX_BYTES }) mounted.url += snapshot.size continue } diff --git a/apps/sim/lib/copilot/tools/server/table/user-table.test.ts b/apps/sim/lib/copilot/tools/server/table/user-table.test.ts index 85d39c44e4d..1e96164e1b5 100644 --- a/apps/sim/lib/copilot/tools/server/table/user-table.test.ts +++ b/apps/sim/lib/copilot/tools/server/table/user-table.test.ts @@ -258,6 +258,7 @@ vi.mock('@/lib/workflows/application/context', () => ({ })) vi.mock('@/lib/workflows/application/resolve-workflow-outputs', () => ({ + loadResolvedDeployedWorkflowOutputs: async () => mockExecuteCopilotWorkflowUseCase(), loadResolvedWorkflowOutputs: async () => mockExecuteCopilotWorkflowUseCase(), resolveWorkflowOutputs: { operation: { id: 'workflows.read' } }, })) @@ -959,6 +960,25 @@ describe('userTableServerTool workflow scope', () => { expect(mockAddWorkflowGroup).not.toHaveBeenCalled() }) + it('does not pass a legacy deployment mode into workflow group creation', async () => { + const result = await userTableServerTool.execute( + { + operation: 'add_workflow_group', + args: { + tableId: 'tbl_1', + workflowId: 'workflow-1', + outputs: [{ blockId: 'block-1', path: 'content' }], + deploymentMode: 'live', + }, + }, + buildToolContext() + ) + + expect(result.success).toBe(true) + expect(mockAddWorkflowGroup).toHaveBeenCalledTimes(1) + expect(mockAddWorkflowGroup.mock.calls[0][0].group).not.toHaveProperty('deploymentMode') + }) + it('conceals unknown application failures from tool output', async () => { mockQueryRows.mockRejectedValueOnce(new Error('database host unavailable')) diff --git a/apps/sim/lib/copilot/tools/server/table/user-table.ts b/apps/sim/lib/copilot/tools/server/table/user-table.ts index 372d3aa9898..d48ac344d42 100644 --- a/apps/sim/lib/copilot/tools/server/table/user-table.ts +++ b/apps/sim/lib/copilot/tools/server/table/user-table.ts @@ -62,7 +62,6 @@ import type { TablePredicateInput, TableSchema, WorkflowGroupDependencies, - WorkflowGroupDeploymentMode, } from '@/lib/table/types' import { viewConfigIdsToNames } from '@/lib/table/views/service' import type { ResolvedSecretTraceProvenanceV1 } from '@/executor/utils/resolved-secret-trace-registry' @@ -132,16 +131,6 @@ function resolveAuthorizedWorkflowOutputs( }) } -/** - * Narrows a raw `deploymentMode` arg to the `'live' | 'deployed'` union, or - * `undefined` when absent/invalid (leaving the group's existing value — which - * itself defaults to `'live'`). Lets Mothership choose whether a group's - * per-cell runs execute the live draft or the latest active deployment. - */ -function parseDeploymentMode(value: unknown): WorkflowGroupDeploymentMode | undefined { - return value === 'live' || value === 'deployed' ? value : undefined -} - /** Validates an optional row limit against the policy for the requested surface operation. */ function limitError(limit: unknown, max?: number): string | null { if (limit === undefined) return null @@ -1242,7 +1231,6 @@ export const userTableServerTool: BaseServerTool const dependencies = args.dependencies as WorkflowGroupDependencies | undefined const name = args.name as string | undefined - const deploymentMode = parseDeploymentMode(args.deploymentMode) assertNotAborted() const autoRun = args.autoRun === true const { table: updated, group } = await executeCopilotCreateWorkflowTableGroup(context, { @@ -1252,7 +1240,6 @@ export const userTableServerTool: BaseServerTool outputs: rawOutputs, name, dependencies, - deploymentMode, autoRun, }) return { @@ -1294,7 +1281,6 @@ export const userTableServerTool: BaseServerTool dependencies: args.dependencies as WorkflowGroupDependencies | undefined, outputs: updateOutputs, mappingUpdates, - deploymentMode: parseDeploymentMode(args.deploymentMode), autoRun: typeof args.autoRun === 'boolean' ? args.autoRun : undefined, }) return { diff --git a/apps/sim/lib/core/async-jobs/backends/trigger-dev.ts b/apps/sim/lib/core/async-jobs/backends/trigger-dev.ts index 4b3960740de..3b103314a62 100644 --- a/apps/sim/lib/core/async-jobs/backends/trigger-dev.ts +++ b/apps/sim/lib/core/async-jobs/backends/trigger-dev.ts @@ -181,6 +181,7 @@ const JOB_TYPE_TO_TASK_ID: Record = { 'workflow-group-cell': 'workflow-group-cell', 'cleanup-logs': 'cleanup-logs', 'cleanup-soft-deletes': 'cleanup-soft-deletes', + 'cleanup-table-row-ttl': 'cleanup-table-row-ttl', 'cleanup-tasks': 'cleanup-tasks', 'run-data-drain': 'run-data-drain', } diff --git a/apps/sim/lib/core/async-jobs/types.ts b/apps/sim/lib/core/async-jobs/types.ts index fb5facc4b13..793ce0d938a 100644 --- a/apps/sim/lib/core/async-jobs/types.ts +++ b/apps/sim/lib/core/async-jobs/types.ts @@ -44,6 +44,7 @@ export type JobType = | 'workflow-group-cell' | 'cleanup-logs' | 'cleanup-soft-deletes' + | 'cleanup-table-row-ttl' | 'cleanup-tasks' | 'run-data-drain' diff --git a/apps/sim/lib/core/config/env.ts b/apps/sim/lib/core/config/env.ts index 74462eb6c72..c7fd7c7616d 100644 --- a/apps/sim/lib/core/config/env.ts +++ b/apps/sim/lib/core/config/env.ts @@ -587,6 +587,7 @@ export const env = createEnv({ SESSION_POLICIES_ENABLED: z.boolean().optional(), // Enable org session policies on self-hosted (bypasses hosted requirements) FORKING_ENABLED: z.boolean().optional(), // Enable workspace forking on self-hosted (bypasses hosted requirements) TABLES_V2_API: z.boolean().optional(), // Enable the v2 tables HTTP API (public /api/v2/tables + internal /api/table/[tableId]/query predicate-grammar route) + TABLE_ROW_TTL: z.boolean().optional(), CREDENTIAL_GROUPS: z.boolean().optional(), // Enable enterprise Credential Groups globally // Organizations - for self-hosted deployments diff --git a/apps/sim/lib/core/config/feature-flags.test.ts b/apps/sim/lib/core/config/feature-flags.test.ts index 6023ac5812e..fefd67589a3 100644 --- a/apps/sim/lib/core/config/feature-flags.test.ts +++ b/apps/sim/lib/core/config/feature-flags.test.ts @@ -12,6 +12,7 @@ const { mockFetch, mockIsPlatformAdmin, envRef } = vi.hoisted(() => ({ APPCONFIG_APPLICATION: 'sim-staging' as string | undefined, APPCONFIG_ENVIRONMENT: 'staging' as string | undefined, TABLES_V2_API: undefined as boolean | undefined, + TABLE_ROW_TTL: undefined as boolean | undefined, CREDENTIAL_GROUPS: undefined as boolean | undefined, }, })) @@ -77,6 +78,7 @@ describe('getFeatureFlags', () => { // All registered flags should be present, disabled (env vars unset in test env) expect(flags['trigger-eu-region']).toEqual({ enabled: false }) expect(flags['tables-v2-api']).toEqual({ enabled: false }) + expect(flags['table-row-ttl']).toEqual({ enabled: false }) expect(flags['credential-groups']).toEqual({ enabled: false }) expect(mockFetch).not.toHaveBeenCalled() }) @@ -104,6 +106,7 @@ describe('getFeatureFlags', () => { const flags = await getFeatureFlags() expect(flags['trigger-eu-region']).toEqual({ enabled: false }) expect(flags['tables-v2-api']).toEqual({ enabled: false }) + expect(flags['table-row-ttl']).toEqual({ enabled: false }) expect(flags['credential-groups']).toEqual({ enabled: false }) }) @@ -238,3 +241,23 @@ describe('tables-v2-api flag', () => { expect(await isFeatureEnabled('tables-v2-api')).toBe(true) }) }) + +describe('table-row-ttl flag', () => { + beforeEach(() => { + vi.clearAllMocks() + setEnvFlags({ isAppConfigEnabled: false }) + envRef.TABLE_ROW_TTL = undefined + }) + + it('uses a global fallback switch off AppConfig', async () => { + expect(await isFeatureEnabled('table-row-ttl')).toBe(false) + + envRef.TABLE_ROW_TTL = true + expect(await isFeatureEnabled('table-row-ttl')).toBe(true) + }) + + it('uses the global AppConfig clause', async () => { + withAppConfig({ 'table-row-ttl': { enabled: true } }) + expect(await isFeatureEnabled('table-row-ttl')).toBe(true) + }) +}) diff --git a/apps/sim/lib/core/config/feature-flags.ts b/apps/sim/lib/core/config/feature-flags.ts index b610508b7ee..e0d457d69c9 100644 --- a/apps/sim/lib/core/config/feature-flags.ts +++ b/apps/sim/lib/core/config/feature-flags.ts @@ -72,6 +72,12 @@ const FEATURE_FLAGS = { 'AppConfig; off-AppConfig falls back to TABLES_V2_API.', fallback: 'TABLES_V2_API', }, + 'table-row-ttl': { + description: + 'Enable TTL columns and the scheduled cleanup that removes expired table rows. ' + + 'Global on/off only; existing TTL data remains readable when disabled.', + fallback: 'TABLE_ROW_TTL', + }, 'credential-groups': { description: 'Workspace-owned collections that gather managed OAuth credentials from external users. ' + diff --git a/apps/sim/lib/core/utils/timezone.test.ts b/apps/sim/lib/core/utils/timezone.test.ts index 935a9061cee..64ce3f793d4 100644 --- a/apps/sim/lib/core/utils/timezone.test.ts +++ b/apps/sim/lib/core/utils/timezone.test.ts @@ -1,11 +1,72 @@ import { describe, expect, it } from 'vitest' import { + formatInstantInTimeZone, getSupportedTimezones, getTimezoneOptions, + getWallClockParts, wallClockNow, zonedClockDate, + zonedWallClock, zonedWallClockToUtc, -} from './timezone' + zonedWallClockWithOffset, +} from '@/lib/core/utils/timezone' + +describe('formatInstantInTimeZone', () => { + it.each([ + ['UTC', '0050-01-15T12:00:00Z', '0050-01-15T12:00:00Z'], + ['UTC', '2026-06-15T00:15:30Z', '2026-06-15T00:15:30Z'], + ['America/Los_Angeles', '2026-06-15T00:15:30Z', '2026-06-14T17:15:30-07:00'], + ['Asia/Tokyo', '2026-06-15T00:15:30Z', '2026-06-15T09:15:30+09:00'], + ['Asia/Kathmandu', '2026-06-15T00:15:30Z', '2026-06-15T06:00:30+05:45'], + ['Australia/Lord_Howe', '2026-06-15T00:15:30Z', '2026-06-15T10:45:30+10:30'], + ])('formats an instant in %s with its exact offset', (timeZone, iso, expected) => { + expect(formatInstantInTimeZone(new Date(iso), timeZone)).toBe(expected) + }) + + it('distinguishes both copies of an autumn daylight-saving hour', () => { + expect(formatInstantInTimeZone(new Date('2026-11-01T05:30:00Z'), 'America/New_York')).toBe( + '2026-11-01T01:30:00-04:00' + ) + expect(formatInstantInTimeZone(new Date('2026-11-01T06:30:00Z'), 'America/New_York')).toBe( + '2026-11-01T01:30:00-05:00' + ) + }) + + it('round-trips the same instant after changing display timezones', () => { + const instant = new Date('2026-11-01T06:30:00Z') + for (const timeZone of [ + 'UTC', + 'America/Los_Angeles', + 'America/New_York', + 'Asia/Kathmandu', + 'Australia/Lord_Howe', + ]) { + const editable = formatInstantInTimeZone(instant, timeZone) + expect(new Date(editable).getTime()).toBe(instant.getTime()) + } + }) + + it('preserves a four-digit low year in naive wall-clock output', () => { + expect(zonedWallClock(new Date('0050-01-15T12:00:00Z'), 'UTC')).toBe('0050-01-15T12:00') + }) +}) + +describe('getWallClockParts', () => { + it('returns the calendar fields of an instant in the requested timezone', () => { + expect(getWallClockParts(new Date('2026-06-15T00:15:30Z'), 'America/Los_Angeles')).toEqual({ + year: 2026, + month: 6, + day: 14, + hour: 17, + minute: 15, + second: 30, + }) + }) + + it('rejects an empty timezone instead of using the runtime local timezone', () => { + expect(() => getWallClockParts(new Date('2026-06-15T00:15:30Z'), '')).toThrow(RangeError) + }) +}) describe('zonedWallClockToUtc', () => { it('treats a UTC wall-clock as the same instant', () => { @@ -14,6 +75,26 @@ describe('zonedWallClockToUtc', () => { ) }) + it.each(['0000', '0001', '0050', '0099'])( + 'preserves the full year %s when resolving a wall-clock', + (year) => { + expect(zonedWallClockToUtc(`${year}-01-15T12:00`, 'UTC').toISOString()).toBe( + `${year}-01-15T12:00:00.000Z` + ) + } + ) + + it('uses the requested low year when resolving historical timezone rules', () => { + const wallClock = '0050-01-15T12:00:00' + + expect(zonedWallClockToUtc(wallClock, 'America/New_York').toISOString()).toBe( + '0050-01-15T16:56:02.000Z' + ) + expect(zonedWallClockWithOffset(wallClock, 'America/New_York')).toBe( + '0050-01-15T12:00:00-04:56' + ) + }) + it('applies a positive (east-of-UTC) offset (Asia/Kolkata, UTC+5:30)', () => { expect(zonedWallClockToUtc('2026-06-15T09:00', 'Asia/Kolkata').toISOString()).toBe( '2026-06-15T03:30:00.000Z' @@ -48,11 +129,104 @@ describe('zonedWallClockToUtc', () => { }) it('resolves a spring-forward gap wall-clock forward by the DST shift', () => { - // 2026-03-08 02:00–02:59 does not exist in America/New_York (EST→EDT). - expect(zonedWallClockToUtc('2026-03-08T02:30', 'America/New_York').toISOString()).toBe( - '2026-03-08T07:30:00.000Z' + const instant = zonedWallClockToUtc('2026-03-08T02:30', 'America/New_York') + const stampedWallClock = zonedWallClockWithOffset('2026-03-08T02:30', 'America/New_York') + + expect(instant.toISOString()).toBe('2026-03-08T07:30:00.000Z') + expect(stampedWallClock).toBe('2026-03-08T02:30-05:00') + expect(new Date(stampedWallClock).toISOString()).toBe(instant.toISOString()) + }) + + it.each([ + [ + 'Europe/Berlin', + '2026-03-29T02:30', + '2026-03-29T01:30:00.000Z', + '2026-03-29T03:30:00+02:00', + '2026-03-29T02:30+01:00', + ], + [ + 'Australia/Lord_Howe', + '2026-10-04T02:15', + '2026-10-03T15:45:00.000Z', + '2026-10-04T02:45:00+11:00', + '2026-10-04T02:15+10:30', + ], + ])( + 'resolves an east-of-UTC spring-forward gap in %s to the first compatible wall-clock', + (timeZone, wallClock, expectedInstant, expectedRenderedWallClock, expectedStampedWallClock) => { + const instant = zonedWallClockToUtc(wallClock, timeZone) + const stampedWallClock = zonedWallClockWithOffset(wallClock, timeZone) + + expect(instant.toISOString()).toBe(expectedInstant) + expect(formatInstantInTimeZone(instant, timeZone)).toBe(expectedRenderedWallClock) + expect(stampedWallClock).toBe(expectedStampedWallClock) + expect(new Date(stampedWallClock).toISOString()).toBe(expectedInstant) + } + ) + + it.each([ + ['America/New_York', '2026-11-01T01:30', '2026-11-01T06:30:00.000Z', '-05:00'], + ['Europe/Berlin', '2026-10-25T02:30', '2026-10-25T01:30:00.000Z', '+01:00'], + ['Australia/Lord_Howe', '2026-04-05T01:45', '2026-04-04T15:15:00.000Z', '+10:30'], + ])( + 'chooses the later post-transition instant for an ambiguous fall-back wall-clock in %s', + (timeZone, wallClock, expectedInstant, expectedOffset) => { + const instant = zonedWallClockToUtc(wallClock, timeZone) + const stampedWallClock = zonedWallClockWithOffset(wallClock, timeZone) + + expect(instant.toISOString()).toBe(expectedInstant) + expect(stampedWallClock).toBe(`${wallClock}${expectedOffset}`) + expect(new Date(stampedWallClock).toISOString()).toBe(expectedInstant) + } + ) + + it.each([ + ['America/New_York', '2026-11-01T01:30', '2026-11-01T05:30:00.000Z', '-04:00'], + ['Europe/Berlin', '2026-10-25T02:30', '2026-10-25T00:30:00.000Z', '+02:00'], + ['Australia/Lord_Howe', '2026-04-05T01:45', '2026-04-04T14:45:00.000Z', '+11:00'], + ])( + 'can choose the earlier instant for an ambiguous fall-back wall-clock in %s', + (timeZone, wallClock, expectedInstant, expectedOffset) => { + const options = { ambiguousTime: 'earlier' as const } + const instant = zonedWallClockToUtc(wallClock, timeZone, options) + const stampedWallClock = zonedWallClockWithOffset(wallClock, timeZone, options) + + expect(instant.toISOString()).toBe(expectedInstant) + expect(stampedWallClock).toBe(`${wallClock}${expectedOffset}`) + expect(new Date(stampedWallClock).toISOString()).toBe(expectedInstant) + } + ) + + it('does not retain timezone state between consecutive resolutions', () => { + const wallClock = '2026-06-15T09:00:30' + + expect(zonedWallClockToUtc(wallClock, 'America/New_York').toISOString()).toBe( + '2026-06-15T13:00:30.000Z' + ) + expect(zonedWallClockToUtc(wallClock, 'Asia/Kathmandu').toISOString()).toBe( + '2026-06-15T03:15:30.000Z' + ) + expect(zonedWallClockToUtc(wallClock, 'America/New_York').toISOString()).toBe( + '2026-06-15T13:00:30.000Z' ) }) + + it('can serialize historical sub-minute offsets toward a later instant', () => { + const wallClock = '1970-01-01T00:00:00' + const timezone = 'Africa/Monrovia' + const exactInstant = zonedWallClockToUtc(wallClock, timezone) + const options = { offsetMinuteRounding: 'floor' as const } + + expect(exactInstant.toISOString()).toBe('1970-01-01T00:44:30.000Z') + expect(zonedWallClockWithOffset(wallClock, timezone, options)).toBe('1970-01-01T00:00:00-00:45') + expect(formatInstantInTimeZone(exactInstant, timezone, options)).toBe( + '1970-01-01T00:00:00-00:45' + ) + expect( + Date.parse(zonedWallClockWithOffset(wallClock, timezone, options)) + ).toBeGreaterThanOrEqual(exactInstant.getTime()) + }) }) describe('wallClockNow', () => { diff --git a/apps/sim/lib/core/utils/timezone.ts b/apps/sim/lib/core/utils/timezone.ts index 297c0a65ec8..e4d2ffea799 100644 --- a/apps/sim/lib/core/utils/timezone.ts +++ b/apps/sim/lib/core/utils/timezone.ts @@ -23,6 +23,51 @@ const COMMON_TIMEZONES = [ 'Australia/Sydney', ] +/** A wall-clock reading of an instant in some timezone. */ +export interface WallClockParts { + year: number + /** 1-based month. */ + month: number + day: number + hour: number + minute: number + second: number +} + +function pad(value: number): string { + return String(value).padStart(2, '0') +} + +/** Formats years 0–9999 using ISO's four-digit representation. */ +export function formatIsoYear(year: number): string { + const serialized = String(year) + return year >= 0 && year <= 9999 ? serialized.padStart(4, '0') : serialized +} + +/** Builds a UTC timestamp without `Date.UTC` remapping years 0–99 to 1900–1999. */ +function utcTimestamp(wall: WallClockParts): number { + if (wall.year < 0 || wall.year > 99) { + return Date.UTC(wall.year, wall.month - 1, wall.day, wall.hour, wall.minute, wall.second) + } + const date = new Date(0) + date.setUTCFullYear(wall.year, wall.month - 1, wall.day) + date.setUTCHours(wall.hour, wall.minute, wall.second, 0) + return date.getTime() +} + +/** RFC 3339 offset suffix: `Z` for zero, else `±HH:MM`. */ +export function formatUtcOffsetSuffix(offsetMinutes: number): string { + if (offsetMinutes === 0) return 'Z' + const sign = offsetMinutes > 0 ? '+' : '-' + const absoluteMinutes = Math.abs(offsetMinutes) + return `${sign}${pad(Math.floor(absoluteMinutes / 60))}:${pad(absoluteMinutes % 60)}` +} + +function offsetMsFromWallClock(instant: Date, wall: WallClockParts): number { + const wallAsUtc = utcTimestamp(wall) + return wallAsUtc - instant.getTime() +} + /** The IANA timezone the current runtime resolves to (e.g. `America/New_York`). */ export function getBrowserTimezone(): string { return Intl.DateTimeFormat().resolvedOptions().timeZone @@ -38,6 +83,11 @@ export function isValidTimezone(timezone: string): boolean { } } +/** Removes control characters and bounds an untrusted timezone before displaying it. */ +export function sanitizeTimezoneForDisplay(timezone: string, maxLength = 64): string { + return truncate(timezone.replace(/[\p{Cc}\p{Zl}\p{Zp}]/gu, ' '), maxLength) +} + /** * Rejects a timezone that is not an IANA name. * @@ -54,7 +104,7 @@ export function assertValidTimezone(timezone: string): void { // Echoed back trimmed and stripped of line breaks: the rejected value came off // a query string, and a raw one carrying newlines or U+2028/U+2029 would forge // extra lines in whatever log or error surface renders the message. - const safe = truncate(timezone.replace(/[\p{Cc}\p{Zl}\p{Zp}]/gu, ' '), 64) + const safe = sanitizeTimezoneForDisplay(timezone) throw new Error(`Invalid timezone: ${safe}. Use an IANA name like "America/Los_Angeles".`) } } @@ -116,22 +166,66 @@ export function getTimezoneOptions(): TimezoneOption[] { } /** - * An instant's wall-clock time in `timeZone` as a naive `yyyy-MM-ddTHH:mm` - * string. Lets callers reason about a user's local date/time without UTC — e.g. - * to recover the local date/time a stored task instant represents in its zone. + * The wall-clock fields of `instant` in `timeZone`, or in the runtime's local + * timezone when omitted. */ -export function zonedWallClock(instant: Date, timeZone: string): string { - const parts = new Intl.DateTimeFormat('en-CA', { +export function getWallClockParts(instant: Date, timeZone?: string): WallClockParts { + if (timeZone === undefined) { + return { + year: instant.getFullYear(), + month: instant.getMonth() + 1, + day: instant.getDate(), + hour: instant.getHours(), + minute: instant.getMinutes(), + second: instant.getSeconds(), + } + } + + const parts = new Intl.DateTimeFormat('en-US', { timeZone, + hourCycle: 'h23', + era: 'short', year: 'numeric', month: '2-digit', day: '2-digit', hour: '2-digit', minute: '2-digit', - hourCycle: 'h23', + second: '2-digit', }).formatToParts(instant) - const get = (type: string) => parts.find((p) => p.type === type)?.value ?? '00' - return `${get('year')}-${get('month')}-${get('day')}T${get('hour')}:${get('minute')}` + const get = (type: string) => Number(parts.find((part) => part.type === type)?.value) + const year = get('year') + const era = parts.find((part) => part.type === 'era')?.value + return { + year: era === 'BC' ? 1 - year : year, + month: get('month'), + day: get('day'), + hour: get('hour'), + minute: get('minute'), + second: get('second'), + } +} + +/** Formats an instant as an RFC 3339 wall time in an IANA timezone. */ +export function formatInstantInTimeZone( + instant: Date, + timeZone: string, + options?: ZonedWallClockOptions +): string { + const wall = getWallClockParts(instant, timeZone) + const wholeSecondInstant = new Date(Math.floor(instant.getTime() / 1000) * 1000) + const exactOffsetMinutes = offsetMsFromWallClock(wholeSecondInstant, wall) / 60_000 + const offsetMinutes = roundOffsetMinutes(exactOffsetMinutes, options) + return `${formatIsoYear(wall.year)}-${pad(wall.month)}-${pad(wall.day)}T${pad(wall.hour)}:${pad(wall.minute)}:${pad(wall.second)}${formatUtcOffsetSuffix(offsetMinutes)}` +} + +/** + * An instant's wall-clock time in `timeZone` as a naive `yyyy-MM-ddTHH:mm` + * string. Lets callers reason about a user's local date/time without UTC — e.g. + * to recover the local date/time a stored task instant represents in its zone. + */ +export function zonedWallClock(instant: Date, timeZone: string): string { + const wall = getWallClockParts(instant, timeZone) + return `${formatIsoYear(wall.year)}-${pad(wall.month)}-${pad(wall.day)}T${pad(wall.hour)}:${pad(wall.minute)}` } /** The current wall-clock time in `timeZone` as a naive `yyyy-MM-ddTHH:mm` string. */ @@ -156,26 +250,59 @@ export function zonedClockDate(instant: Date, timeZone: string): Date { /** The UTC offset (ms, east-positive) of `timeZone` at a given instant. */ function timezoneOffsetMs(instant: Date, timeZone: string): number { - const parts = new Intl.DateTimeFormat('en-US', { - timeZone, - hourCycle: 'h23', - year: 'numeric', - month: '2-digit', - day: '2-digit', - hour: '2-digit', - minute: '2-digit', - second: '2-digit', - }).formatToParts(instant) - const get = (type: string) => Number(parts.find((p) => p.type === type)?.value) - const asUtc = Date.UTC( - get('year'), - get('month') - 1, - get('day'), - get('hour'), - get('minute'), - get('second') + return offsetMsFromWallClock(instant, getWallClockParts(instant, timeZone)) +} + +interface ZonedWallClockResolution { + instant: Date + offsetMinutes: number +} + +export interface ZonedWallClockOptions { + /** Which real instant to use when the wall clock occurs twice during a DST fall-back. */ + ambiguousTime?: 'earlier' | 'later' + /** How to serialize rare historical offsets containing seconds into RFC 3339 minutes. */ + offsetMinuteRounding?: 'nearest' | 'floor' +} + +function roundOffsetMinutes(exactOffsetMinutes: number, options?: ZonedWallClockOptions): number { + return options?.offsetMinuteRounding === 'floor' + ? Math.floor(exactOffsetMinutes) + : Math.round(exactOffsetMinutes) +} + +function resolveZonedWallClock( + wallClock: string, + timeZone: string, + options?: ZonedWallClockOptions +): ZonedWallClockResolution { + const [datePart, timePart] = wallClock.split('T') + const [year, month, day] = datePart.split('-').map(Number) + const [hour, minute, second = 0] = timePart.split(':').map(Number) + const utcGuess = utcTimestamp({ year, month, day, hour, minute, second }) + const dayMs = 24 * 60 * 60 * 1000 + const offsets = new Set( + [-dayMs, 0, dayMs].map((distance) => timezoneOffsetMs(new Date(utcGuess + distance), timeZone)) ) - return asUtc - instant.getTime() + const candidates = [...offsets].map((offset) => { + const instantMs = utcGuess - offset + const actualOffset = timezoneOffsetMs(new Date(instantMs), timeZone) + return { instantMs, wallClockMs: instantMs + actualOffset } + }) + const exactCandidate = candidates + .filter(({ wallClockMs }) => wallClockMs === utcGuess) + .sort((a, b) => + options?.ambiguousTime === 'earlier' ? a.instantMs - b.instantMs : b.instantMs - a.instantMs + )[0] + const compatibleCandidate = candidates + .filter(({ wallClockMs }) => wallClockMs > utcGuess) + .sort((a, b) => a.wallClockMs - b.wallClockMs || a.instantMs - b.instantMs)[0] + const chosenCandidate = exactCandidate ?? compatibleCandidate ?? candidates[0] + const instantMs = chosenCandidate.instantMs + return { + instant: new Date(instantMs), + offsetMinutes: (utcGuess - instantMs) / 60_000, + } } /** @@ -184,23 +311,28 @@ function timezoneOffsetMs(instant: Date, timeZone: string): number { * whose own offset reproduces the requested wall-clock, which is correct for any * date (including future ones whose offset differs from today's) and across DST: * a naive single pass reads the offset on the wrong side of a same-day boundary - * — notably the autumn fall-back hour — and lands an hour off. For an ambiguous - * fall-back wall-clock the later (post-transition) instant is chosen; a + * — notably the autumn fall-back hour — and lands an hour off. An ambiguous + * fall-back wall-clock defaults to the later, post-transition instant, but + * callers preserving earlier semantics may request the earlier instant. A * wall-clock in the spring-forward gap (a nonexistent local hour) has no * self-consistent instant and resolves forward by the DST shift, matching how * calendar apps treat that once-a-year hour. */ -export function zonedWallClockToUtc(wallClock: string, timeZone: string): Date { - const [datePart, timePart] = wallClock.split('T') - const [year, month, day] = datePart.split('-').map(Number) - const [hour, minute, second = 0] = timePart.split(':').map(Number) - const utcGuess = Date.UTC(year, month - 1, day, hour, minute, second) - const guessOffset = timezoneOffsetMs(new Date(utcGuess), timeZone) - const candidate = utcGuess - guessOffset - const candidateOffset = timezoneOffsetMs(new Date(candidate), timeZone) - if (candidateOffset === guessOffset) return new Date(candidate) - const adjusted = utcGuess - candidateOffset - return timezoneOffsetMs(new Date(adjusted), timeZone) === candidateOffset - ? new Date(adjusted) - : new Date(candidate) +export function zonedWallClockToUtc( + wallClock: string, + timeZone: string, + options?: ZonedWallClockOptions +): Date { + return resolveZonedWallClock(wallClock, timeZone, options).instant +} + +/** Stamps a naive wall-clock with the offset selected by the shared timezone resolver. */ +export function zonedWallClockWithOffset( + wallClock: string, + timeZone: string, + options?: ZonedWallClockOptions +): string { + const resolution = resolveZonedWallClock(wallClock, timeZone, options) + const offsetMinutes = roundOffsetMinutes(resolution.offsetMinutes, options) + return `${wallClock}${formatUtcOffsetSuffix(offsetMinutes)}` } diff --git a/apps/sim/lib/credential-groups/application/authorization.test.ts b/apps/sim/lib/credential-groups/application/authorization.test.ts index 052dfba31f9..fec8b1a7bbb 100644 --- a/apps/sim/lib/credential-groups/application/authorization.test.ts +++ b/apps/sim/lib/credential-groups/application/authorization.test.ts @@ -20,7 +20,10 @@ vi.mock('@/lib/resource-policies/repository', () => ({ requireResourcePolicy: mocks.requirePolicy, })) -import { requireCredentialGroupCredentialAccess } from '@/lib/credential-groups/application/authorization' +import { + requireCredentialGroupCredentialAccess, + requireCredentialGroupWorkflowActor, +} from '@/lib/credential-groups/application/authorization' const context = { workspaceId: 'workspace-1', @@ -216,3 +219,60 @@ describe('requireCredentialGroupCredentialAccess', () => { expect(mocks.loadEnrollmentAccess).not.toHaveBeenCalled() }) }) + +describe('requireCredentialGroupWorkflowActor', () => { + it('returns the external subject a Slack-triggered run acts as', () => { + expect(requireCredentialGroupWorkflowActor(executorPrincipal())).toEqual({ + kind: 'external_user', + provider: 'slack', + tenantId: 'T123', + subjectId: 'U123', + }) + }) + + it('returns no subject for an actorless deployed run', () => { + const principal = executorPrincipal() + principal.delegationContext!.principal = { + kind: 'system', + serviceId: 'schedule', + workspaceId: 'workspace-1', + workflowId: 'root-workflow', + } + + expect(requireCredentialGroupWorkflowActor(principal)).toBeNull() + }) + + it('returns the Sim subject a session-actor run acts as', () => { + const principal = executorPrincipal() + principal.subjectUserId = 'user-1' + principal.delegationContext!.principal = { + kind: 'session', + userId: 'user-1', + sessionId: 'session-1', + } + + expect(requireCredentialGroupWorkflowActor(principal)).toEqual({ + kind: 'sim_user', + userId: 'user-1', + }) + }) + + it('rejects a delegation whose asserted subject contradicts its run', () => { + const invented = executorPrincipal() + invented.subjectUserId = 'invented-user' + expect(() => requireCredentialGroupWorkflowActor(invented)).toThrow( + 'Credential Group actor access required' + ) + + const mismatched = executorPrincipal() + mismatched.subjectUserId = 'user-2' + mismatched.delegationContext!.principal = { + kind: 'session', + userId: 'user-1', + sessionId: 'session-1', + } + expect(() => requireCredentialGroupWorkflowActor(mismatched)).toThrow( + 'Credential Group actor access required' + ) + }) +}) diff --git a/apps/sim/lib/credential-groups/application/authorization.ts b/apps/sim/lib/credential-groups/application/authorization.ts index 2cde1459083..13cc444e606 100644 --- a/apps/sim/lib/credential-groups/application/authorization.ts +++ b/apps/sim/lib/credential-groups/application/authorization.ts @@ -1,5 +1,6 @@ import { type Principal, + type PrincipalSubject, resolvePrincipalSubject, type WorkflowExecutionAuthority, type WorkflowExecutionPrincipal, @@ -67,16 +68,19 @@ function requireConsistentWorkflowSubject( return subject } -export function requireCredentialGroupWorkflowSubject(principal: Principal): string { - const subject = resolvePrincipalSubject(requireWorkflowExecutionPrincipal(principal)) - if ( - subject?.kind !== 'sim_user' || - principal.kind !== 'delegated' || - principal.subjectUserId !== subject.userId - ) { - throw new OrchestrationError('forbidden', 'Credential Group user access required') - } - return subject.userId +/** + * Asserts the delegation still names the subject its run was minted for, without + * requiring that subject to be a Sim user. + * + * A Slack-triggered run's subject is the external Slack user, and a scheduled, + * public-API, or subject-less webhook run has no subject at all. Neither is + * representable as a Sim user, and neither is what authorizes the call — for an + * actorless caller that is the deployment the workspace layer already checked. + * Whoever the run acts as is attribution only; an invitation issued with no Sim + * user simply records none. + */ +export function requireCredentialGroupWorkflowActor(principal: Principal): PrincipalSubject | null { + return requireConsistentWorkflowSubject(principal, requireWorkflowExecutionPrincipal(principal)) } export async function requireCredentialGroupCredentialAccess( diff --git a/apps/sim/lib/credential-groups/application/list-groups.ts b/apps/sim/lib/credential-groups/application/list-groups.ts index d33938db34e..75599f6734e 100644 --- a/apps/sim/lib/credential-groups/application/list-groups.ts +++ b/apps/sim/lib/credential-groups/application/list-groups.ts @@ -2,7 +2,7 @@ import { defineAuthorizedWorkspaceUseCase } from '@/lib/core/application' import { OrchestrationError } from '@/lib/core/orchestration/types' import { credentialGroupWorkspaceDelegationPolicy, - requireCredentialGroupWorkflowSubject, + requireCredentialGroupWorkflowActor, } from '@/lib/credential-groups/application/authorization' import { requireCredentialGroupsAvailable, @@ -35,7 +35,7 @@ export const listCredentialGroupsForWorkflow = defineAuthorizedWorkspaceUseCase( resolveCredentialGroupWorkspaceContext(input.workspaceId), authorizationOptions: { delegation: credentialGroupWorkspaceDelegationPolicy }, authorizeResource({ principal }) { - requireCredentialGroupWorkflowSubject(principal) + requireCredentialGroupWorkflowActor(principal) }, execute: async ({ input, context }): Promise => { if ( diff --git a/apps/sim/lib/credential-groups/application/list-people.ts b/apps/sim/lib/credential-groups/application/list-people.ts index 10338d8defe..6d5d2ae9043 100644 --- a/apps/sim/lib/credential-groups/application/list-people.ts +++ b/apps/sim/lib/credential-groups/application/list-people.ts @@ -3,7 +3,7 @@ import { defineAuthorizedWorkspaceUseCase } from '@/lib/core/application' import { OrchestrationError } from '@/lib/core/orchestration/types' import { credentialGroupDelegationPolicy, - requireCredentialGroupWorkflowSubject, + requireCredentialGroupWorkflowActor, } from '@/lib/credential-groups/application/authorization' import { requireCredentialGroupsAvailable, @@ -38,7 +38,7 @@ export const listCredentialGroupPeople = defineAuthorizedWorkspaceUseCase({ resolveCredentialGroupContext(input.credentialGroupId), authorizationOptions: { delegation: credentialGroupDelegationPolicy }, authorizeResource({ principal }) { - requireCredentialGroupWorkflowSubject(principal) + requireCredentialGroupWorkflowActor(principal) }, execute: async ({ input, context }) => { if (context.status !== 'active') { diff --git a/apps/sim/lib/credential-groups/application/send-invite.test.ts b/apps/sim/lib/credential-groups/application/send-invite.test.ts new file mode 100644 index 00000000000..d72495678fa --- /dev/null +++ b/apps/sim/lib/credential-groups/application/send-invite.test.ts @@ -0,0 +1,199 @@ +/** + * @vitest-environment node + */ +import type { WorkflowExecutionDelegatedPrincipal } from '@sim/auth/principal' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const mocks = vi.hoisted(() => ({ + inviteEnrollment: vi.fn(), + loadInviter: vi.fn(), + requireAvailable: vi.fn(), + resolveGroup: vi.fn(), + resolvePermission: vi.fn(), +})) + +vi.mock('@/lib/credential-groups/application/context', () => ({ + requireCredentialGroupsAvailable: mocks.requireAvailable, + resolveCredentialGroupContext: mocks.resolveGroup, +})) + +vi.mock('@/lib/credential-groups/enrollments', () => ({ + inviteCredentialGroupEnrollment: mocks.inviteEnrollment, + loadCredentialGroupInviterIdentity: mocks.loadInviter, + CredentialGroupEnrollmentError: class CredentialGroupEnrollmentError extends Error { + constructor( + message: string, + readonly status: 400 | 404 | 409 | 502 + ) { + super(message) + } + }, +})) + +vi.mock('@sim/platform-authz/workspace', () => ({ + permissionSatisfies: (permission: string | null, required: string) => + permission === 'admin' || permission === required, + resolveEffectiveWorkspacePermission: mocks.resolvePermission, +})) + +import { sendCredentialGroupInvite } from '@/lib/credential-groups/application/send-invite' + +const context = { + workspaceId: 'workspace-1', + workspaceOrganizationId: null, + allowPersonalApiKeys: true, + billedAccountUserId: 'billing-owner-1', + credentialGroupId: 'group-1', + name: 'Support', + status: 'active' as const, + options: [], +} + +function executorPrincipal(): WorkflowExecutionDelegatedPrincipal { + return { + kind: 'delegated', + serviceId: 'executor', + subjectUserId: 'admin-1', + workspaceId: 'workspace-1', + delegationId: 'delegation-1', + audience: 'sim:credential-groups', + issuedAt: new Date(Date.now() - 1_000), + expiresAt: new Date(Date.now() + 60_000), + resourceScope: { credentialGroupId: 'group-1' }, + delegationContext: { + kind: 'workflow_execution', + workflowId: 'workflow-1', + principal: { kind: 'session', userId: 'admin-1', sessionId: 'session-1' }, + currentWorkflow: { workflowId: 'workflow-1', mode: 'draft' }, + }, + } +} + +/** A deployed run whose only actor is the external identity that triggered it. */ +function unattendedPrincipal( + principal: NonNullable['principal'] +): WorkflowExecutionDelegatedPrincipal { + const { subjectUserId: _subject, ...base } = executorPrincipal() + return { + ...base, + delegationContext: { + kind: 'workflow_execution', + workflowId: 'workflow-1', + principal, + currentWorkflow: { + workflowId: 'workflow-1', + mode: 'deployment', + deploymentVersionId: 'version-1', + }, + }, + } +} + +function slackPrincipal(): WorkflowExecutionDelegatedPrincipal { + return unattendedPrincipal({ + kind: 'system', + serviceId: 'webhook', + workspaceId: 'workspace-1', + workflowId: 'workflow-1', + webhookId: 'webhook-1', + provider: 'slack', + subject: { kind: 'external_user', provider: 'slack', tenantId: 'T123', subjectId: 'U123' }, + }) +} + +function invite(principal: WorkflowExecutionDelegatedPrincipal) { + return sendCredentialGroupInvite.execute({ + principal, + input: { credentialGroupId: 'group-1', email: 'person@example.com' }, + }) +} + +describe('sendCredentialGroupInvite', () => { + beforeEach(() => { + vi.clearAllMocks() + mocks.resolveGroup.mockResolvedValue(context) + mocks.resolvePermission.mockResolvedValue('admin') + mocks.requireAvailable.mockResolvedValue(undefined) + mocks.loadInviter.mockResolvedValue({ name: 'Ada Lovelace', email: 'ada@example.com' }) + mocks.inviteEnrollment.mockResolvedValue({ + id: 'enrollment-1', + email: 'person@example.com', + status: 'invited', + }) + }) + + it('invites without naming an inviter on a Slack-triggered run', async () => { + const result = await invite(slackPrincipal()) + + expect(result.enrollment.id).toBe('enrollment-1') + expect(mocks.loadInviter).not.toHaveBeenCalled() + expect(mocks.inviteEnrollment).toHaveBeenCalledWith( + 'workspace-1', + 'group-1', + undefined, + undefined, + 'person@example.com' + ) + }) + + it('invites without naming an inviter on an actorless run', async () => { + await invite( + unattendedPrincipal({ + kind: 'system', + serviceId: 'schedule', + workspaceId: 'workspace-1', + workflowId: 'workflow-1', + }) + ) + + expect(mocks.inviteEnrollment).toHaveBeenCalledWith( + 'workspace-1', + 'group-1', + undefined, + undefined, + 'person@example.com' + ) + }) + + it('names the human a session-actor run acts as', async () => { + await invite(executorPrincipal()) + + expect(mocks.loadInviter).toHaveBeenCalledWith('admin-1') + expect(mocks.inviteEnrollment).toHaveBeenCalledWith( + 'workspace-1', + 'group-1', + 'admin-1', + 'Ada Lovelace', + 'person@example.com' + ) + }) + + it('falls back to the inviter email when they have no name', async () => { + mocks.loadInviter.mockResolvedValue({ name: ' ', email: 'ada@example.com' }) + + await invite(executorPrincipal()) + + expect(mocks.inviteEnrollment).toHaveBeenCalledWith( + 'workspace-1', + 'group-1', + 'admin-1', + 'ada@example.com', + 'person@example.com' + ) + }) + + it('requires the current subject to remain a workspace admin', async () => { + mocks.resolvePermission.mockResolvedValue('write') + + await expect(invite(executorPrincipal())).rejects.toMatchObject({ code: 'forbidden' }) + expect(mocks.inviteEnrollment).not.toHaveBeenCalled() + }) + + it('rejects a delegation asserting a subject its run never had', async () => { + const spoofed = slackPrincipal() + spoofed.subjectUserId = 'invented-user' + + await expect(invite(spoofed)).rejects.toMatchObject({ code: 'forbidden' }) + expect(mocks.inviteEnrollment).not.toHaveBeenCalled() + }) +}) diff --git a/apps/sim/lib/credential-groups/application/send-invite.ts b/apps/sim/lib/credential-groups/application/send-invite.ts index 31ed72e9330..3e970b81675 100644 --- a/apps/sim/lib/credential-groups/application/send-invite.ts +++ b/apps/sim/lib/credential-groups/application/send-invite.ts @@ -1,10 +1,11 @@ import { AuditAction, AuditResourceType } from '@sim/audit' +import { resolvePrincipalSubjectUserId } from '@sim/auth/principal' import { isValidEmailSyntax, normalizeEmail } from '@sim/utils/string' import { defineAuthorizedWorkspaceUseCase } from '@/lib/core/application' import { OrchestrationError } from '@/lib/core/orchestration/types' import { credentialGroupDelegationPolicy, - requireCredentialGroupWorkflowSubject, + requireCredentialGroupWorkflowActor, } from '@/lib/credential-groups/application/authorization' import { requireCredentialGroupsAvailable, @@ -28,7 +29,7 @@ export const sendCredentialGroupInvite = defineAuthorizedWorkspaceUseCase({ resolveCredentialGroupContext(input.credentialGroupId), authorizationOptions: { delegation: credentialGroupDelegationPolicy }, authorizeResource({ principal }) { - requireCredentialGroupWorkflowSubject(principal) + requireCredentialGroupWorkflowActor(principal) }, execute: async ({ principal, input, context }) => { if (context.status !== 'active') { @@ -40,12 +41,11 @@ export const sendCredentialGroupInvite = defineAuthorizedWorkspaceUseCase({ } await requireCredentialGroupsAvailable(context.workspaceId) - const userId = requireCredentialGroupWorkflowSubject(principal) - const inviter = await loadCredentialGroupInviterIdentity(userId) + // Attribution, not authority. An actorless or Slack-triggered run names no + // inviter rather than borrowing its actor, so the email claims no one invited. + const userId = resolvePrincipalSubjectUserId(principal) + const inviter = userId ? await loadCredentialGroupInviterIdentity(userId) : null const inviterName = inviter?.name?.trim() || inviter?.email - if (!inviterName) { - throw new OrchestrationError('conflict', 'Inviting user has no display identity') - } try { const enrollment = await inviteCredentialGroupEnrollment( diff --git a/apps/sim/lib/credential-groups/enrollments.ts b/apps/sim/lib/credential-groups/enrollments.ts index 91e661c7238..3f99519ab2f 100644 --- a/apps/sim/lib/credential-groups/enrollments.ts +++ b/apps/sim/lib/credential-groups/enrollments.ts @@ -72,7 +72,8 @@ interface IssuedInvitation { } export interface PublicCredentialGroupEnrollment { - inviterName: string + /** Null when the invitation was issued by a workflow or a since-deleted user. */ + inviterName: string | null workspaceName: string credentialGroupName: string options: Array< @@ -426,8 +427,10 @@ async function issueInvitation( async function sendInvitation( context: InvitationContext, - userId: string, - inviterName: string, + /** See {@link issueInvitation}: the issuer is attribution, never the authority. */ + userId: string | undefined, + /** Absent when a workflow issued the invitation — the copy drops the inviter. */ + inviterName: string | undefined, email: string, options: SendInvitationOptions ): Promise { @@ -658,8 +661,10 @@ export async function loadCredentialGroupInviterIdentity( export async function inviteCredentialGroupEnrollment( workspaceId: string, groupId: string, - userId: string, - inviterName: string, + /** See {@link issueInvitation}: the issuer is attribution, never the authority. */ + userId: string | undefined, + /** See {@link sendInvitation}: absent for a workflow-issued invitation. */ + inviterName: string | undefined, email: string ): Promise { const context = await getInvitationContext(workspaceId, groupId) @@ -789,7 +794,7 @@ async function buildPublicCredentialGroupEnrollment( ) return { - inviterName: row.inviterName ?? 'A workspace admin', + inviterName: row.inviterName, workspaceName: row.workspaceName, credentialGroupName: row.groupName, options: await Promise.all( diff --git a/apps/sim/lib/execution/payloads/sandbox-file-mount-ref.ts b/apps/sim/lib/execution/payloads/sandbox-file-mount-ref.ts new file mode 100644 index 00000000000..55555db2163 --- /dev/null +++ b/apps/sim/lib/execution/payloads/sandbox-file-mount-ref.ts @@ -0,0 +1,124 @@ +import { isUserFileWithMetadata } from '@/lib/core/utils/user-file' +import type { UserFile } from '@/executor/types' + +export const SANDBOX_FILE_MOUNT_REF_MARKER = '__simSandboxFileMount' +export const SANDBOX_FILE_MOUNT_REF_VERSION = 1 + +/** + * A request to place one file on the sandbox filesystem, standing in for the + * path until the sandbox exists. + * + * Emitted when code references ``. Reference resolution happens + * long before a sandbox is created, and mount paths are only known once the whole + * set is planned (they are sanitized and de-duplicated together), so the resolver + * leaves this marker and the function runtime swaps in the real path. + * + * Same shape as {@link LargeValueRef}: a marker a later layer materializes. It + * exists only where the caller wrote `.path`, which is what keeps a bare + * `` reference — the common case, and the one that runs fine in the + * isolated VM — from being dragged into a remote sandbox it never needed. + */ +export interface SandboxFileMountRef { + [SANDBOX_FILE_MOUNT_REF_MARKER]: true + version: typeof SANDBOX_FILE_MOUNT_REF_VERSION + file: UserFile +} + +export function createSandboxFileMountRef(file: UserFile): SandboxFileMountRef { + return { + [SANDBOX_FILE_MOUNT_REF_MARKER]: true, + version: SANDBOX_FILE_MOUNT_REF_VERSION, + file, + } +} + +export function isSandboxFileMountRef(value: unknown): value is SandboxFileMountRef { + if (!value || typeof value !== 'object') return false + + const candidate = value as Record + return ( + candidate[SANDBOX_FILE_MOUNT_REF_MARKER] === true && + candidate.version === SANDBOX_FILE_MOUNT_REF_VERSION && + isUserFileWithMetadata(candidate.file) + ) +} + +/** + * Replaces every mount marker in a value with whatever `resolvePath` returns for + * its file, leaving the rest of the structure untouched. + * + * Rebuilds containers rather than mutating them: the same resolved block output + * can be shared with other consumers, and a marker can sit anywhere inside a + * referenced object, not only at the top level. + */ +export function replaceSandboxFileMountRefs( + value: unknown, + resolvePath: (file: UserFile) => string, + seen = new WeakMap() +): unknown { + if (!value || typeof value !== 'object') return value + if (isSandboxFileMountRef(value)) return resolvePath(value.file) + + const existing = seen.get(value) + if (existing !== undefined) return existing + + if (Array.isArray(value)) { + const next: unknown[] = [] + seen.set(value, next) + for (const item of value) next.push(replaceSandboxFileMountRefs(item, resolvePath, seen)) + return next + } + + // Only plain containers are rebuilt. A Date, Buffer, Map, or class instance + // has no own enumerable entries worth walking, and reconstructing one from + // Object.entries would quietly replace it with a stripped plain object — a + // Date becoming `{}` on its way to the sandbox. Such a value cannot hold a + // mount marker anyway, so passing it through is both safer and complete. + const prototype = Object.getPrototypeOf(value) + if (prototype !== Object.prototype && prototype !== null) return value + + const next: Record = {} + seen.set(value, next) + for (const [key, item] of Object.entries(value)) { + // defineProperty, not assignment: a own `__proto__` key would otherwise hit + // Object.prototype's setter and vanish before the value reaches the sandbox. + Object.defineProperty(next, key, { + value: replaceSandboxFileMountRefs(item, resolvePath, seen), + enumerable: true, + writable: true, + configurable: true, + }) + } + return next +} + +/** Every file a value asks to have mounted, in first-seen order. */ +export function collectSandboxFileMountRefs( + value: unknown, + found: UserFile[] = [], + seen = new WeakSet() +): UserFile[] { + if (!value || typeof value !== 'object') return found + if (isSandboxFileMountRef(value)) { + found.push(value.file) + return found + } + if (seen.has(value)) return found + seen.add(value) + + if (Array.isArray(value)) { + for (const item of value) collectSandboxFileMountRefs(item, found, seen) + return found + } + + // Same plain-container rule the replacement pass applies. The two walks have to + // agree on the tree: a marker counted here but skipped there would mount a file + // whose reference never became a path. + const prototype = Object.getPrototypeOf(value) + if (prototype !== Object.prototype && prototype !== null) return found + + for (const item of Object.values(value)) { + collectSandboxFileMountRefs(item, found, seen) + } + return found +} diff --git a/apps/sim/lib/execution/remote-sandbox/conformance.test.ts b/apps/sim/lib/execution/remote-sandbox/conformance.test.ts index 0f4271033ef..e6f915f7dc3 100644 --- a/apps/sim/lib/execution/remote-sandbox/conformance.test.ts +++ b/apps/sim/lib/execution/remote-sandbox/conformance.test.ts @@ -9,6 +9,7 @@ import { Readable } from 'node:stream' import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' import { CodeLanguage } from '@/lib/execution/languages' +import { SANDBOX_OUTPUT_DIR_SENTINEL } from '@/lib/execution/remote-sandbox/sandbox-paths' const { mockResolveSandbox, @@ -22,12 +23,14 @@ const { mockE2BFilesRead, mockE2BFilesRemove, mockE2BFilesWrite, + mockE2BFilesList, mockE2BKill, mockDaytonaCreate, mockInterpreterRunCode, mockProcessCodeRun, mockExecuteCommand, mockGetFileDetails, + mockListFiles, mockUploadFile, mockDownloadFile, mockDownloadFileStream, @@ -71,12 +74,14 @@ const { mockE2BFilesRead: vi.fn(), mockE2BFilesRemove: vi.fn(), mockE2BFilesWrite: vi.fn(), + mockE2BFilesList: vi.fn(), mockE2BKill: vi.fn(), mockDaytonaCreate: vi.fn(), mockInterpreterRunCode: vi.fn(), mockProcessCodeRun: vi.fn(), mockExecuteCommand: vi.fn(), mockGetFileDetails: vi.fn(), + mockListFiles: vi.fn(), mockUploadFile: vi.fn(), mockDownloadFile: vi.fn(), mockDownloadFileStream: vi.fn(), @@ -118,12 +123,21 @@ import { SIM_RESULT_PREFIX, withPiSandbox, } from '@/lib/execution/remote-sandbox' -import { daytonaProvider } from '@/lib/execution/remote-sandbox/daytona' -import { E2B_MAX_SANDBOX_LIFETIME_MS, e2bProvider } from '@/lib/execution/remote-sandbox/e2b' +import { + daytonaProvider, + resolveDaytonaSandboxLifetimeMs, +} from '@/lib/execution/remote-sandbox/daytona' +import { + E2B_MAX_SANDBOX_LIFETIME_MS, + e2bProvider, + resolveE2BSandboxLifetimeMs, +} from '@/lib/execution/remote-sandbox/e2b' import { MAX_SANDBOX_OUTPUT_BYTES, + MAX_SANDBOX_OUTPUT_FILES, MAX_SANDBOX_PROCESS_OUTPUT_BYTES, MAX_SANDBOX_STREAMED_OUTPUT_TAIL_BYTES, + readTrustedSandboxOutputCost, } from '@/lib/execution/remote-sandbox/output-limits' import { PI_SANDBOX_MIN_LIFETIME_MS, @@ -134,6 +148,27 @@ import { resolveProvider } from '@/lib/execution/remote-sandbox/provider' type Provider = 'e2b' | 'daytona' const PROVIDERS: Provider[] = ['e2b', 'daytona'] +describe('provider-effective sandbox lifetimes', () => { + it('matches E2B second and Daytona minute rounding', () => { + expect(resolveE2BSandboxLifetimeMs(1001)).toBe(2000) + expect(resolveDaytonaSandboxLifetimeMs(1001)).toBe(60_000) + }) + + it.each(PROVIDERS)('reports the %s SDK create dispatch time', async (provider) => { + useProvider(provider) + const onProviderRequestStarted = vi.fn() + const createMock = provider === 'e2b' ? mockE2BCreate : mockDaytonaCreate + + await resolveProvider().create('code', { lifetimeMs: 1000, onProviderRequestStarted }) + + expect(onProviderRequestStarted).toHaveBeenCalledOnce() + expect(onProviderRequestStarted).toHaveBeenCalledWith(expect.any(Number)) + expect(onProviderRequestStarted.mock.invocationCallOrder[0]).toBeLessThan( + createMock.mock.invocationCallOrder[0] + ) + }) +}) + /** Points the shared layer at one provider via the SANDBOX_PROVIDER env var. */ function useProvider(provider: Provider) { mockEnv.SANDBOX_PROVIDER = provider @@ -266,6 +301,7 @@ beforeEach(() => { read: mockE2BFilesRead, remove: mockE2BFilesRemove, write: mockE2BFilesWrite, + list: mockE2BFilesList, }, kill: mockE2BKill, }) @@ -296,6 +332,7 @@ beforeEach(() => { downloadFile: mockDownloadFile, downloadFileStream: mockDownloadFileStream, getFileDetails: mockGetFileDetails, + listFiles: mockListFiles, }, delete: mockDelete, }) @@ -330,6 +367,47 @@ describe.each(PROVIDERS)('sandbox conformance [%s]', (provider) => { expect(res.result).toEqual({ ok: true }) expect(res.stdout).toBe('hello') expect(res.error).toBeUndefined() + expect(res.cost).toBeUndefined() + }) + + it('adds provider cost to a metered successful code result', async () => { + stubCodeRun(provider, `${SIM_RESULT_PREFIX}{"ok":true}`) + let now = 1_800_000_000_000 + const nowSpy = vi.spyOn(Date, 'now').mockImplementation(() => ++now) + + try { + const res = await executeInSandbox({ + code: 'x', + language: CodeLanguage.Python, + timeoutMs: 1000, + meterUsage: true, + }) + + expect(res.cost).toEqual({ input: 0, output: 0, total: expect.any(Number) }) + expect(res.cost?.total).toBeGreaterThan(0) + } finally { + nowSpy.mockRestore() + } + }) + + it('adds provider cost to a metered successful shell result', async () => { + stubShellCommand(provider, 'ok', '', 0) + let now = 1_800_000_000_000 + const nowSpy = vi.spyOn(Date, 'now').mockImplementation(() => ++now) + + try { + const res = await executeShellInSandbox({ + code: 'echo ok', + envs: {}, + timeoutMs: 1000, + meterUsage: true, + }) + + expect(res.cost).toEqual({ input: 0, output: 0, total: expect.any(Number) }) + expect(res.cost?.total).toBeGreaterThan(0) + } finally { + nowSpy.mockRestore() + } }) it('takes the LAST marker so user output cannot shadow the real result', async () => { @@ -354,10 +432,12 @@ describe.each(PROVIDERS)('sandbox conformance [%s]', (provider) => { code: 'x', language: CodeLanguage.Python, timeoutMs: 1000, + meterUsage: true, }) expect(res.result).toBeNull() expect(res.error).toContain('corrupted in transport') + expect(res.cost).toBeUndefined() }) it('survives a large single-line payload without chunk corruption', async () => { @@ -386,13 +466,19 @@ describe.each(PROVIDERS)('sandbox conformance [%s]', (provider) => { }) } - await expect( - executeInSandbox({ code: 'x', language: CodeLanguage.Python, timeoutMs: 1000 }) - ).rejects.toMatchObject({ + const error = await executeInSandbox({ + code: 'x', + language: CodeLanguage.Python, + timeoutMs: 1000, + meterUsage: true, + }).catch((error: unknown) => error) + + expect(error).toMatchObject({ code: 'sandbox_output_limit_exceeded', outputKind: 'process', limitBytes: MAX_SANDBOX_PROCESS_OUTPUT_BYTES, }) + expect(readTrustedSandboxOutputCost(error)).toBeUndefined() expect(provider === 'e2b' ? mockE2BKill : mockDelete).toHaveBeenCalledTimes(1) }) @@ -495,11 +581,13 @@ describe.each(PROVIDERS)('sandbox conformance [%s]', (provider) => { code: 'raise ValueError("boom")', language: CodeLanguage.Python, timeoutMs: 1000, + meterUsage: true, }) expect(res.error).toBe('ValueError: boom') expect(res.stdout).toContain('ValueError: boom') expect(res.result).toBeNull() + expect(res.cost).toEqual({ input: 0, output: 0, total: expect.any(Number) }) }) it('normalizes Python code budget expiry to a typed timeout abort', async () => { @@ -600,6 +688,252 @@ describe.each(PROVIDERS)('sandbox conformance [%s]', (provider) => { ).rejects.toThrow(/Failed to fetch mounted file/) }) + /** Stubs one directory listing in whichever shape the provider returns. */ + function stubOutputDirListing( + entries: Array<{ path: string; size: number; kind?: 'file' | 'dir' }> + ) { + if (provider === 'e2b') { + mockE2BFilesList.mockResolvedValueOnce( + entries.map((entry) => ({ + name: entry.path.split('/').pop(), + path: entry.path, + size: entry.size, + type: entry.kind === 'dir' ? 'dir' : 'file', + })) + ) + } else { + mockListFiles.mockResolvedValueOnce( + entries.map((entry) => ({ + name: entry.path.split('/').pop(), + path: entry.path, + size: entry.size, + isDir: entry.kind === 'dir', + mode: entry.kind === 'dir' ? 'drwxr-xr-x' : '-rw-r--r--', + })) + ) + } + } + + it('bills a completed run whose harvest produced more files than it can export', async () => { + // The sandbox executed and was paid for; the refusal is about what the code + // wrote, so it belongs with the post-completion export failures rather than + // the provider failures the policy absorbs. + stubCodeRun(provider, `${SIM_RESULT_PREFIX}null`) + stubOutputDirListing( + Array.from({ length: MAX_SANDBOX_OUTPUT_FILES + 1 }, (_, index) => ({ + path: `/tmp/sim/outputs/file-${index}.txt`, + size: 1, + })) + ) + + const error = await executeInSandbox({ + code: 'x', + language: CodeLanguage.Python, + timeoutMs: 1000, + outputSandboxDir: '/tmp/sim/outputs', + meterUsage: true, + }).catch((error: unknown) => error) + + expect(error).toMatchObject({ code: 'sandbox_output_not_exportable' }) + expect(readTrustedSandboxOutputCost(error)).toEqual({ + input: 0, + output: 0, + total: expect.any(Number), + }) + }) + + it('creates the output directory before user code runs', async () => { + stubCodeRun(provider, `__SIM_RESULT__=${JSON.stringify('done')}`) + stubOutputDirListing([]) + + await executeInSandbox({ + code: 'x', + language: CodeLanguage.Python, + timeoutMs: 1000, + outputSandboxDir: '/tmp/sim/outputs', + }) + + // Regression guard. `outputSandboxDir` is this layer's contract, so this + // layer has to create the directory: when creation lived in the caller's + // runtime prologue instead, calling executeInSandbox directly left user + // code writing into a directory that did not exist, and every write was + // ENOENT. The sentinel must be written before the code file that runs. + const writeMock = provider === 'e2b' ? mockE2BFilesWrite : mockUploadFile + const writtenPaths = writeMock.mock.calls.map((call) => + provider === 'e2b' ? call[0] : call[1] + ) + const sentinelIndex = writtenPaths.findIndex((path: string) => + path?.includes('/tmp/sim/outputs/.sim-keep') + ) + const codeIndex = writtenPaths.findIndex((path: string) => path?.includes('.sim-function-')) + expect(sentinelIndex).toBeGreaterThanOrEqual(0) + expect(codeIndex).toBeGreaterThanOrEqual(0) + expect(sentinelIndex).toBeLessThan(codeIndex) + }) + + it('keeps the directory sentinel out of the harvest', async () => { + stubCodeRun(provider, `__SIM_RESULT__=${JSON.stringify('done')}`) + stubOutputDirListing([ + { path: `/tmp/sim/outputs/${SANDBOX_OUTPUT_DIR_SENTINEL}`, size: 0 }, + { path: '/tmp/sim/outputs/real.txt', size: 4 }, + ]) + stubOutputFileSizes(provider, 4) + stubOutputFileRead(provider, 'real') + + const result = await executeInSandbox({ + code: 'x', + language: CodeLanguage.Python, + timeoutMs: 1000, + outputSandboxDir: '/tmp/sim/outputs', + }) + + expect(result.collectedFiles?.map((file) => file.relativePath)).toEqual(['real.txt']) + }) + + it('harvests files written to the output directory', async () => { + stubCodeRun(provider, `__SIM_RESULT__=${JSON.stringify('done')}`) + stubOutputDirListing([{ path: '/tmp/sim/outputs/report.csv', size: 5 }]) + stubOutputFileSizes(provider, 5) + stubOutputFileRead(provider, 'a,b\n1') + + const result = await executeInSandbox({ + code: 'x', + language: CodeLanguage.Python, + timeoutMs: 1000, + outputSandboxDir: '/tmp/sim/outputs', + }) + + expect(result.collectedFiles).toEqual([ + { + path: '/tmp/sim/outputs/report.csv', + relativePath: 'report.csv', + // Always base64, so an arbitrary harvested filename can never be + // decoded as utf8 and silently corrupted. + contentBase64: Buffer.from('a,b\n1').toString('base64'), + byteLength: 5, + }, + ]) + }) + + it('excludes directories from the harvest', async () => { + stubCodeRun(provider, `__SIM_RESULT__=${JSON.stringify('done')}`) + stubOutputDirListing([{ path: '/tmp/sim/outputs/nested', size: 0, kind: 'dir' }]) + + const result = await executeInSandbox({ + code: 'x', + language: CodeLanguage.Python, + timeoutMs: 1000, + outputSandboxDir: '/tmp/sim/outputs', + }) + + expect(result.collectedFiles).toBeUndefined() + }) + + it('refuses a harvest whose nesting outran the listing depth', async () => { + stubCodeRun(provider, `__SIM_RESULT__=${JSON.stringify('done')}`) + // A directory reported at the traversal limit still holds unlisted files. + // Returning the shallow ones would drop the rest without a word. + const deep = Array.from({ length: 12 }, (_, index) => `l${index + 1}`).join('/') + stubOutputDirListing([ + { path: '/tmp/sim/outputs/shallow.txt', size: 4 }, + { path: `/tmp/sim/outputs/${deep}`, size: 0, kind: 'dir' }, + ]) + + await expect( + executeInSandbox({ + code: 'x', + language: CodeLanguage.Python, + timeoutMs: 1000, + outputSandboxDir: '/tmp/sim/outputs', + }) + ).rejects.toThrow(/nested deeper than 12 levels/) + }) + + it('refuses a harvest over the output file count rather than truncating it', async () => { + stubCodeRun(provider, `__SIM_RESULT__=${JSON.stringify('done')}`) + stubOutputDirListing( + Array.from({ length: 21 }, (_, index) => ({ + path: `/tmp/sim/outputs/file-${index}.txt`, + size: 1, + })) + ) + + await expect( + executeInSandbox({ + code: 'x', + language: CodeLanguage.Python, + timeoutMs: 1000, + outputSandboxDir: '/tmp/sim/outputs', + }) + ).rejects.toThrow(/over the 20-file export limit/) + }) + + it('spends one file ceiling across declared and harvested outputs', async () => { + // The limit is what an execution exports, not what one directory holds, so a + // request that both declares and harvests cannot take 20 of each. + stubCodeRun(provider, `__SIM_RESULT__=${JSON.stringify('done')}`) + stubOutputFileSizes(provider, 1, 1) + stubOutputDirListing( + Array.from({ length: MAX_SANDBOX_OUTPUT_FILES - 1 }, (_, index) => ({ + path: `/tmp/sim/outputs/file-${index}.txt`, + size: 1, + })) + ) + + await expect( + executeInSandbox({ + code: 'x', + language: CodeLanguage.Python, + timeoutMs: 1000, + outputSandboxPaths: ['/out/first.txt', '/out/second.txt'], + outputSandboxDir: '/tmp/sim/outputs', + }) + ).rejects.toThrow(/produced 21 files .* over the 20-file export limit/) + }) + + it('does not charge a declared path inside the harvest directory to the ceiling twice', async () => { + // The directory holds exactly the limit and the request names one of those + // files. Charging it on both sides would refuse a run exporting 20 files. + stubCodeRun(provider, `__SIM_RESULT__=${JSON.stringify('done')}`) + // One inspection for the declared path, then one per file actually read. + stubOutputFileSizes(provider, ...Array.from({ length: MAX_SANDBOX_OUTPUT_FILES + 1 }, () => 1)) + stubOutputDirListing( + Array.from({ length: MAX_SANDBOX_OUTPUT_FILES }, (_, index) => ({ + path: `/tmp/sim/outputs/file-${index}.txt`, + size: 1, + })) + ) + for (let index = 0; index < MAX_SANDBOX_OUTPUT_FILES; index += 1) { + stubOutputFileRead(provider, 'x') + } + + const result = await executeInSandbox({ + code: 'x', + language: CodeLanguage.Python, + timeoutMs: 1000, + outputSandboxPath: '/tmp/sim/outputs/file-0.txt', + outputSandboxDir: '/tmp/sim/outputs', + }) + + // Exported once as a declared path, rather than a second time as a harvest. + expect(Object.keys(result.exportedFiles ?? {})).toEqual(['/tmp/sim/outputs/file-0.txt']) + expect(result.collectedFiles).toHaveLength(MAX_SANDBOX_OUTPUT_FILES - 1) + expect(result.collectedFiles?.map((file) => file.relativePath)).not.toContain('file-0.txt') + }) + + it('does not list the output directory when no harvest was requested', async () => { + stubCodeRun(provider, `__SIM_RESULT__=${JSON.stringify('done')}`) + + const result = await executeInSandbox({ + code: 'x', + language: CodeLanguage.Python, + timeoutMs: 1000, + }) + + expect(result.collectedFiles).toBeUndefined() + expect(provider === 'e2b' ? mockE2BFilesList : mockListFiles).not.toHaveBeenCalled() + }) + it('materializes private code inputs after dependencies and user files', async () => { const privateText = 'line one\n"quoted"\\slash\0tail' const privateBytes = Uint8Array.from([0, 10, 34, 92, 255]).buffer @@ -816,11 +1150,17 @@ describe.each(PROVIDERS)('sandbox conformance [%s]', (provider) => { */ stubShellCommand(provider, provider === 'daytona' ? 'boom detail' : '', 'boom detail', 3) - const res = await executeShellInSandbox({ code: 'false', envs: {}, timeoutMs: 1000 }) + const res = await executeShellInSandbox({ + code: 'false', + envs: {}, + timeoutMs: 1000, + meterUsage: true, + }) expect(res.result).toBeNull() expect(res.error).toContain('boom detail') expect(res.stdout).toContain('boom detail') + expect(res.cost).toEqual({ input: 0, output: 0, total: expect.any(Number) }) }) it('terminates shell execution when streamed process output exceeds the byte budget', async () => { @@ -922,18 +1262,24 @@ describe.each(PROVIDERS)('sandbox conformance [%s]', (provider) => { stubCodeRun(provider, `${SIM_RESULT_PREFIX}null`) stubOutputFileSizes(provider, MAX_SANDBOX_OUTPUT_BYTES + 1) - await expect( - executeInSandbox({ - code: 'x', - language: CodeLanguage.Python, - timeoutMs: 1000, - outputSandboxPath: '/out/report.txt', - }) - ).rejects.toMatchObject({ + const error = await executeInSandbox({ + code: 'x', + language: CodeLanguage.Python, + timeoutMs: 1000, + outputSandboxPath: '/out/report.txt', + meterUsage: true, + }).catch((error: unknown) => error) + + expect(error).toMatchObject({ code: 'sandbox_output_limit_exceeded', attemptedBytes: MAX_SANDBOX_OUTPUT_BYTES + 1, limitBytes: MAX_SANDBOX_OUTPUT_BYTES, }) + expect(readTrustedSandboxOutputCost(error)).toEqual({ + input: 0, + output: 0, + total: expect.any(Number), + }) expect(provider === 'e2b' ? mockE2BFilesRead : mockDownloadFileStream).not.toHaveBeenCalled() }) @@ -986,15 +1332,96 @@ describe.each(PROVIDERS)('sandbox conformance [%s]', (provider) => { mockGetFileDetails.mockResolvedValueOnce({ size: 1, isDir: false, mode: 'prw-r--r--' }) } - await expect( - executeInSandbox({ - code: 'x', - language: CodeLanguage.Python, + const error = await executeInSandbox({ + code: 'x', + language: CodeLanguage.Python, + timeoutMs: 1000, + outputSandboxPath: '/out/link.txt', + meterUsage: true, + }).catch((error: unknown) => error) + + expect(error).toMatchObject({ code: 'sandbox_output_file_invalid' }) + expect(readTrustedSandboxOutputCost(error)).toEqual({ + input: 0, + output: 0, + total: expect.any(Number), + }) + expect(provider === 'e2b' ? mockE2BFilesRead : mockDownloadFileStream).not.toHaveBeenCalled() + }) + + it.each(['oversized', 'non-regular'] as const)( + 'retains metered shell cost for a completed execution with %s output', + async (failure) => { + stubShellCommand(provider, '', '', 0) + if (failure === 'oversized') { + stubOutputFileSizes(provider, MAX_SANDBOX_OUTPUT_BYTES + 1) + } else if (provider === 'e2b') { + mockE2BFilesGetInfo.mockResolvedValueOnce({ size: 1, type: 'symlink' }) + } else { + mockGetFileDetails.mockResolvedValueOnce({ size: 1, isDir: false, mode: 'prw-r--r--' }) + } + + const error = await executeShellInSandbox({ + code: 'echo done', + envs: {}, timeoutMs: 1000, - outputSandboxPath: '/out/link.txt', + outputSandboxPath: '/out/result.txt', + meterUsage: true, + }).catch((error: unknown) => error) + + expect(error).toMatchObject({ + code: + failure === 'oversized' ? 'sandbox_output_limit_exceeded' : 'sandbox_output_file_invalid', }) - ).rejects.toMatchObject({ code: 'sandbox_output_file_invalid' }) - expect(provider === 'e2b' ? mockE2BFilesRead : mockDownloadFileStream).not.toHaveBeenCalled() + expect(readTrustedSandboxOutputCost(error)).toEqual({ + input: 0, + output: 0, + total: expect.any(Number), + }) + } + ) + + it('does not attach cost to a generic provider failure during output collection', async () => { + stubCodeRun(provider, `${SIM_RESULT_PREFIX}null`) + stubOutputFileSizes(provider, 1, 1) + const failure = new Error('provider file read failed') + if (provider === 'e2b') { + mockE2BFilesRead.mockRejectedValueOnce(failure) + } else { + mockDownloadFileStream.mockRejectedValueOnce(failure) + } + + const error = await executeInSandbox({ + code: 'x', + language: CodeLanguage.Python, + timeoutMs: 1000, + outputSandboxPath: '/out/result.txt', + meterUsage: true, + }).catch((error: unknown) => error) + + expect(error).toBe(failure) + expect(readTrustedSandboxOutputCost(error)).toBeUndefined() + }) + + it('does not attach cost to a generic provider failure during output inspection', async () => { + stubCodeRun(provider, `${SIM_RESULT_PREFIX}null`) + const failure = new Error('provider file metadata failed') + if (provider === 'e2b') { + mockE2BFilesGetInfo.mockRejectedValueOnce(failure) + } else { + mockGetFileDetails.mockRejectedValueOnce(failure) + } + + const error = await executeInSandbox({ + code: 'x', + language: CodeLanguage.Python, + timeoutMs: 1000, + outputSandboxPath: '/out/result.txt', + meterUsage: true, + }).catch((error: unknown) => error) + + expect(error).toBe(failure) + expect(readTrustedSandboxOutputCost(error)).toBeUndefined() }) it('does not return code results when cancellation arrives during output collection', async () => { @@ -1313,6 +1740,58 @@ describe('provider stream recovery', () => { expect(mockGetSessionCommand).toHaveBeenCalledWith(expect.any(String), 'cmd_1') }) + it('fails an at-most-once Daytona run closed when final status has no exit code', async () => { + mockGetSessionCommand.mockResolvedValueOnce({}) + const sandbox = await daytonaProvider.create('code') + + await expect( + sandbox.runCommand('node function.js', { timeoutMs: 1000, atMostOnce: true }) + ).rejects.toMatchObject({ + name: 'SandboxLaunchIndeterminateError', + retryable: false, + code: 'sandbox_launch_indeterminate', + }) + }) + + it('fails an at-most-once Daytona run closed when its readiness handshake never completes', async () => { + mockGetSessionCommandLogs.mockResolvedValueOnce(undefined) + mockGetSessionCommand.mockResolvedValueOnce({ exitCode: 78 }) + const sandbox = await daytonaProvider.create('code') + + await expect( + sandbox.runCommand('node function.js', { timeoutMs: 1000, atMostOnce: true }) + ).rejects.toMatchObject({ + name: 'SandboxLaunchIndeterminateError', + retryable: false, + code: 'sandbox_launch_indeterminate', + }) + expect(mockSendSessionCommandInput).not.toHaveBeenCalled() + }) + + it('fails an at-most-once Daytona run closed when final status lookup fails', async () => { + mockGetSessionCommand.mockRejectedValueOnce(new Error('control plane unavailable')) + const sandbox = await daytonaProvider.create('code') + + await expect( + sandbox.runCommand('node function.js', { timeoutMs: 1000, atMostOnce: true }) + ).rejects.toMatchObject({ + name: 'SandboxLaunchIndeterminateError', + retryable: false, + code: 'sandbox_launch_indeterminate', + }) + }) + + it('preserves a pre-dispatch Daytona failure for at-most-once runs', async () => { + const failure = new Error('session unavailable') + mockCreateSession.mockRejectedValueOnce(failure) + const sandbox = await daytonaProvider.create('code') + + await expect( + sandbox.runCommand('node function.js', { timeoutMs: 1000, atMostOnce: true }) + ).rejects.toBe(failure) + expect(mockExecuteSessionCommand).not.toHaveBeenCalled() + }) + it('keeps the original Daytona deadline while recovering a disconnected stream', async () => { mockGetSessionCommandLogs .mockRejectedValueOnce(new Error('stream disconnected')) @@ -2253,10 +2732,12 @@ describe('Pi sandbox lifetime', () => { code: 'x', language: CodeLanguage.Python, timeoutMs: 7 * 24 * 60 * 60 * 1000, + meterUsage: true, }) expect(result.error).toContain('E2B reached its 24-hour limit') expect(result.error).toContain('workflow timeout may be longer') + expect(result.cost).toBeUndefined() expect(mockRecordSandboxProviderLimit).toHaveBeenCalledWith({ provider: 'e2b', operation: 'code', @@ -2280,9 +2761,11 @@ describe('Pi sandbox lifetime', () => { const result = await executeShellInSandbox({ code: 'sleep infinity', timeoutMs: 7 * 24 * 60 * 60 * 1000, + meterUsage: true, }) expect(result.error).toContain('E2B reached its 24-hour limit') + expect(result.cost).toBeUndefined() expect(mockRecordSandboxProviderLimit).toHaveBeenCalledWith({ provider: 'e2b', operation: 'command', diff --git a/apps/sim/lib/execution/remote-sandbox/daytona.ts b/apps/sim/lib/execution/remote-sandbox/daytona.ts index 0df2ae61443..11df31c48f5 100644 --- a/apps/sim/lib/execution/remote-sandbox/daytona.ts +++ b/apps/sim/lib/execution/remote-sandbox/daytona.ts @@ -27,11 +27,13 @@ import { SandboxProcessOutputBudget, tailStreamedSandboxOutput, } from '@/lib/execution/remote-sandbox/output-limits' +import { resolveSandboxDirectoryEntryPath } from '@/lib/execution/remote-sandbox/sandbox-paths' import type { CreateSandboxOptions, RunCommandOptions, SandboxCodeResult, SandboxCommandResult, + SandboxDirectoryEntry, SandboxHandle, SandboxKind, SandboxProvider, @@ -41,6 +43,11 @@ const logger = createLogger('DaytonaSandboxProvider') const DAYTONA_DEFAULT_SANDBOX_TTL_MS = 24 * 60 * 60 * 1000 const DAYTONA_STREAM_READY_MARKER = '__SIM_DAYTONA_STREAM_READY__' +/** Daytona expresses sandbox TTLs as whole minutes. */ +export function resolveDaytonaSandboxLifetimeMs(lifetimeMs: number): number { + return Math.max(1, Math.ceil(lifetimeMs / 60_000)) * 60_000 +} + /** Daytona expresses every timeout in seconds; the rest of Sim works in milliseconds. */ function toSeconds(timeoutMs: number): number { return Math.max(1, Math.ceil(timeoutMs / 1000)) @@ -294,6 +301,7 @@ class DaytonaSandboxHandle implements SandboxHandle { // must never have. const finalStdout = () => (retainStdout ? stdout : tailStreamedSandboxOutput(stdout)) const finalStderr = () => (retainStderr ? stderr : tailStreamedSandboxOutput(stderr)) + let commandDispatched = false try { await this.sandbox.process.createSession(sessionId) sessionCreated = true @@ -332,6 +340,7 @@ class DaytonaSandboxHandle implements SandboxHandle { if (typeof commandId !== 'string' || commandId.length === 0) { throw new SandboxLaunchIndeterminateError('Daytona') } + commandDispatched = true // Accumulate the streamed chunks as well as forwarding them: callers read // markers out of stdout (the Pi cloud flow parses __BASE_SHA__/__CHANGED__) // and format failures from stderr, so returning empty strings here would @@ -653,7 +662,14 @@ class DaytonaSandboxHandle implements SandboxHandle { } const finished = await this.sandbox.process.getSessionCommand(sessionId, commandId) - const exitCode = finished.exitCode ?? 0 + if (options.atMostOnce && !releaseRequested) { + throw new SandboxLaunchIndeterminateError('Daytona') + } + const exitCode = finished.exitCode + if (typeof exitCode !== 'number' || !Number.isFinite(exitCode)) { + if (options.atMostOnce) throw new SandboxLaunchIndeterminateError('Daytona') + return { stdout: finalStdout(), stderr: finalStderr(), exitCode: 0 } + } return { stdout: finalStdout(), stderr: finalStderr(), exitCode } } catch (error) { if (isSandboxOutputLimitError(error)) { @@ -674,6 +690,12 @@ class DaytonaSandboxHandle implements SandboxHandle { timedOut: true, } } + if (options.atMostOnce) { + if (commandDispatched) { + throw new SandboxLaunchIndeterminateError('Daytona', { cause: error }) + } + throw error + } if (operation === 'code') throw error return { stdout: finalStdout(), stderr: finalStderr() || getErrorMessage(error), exitCode: 1 } } finally { @@ -742,6 +764,24 @@ class DaytonaSandboxHandle implements SandboxHandle { await this.sandbox.fs.uploadFile(buffer, path) } + async listFiles(path: string, options?: { depth?: number }): Promise { + const entries = await this.sandbox.fs.listFiles(path, { + ...(options?.depth !== undefined ? { depth: options.depth } : {}), + }) + + const files: SandboxDirectoryEntry[] = [] + for (const entry of entries) { + const resolved = resolveSandboxDirectoryEntryPath(path, entry.path ?? entry.name) + if (!resolved) continue + files.push({ + ...resolved, + kind: entry.isDir ? 'directory' : 'file', + size: entry.size, + }) + } + return files + } + async kill(): Promise { if (this.killed) return if (!this.killPromise) { @@ -775,6 +815,7 @@ function shellQuote(value: string): string { export const daytonaProvider: SandboxProvider = { id: 'daytona', dependencyStrategy: 'runtime', + resolveLifetimeMs: resolveDaytonaSandboxLifetimeMs, async create(kind: SandboxKind, options?: CreateSandboxOptions): Promise { const apiKey = env.DAYTONA_API_KEY if (!apiKey) { @@ -790,11 +831,11 @@ export const daytonaProvider: SandboxProvider = { snapshot, language: toDaytonaLanguage(language), ephemeral: true, - ttlMinutes: Math.max( - 1, - Math.ceil((options?.lifetimeMs ?? DAYTONA_DEFAULT_SANDBOX_TTL_MS) / 60_000) - ), + ttlMinutes: + resolveDaytonaSandboxLifetimeMs(options?.lifetimeMs ?? DAYTONA_DEFAULT_SANDBOX_TTL_MS) / + 60_000, } + options?.onProviderRequestStarted?.(Date.now()) const sandbox = await daytona.create(createOptions) return new DaytonaSandboxHandle(sandbox, language) diff --git a/apps/sim/lib/execution/remote-sandbox/e2b.ts b/apps/sim/lib/execution/remote-sandbox/e2b.ts index db387fd93e7..c8922b2f3d6 100644 --- a/apps/sim/lib/execution/remote-sandbox/e2b.ts +++ b/apps/sim/lib/execution/remote-sandbox/e2b.ts @@ -43,6 +43,7 @@ import { SandboxProcessOutputBudget, tailStreamedSandboxOutput, } from '@/lib/execution/remote-sandbox/output-limits' +import { resolveSandboxDirectoryEntryPath } from '@/lib/execution/remote-sandbox/sandbox-paths' import { quoteDependency, type SandboxSpec, @@ -55,6 +56,7 @@ import type { RunCommandOptions, SandboxCodeResult, SandboxCommandResult, + SandboxDirectoryEntry, SandboxHandle, SandboxImageBuild, SandboxImageBuilder, @@ -95,6 +97,11 @@ export const E2B_SANDBOX_MATERIALIZER_REVISION = FUNCTION_SANDBOX_MATERIALIZER_R /** Maximum continuous sandbox lifetime supported by E2B. */ export const E2B_MAX_SANDBOX_LIFETIME_MS = 24 * 60 * 60 * 1000 +/** E2B sends sandbox lifetimes as whole seconds. */ +export function resolveE2BSandboxLifetimeMs(lifetimeMs: number): number { + return Math.min(Math.ceil(lifetimeMs / 1000) * 1000, E2B_MAX_SANDBOX_LIFETIME_MS) +} + const E2B_PROVIDER_LIMIT_ERROR = 'E2B reached its 24-hour limit for a single sandbox execution. The workflow timeout may be longer, but this Function call must finish within 24 hours.' const E2B_TIMEOUT_MESSAGE_PATTERN = @@ -365,7 +372,7 @@ class E2BSandboxHandle implements SandboxHandle { return { text: '', stdout: result.stdout, stderr: result.stderr, timedOut: true } } if (result.exitCode !== 0) { - if (result.stderr === E2B_PROVIDER_LIMIT_ERROR) { + if (result.providerFailure === 'provider_limit') { return { text: '', stdout: result.stdout, @@ -375,6 +382,7 @@ class E2BSandboxHandle implements SandboxHandle { value: E2B_PROVIDER_LIMIT_ERROR, traceback: E2B_PROVIDER_LIMIT_ERROR, }, + providerFailure: result.providerFailure, } } return processCodeFailure(result) @@ -534,7 +542,12 @@ class E2BSandboxHandle implements SandboxHandle { if (isNonRetryableExecutionError(error)) throw error if (reachedE2BProviderLimit(error, this.providerLimitAtMs, options.signal)) { recordSandboxProviderLimit({ provider: 'e2b', operation }) - return { stdout: '', stderr: E2B_PROVIDER_LIMIT_ERROR, exitCode: 1 } + return { + stdout: '', + stderr: E2B_PROVIDER_LIMIT_ERROR, + exitCode: 1, + providerFailure: 'provider_limit', + } } // The SDK throws on non-zero exit; callers want the streams, not a throw. const failure = error as { @@ -628,6 +641,25 @@ class E2BSandboxHandle implements SandboxHandle { await this.sandbox.files.write(path, content as string) } + async listFiles(path: string, options?: { depth?: number }): Promise { + const entries = await this.sandbox.files.list(path, { + ...(options?.depth !== undefined ? { depth: options.depth } : {}), + }) + + const files: SandboxDirectoryEntry[] = [] + for (const entry of entries) { + if (entry.type !== 'file' && entry.type !== 'dir') continue + const resolved = resolveSandboxDirectoryEntryPath(path, entry.path) + if (!resolved) continue + files.push({ + ...resolved, + kind: entry.type === 'dir' ? 'directory' : 'file', + size: entry.size, + }) + } + return files + } + async kill(): Promise { if (this.killed) return if (!this.killPromise) { @@ -842,6 +874,7 @@ export const e2bProvider: SandboxProvider = { id: 'e2b', dependencyStrategy: 'prebuilt', images: e2bImages, + resolveLifetimeMs: resolveE2BSandboxLifetimeMs, async create(kind: SandboxKind, options?: CreateSandboxOptions): Promise { const apiKey = env.E2B_API_KEY if (!apiKey) { @@ -860,7 +893,9 @@ export const e2bProvider: SandboxProvider = { // default — longer than the lifetime it asked for, which is the opposite of // what it requested. const effectiveLifetimeMs = - options?.lifetimeMs !== undefined ? e2bTimeoutMs(options.lifetimeMs) : undefined + options?.lifetimeMs !== undefined + ? resolveE2BSandboxLifetimeMs(options.lifetimeMs) + : undefined const createOptions = { apiKey, ...(effectiveLifetimeMs !== undefined ? { timeoutMs: effectiveLifetimeMs } : {}), @@ -868,6 +903,7 @@ export const e2bProvider: SandboxProvider = { const { Sandbox } = await import('@e2b/code-interpreter') const lifetimeStartedAtMs = Date.now() + options?.onProviderRequestStarted?.(lifetimeStartedAtMs) const sandbox = await Sandbox.create(templateName, createOptions) return new E2BSandboxHandle( diff --git a/apps/sim/lib/execution/remote-sandbox/function-resources.ts b/apps/sim/lib/execution/remote-sandbox/function-resources.ts index afbe013c32f..9061d043da9 100644 --- a/apps/sim/lib/execution/remote-sandbox/function-resources.ts +++ b/apps/sim/lib/execution/remote-sandbox/function-resources.ts @@ -2,6 +2,7 @@ export const FUNCTION_SANDBOX_CPU_COUNT = 2 export const FUNCTION_SANDBOX_MEMORY_GB = 4 export const FUNCTION_SANDBOX_MEMORY_MB = FUNCTION_SANDBOX_MEMORY_GB * 1024 +export const FUNCTION_DAYTONA_DISK_GB = 10 /** Bump when custom dependency-layer rendering changes without a semantic spec change. */ export const FUNCTION_SANDBOX_MATERIALIZER_REVISION = 2 diff --git a/apps/sim/lib/execution/remote-sandbox/image-registry.test.ts b/apps/sim/lib/execution/remote-sandbox/image-registry.test.ts index 250daf087ef..dd799a39c54 100644 --- a/apps/sim/lib/execution/remote-sandbox/image-registry.test.ts +++ b/apps/sim/lib/execution/remote-sandbox/image-registry.test.ts @@ -119,13 +119,10 @@ import { cleanupSandboxImages, ensureSandboxImage, FAILED_BUILD_RETRY_COOLDOWN_MS, - LEGACY_SANDBOX_IMAGE_BUILD_TASK_ID, - PREVIOUS_SANDBOX_IMAGE_BUILD_TASK_ID, releaseSandboxImage, runSandboxImageBuild, SANDBOX_IMAGE_BUILD_TASK_ID, sandboxBuildIdempotencyKey, - sandboxImageBuildTaskIds, } from '@/lib/execution/remote-sandbox/image-registry' const READY_IMAGE = { @@ -519,15 +516,8 @@ describe('runSandboxImageBuild attempt ownership', () => { systemPackages: [], } - it('routes each renderer revision to a distinct Trigger.dev task ID', () => { - expect(SANDBOX_IMAGE_BUILD_TASK_ID).toBe('sandbox-image-build-v2') - expect(PREVIOUS_SANDBOX_IMAGE_BUILD_TASK_ID).toBe('sandbox-image-build-v1') - expect(LEGACY_SANDBOX_IMAGE_BUILD_TASK_ID).toBe('sandbox-image-build') - expect(sandboxImageBuildTaskIds(2)).toEqual({ - current: 'sandbox-image-build-v2', - previous: 'sandbox-image-build-v1', - legacy: 'sandbox-image-build', - }) + it('uses one stable Trigger.dev task ID', () => { + expect(SANDBOX_IMAGE_BUILD_TASK_ID).toBe('sandbox-image-build') }) it('refuses an app-new/task-old renderer mismatch before claiming the row', async () => { diff --git a/apps/sim/lib/execution/remote-sandbox/image-registry.ts b/apps/sim/lib/execution/remote-sandbox/image-registry.ts index 9fabd8f0150..3f80df3786f 100644 --- a/apps/sim/lib/execution/remote-sandbox/image-registry.ts +++ b/apps/sim/lib/execution/remote-sandbox/image-registry.ts @@ -17,7 +17,6 @@ import { providerBuildError, type SandboxBuildError, } from '@/lib/execution/remote-sandbox/build-errors' -import { FUNCTION_SANDBOX_MATERIALIZER_REVISION } from '@/lib/execution/remote-sandbox/function-resources' import { resolveProvider } from '@/lib/execution/remote-sandbox/provider' import { invalidateSandboxResolution } from '@/lib/execution/remote-sandbox/resolve' import type { SandboxSpec } from '@/lib/execution/remote-sandbox/sandbox-spec' @@ -44,23 +43,7 @@ const STALE_BUILD_MS = BUILD_POLL_CAP_MS * 2 const POLL_BASE_MS = 3_000 const POLL_MAX_MS = 20_000 -/** Task IDs kept live together so either web-first or worker-first rollouts drain safely. */ -export function sandboxImageBuildTaskIds(rendererRevision: number): { - current: string - previous?: string - legacy: string -} { - return { - current: `sandbox-image-build-v${rendererRevision}`, - ...(rendererRevision > 1 ? { previous: `sandbox-image-build-v${rendererRevision - 1}` } : {}), - legacy: 'sandbox-image-build', - } -} - -const SANDBOX_IMAGE_TASK_IDS = sandboxImageBuildTaskIds(FUNCTION_SANDBOX_MATERIALIZER_REVISION) -export const SANDBOX_IMAGE_BUILD_TASK_ID = SANDBOX_IMAGE_TASK_IDS.current -export const PREVIOUS_SANDBOX_IMAGE_BUILD_TASK_ID = SANDBOX_IMAGE_TASK_IDS.previous -export const LEGACY_SANDBOX_IMAGE_BUILD_TASK_ID = SANDBOX_IMAGE_TASK_IDS.legacy +export const SANDBOX_IMAGE_BUILD_TASK_ID = 'sandbox-image-build' export interface SandboxImageBuildPayload { provider: SandboxProviderId diff --git a/apps/sim/lib/execution/remote-sandbox/index.ts b/apps/sim/lib/execution/remote-sandbox/index.ts index 7055a88ec2e..3b4e92015b8 100644 --- a/apps/sim/lib/execution/remote-sandbox/index.ts +++ b/apps/sim/lib/execution/remote-sandbox/index.ts @@ -1,6 +1,11 @@ import { createLogger } from '@sim/logger' import { getErrorMessage } from '@sim/utils/errors' import { generateShortId } from '@sim/utils/id' +import { + createSandboxPricing, + priceSandboxUsage, + type SandboxPricing, +} from '@/lib/billing/sandbox-pricing' import { createTimeoutAbortController, getRemainingExecutionMs, @@ -10,10 +15,17 @@ import { recordSandboxTeardownFailure } from '@/lib/core/execution-limits/metric import { buildJavaScriptRuntimeBindingsSource } from '@/lib/execution/code-placeholders/javascript-runtime' import { SANDBOX_SYSTEM_PATH } from '@/lib/execution/remote-sandbox/cli-tools.server' import { + attachTrustedSandboxOutputCost, isSandboxOutputFileError, isSandboxOutputLimitError, + isSandboxOutputNotExportableError, MAX_SANDBOX_OUTPUT_BYTES, + MAX_SANDBOX_OUTPUT_FILES, MAX_SANDBOX_PROCESS_OUTPUT_BYTES, + MAX_SANDBOX_URL_MOUNT_BYTES, + SandboxOutputDepthError, + SandboxOutputDirectoryMissingError, + SandboxOutputFileCountError, SandboxOutputLimitError, } from '@/lib/execution/remote-sandbox/output-limits' import { resolvePiSandboxLifetimeMs } from '@/lib/execution/remote-sandbox/pi-lifetime' @@ -25,20 +37,31 @@ import { repairMissingSandboxImage, resolveWorkspaceSandbox, } from '@/lib/execution/remote-sandbox/resolve' +import { + SANDBOX_OUTPUT_DIR_MAX_DEPTH, + SANDBOX_OUTPUT_DIR_SENTINEL, +} from '@/lib/execution/remote-sandbox/sandbox-paths' import type { CreateSandboxOptions, SandboxCodeResult, + SandboxCollectedFile, SandboxCommandResult, + SandboxCostSink, + SandboxDirectoryEntry, + SandboxExecutionCost, SandboxExecutionRequest, SandboxExecutionResult, SandboxFile, SandboxHandle, SandboxKind, SandboxPrivateInput, + SandboxProvider, + SandboxProviderId, SandboxShellExecutionRequest, } from '@/lib/execution/remote-sandbox/types' export type { + SandboxCostSink, SandboxExecutionRequest, SandboxExecutionResult, SandboxFile, @@ -48,14 +71,41 @@ export type { const logger = createLogger('RemoteSandbox') +interface CreatedSandbox { + sandbox: SandboxHandle + providerId: SandboxProviderId + startedAtMs: number + effectiveLifetimeMs?: number + pricing?: SandboxPricing +} + async function createSandbox( kind: SandboxKind, - options?: CreateSandboxOptions -): Promise { - const provider = resolveProvider() - const sandbox = await provider.create(kind, options) + options?: CreateSandboxOptions, + meterUsage = false, + provider: SandboxProvider = resolveProvider() +): Promise { + const effectiveLifetimeMs = + options?.lifetimeMs !== undefined ? provider.resolveLifetimeMs(options.lifetimeMs) : undefined + if (meterUsage && effectiveLifetimeMs === undefined) { + throw new Error('Metered sandbox execution requires a provider lifetime') + } + const pricing = meterUsage ? createSandboxPricing(provider.id) : undefined + let startedAtMs = Date.now() + const providerOptions = { + ...options, + ...(effectiveLifetimeMs !== undefined ? { lifetimeMs: effectiveLifetimeMs } : {}), + ...(meterUsage ? { onProviderRequestStarted: (value: number) => (startedAtMs = value) } : {}), + } + const sandbox = await provider.create(kind, providerOptions) logger.info('Created sandbox', { provider: provider.id, kind, sandboxId: sandbox.sandboxId }) - return sandbox + return { + sandbox, + providerId: provider.id, + startedAtMs, + ...(effectiveLifetimeMs !== undefined ? { effectiveLifetimeMs } : {}), + ...(pricing ? { pricing } : {}), + } } /** @@ -72,10 +122,11 @@ async function createSelectedSandbox( kind: SandboxKind, options: CreateSandboxOptions, selected: ResolvedSandbox | null, - signal: AbortSignal -): Promise { + signal: AbortSignal, + meterUsage = false +): Promise { try { - return await createSandbox(kind, options) + return await createSandbox(kind, options, meterUsage) } catch (error) { throwIfAborted(signal) if (!selected) throw error @@ -155,10 +206,13 @@ function throwIfSandboxTimedOut(result: { timedOut?: boolean }): void { if (result.timedOut) throw new DOMException('timeout', 'AbortError') } -function bindSandboxAbort(sandbox: SandboxHandle, signal?: AbortSignal) { +function bindSandboxAbort( + sandbox: SandboxHandle, + provider: SandboxProviderId, + signal?: AbortSignal +) { let killed = false let killPromise: Promise | null = null - const provider = resolveProvider().id const kill = (reason: 'cleanup' | 'cancellation' | 'timeout'): Promise => { if (killed) return Promise.resolve() if (!killPromise) { @@ -200,6 +254,52 @@ function bindSandboxAbort(sandbox: SandboxHandle, signal?: AbortSignal) { } } +function calculateSandboxCost( + created: CreatedSandbox, + cleanupStartedAtMs: number +): SandboxExecutionCost | undefined { + if (!created.pricing || created.effectiveLifetimeMs === undefined) return undefined + const usage = priceSandboxUsage( + created.pricing, + cleanupStartedAtMs - created.startedAtMs, + created.effectiveLifetimeMs + ) + return { input: 0, output: 0, total: usage.billedCost } +} + +/** + * Fetches one URL mount inside the sandbox, bounded by MAX_BYTES. + * + * Three mechanisms, because no one of them is sufficient on its own. + * `--max-filesize` refuses an oversized object before a byte moves, but only when + * the response declares a Content-Length — a chunked or length-less reply walks + * straight past it. `head -c` therefore caps what can ever reach the disk at one + * byte over the limit, so a mis-declared object cannot fill the sandbox while we + * wait to notice. The final size check is what turns that truncated file into a + * refusal rather than a silently corrupted mount. + * + * curl's exit status travels through a file because its status is lost in a + * pipeline, and losing it would let a 403 on an expired URL look like a + * successful empty download. The size check is consulted first: when `head` + * closes the pipe early curl dies of EPIPE, and "over the limit" is the useful + * message there, not the write error it provokes. + * + * MAX_BYTES, URL, DST, and DIR all arrive as environment variables, never + * interpolated, so a presigned query string cannot break out of the command. + */ +const FETCH_URL_MOUNT_COMMAND = [ + 'set -e', + '[ -n "$DIR" ] && mkdir -p "$DIR"', + 'STATUS_FILE=$(mktemp)', + 'STATUS=0', + '{ curl -fsS --retry 3 --retry-connrefused --max-time 300 --max-filesize "$MAX_BYTES" "$URL" || STATUS=$?; echo "$STATUS" > "$STATUS_FILE"; } | head -c "$(( MAX_BYTES + 1 ))" > "$DST"', + 'STATUS=$(cat "$STATUS_FILE")', + 'rm -f "$STATUS_FILE"', + 'SIZE=$(wc -c < "$DST")', + 'if [ "$SIZE" -gt "$MAX_BYTES" ]; then rm -f "$DST"; echo "mounted file exceeds the $MAX_BYTES byte limit" >&2; exit 1; fi', + 'if [ "$STATUS" -ne 0 ]; then rm -f "$DST"; echo "curl exited $STATUS" >&2; exit 1; fi', +].join('\n') + /** * Materializes sandbox input files before user code runs. `content` entries are written inline; * `url` entries are fetched from inside the sandbox via `curl` — their bytes never pass through the @@ -220,16 +320,24 @@ async function writeSandboxInputs( const dir = file.path.slice(0, file.path.lastIndexOf('/')) let result: SandboxCommandResult try { - result = await sandbox.runCommand( - 'set -e; [ -n "$DIR" ] && mkdir -p "$DIR"; curl -fsS --retry 3 --retry-connrefused --max-time 300 "$URL" -o "$DST"', - { - envs: { URL: file.url, DST: file.path, DIR: dir }, - timeoutMs: Math.min(300_000, remainingSandboxBudgetMs(opts.signal)), - maxOutputBytes: MAX_SANDBOX_PROCESS_OUTPUT_BYTES, - signal: opts.signal, - rootUser: opts.rootUser, - } - ) + result = await sandbox.runCommand(FETCH_URL_MOUNT_COMMAND, { + envs: { + URL: file.url, + DST: file.path, + DIR: dir, + // Clamped, not just defaulted: `sandboxFiles` reaches this layer from + // the request body, so a declared ceiling is a caller's number. It may + // lower the limit for its own mount but never raise it past the one + // this layer guarantees. + MAX_BYTES: String( + Math.min(file.maxBytes ?? MAX_SANDBOX_URL_MOUNT_BYTES, MAX_SANDBOX_URL_MOUNT_BYTES) + ), + }, + timeoutMs: Math.min(300_000, remainingSandboxBudgetMs(opts.signal)), + maxOutputBytes: MAX_SANDBOX_PROCESS_OUTPUT_BYTES, + signal: opts.signal, + rootUser: opts.rootUser, + }) } catch (error) { throwIfAborted(opts.signal) throw new Error( @@ -406,14 +514,14 @@ async function readSandboxOutputFile( logger.warn('Failed to read requested sandbox output file', { sandboxId: sandbox.sandboxId, }) - return undefined + throw error } } async function inspectSandboxOutputFileSize( sandbox: SandboxHandle, outputSandboxPath: string -): Promise { +): Promise { try { const size = await sandbox.getFileSize(outputSandboxPath) if (!Number.isSafeInteger(size) || size < 0) { @@ -425,7 +533,7 @@ async function inspectSandboxOutputFileSize( logger.warn('Failed to inspect requested sandbox output file', { sandboxId: sandbox.sandboxId, }) - return undefined + throw error } } @@ -441,17 +549,96 @@ function requestedOutputSandboxPaths(req: { ] } +/** + * Enumerates the harvest directory, refusing anything it cannot return in full — + * too many files, or nesting past what the listing reaches — before a single + * byte is read. Sorted so a multi-file result is stable run to run rather than + * inheriting whatever order the provider happened to return. + * + * `declaredPaths` are the files the request already named. One sitting inside the + * directory is dropped rather than harvested a second time, and the rest count + * toward the ceiling: the limit is what one execution exports, not what one + * directory holds, so declaring and harvesting cannot spend it twice. + */ +async function listOutputDirectoryFiles( + sandbox: SandboxHandle, + outputSandboxDir: string, + declaredPaths: ReadonlySet, + signal: AbortSignal +): Promise { + let listed: SandboxDirectoryEntry[] + try { + listed = await sandbox.listFiles(outputSandboxDir, { depth: SANDBOX_OUTPUT_DIR_MAX_DEPTH }) + } catch (error) { + // The directory is created before user code runs, so the only way it can be + // missing now is that the code removed it. Providers report that as a raw + // `lstat ... no such file or directory`, which reads like a Sim fault; say + // what actually happened instead. Anything else propagates untouched rather + // than being flattened into "produced nothing". + if (/not_?found|no such file|ENOENT/i.test(getErrorMessage(error))) { + throw new SandboxOutputDirectoryMissingError(outputSandboxDir) + } + throw error + } + const entries = listed.filter((entry) => entry.relativePath !== SANDBOX_OUTPUT_DIR_SENTINEL) + remainingSandboxBudgetMs(signal) + + // A directory sitting exactly at the traversal limit still has unlisted + // contents, and the providers report no truncation of their own. Refuse + // rather than return a partial harvest: a file the code wrote and the caller + // never receives is worse than an error naming the reason. + const truncatedAt = entries.find( + (entry) => + entry.kind === 'directory' && + entry.relativePath.split('/').length >= SANDBOX_OUTPUT_DIR_MAX_DEPTH + ) + if (truncatedAt) { + throw new SandboxOutputDepthError( + `${outputSandboxDir}/${truncatedAt.relativePath}`, + SANDBOX_OUTPUT_DIR_MAX_DEPTH + ) + } + + const files = entries.filter((entry) => entry.kind === 'file' && !declaredPaths.has(entry.path)) + const exported = declaredPaths.size + files.length + if (exported > MAX_SANDBOX_OUTPUT_FILES) { + throw new SandboxOutputFileCountError(exported, outputSandboxDir) + } + return files.sort((a, b) => a.path.localeCompare(b.path)) +} + +/** + * Brings the harvest directory into existence before user code runs. + * + * Owned here rather than by the caller's runtime prologue because + * `outputSandboxDir` is this layer's contract: a caller that asks for a harvest + * must not also have to know it is responsible for creating the directory, or + * the first write in their code is ENOENT. + */ +async function ensureSandboxOutputDir( + sandbox: SandboxHandle, + outputSandboxDir: string | undefined, + signal: AbortSignal +): Promise { + if (!outputSandboxDir) return + await sandbox.writeFile(`${outputSandboxDir}/${SANDBOX_OUTPUT_DIR_SENTINEL}`, '') + remainingSandboxBudgetMs(signal) +} + async function collectExportedFiles( sandbox: SandboxHandle, - req: { outputSandboxPath?: string; outputSandboxPaths?: string[] }, + req: { outputSandboxPath?: string; outputSandboxPaths?: string[]; outputSandboxDir?: string }, options: { signal: AbortSignal } -): Promise<{ exportedFiles?: Record; exportedFileContent?: string }> { +): Promise<{ + exportedFiles?: Record + exportedFileContent?: string + collectedFiles?: SandboxCollectedFile[] +}> { const readablePaths: string[] = [] let totalOutputBytes = 0 for (const outputSandboxPath of requestedOutputSandboxPaths(req)) { const size = await inspectSandboxOutputFileSize(sandbox, outputSandboxPath) remainingSandboxBudgetMs(options.signal) - if (size === undefined) continue totalOutputBytes += size if (totalOutputBytes > MAX_SANDBOX_OUTPUT_BYTES) { throw new SandboxOutputLimitError(totalOutputBytes) @@ -459,6 +646,22 @@ async function collectExportedFiles( readablePaths.push(outputSandboxPath) } + // Sized into the same running total as the declared paths, so an execution + // cannot spend the byte ceiling twice by both declaring and harvesting. The + // listing applies the same rule to the file-count ceiling and drops a declared + // path that happens to sit inside the harvest directory — double-billing it + // would reject a single output larger than half the ceiling as oversized. + const declaredPaths = new Set(readablePaths) + const discovered = req.outputSandboxDir + ? await listOutputDirectoryFiles(sandbox, req.outputSandboxDir, declaredPaths, options.signal) + : [] + for (const entry of discovered) { + totalOutputBytes += entry.size + if (totalOutputBytes > MAX_SANDBOX_OUTPUT_BYTES) { + throw new SandboxOutputLimitError(totalOutputBytes) + } + } + const exportedFiles: Record = {} let readOutputBytes = 0 for (const outputSandboxPath of readablePaths) { @@ -484,9 +687,45 @@ async function collectExportedFiles( throw error } } + + const collectedFiles: SandboxCollectedFile[] = [] + for (const entry of discovered) { + try { + // Always base64: a harvested filename is arbitrary, and the extension + // allowlist that picks an encoding for a declared path would decode a + // `.parquet` or an extensionless binary as utf8 — substituting U+FFFD and + // delivering corruption that still looks like a valid file. + const file = await sandbox.readFileWithLimit(entry.path, { + maxBytes: MAX_SANDBOX_OUTPUT_BYTES - readOutputBytes, + encoding: 'base64', + signal: options.signal, + }) + remainingSandboxBudgetMs(options.signal) + readOutputBytes += file.byteLength + collectedFiles.push({ + path: entry.path, + relativePath: entry.relativePath, + contentBase64: file.content, + byteLength: file.byteLength, + }) + } catch (error) { + if (isSandboxOutputLimitError(error)) { + throw new SandboxOutputLimitError( + readOutputBytes + error.attemptedBytes, + MAX_SANDBOX_OUTPUT_BYTES + ) + } + // Unlike a declared path, a harvested file was just observed to exist, so + // a failed read is an anomaly rather than a caller mistake. Dropping it + // would silently lose output the code successfully produced. + throw error + } + } + return { exportedFileContent: req.outputSandboxPath ? exportedFiles[req.outputSandboxPath] : undefined, exportedFiles: Object.keys(exportedFiles).length ? exportedFiles : undefined, + collectedFiles: collectedFiles.length ? collectedFiles : undefined, } } @@ -506,6 +745,33 @@ function installBudgetMs(timeoutMs: number): number { return Math.max(0, Math.min(RUNTIME_INSTALL_TIMEOUT_MS, timeoutMs - MIN_CODE_BUDGET_MS)) } +/** + * Held back from the code's own budget when an execution will export files. + * + * The export runs after the code succeeds and draws on the same wall clock, so + * without a reserve a long install plus long-running code can time out during + * the read — destroying work the code already finished, under an error that + * only says "timeout". + */ +const MIN_EXPORT_BUDGET_MS = 10_000 + +/** + * The budget handed to user code, less an export reserve when this request will + * read files back. Short budgets are left alone: taking the reserve out of one + * would starve the code to buy time for an export it never reaches. + */ +function codeBudgetMs( + req: { outputSandboxPath?: string; outputSandboxPaths?: string[]; outputSandboxDir?: string }, + signal: AbortSignal +): number { + const remainingMs = remainingSandboxBudgetMs(signal) + const exportsFiles = Boolean( + req.outputSandboxDir || req.outputSandboxPath || req.outputSandboxPaths?.length + ) + if (!exportsFiles || remainingMs <= MIN_EXPORT_BUDGET_MS * 2) return remainingMs + return remainingMs - MIN_EXPORT_BUDGET_MS +} + /** * Installs a runtime sandbox's dependencies out of the caller's budget and * uses the shared wall-clock budget, so creation and every later phase consume @@ -547,7 +813,7 @@ async function executeInSandboxWithinBudget( }) throwIfAborted(signal) - const sandbox = await createSelectedSandbox( + const created = await createSelectedSandbox( kind, { language, @@ -555,10 +821,14 @@ async function executeInSandboxWithinBudget( lifetimeMs: remainingSandboxBudgetMs(signal), }, selected, - signal + signal, + req.meterUsage ) + const sandbox = created.sandbox const sandboxId = sandbox.sandboxId - const abortBinding = bindSandboxAbort(sandbox, signal) + const abortBinding = bindSandboxAbort(sandbox, created.providerId, signal) + let billableResult: SandboxExecutionResult | undefined + let billableOutputError: unknown try { throwIfAborted(signal) @@ -568,6 +838,7 @@ async function executeInSandboxWithinBudget( // await provisionWithinBudget(sandbox, selected, signal) await writeSandboxInputs(sandbox, req.sandboxFiles, { signal }) + await ensureSandboxOutputDir(sandbox, req.outputSandboxDir, signal) const privateInputEnvironment = await writeSandboxPrivateInputs( sandbox, req.privateInputs, @@ -583,7 +854,7 @@ async function executeInSandboxWithinBudget( let execution: SandboxCodeResult try { execution = await sandbox.runCode(code, { - timeoutMs: remainingSandboxBudgetMs(signal), + timeoutMs: codeBudgetMs(req, signal), javascriptPreload: buildJavaScriptRuntimeBindingsSource(req.runtimeBindings ?? []), maxOutputBytes: MAX_SANDBOX_PROCESS_OUTPUT_BYTES, signal, @@ -602,12 +873,14 @@ async function executeInSandboxWithinBudget( sandboxId, hasTraceback: Boolean(execution.error.traceback), }) - return { + const executionResult = { result: null, stdout: execution.error.traceback || errorMessage, error: errorMessage, sandboxId, } + if (execution.providerFailure !== 'provider_limit') billableResult = executionResult + return executionResult } // Distinct sources (final-expression text, stdout, stderr) join with '\n' so @@ -636,19 +909,47 @@ async function executeInSandboxWithinBudget( } } - const { exportedFiles, exportedFileContent } = await collectExportedFiles(sandbox, req, { - signal, - }) - throwIfAborted(signal) - - return { + billableResult = { result: extraction.result, stdout: cleanedStdout, sandboxId, - exportedFileContent, - exportedFiles, } + try { + const { exportedFiles, exportedFileContent, collectedFiles } = await collectExportedFiles( + sandbox, + req, + { signal } + ) + throwIfAborted(signal) + billableResult.exportedFileContent = exportedFileContent + billableResult.exportedFiles = exportedFiles + billableResult.collectedFiles = collectedFiles + } catch (error) { + /* + * A harvest that cannot return what the run produced — too many files, too + * deep, or an output directory the code deleted — is the caller's to fix + * and arrives only after the sandbox has already executed. It belongs with + * the other post-completion export failures the policy bills, not with the + * provider failures it absorbs; leaving it out let a completed run whose + * code wrote one file too many go free. + */ + if ( + isSandboxOutputLimitError(error) || + isSandboxOutputFileError(error) || + isSandboxOutputNotExportableError(error) + ) { + billableOutputError = error + } + throw error + } + return billableResult } finally { + const cleanupStartedAtMs = Date.now() + const cost = calculateSandboxCost(created, cleanupStartedAtMs) + if (cost && billableResult) billableResult.cost = cost + if (cost && billableOutputError) { + attachTrustedSandboxOutputCost(billableOutputError, cost) + } abortBinding.detach() await abortBinding.cleanup() } @@ -677,14 +978,18 @@ async function executeShellInSandboxWithinBudget( }) throwIfAborted(signal) - const sandbox = await createSelectedSandbox( + const created = await createSelectedSandbox( kind, { imageRef: selected?.imageRef, lifetimeMs: remainingSandboxBudgetMs(signal) }, selected, - signal + signal, + req.meterUsage ) + const sandbox = created.sandbox const sandboxId = sandbox.sandboxId - const abortBinding = bindSandboxAbort(sandbox, signal) + const abortBinding = bindSandboxAbort(sandbox, created.providerId, signal) + let billableResult: SandboxExecutionResult | undefined + let billableOutputError: unknown try { throwIfAborted(signal) @@ -696,6 +1001,7 @@ async function executeShellInSandboxWithinBudget( rootUser: true, signal, }) + await ensureSandboxOutputDir(sandbox, req.outputSandboxDir, signal) const privateInputEnvironment = await writeSandboxPrivateInputs( sandbox, req.privateInputs, @@ -711,7 +1017,7 @@ async function executeShellInSandboxWithinBudget( PATH: selected?.envs?.PATH ?? SANDBOX_SYSTEM_PATH, ...privateInputEnvironment, }, - timeoutMs: remainingSandboxBudgetMs(signal), + timeoutMs: codeBudgetMs(req, signal), maxOutputBytes: MAX_SANDBOX_PROCESS_OUTPUT_BYTES, signal, rootUser: true, @@ -735,7 +1041,9 @@ async function executeShellInSandboxWithinBudget( sandboxId, exitCode: result.exitCode, }) - return { result: null, stdout, error: errorMessage, sandboxId } + const executionResult = { result: null, stdout, error: errorMessage, sandboxId } + if (result.providerFailure !== 'provider_limit') billableResult = executionResult + return executionResult } // Shell scripts have no wrapper: any __SIM_RESULT__ line is user-authored @@ -744,19 +1052,47 @@ async function executeShellInSandboxWithinBudget( const extraction = extractSimResult(stdout) const parsed = extraction.parseFailed ? extraction.rawPayload : extraction.result - const { exportedFiles, exportedFileContent } = await collectExportedFiles(sandbox, req, { - signal, - }) - throwIfAborted(signal) - - return { + billableResult = { result: parsed, stdout: extraction.cleanedStdout, sandboxId, - exportedFileContent, - exportedFiles, } + try { + const { exportedFiles, exportedFileContent, collectedFiles } = await collectExportedFiles( + sandbox, + req, + { signal } + ) + throwIfAborted(signal) + billableResult.exportedFileContent = exportedFileContent + billableResult.exportedFiles = exportedFiles + billableResult.collectedFiles = collectedFiles + } catch (error) { + /* + * A harvest that cannot return what the run produced — too many files, too + * deep, or an output directory the code deleted — is the caller's to fix + * and arrives only after the sandbox has already executed. It belongs with + * the other post-completion export failures the policy bills, not with the + * provider failures it absorbs; leaving it out let a completed run whose + * code wrote one file too many go free. + */ + if ( + isSandboxOutputLimitError(error) || + isSandboxOutputFileError(error) || + isSandboxOutputNotExportableError(error) + ) { + billableOutputError = error + } + throw error + } + return billableResult } finally { + const cleanupStartedAtMs = Date.now() + const cost = calculateSandboxCost(created, cleanupStartedAtMs) + if (cost && billableResult) billableResult.cost = cost + if (cost && billableOutputError) { + attachTrustedSandboxOutputCost(billableOutputError, cost) + } abortBinding.detach() await abortBinding.cleanup() } @@ -813,12 +1149,13 @@ export interface PiSandboxRunner { * caller's sandbox body, which would have buried the change in whitespace. */ export async function withPiSandbox( - options: { lifetimeMs?: number }, + options: { lifetimeMs?: number; cost?: SandboxCostSink }, fn: (runner: PiSandboxRunner) => Promise ): Promise { const lifetimeMs = options.lifetimeMs !== undefined ? options.lifetimeMs : resolvePiSandboxLifetimeMs() - const sandbox = await createSandbox('pi', { lifetimeMs }) + const created = await createSandbox('pi', { lifetimeMs }, Boolean(options.cost)) + const { sandbox } = created logger.info('Started Pi sandbox', { sandboxId: sandbox.sandboxId, lifetimeMs }) const runner: PiSandboxRunner = { @@ -835,9 +1172,31 @@ export async function withPiSandbox( writeFile: (path, content) => sandbox.writeFile(path, content), } + let sessionCompleted = false try { - return await fn(runner) + const result = await fn(runner) + sessionCompleted = true + return result } finally { + /* + * Charged only for a session that ran to completion, which is the same rule + * the Function path applies to its own outcomes: a run whose sandbox never + * delivered is not billed, because a charge nobody can tie to delivered work + * is not one worth defending. A session that ends by throwing — a provider + * crash, a lifetime limit, a cancellation — is absorbed, and a create that + * throws never reaches here at all. + * + * A command exiting non-zero is not a failure by this rule. `fn` returns + * normally there, the agent produced its answer, and the Function path bills + * its own non-zero exits for the same reason. + * + * Measured up to teardown rather than to the last command, so the window + * covers the whole time the provider held the sandbox. + */ + if (sessionCompleted) { + const cost = calculateSandboxCost(created, Date.now()) + if (cost && options.cost) options.cost.total += cost.total + } try { await sandbox.kill() } catch { diff --git a/apps/sim/lib/execution/remote-sandbox/output-limits.ts b/apps/sim/lib/execution/remote-sandbox/output-limits.ts index 91fc5cb3616..e260660aa19 100644 --- a/apps/sim/lib/execution/remote-sandbox/output-limits.ts +++ b/apps/sim/lib/execution/remote-sandbox/output-limits.ts @@ -1,5 +1,27 @@ +import type { SandboxExecutionCost } from '@/lib/execution/remote-sandbox/types' + export const MAX_SANDBOX_OUTPUT_BYTES = 50 * 1024 * 1024 +/** + * Hard ceiling on a single URL-mounted input, enforced inside the sandbox by + * `curl --max-filesize` against the bytes actually served. + * + * The planner checks a recorded size first for a fast, well-worded failure; this + * is the backstop for when that size understates the stored object, and it is + * what a URL mount falls back to when the caller declares no ceiling of its own. + * URL bytes never enter the web process, so the resource being bounded is + * sandbox disk. + */ +export const MAX_SANDBOX_URL_MOUNT_BYTES = 500 * 1024 * 1024 + +/** + * How many files one execution may export, whether declared by path or + * discovered by harvesting the output directory. Exceeding it is an error rather + * than a truncation: silently returning the first 20 of 100 files reads as + * success while losing the rest. + */ +export const MAX_SANDBOX_OUTPUT_FILES = 20 + /** * Maximum combined stdout, stderr, result text, and structured error text kept * for one sandbox operation. Function results larger than this should be @@ -52,6 +74,51 @@ export function appendStreamedSandboxOutput(current: string, chunk: string): str export const SANDBOX_OUTPUT_LIMIT_CODE = 'sandbox_output_limit_exceeded' as const export const SANDBOX_OUTPUT_FILE_INVALID_CODE = 'sandbox_output_file_invalid' as const +/** + * The harvest cannot return what the run produced — too many files, or nested + * past what the listing reaches. Both are the caller's to fix and neither is + * retryable, so they share a code and are reported as one 400. + */ +export const SANDBOX_OUTPUT_NOT_EXPORTABLE_CODE = 'sandbox_output_not_exportable' as const + +/** More files in the harvest directory than one execution may export. */ +export class SandboxOutputFileCountError extends Error { + readonly code = SANDBOX_OUTPUT_NOT_EXPORTABLE_CODE + + constructor(observedFiles: number, directory: string, limit = MAX_SANDBOX_OUTPUT_FILES) { + super( + `Sandbox produced ${observedFiles} files in ${directory}, over the ${limit}-file export limit. Write fewer files, or archive them into a single .zip.` + ) + this.name = 'SandboxOutputFileCountError' + } +} + +/** Harvest directory nested deeper than the listing can reach. */ +export class SandboxOutputDepthError extends Error { + readonly code = SANDBOX_OUTPUT_NOT_EXPORTABLE_CODE + + constructor(directoryPath: string, maxDepth: number) { + super( + `Sandbox output "${directoryPath}" is nested deeper than ${maxDepth} levels, so its contents cannot be returned. Write results closer to the top of the output directory, or archive the tree into a single file.` + ) + this.name = 'SandboxOutputDepthError' + } +} + +const trustedSandboxOutputCosts = new WeakMap() + +/** Associates Sim-calculated cost with a trusted post-execution output error. */ +export function attachTrustedSandboxOutputCost(error: unknown, cost: SandboxExecutionCost): void { + if (typeof error !== 'object' || error === null) return + trustedSandboxOutputCosts.set(error, cost) +} + +/** Reads cost only when the sandbox lifecycle attached it after a completed execution. */ +export function readTrustedSandboxOutputCost(error: unknown): SandboxExecutionCost | undefined { + return typeof error === 'object' && error !== null + ? trustedSandboxOutputCosts.get(error) + : undefined +} export class SandboxOutputFileError extends Error { readonly code = SANDBOX_OUTPUT_FILE_INVALID_CODE @@ -136,3 +203,28 @@ export function isSandboxOutputFileError(error: unknown): error is SandboxOutput (error as { code?: unknown }).code === SANDBOX_OUTPUT_FILE_INVALID_CODE) ) } + +/** The harvest directory was removed by the code that was supposed to fill it. */ +export class SandboxOutputDirectoryMissingError extends Error { + readonly code = SANDBOX_OUTPUT_NOT_EXPORTABLE_CODE + + constructor(directoryPath: string) { + super( + `The sandbox output directory ${directoryPath} no longer exists — the code deleted it. Write files into it rather than replacing it; no files could be returned from this run.` + ) + this.name = 'SandboxOutputDirectoryMissingError' + } +} + +export function isSandboxOutputNotExportableError( + error: unknown +): error is + | SandboxOutputFileCountError + | SandboxOutputDepthError + | SandboxOutputDirectoryMissingError { + return ( + typeof error === 'object' && + error !== null && + (error as { code?: unknown }).code === SANDBOX_OUTPUT_NOT_EXPORTABLE_CODE + ) +} diff --git a/apps/sim/lib/execution/remote-sandbox/pi-sandbox-billing.smoke.test.ts b/apps/sim/lib/execution/remote-sandbox/pi-sandbox-billing.smoke.test.ts new file mode 100644 index 00000000000..648eb64e491 --- /dev/null +++ b/apps/sim/lib/execution/remote-sandbox/pi-sandbox-billing.smoke.test.ts @@ -0,0 +1,96 @@ +/** + * @vitest-environment node + * + * Checks that a Pi session's sandbox is actually metered against a real provider. + * + * The handler-level test mocks the backend and writes into the sink by hand, so + * it proves the wiring from a backend to the block's cost and nothing else. It + * would still pass if `withPiSandbox` never metered at all — which is exactly + * the bug this path had. Only a real Pi sandbox shows that creation is metered, + * that teardown reports, and that the amount tracks the session's real lifetime. + * + * Enable with `SANDBOX_BILLING_SMOKE=1`, against whichever provider + * `SANDBOX_PROVIDER` selects. Needs that provider's Pi image configured + * (`E2B_PI_TEMPLATE_ID` / `DAYTONA_PI_SNAPSHOT_ID`). + */ +import { describe, expect, it } from 'vitest' +import { createSandboxPricing } from '@/lib/billing/sandbox-pricing' +import { withPiSandbox } from '@/lib/execution/remote-sandbox' +import { resolveProvider } from '@/lib/execution/remote-sandbox/provider' +import type { SandboxCostSink } from '@/lib/execution/remote-sandbox/types' + +const smokeEnabled = process.env.SANDBOX_BILLING_SMOKE === '1' +const CASE_TIMEOUT_MS = 5 * 60_000 + +/** Long enough that provisioning jitter cannot dominate the measured session. */ +const SLEEP_SECONDS = 5 +/** Well under any provider ceiling, so the lifetime cap never clamps the charge. */ +const LIFETIME_MS = 10 * 60_000 + +describe.skipIf(!smokeEnabled)('pi sandbox billing smoke', () => { + it( + 'bills the session a Pi sandbox was held for', + async () => { + const pricing = createSandboxPricing(resolveProvider().id) + const usdPerBilledSecond = + (pricing.resources.vcpu * pricing.rates.cpuUsdPerVcpuSecond + + pricing.resources.memoryGiB * pricing.rates.memoryUsdPerGiBSecond + + pricing.resources.diskGiB * pricing.rates.diskUsdPerGiBSecond) * + pricing.multiplier + + const sandboxCost: SandboxCostSink = { total: 0 } + const wallClockStartedAtMs = Date.now() + const exitCode = await withPiSandbox( + { lifetimeMs: LIFETIME_MS, cost: sandboxCost }, + async (runner) => { + const result = await runner.run(`sleep ${SLEEP_SECONDS}; echo held`, { + envs: {}, + timeoutMs: CASE_TIMEOUT_MS, + }) + return result.exitCode + } + ) + const wallClockMs = Date.now() - wallClockStartedAtMs + + expect(exitCode).toBe(0) + expect(sandboxCost.total).toBeGreaterThanOrEqual(SLEEP_SECONDS * usdPerBilledSecond) + expect(sandboxCost.total).toBeLessThanOrEqual((wallClockMs / 1000) * usdPerBilledSecond) + }, + CASE_TIMEOUT_MS + ) + + it( + 'bills nothing for a session that ended by throwing', + async () => { + // Mirrors the Function path: a sandbox that never delivered is absorbed + // rather than charged. Covers a provider crash, a lifetime limit, and a + // cancellation alike, since all three reach here the same way. + const sandboxCost: SandboxCostSink = { total: 0 } + + await expect( + withPiSandbox({ lifetimeMs: LIFETIME_MS, cost: sandboxCost }, async (runner) => { + await runner.run('echo started', { envs: {}, timeoutMs: CASE_TIMEOUT_MS }) + throw new Error('session failed after the sandbox was provisioned') + }) + ).rejects.toThrow('session failed after the sandbox was provisioned') + + expect(sandboxCost.total).toBe(0) + }, + CASE_TIMEOUT_MS + ) + + it( + 'bills nothing when no sink is supplied', + async () => { + // The mothership and any other internal caller must stay free, and the + // absence of a sink is the whole mechanism keeping them that way. + const held = await withPiSandbox({ lifetimeMs: LIFETIME_MS }, async (runner) => { + const result = await runner.run('echo held', { envs: {}, timeoutMs: CASE_TIMEOUT_MS }) + return result.exitCode + }) + + expect(held).toBe(0) + }, + CASE_TIMEOUT_MS + ) +}) diff --git a/apps/sim/lib/execution/remote-sandbox/sandbox-billing.smoke.test.ts b/apps/sim/lib/execution/remote-sandbox/sandbox-billing.smoke.test.ts new file mode 100644 index 00000000000..6b5ebfa63ee --- /dev/null +++ b/apps/sim/lib/execution/remote-sandbox/sandbox-billing.smoke.test.ts @@ -0,0 +1,78 @@ +/** + * @vitest-environment node + * + * Checks the metered amount against a real provider run. + * + * `sandbox-pricing.test.ts` pins the arithmetic and the conformance suite proves + * a cost is produced, attached, and routed — but that suite stubs the provider + * and mocks `Date.now()`, so its clock advances one millisecond per call. Under + * those conditions `total > 0` is the strongest claim available, and it would + * hold just as well if the metered window measured the wrong instants. Only a + * real run can show that the window tracks the sandbox's actual lifetime. + * + * Enable with `SANDBOX_BILLING_SMOKE=1`. Runs against whichever provider + * `SANDBOX_PROVIDER` selects, so point it at each in turn to cover both. + */ +import { describe, expect, it } from 'vitest' +import { createSandboxPricing } from '@/lib/billing/sandbox-pricing' +import { CodeLanguage } from '@/lib/execution/languages' +import { executeInSandbox } from '@/lib/execution/remote-sandbox' +import { resolveProvider } from '@/lib/execution/remote-sandbox/provider' + +const smokeEnabled = process.env.SANDBOX_BILLING_SMOKE === '1' +const CASE_TIMEOUT_MS = 5 * 60_000 +const RUN_TIMEOUT_MS = 4 * 60_000 + +/** Long enough that provisioning jitter cannot dominate the measured runtime. */ +const SLEEP_SECONDS = 5 + +describe.skipIf(!smokeEnabled)('sandbox billing smoke', () => { + it( + 'bills the sandbox lifetime at the provider rate', + async () => { + const pricing = createSandboxPricing(resolveProvider().id) + const usdPerSecond = + pricing.resources.vcpu * pricing.rates.cpuUsdPerVcpuSecond + + pricing.resources.memoryGiB * pricing.rates.memoryUsdPerGiBSecond + + pricing.resources.diskGiB * pricing.rates.diskUsdPerGiBSecond + const usdPerBilledSecond = usdPerSecond * pricing.multiplier + + const wallClockStartedAtMs = Date.now() + const result = await executeInSandbox({ + code: `import time\ntime.sleep(${SLEEP_SECONDS})\nprint("slept")`, + language: CodeLanguage.Python, + timeoutMs: RUN_TIMEOUT_MS, + meterUsage: true, + }) + const wallClockMs = Date.now() - wallClockStartedAtMs + + expect(result.cost).toEqual({ input: 0, output: 0, total: expect.any(Number) }) + const billed = result.cost?.total ?? 0 + + /** + * The window opens immediately before the provider create call and closes + * before teardown, so it has to cover the sleep and cannot exceed the whole + * call measured from out here. A rate error, a wrong resource constant, or a + * window anchored to the wrong instant all land outside these bounds — which + * an `expect.any(Number)` assertion cannot see. + */ + expect(billed).toBeGreaterThanOrEqual(SLEEP_SECONDS * usdPerBilledSecond) + expect(billed).toBeLessThanOrEqual((wallClockMs / 1000) * usdPerBilledSecond) + }, + CASE_TIMEOUT_MS + ) + + it( + 'bills nothing when the caller did not ask for metering', + async () => { + const result = await executeInSandbox({ + code: 'print("unmetered")', + language: CodeLanguage.Python, + timeoutMs: RUN_TIMEOUT_MS, + }) + + expect(result.cost).toBeUndefined() + }, + CASE_TIMEOUT_MS + ) +}) diff --git a/apps/sim/lib/execution/remote-sandbox/sandbox-files.smoke.test.ts b/apps/sim/lib/execution/remote-sandbox/sandbox-files.smoke.test.ts new file mode 100644 index 00000000000..72f60ec4087 --- /dev/null +++ b/apps/sim/lib/execution/remote-sandbox/sandbox-files.smoke.test.ts @@ -0,0 +1,348 @@ +/** + * @vitest-environment node + * + * End-to-end file I/O against a real sandbox provider. + * + * The conformance suite proves both adapters agree on a mocked SDK; this proves + * the contract survives the actual provider — that a mount really lands where + * the code expects, that the output directory really exists before user code + * runs, and that harvested bytes really come back unchanged. + * + * Enable with `SANDBOX_FILES_SMOKE=1`. Requires `E2B_API_KEY` and + * `E2B_FUNCTION_TEMPLATE_ID`; set `SANDBOX_PROVIDER=daytona` (with + * `DAYTONA_API_KEY` and `DAYTONA_SHELL_SNAPSHOT_ID`) to run the same table + * against Daytona instead. Each case creates and destroys one sandbox. + */ +import { createHash } from 'node:crypto' +import { describe, expect, it } from 'vitest' +import { CodeLanguage } from '@/lib/execution/languages' +import { + executeInSandbox, + executeShellInSandbox, + SIM_RESULT_PREFIX, +} from '@/lib/execution/remote-sandbox' +import { SANDBOX_INPUT_DIR, SANDBOX_OUTPUT_DIR } from '@/lib/execution/remote-sandbox/sandbox-paths' + +const smokeEnabled = process.env.SANDBOX_FILES_SMOKE === '1' +const CASE_TIMEOUT_MS = 5 * 60_000 +const RUN_TIMEOUT_MS = 4 * 60_000 + +/** Bytes that a UTF-8 round trip would destroy — the corruption we must not see. */ +const BINARY_FIXTURE = Buffer.from([ + 0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 0x00, 0xff, 0xfe, 0x80, 0x7f, 0xc3, 0x28, +]) + +function sha256(buffer: Buffer): string { + return createHash('sha256').update(buffer).digest('hex') +} + +function decode(contentBase64: string): Buffer { + return Buffer.from(contentBase64, 'base64') +} + +/** + * Emits the result marker by hand. These cases drive the sandbox layer directly, + * below the wrapper `execute-request` builds, so `__sim_result__` and a bare + * `return` are not available here — the marker is what proves the code ran to + * completion rather than dying partway. + */ +function pythonResult(expression: string): string { + return `import json; print('${SIM_RESULT_PREFIX}' + json.dumps(${expression}))` +} + +function javascriptResult(expression: string): string { + return `console.log('\\n${SIM_RESULT_PREFIX}' + JSON.stringify(${expression}))` +} + +describe.skipIf(!smokeEnabled)('sandbox file I/O smoke', () => { + it( + 'mounts inputs, harvests outputs, and preserves binary bytes exactly', + async () => { + const result = await executeInSandbox({ + code: [ + 'import os, shutil', + `text = open(os.path.join(${JSON.stringify(SANDBOX_INPUT_DIR)}, 'notes.txt')).read()`, + `blob = open(os.path.join(${JSON.stringify(SANDBOX_INPUT_DIR)}, 'fixture.bin'), 'rb').read()`, + `out = ${JSON.stringify(SANDBOX_OUTPUT_DIR)}`, + "open(os.path.join(out, 'echo.txt'), 'w').write(text.upper())", + // Copied byte-for-byte so any encoding mistake anywhere in the round + // trip shows up as a hash mismatch rather than a plausible-looking file. + "open(os.path.join(out, 'copy.bin'), 'wb').write(blob)", + "os.makedirs(os.path.join(out, 'nested'), exist_ok=True)", + "open(os.path.join(out, 'nested', 'deep.txt'), 'w').write('nested')", + "open(os.path.join(out, 'empty.txt'), 'w').write('')", + pythonResult('{"len": len(blob)}'), + ].join('\n'), + language: CodeLanguage.Python, + timeoutMs: RUN_TIMEOUT_MS, + sandboxFiles: [ + { path: `${SANDBOX_INPUT_DIR}/notes.txt`, content: 'hello sandbox' }, + { + path: `${SANDBOX_INPUT_DIR}/fixture.bin`, + content: BINARY_FIXTURE.toString('base64'), + encoding: 'base64', + }, + ], + outputSandboxDir: SANDBOX_OUTPUT_DIR, + }) + + expect(result.error).toBeUndefined() + expect(result.result).toEqual({ len: BINARY_FIXTURE.length }) + + const byPath = new Map((result.collectedFiles ?? []).map((file) => [file.relativePath, file])) + expect([...byPath.keys()].sort()).toEqual([ + 'copy.bin', + 'echo.txt', + 'empty.txt', + 'nested/deep.txt', + ]) + + expect(decode(byPath.get('echo.txt')!.contentBase64).toString('utf8')).toBe('HELLO SANDBOX') + expect(sha256(decode(byPath.get('copy.bin')!.contentBase64))).toBe(sha256(BINARY_FIXTURE)) + expect(byPath.get('copy.bin')!.byteLength).toBe(BINARY_FIXTURE.length) + expect(decode(byPath.get('nested/deep.txt')!.contentBase64).toString('utf8')).toBe('nested') + expect(byPath.get('empty.txt')!.byteLength).toBe(0) + }, + CASE_TIMEOUT_MS + ) + + it( + 'reads a mounted file and creates the output directory before JavaScript user code runs', + async () => { + const result = await executeInSandbox({ + code: [ + "import { readFileSync, writeFileSync, existsSync } from 'node:fs'", + `const out = ${JSON.stringify(SANDBOX_OUTPUT_DIR)}`, + // Asserted from inside the sandbox: if the directory were not created + // before user code, the very first write is ENOENT. + 'if (!existsSync(out)) throw new Error("output dir missing before user code")', + // The point of resolving `` to a path rather than + // inlining bytes is that every language can just open it. Python and + // Shell prove that in the cases either side of this one. + `const seed = readFileSync(${JSON.stringify(`${SANDBOX_INPUT_DIR}/seed.txt`)}, 'utf8')`, + 'writeFileSync(out + "/from-js.json", JSON.stringify({ seed }))', + javascriptResult('{ wrote: true }'), + ].join('\n'), + language: CodeLanguage.JavaScript, + timeoutMs: RUN_TIMEOUT_MS, + sandboxFiles: [{ path: `${SANDBOX_INPUT_DIR}/seed.txt`, content: 'js seed' }], + outputSandboxDir: SANDBOX_OUTPUT_DIR, + }) + + expect(result.error).toBeUndefined() + expect(result.collectedFiles).toHaveLength(1) + expect(result.collectedFiles?.[0].relativePath).toBe('from-js.json') + expect(JSON.parse(decode(result.collectedFiles![0].contentBase64).toString('utf8'))).toEqual({ + seed: 'js seed', + }) + }, + CASE_TIMEOUT_MS + ) + + it( + 'creates the output directory before shell user code runs', + async () => { + const result = await executeShellInSandbox({ + code: [ + `test -d ${SANDBOX_OUTPUT_DIR} || { echo "output dir missing" >&2; exit 1; }`, + `cp ${SANDBOX_INPUT_DIR}/seed.txt ${SANDBOX_OUTPUT_DIR}/from-shell.txt`, + `echo "${SIM_RESULT_PREFIX}\\"done\\""`, + ].join('\n'), + envs: {}, + timeoutMs: RUN_TIMEOUT_MS, + sandboxFiles: [{ path: `${SANDBOX_INPUT_DIR}/seed.txt`, content: 'shell seed' }], + outputSandboxDir: SANDBOX_OUTPUT_DIR, + }) + + expect(result.error).toBeUndefined() + expect(result.collectedFiles).toHaveLength(1) + expect(decode(result.collectedFiles![0].contentBase64).toString('utf8')).toBe('shell seed') + }, + CASE_TIMEOUT_MS + ) + + it( + 'returns nothing rather than failing when the code writes no files', + async () => { + const result = await executeInSandbox({ + code: pythonResult('"no files"'), + language: CodeLanguage.Python, + timeoutMs: RUN_TIMEOUT_MS, + outputSandboxDir: SANDBOX_OUTPUT_DIR, + }) + + expect(result.error).toBeUndefined() + expect(result.result).toBe('no files') + // "Produced nothing" is an ordinary outcome; the directory exists because + // the prologue made it, so listing it must succeed and come back empty. + expect(result.collectedFiles).toBeUndefined() + }, + CASE_TIMEOUT_MS + ) + + it( + 'skips directories and follows symlinks identically on either provider', + async () => { + const result = await executeInSandbox({ + code: [ + 'import os', + `out = ${JSON.stringify(SANDBOX_OUTPUT_DIR)}`, + "open(os.path.join(out, 'real.txt'), 'w').write('real')", + "os.symlink('/etc/passwd', os.path.join(out, 'linked.txt'))", + "os.makedirs(os.path.join(out, 'adir'), exist_ok=True)", + pythonResult('"planted"'), + ].join('\n'), + language: CodeLanguage.Python, + timeoutMs: RUN_TIMEOUT_MS, + outputSandboxDir: SANDBOX_OUTPUT_DIR, + }) + + expect(result.error).toBeUndefined() + // Followed rather than excluded, and the same on both providers — Daytona + // resolves links in its listing with no field that would reveal one, and + // the code could copy the target's bytes into the directory itself + // anyway. The empty directory is skipped on both. + expect((result.collectedFiles ?? []).map((file) => file.relativePath).sort()).toEqual([ + 'linked.txt', + 'real.txt', + ]) + }, + CASE_TIMEOUT_MS + ) + + it( + 'refuses a harvest over the file-count limit instead of truncating it', + async () => { + await expect( + executeInSandbox({ + code: [ + 'import os', + `out = ${JSON.stringify(SANDBOX_OUTPUT_DIR)}`, + 'for i in range(21):', + " open(os.path.join(out, f'file-{i}.txt'), 'w').write(str(i))", + pythonResult('"wrote 21"'), + ].join('\n'), + language: CodeLanguage.Python, + timeoutMs: RUN_TIMEOUT_MS, + outputSandboxDir: SANDBOX_OUTPUT_DIR, + }) + ).rejects.toThrow(/over the 20-file export limit/) + }, + CASE_TIMEOUT_MS + ) + + it( + 'probe: how deep a nested output is still harvested', + async () => { + const result = await executeInSandbox({ + code: [ + 'import os', + `out = ${JSON.stringify(SANDBOX_OUTPUT_DIR)}`, + 'for depth in range(1, 6):', + " d = os.path.join(out, *[f'l{i}' for i in range(1, depth + 1)])", + ' os.makedirs(d, exist_ok=True)', + " open(os.path.join(d, 'leaf.txt'), 'w').write(str(depth))", + pythonResult('"nested"'), + ].join('\n'), + language: CodeLanguage.Python, + timeoutMs: RUN_TIMEOUT_MS, + outputSandboxDir: SANDBOX_OUTPUT_DIR, + }) + + expect(result.error).toBeUndefined() + // Nesting deeper than the listing depth must not vanish silently — losing + // a file the code successfully wrote is worse than refusing the harvest. + expect((result.collectedFiles ?? []).map((file) => file.relativePath).sort()).toEqual([ + 'l1/l2/l3/l4/l5/leaf.txt', + 'l1/l2/l3/l4/leaf.txt', + 'l1/l2/l3/leaf.txt', + 'l1/l2/leaf.txt', + 'l1/leaf.txt', + ]) + }, + CASE_TIMEOUT_MS + ) + + it( + 'probe: a file name containing a newline survives the listing', + async () => { + const result = await executeInSandbox({ + code: [ + 'import os', + `out = ${JSON.stringify(SANDBOX_OUTPUT_DIR)}`, + `open(os.path.join(out, 'we\\nird.txt'), 'w').write('newline name')`, + pythonResult('"newline"'), + ].join('\n'), + language: CodeLanguage.Python, + timeoutMs: RUN_TIMEOUT_MS, + outputSandboxDir: SANDBOX_OUTPUT_DIR, + }) + + expect(result.error).toBeUndefined() + // A structured listing has no delimiter to corrupt, unlike the `find` + // manifest this deliberately avoids. + expect(result.collectedFiles).toHaveLength(1) + expect(decode(result.collectedFiles![0].contentBase64).toString('utf8')).toBe('newline name') + }, + CASE_TIMEOUT_MS + ) + + it( + 'names the cause when user code deletes the output directory', + async () => { + await expect( + executeInSandbox({ + code: [ + 'import shutil', + `shutil.rmtree(${JSON.stringify(SANDBOX_OUTPUT_DIR)})`, + pythonResult('"deleted"'), + ].join('\n'), + language: CodeLanguage.Python, + timeoutMs: RUN_TIMEOUT_MS, + outputSandboxDir: SANDBOX_OUTPUT_DIR, + }) + // Without this the caller sees a raw `lstat ... no such file or + // directory`, which reads like a platform fault rather than their own + // `rmtree`. + ).rejects.toThrow(/no longer exists — the code deleted it/) + }, + CASE_TIMEOUT_MS + ) + + it( + 'round-trips awkward file names', + async () => { + const result = await executeInSandbox({ + code: [ + 'import os', + `out = ${JSON.stringify(SANDBOX_OUTPUT_DIR)}`, + "open(os.path.join(out, 'Q4 Sales (Final).csv'), 'w').write('a,b')", + "open(os.path.join(out, 'rapport-café.txt'), 'w', encoding='utf-8').write('café')", + "open(os.path.join(out, 'archive.tar.gz'), 'wb').write(b'\\x1f\\x8b\\x08')", + "open(os.path.join(out, 'noext'), 'wb').write(b'\\x00\\x01\\x02')", + pythonResult('"named"'), + ].join('\n'), + language: CodeLanguage.Python, + timeoutMs: RUN_TIMEOUT_MS, + outputSandboxDir: SANDBOX_OUTPUT_DIR, + }) + + expect(result.error).toBeUndefined() + const byPath = new Map((result.collectedFiles ?? []).map((file) => [file.relativePath, file])) + expect([...byPath.keys()].sort()).toEqual([ + 'Q4 Sales (Final).csv', + 'archive.tar.gz', + 'noext', + 'rapport-café.txt', + ]) + // Extension-less and gzip content must survive: neither is in the + // allowlist that decides encoding for a declared output path. + expect(decode(byPath.get('noext')!.contentBase64)).toEqual(Buffer.from([0, 1, 2])) + expect(decode(byPath.get('archive.tar.gz')!.contentBase64)).toEqual( + Buffer.from([0x1f, 0x8b, 0x08]) + ) + expect(decode(byPath.get('rapport-café.txt')!.contentBase64).toString('utf8')).toBe('café') + }, + CASE_TIMEOUT_MS + ) +}) diff --git a/apps/sim/lib/execution/remote-sandbox/sandbox-paths.ts b/apps/sim/lib/execution/remote-sandbox/sandbox-paths.ts new file mode 100644 index 00000000000..d6bddf3eeb1 --- /dev/null +++ b/apps/sim/lib/execution/remote-sandbox/sandbox-paths.ts @@ -0,0 +1,106 @@ +/** + * Filesystem contract shared by every layer that touches a sandbox mount: the + * resolver that plans mount paths, the sandbox layer that creates the output + * directory and enumerates it, and the tool description that teaches a model + * where to write. + * + * Deliberately under `/tmp` rather than a home directory. E2B's default user is + * `user` with workdir `/home/user`, but the Daytona image is built from + * `python:3.13-slim-trixie` with no `useradd`, `USER`, or `WORKDIR`, so + * `/home/user` does not exist there and Daytona resolves relative paths against + * its own working directory. `/tmp` is present and writable by any user on any + * Linux image, which keeps one literal correct on both providers — and lets the + * tool description name that literal to the model instead of a path that has to + * be resolved per provider before it can be quoted. + */ + +/** + * Where mounted input files are materialized before user code runs. + * + * Both directories sit under `/tmp/sim/`, while the runtime's own scratch files + * (`/tmp/.sim-private-input-*`, `/tmp/.sim-env-*`, `/tmp/.sim-command-*`) are + * dotfiles at the `/tmp` root — so enumerating the output directory cannot reach + * them. + */ +export const SANDBOX_INPUT_DIR = '/tmp/sim/inputs' + +/** Files user code writes here are harvested back as platform file objects. */ +export const SANDBOX_OUTPUT_DIR = '/tmp/sim/outputs' + +/** + * Sentinel written to bring the output directory into existence before user code + * runs, and skipped when the directory is harvested. + * + * A directory cannot be created through the providers' filesystem APIs directly, + * but writing a file creates its parents — the same trick the Copilot directory + * mount already uses to materialize an empty folder. Doing it this way keeps the + * cost at one filesystem write; a `mkdir -p` command would instead cost a whole + * session on Daytona, which creates and tears one down per command. + * + * The suffix is not decoration: the harvest filters this name out, so a plainer + * one like `.sim-keep` would silently swallow a user file that happened to share + * it. + */ +export const SANDBOX_OUTPUT_DIR_SENTINEL = '.sim-keep-97f2c1a4' + +/** + * How deep the output directory is enumerated, counted in path segments — a file + * at `a/b/leaf.txt` is depth 3. + * + * Set far above any plausible layout rather than close to it, because the + * providers' listings take a depth and give no signal that they stopped. A file + * below the limit is one the code successfully wrote and the caller never + * receives, so the harvest also refuses outright when it sees a directory + * sitting at the limit — that entry is the evidence the listing was cut short. + */ +export const SANDBOX_OUTPUT_DIR_MAX_DEPTH = 12 + +/** + * How many files one Function block invocation may mount. Far below the Copilot + * ceiling: a block names its inputs one at a time, so a large count is a mistake + * rather than a legitimate bulk mount. + * + * Lives here, with the other mount bounds, because the boundary contract needs it + * too — and this module imports nothing, so a contract can read it without + * pulling the server-only mount resolver into a client-reachable graph. + */ +export const MAX_BLOCK_MOUNTED_FILES = 20 + +/** Trailing-slash-insensitive directory prefix, for joining and stripping. */ +function withTrailingSlash(dir: string): string { + return dir.endsWith('/') ? dir : `${dir}/` +} + +/** + * Resolves one provider directory entry to an absolute path plus its path + * relative to the listed directory. + * + * Providers disagree on whether a listing reports absolute or directory-relative + * paths, and Daytona resolves relative paths against its own working directory + * rather than the listed one — so a relative entry is joined to the directory we + * asked for instead of being trusted as-is. Returns null when the result escapes + * that directory, which is what keeps a `..` component in a provider-reported + * name from reaching a reader. + */ +export function resolveSandboxDirectoryEntryPath( + dir: string, + reportedPath: string +): { path: string; relativePath: string } | null { + const prefix = withTrailingSlash(dir) + const absolute = reportedPath.startsWith('/') ? reportedPath : `${prefix}${reportedPath}` + + const segments: string[] = [] + for (const segment of absolute.split('/')) { + if (segment === '' || segment === '.') continue + if (segment === '..') { + if (segments.length === 0) return null + segments.pop() + continue + } + segments.push(segment) + } + const normalized = `/${segments.join('/')}` + + if (!normalized.startsWith(prefix)) return null + return { path: normalized, relativePath: normalized.slice(prefix.length) } +} diff --git a/apps/sim/lib/execution/remote-sandbox/types.ts b/apps/sim/lib/execution/remote-sandbox/types.ts index c2f4e7b2b28..10732ad35ae 100644 --- a/apps/sim/lib/execution/remote-sandbox/types.ts +++ b/apps/sim/lib/execution/remote-sandbox/types.ts @@ -18,7 +18,20 @@ export type SandboxProviderId = 'e2b' | 'daytona' */ export type SandboxFile = | { type?: 'content'; path: string; content: string; encoding?: 'base64' } - | { type: 'url'; path: string; url: string } + | { + type: 'url' + path: string + url: string + /** + * Ceiling enforced on the bytes actually transferred, rather than on a size + * the caller reported. A caller's pre-read check is a fast, well-worded + * failure; this is what makes it true when the recorded size understates + * the stored object. Optional only because it crosses the wire; a mount + * that omits it still gets `MAX_SANDBOX_URL_MOUNT_BYTES`, so the cap + * cannot be skipped by omission. + */ + maxBytes?: number + } /** * An internal runtime payload materialized at an opaque sandbox path. @@ -47,12 +60,20 @@ export interface SandboxExecutionRequest { * (mothership-docs) that has python-pptx/docx/openpyxl/reportlab installed. */ sandboxKind?: 'code' | 'mothership' | 'doc' + /** + * Harvest every regular file under this directory after the code succeeds. + * Unlike {@link outputSandboxPaths}, the paths are discovered rather than + * declared, so a model that only authors `code` can still return files. + */ + outputSandboxDir?: string /** Scope for {@link sandboxId}; a sandbox from another workspace is rejected. */ workspaceId?: string /** Workspace sandbox whose dependency set this execution runs against. */ sandboxId?: string /** Cancels the provider sandbox when the caller's execution budget expires. */ signal?: AbortSignal + /** Adds the remote provider cost to a completed, billable Function outcome. */ + meterUsage?: boolean } export interface SandboxShellExecutionRequest { @@ -70,12 +91,34 @@ export interface SandboxShellExecutionRequest { * they run in the doc image (mothership-docs). */ sandboxKind?: 'shell' | 'mothership' | 'doc' + /** See {@link SandboxExecutionRequest.outputSandboxDir}. */ + outputSandboxDir?: string /** Scope for {@link sandboxId}; a sandbox from another workspace is rejected. */ workspaceId?: string /** Workspace sandbox whose dependency set this execution runs against. */ sandboxId?: string /** Cancels the provider sandbox when the caller's execution budget expires. */ signal?: AbortSignal + /** Adds the remote provider cost to a completed, billable Function outcome. */ + meterUsage?: boolean +} + +export interface SandboxExecutionCost { + input: number + output: number + total: number +} + +/** + * Running total a caller accumulates sandbox charges into. + * + * A long-lived sandbox reports its cost when it is torn down, which is after the + * value its caller cares about has already been returned. Handing the layer a + * sink lets the charge land without reshaping every return type between here and + * the block that owns the bill. + */ +export interface SandboxCostSink { + total: number } export interface SandboxExecutionResult { @@ -85,6 +128,25 @@ export interface SandboxExecutionResult { error?: string exportedFileContent?: string exportedFiles?: Record + /** + * Files discovered under {@link SandboxExecutionRequest.outputSandboxDir}. + * + * Always base64, never utf8: the extension allowlist that decides encoding for + * a declared path cannot classify an arbitrary harvested filename, and + * decoding real binary as utf8 substitutes U+FFFD silently — corruption that + * arrives looking like a valid file. Base64 is lossless for any byte + * sequence, and the byte budget is enforced on the decoded length. + */ + collectedFiles?: SandboxCollectedFile[] + cost?: SandboxExecutionCost +} + +/** One harvested output file, carried as base64 with its decoded length. */ +export interface SandboxCollectedFile { + path: string + relativePath: string + contentBase64: string + byteLength: number } /** Result of one command run inside a sandbox. */ @@ -94,6 +156,8 @@ export interface SandboxCommandResult { exitCode: number /** The provider stopped the command because its supplied execution budget elapsed. */ timedOut?: boolean + /** The provider ended execution for an infrastructure reason, not a user-process outcome. */ + providerFailure?: 'provider_limit' } /** @@ -117,6 +181,8 @@ export interface SandboxCodeResult { error?: SandboxCodeError /** The provider stopped the code runner because its supplied execution budget elapsed. */ timedOut?: boolean + /** The provider ended execution for an infrastructure reason, not a user-program outcome. */ + providerFailure?: 'provider_limit' } export interface RunCommandOptions { @@ -180,9 +246,50 @@ export interface SandboxHandle { * delivered without any shell parsing. */ writeFile(path: string, content: string | ArrayBuffer): Promise + /** + * Lists regular files under a directory, recursively to `depth`. + * + * Uses each provider's filesystem API rather than shelling out to `find`. + * A shell listing would cost a session per call on Daytona (its + * `runCommand` creates one, writes an env file, executes, then deletes it), + * depend on GNU coreutils that a future base image need not carry, and be + * corrupted by a filename containing a newline — which user code controls. + * + * Symlinks are followed, not excluded. Daytona's listing resolves them and + * reports no field distinguishing one from a regular file, so excluding them + * is only possible on E2B — and doing it there alone would be a cross-provider + * divergence that reads as a security property while providing none. It + * provides none because the harvest is not a privilege boundary: it runs as + * the same identity as the code, which can already read any file the sandbox + * can and copy the bytes into the output directory itself. + * + * Directories are returned alongside files rather than filtered out, because + * a directory sitting at the traversal limit is the only evidence that the + * listing was cut short — see the truncation check in the harvest. + * + * Errors propagate rather than degrading to an empty list. The output + * directory is created before user code runs, so a listing failure is a real + * fault, and reporting it as "produced nothing" would turn a transient + * provider error into silent loss of the caller's files. + */ + listFiles(path: string, options?: { depth?: number }): Promise kill(): Promise } +/** One entry discovered by {@link SandboxHandle.listFiles}. */ +export interface SandboxDirectoryEntry { + /** Absolute path inside the sandbox. */ + path: string + /** Path relative to the listed directory, retaining any subdirectories. */ + relativePath: string + kind: 'file' | 'directory' + /** + * Provider-reported size. Advisory only — the read re-enforces its own limit, + * since the file can change between listing and read. + */ + size: number +} + export interface CreateSandboxOptions { /** Bound at creation — see {@link SandboxHandle.runCode}. */ language?: CodeLanguage @@ -199,6 +306,8 @@ export interface CreateSandboxOptions { * and creates the sandbox as ephemeral. */ lifetimeMs?: number + /** Reports the instant immediately before the provider SDK create request is dispatched. */ + onProviderRequestStarted?: (startedAtMs: number) => void } /** @@ -286,5 +395,7 @@ export interface SandboxProvider { readonly dependencyStrategy: SandboxDependencyStrategy /** Present exactly when {@link dependencyStrategy} is `prebuilt`. */ readonly images?: SandboxImageBuilder + /** Resolves the provider's rounded lifetime for both creation and metering. */ + resolveLifetimeMs(lifetimeMs: number): number create(kind: SandboxKind, options?: CreateSandboxOptions): Promise } diff --git a/apps/sim/lib/execution/sim-helpers.smoke.test.ts b/apps/sim/lib/execution/sim-helpers.smoke.test.ts new file mode 100644 index 00000000000..9e66b0b4a55 --- /dev/null +++ b/apps/sim/lib/execution/sim-helpers.smoke.test.ts @@ -0,0 +1,233 @@ +/** + * @vitest-environment node + * + * The `sim.*` helper namespace, exercised in a real isolate. + * + * `isolated-vm.test.ts` mocks the spawn, so it never proves the namespace is + * reachable from user code — only that the process plumbing is called. These + * cases run the actual worker and assert a value crosses the boundary in both + * directions, which is the only way the frozen `global.sim` shim and the + * broker's JSON marshalling are covered at all. + * + * Enable with `SIM_HELPERS_SMOKE=1`. Needs `isolated-vm` installed for the + * running Node (prebuilds exist for 22/24 only; other versions source-build). + */ +import { describe, expect, it } from 'vitest' +import { executeInIsolatedVM, type IsolatedVMBrokerHandler } from '@/lib/execution/isolated-vm' + +const smokeEnabled = process.env.SIM_HELPERS_SMOKE === '1' +const CASE_TIMEOUT_MS = 60_000 + +const FILE = { + id: 'file_1', + name: 'notes.txt', + url: 'https://storage.example/notes.txt', + size: 11, + type: 'text/plain', + key: 'execution/ws/wf/exec/abc/notes.txt', + context: 'execution', +} + +/** Records what user code asked for, and answers the way the runtime does. */ +function recordingBrokers(): { + brokers: Record + calls: Array<{ name: string; args: unknown }> +} { + const calls: Array<{ name: string; args: unknown }> = [] + const record = + (name: string, reply: (args: any) => unknown): IsolatedVMBrokerHandler => + async (args: any) => { + calls.push({ name, args }) + return reply(args) + } + + return { + calls, + brokers: { + 'sim.files.readText': record('sim.files.readText', () => 'hello world'), + 'sim.files.readBase64': record('sim.files.readBase64', () => + Buffer.from('hello world').toString('base64') + ), + 'sim.files.readTextChunk': record('sim.files.readTextChunk', (args) => ({ + content: 'hello'.slice(0, args?.options?.length ?? 5), + offset: args?.options?.offset ?? 0, + })), + 'sim.values.read': record('sim.values.read', () => ({ rows: [1, 2, 3] })), + 'sim.values.readArray': record('sim.values.readArray', () => [{ a: 1 }, { a: 2 }]), + }, + } +} + +function run(code: string, brokers: Record) { + return executeInIsolatedVM( + { + code, + params: {}, + envVars: {}, + contextVariables: { simFile: FILE }, + timeoutMs: 20_000, + requestId: 'sim-helpers-smoke', + }, + { brokers } + ) +} + +describe.skipIf(!smokeEnabled)('sim.* helpers in a real isolate', () => { + it( + 'exposes sim.files reads to user code and returns their values', + async () => { + const { brokers, calls } = recordingBrokers() + + const result = await run( + [ + 'const text = await sim.files.readText(simFile)', + 'const b64 = await sim.files.readBase64(simFile)', + 'return { text, b64 }', + ].join('\n'), + brokers + ) + + expect(result.error).toBeUndefined() + expect(result.result).toEqual({ + text: 'hello world', + b64: Buffer.from('hello world').toString('base64'), + }) + // The file object must cross intact — the broker authorizes on its `key`, + // so a shim that dropped fields would fail open at the wrong layer. + expect(calls.map((call) => call.name)).toEqual(['sim.files.readText', 'sim.files.readBase64']) + expect((calls[0].args as { file: typeof FILE }).file).toEqual(FILE) + }, + CASE_TIMEOUT_MS + ) + + it( + 'passes options through and returns structured chunk results', + async () => { + const { brokers, calls } = recordingBrokers() + + const result = await run( + 'return await sim.files.readTextChunk(simFile, { offset: 0, length: 5 })', + brokers + ) + + expect(result.error).toBeUndefined() + expect(result.result).toEqual({ content: 'hello', offset: 0 }) + expect((calls[0].args as { options: unknown }).options).toEqual({ offset: 0, length: 5 }) + }, + CASE_TIMEOUT_MS + ) + + it( + 'exposes sim.values reads for offloaded large values', + async () => { + const { brokers } = recordingBrokers() + + const result = await run( + [ + 'const value = await sim.values.read({ __simLargeValueRef: true })', + 'const rows = await sim.values.readArray({ __simLargeValueRef: true })', + 'return { value, rowCount: rows.length }', + ].join('\n'), + brokers + ) + + expect(result.error).toBeUndefined() + expect(result.result).toEqual({ value: { rows: [1, 2, 3] }, rowCount: 2 }) + }, + CASE_TIMEOUT_MS + ) + + it( + 'surfaces a broker rejection as an ordinary error the code can catch', + async () => { + const brokers: Record = { + 'sim.files.readText': async () => { + throw new Error('File is not available in this execution.') + }, + } + + const result = await run( + [ + 'try {', + ' await sim.files.readText(simFile)', + ' return { caught: false }', + '} catch (error) {', + ' return { caught: true, message: String(error.message) }', + '}', + ].join('\n'), + brokers + ) + + // A denied read has to reach user code as a catchable error, not kill the + // isolate — the same file may be optional to the script. + expect(result.error).toBeUndefined() + expect(result.result).toMatchObject({ caught: true }) + expect((result.result as { message: string }).message).toContain('not available') + }, + CASE_TIMEOUT_MS + ) + + it( + 'pins which globals the fast local runtime actually provides', + async () => { + const { brokers } = recordingBrokers() + + const result = await run( + [ + 'const names = ["sim","fetch","console","JSON","Uint8Array",', + ' "Buffer","require","process","atob","TextDecoder","crypto","setTimeout"]', + 'const out = {}', + 'for (const name of names) out[name] = typeof globalThis[name] !== "undefined"', + 'return out', + ].join('\n'), + brokers + ) + + expect(result.error).toBeUndefined() + // The isolate/sandbox split made concrete. The fast runtime is plain + // ECMAScript plus `fetch` and `sim.*` — no Node built-ins, and notably no + // `crypto`, `TextDecoder`, or even `setTimeout`. Reaching for any of them + // is what makes a block need an import, which is what moves it to the + // slower remote sandbox. The block tip documents exactly this list, so + // pin it here rather than letting it drift. + expect(result.result).toEqual({ + sim: true, + fetch: true, + console: true, + JSON: true, + Uint8Array: true, + Buffer: false, + require: false, + process: false, + atob: false, + TextDecoder: false, + crypto: false, + setTimeout: false, + }) + }, + CASE_TIMEOUT_MS + ) + + it( + 'freezes the namespace so user code cannot replace a helper', + async () => { + const { brokers } = recordingBrokers() + + const result = await run( + [ + 'let replaced = true', + 'try { sim.files.readText = () => "spoofed" } catch { replaced = false }', + 'const text = await sim.files.readText(simFile)', + 'return { replaced, text }', + ].join('\n'), + brokers + ) + + expect(result.error).toBeUndefined() + // Whether the assignment throws or is silently ignored, the real helper + // must still be the one that runs. + expect((result.result as { text: string }).text).toBe('hello world') + }, + CASE_TIMEOUT_MS + ) +}) diff --git a/apps/sim/lib/function-execution/execute-request.test.ts b/apps/sim/lib/function-execution/execute-request.test.ts index 2fd1b35eaa5..dab70ebe860 100644 --- a/apps/sim/lib/function-execution/execute-request.test.ts +++ b/apps/sim/lib/function-execution/execute-request.test.ts @@ -23,6 +23,7 @@ import { PRIVATE_SECRET_PROVENANCE_HEADER, } from '@/lib/execution/private-tool-metadata' import { + attachTrustedSandboxOutputCost, MAX_SANDBOX_OUTPUT_BYTES, SandboxOutputFileError, SandboxOutputLimitError, @@ -84,7 +85,11 @@ vi.mock('@/lib/copilot/request/tools/files', () => ({ md: 'text/markdown', html: 'text/html', }, - normalizeOutputWorkspaceFileName: vi.fn((p: string) => p.replace(/^files\//, '')), + normalizeOutputWorkspaceFileName: vi.fn((p: string) => { + const normalized = p.trim().replace(/^\/+|\/+$/g, '') + if (!normalized) throw new Error('Output path must include a file name') + return normalized.replace(/^files\//, '') + }), resolveOutputFormat: vi.fn(() => 'json'), getOutputFileDeclarations: vi.fn((params: Record) => { if (Array.isArray(params.outputs?.files)) { @@ -143,6 +148,34 @@ vi.mock('@/lib/uploads', () => ({ vi.mock('@/lib/workflows/utils', () => workflowsUtilsMock) +/** + * Only the I/O half is stubbed. Path naming, transports, ceilings and + * authorization are covered against the real implementation in + * `sandbox-mounts.test.ts`; what matters here is the wiring — that a marker + * becomes a mount and that the context variable ends up holding the path. + */ +vi.mock('@/lib/function-execution/sandbox-mounts', () => ({ + planUserFileMounts: (files: Array<{ key: string; name: string }>) => + files.map((userFile) => ({ userFile, mountPath: `/tmp/sim/inputs/${userFile.name}` })), + resolveUserFileMounts: async ({ + planned, + }: { + planned: Array<{ userFile: { name: string }; mountPath: string }> + }) => ({ + sandboxFiles: planned.map(({ mountPath }) => ({ + type: 'url' as const, + path: mountPath, + url: 'https://presigned.example/object', + })), + manifest: planned.map(({ userFile, mountPath }) => ({ + name: userFile.name, + path: mountPath, + size: 1, + type: 'application/pdf', + })), + }), +})) + import { validateProxyUrl } from '@/lib/core/security/input-validation' import { clearLargeValueCacheForTests } from '@/lib/execution/payloads/cache' import { isLargeArrayManifest } from '@/lib/execution/payloads/large-array-manifest-metadata' @@ -192,6 +225,25 @@ async function POST(request: NextRequest): Promise { afterAll(resetEnvFlagsMock) +/** + * A `` reference as it reaches the function runtime: the resolver + * leaves a mount marker in the context variables, which is what asks this run for a + * sandbox filesystem. + */ +const MOUNT_REF = { + __simSandboxFileMount: true, + version: 1, + file: { + id: 'file_1', + name: 'doc.pdf', + url: 'https://storage.example/doc.pdf', + size: 12, + type: 'application/pdf', + key: 'execution/workspace-1/wf-1/exec-1/abc/doc.pdf', + context: 'execution', + }, +} + describe('Function execution request', () => { beforeEach(() => { vi.clearAllMocks() @@ -281,6 +333,7 @@ describe('Function execution request', () => { result: 'done', stdout: 'ok', sandboxId: 'sandbox-123', + cost: { input: 0, output: 0, total: 0.00012345 }, exportedFiles: { '/tmp/out.txt': 'owned by attacker' }, }) mockWriteWorkspaceFileByPath.mockRejectedValueOnce( @@ -291,6 +344,8 @@ describe('Function execution request', () => { code: 'print("done")', language: 'python', workspaceId: 'workspace-victim', + workflowId: 'workflow-1', + executionId: 'execution-1', outputs: { files: [{ path: 'files/README.md', mode: 'overwrite', sandboxPath: '/tmp/out.txt' }], }, @@ -301,6 +356,7 @@ describe('Function execution request', () => { expect(response.status).toBe(403) expect(data).toHaveProperty('error', 'Insufficient workspace permissions') + expect(data.output.cost).toEqual({ input: 0, output: 0, total: 0.00012345 }) expect(mockWriteWorkspaceFileByPath).toHaveBeenCalledTimes(1) }) @@ -320,6 +376,113 @@ describe('Function execution request', () => { expect(mockExecuteShellInSandbox).not.toHaveBeenCalled() }) + it.each([ + { language: 'python', code: 'return 42', kind: 'code' }, + { language: 'shell', code: 'echo ready', kind: 'shell' }, + ])( + 'meters a standard workflow Function $kind sandbox and preserves its cost', + async ({ language, code, kind }) => { + envFlagsMock.isRemoteSandboxEnabled = true + const cost = { input: 0, output: 0, total: 0.00012345 } + const executeSandbox = kind === 'shell' ? mockExecuteShellInSandbox : mockExecuteInSandbox + executeSandbox.mockResolvedValueOnce({ + result: 42, + stdout: 'ready', + sandboxId: `sandbox-${kind}`, + cost, + }) + + const response = await POST( + createMockRequest('POST', { + code, + language, + workflowId: 'workflow-1', + workspaceId: 'workspace-1', + executionId: 'execution-1', + }) + ) + const data = await response.json() + + expect(response.status).toBe(200) + expect(executeSandbox).toHaveBeenCalledWith(expect.objectContaining({ meterUsage: true })) + expect(data.output.cost).toEqual(cost) + } + ) + + it.each([ + { + language: 'javascript', + code: 'import "node:path"\nthrow new Error("boom")', + kind: 'code', + }, + { language: 'python', code: 'raise ValueError("boom")', kind: 'code' }, + { language: 'shell', code: 'exit 1', kind: 'shell' }, + ])( + 'preserves sandbox cost in a failed remote $language Function response', + async ({ language, code, kind }) => { + envFlagsMock.isRemoteSandboxEnabled = true + const cost = { input: 0, output: 0, total: 0.00012345 } + const executeSandbox = kind === 'shell' ? mockExecuteShellInSandbox : mockExecuteInSandbox + executeSandbox.mockResolvedValueOnce({ + result: null, + stdout: 'boom', + error: 'boom', + sandboxId: `sandbox-${language}`, + cost, + }) + + const response = await POST( + createMockRequest('POST', { + code, + language, + workflowId: 'workflow-1', + workspaceId: 'workspace-1', + executionId: 'execution-1', + }) + ) + const data = await response.json() + + expect(response.status).toBe(422) + expect(executeSandbox).toHaveBeenCalledWith(expect.objectContaining({ meterUsage: true })) + expect(data.output.cost).toEqual(cost) + } + ) + + it('does not meter a non-workflow remote Function call', async () => { + envFlagsMock.isRemoteSandboxEnabled = true + + const response = await POST( + createMockRequest('POST', { + code: 'import path from "node:path"\nreturn path.sep', + language: 'javascript', + }) + ) + + expect(response.status).toBe(200) + expect(mockExecuteInSandbox).toHaveBeenCalledWith( + expect.objectContaining({ meterUsage: false }) + ) + }) + + it('keeps a custom Function tool local even when workflow context is present', async () => { + envFlagsMock.isRemoteSandboxEnabled = true + + const response = await POST( + createMockRequest('POST', { + code: 'return 42', + language: 'python', + workflowId: 'workflow-1', + workspaceId: 'workspace-1', + executionId: 'execution-1', + isCustomTool: true, + }) + ) + + expect(response.status).toBe(200) + expect(mockExecuteInIsolatedVM).toHaveBeenCalledOnce() + expect(mockExecuteInSandbox).not.toHaveBeenCalled() + }) + it('does not accept a Mothership sandbox profile from the request body', async () => { const req = createMockRequest('POST', { code: 'return "test"', @@ -375,6 +538,7 @@ describe('Function execution request', () => { expect.objectContaining({ language, sandboxKind: 'mothership', + meterUsage: false, }) ) expect(mockExecuteInIsolatedVM).not.toHaveBeenCalled() @@ -396,7 +560,7 @@ describe('Function execution request', () => { expect(response.status).toBe(200) expect(mockExecuteShellInSandbox).toHaveBeenCalledWith( - expect.objectContaining({ sandboxKind: 'mothership' }) + expect.objectContaining({ sandboxKind: 'mothership', meterUsage: false }) ) }) @@ -697,6 +861,7 @@ describe('Function execution request', () => { result: 'done', stdout: 'ok', sandboxId: 'sandbox-123', + cost: { input: 0, output: 0, total: 0.00023456 }, exportedFiles: { '/home/user/chart.png': 'iVBORw0KGgo=', '/home/user/summary.json': '{"ok":true}', @@ -707,6 +872,8 @@ describe('Function execution request', () => { code: 'print("done")', language: 'python', workspaceId: 'workspace-1', + workflowId: 'workflow-1', + executionId: 'execution-1', outputs: { files: [ { @@ -753,6 +920,7 @@ describe('Function execution request', () => { }) ) expect(data.output.result.files).toHaveLength(2) + expect(data.output.cost).toEqual({ input: 0, output: 0, total: 0.00023456 }) expect(data.resources).toEqual([ expect.objectContaining({ path: 'files/reports/chart.png' }), expect.objectContaining({ path: 'files/reports/summary.json' }), @@ -1331,9 +1499,10 @@ describe('Function execution request', () => { it('preserves output-limit classification from provider-side size inspection', async () => { envFlagsMock.isRemoteSandboxEnabled = true - mockExecuteInSandbox.mockRejectedValueOnce( - new SandboxOutputLimitError(MAX_SANDBOX_OUTPUT_BYTES + 1) - ) + const error = new SandboxOutputLimitError(MAX_SANDBOX_OUTPUT_BYTES + 1) + const cost = { input: 0, output: 0, total: 0.00023456 } + attachTrustedSandboxOutputCost(error, cost) + mockExecuteInSandbox.mockRejectedValueOnce(error) const req = createMockRequest('POST', { code: 'print("done")', @@ -1354,6 +1523,7 @@ describe('Function execution request', () => { expect(response.status).toBe(400) expect(data.error).toBe(`Sandbox output files exceed ${MAX_SANDBOX_OUTPUT_BYTES} bytes total`) + expect(data.output.cost).toEqual(cost) expect(mockWriteWorkspaceFileByPath).not.toHaveBeenCalled() }) @@ -1375,15 +1545,41 @@ describe('Function execution request', () => { expect(response.status).toBe(400) expect(data.error).toContain('must reference a regular file') + expect(data.output.cost).toBeUndefined() expect(mockWriteWorkspaceFileByPath).not.toHaveBeenCalled() }) + it.each(['/', '///', ' / '])( + 'rejects malformed workspace output destination %j before sandbox execution', + async (path) => { + envFlagsMock.isRemoteSandboxEnabled = true + + const response = await POST( + createMockRequest('POST', { + code: 'print("done")', + language: 'python', + workspaceId: 'workspace-1', + outputs: { + files: [{ path, sandboxPath: '/out/report.json' }], + }, + }) + ) + const data = await response.json() + + expect(response.status).toBe(400) + expect(data.error).toBe('Output path must include a file name') + expect(mockExecuteInSandbox).not.toHaveBeenCalled() + expect(mockExecuteShellInSandbox).not.toHaveBeenCalled() + } + ) + it('prevalidates all sandbox output destinations before writing any files', async () => { envFlagsMock.isRemoteSandboxEnabled = true mockExecuteInSandbox.mockResolvedValueOnce({ result: 'done', stdout: 'ok', sandboxId: 'sandbox-123', + cost: { input: 0, output: 0, total: 0.00023456 }, exportedFiles: { '/home/user/first.json': '{"first":true}', '/home/user/second.json': '{"second":true}', @@ -1397,6 +1593,8 @@ describe('Function execution request', () => { code: 'print("done")', language: 'python', workspaceId: 'workspace-1', + workflowId: 'workflow-1', + executionId: 'execution-1', outputs: { files: [ { @@ -1419,6 +1617,7 @@ describe('Function execution request', () => { expect(response.status).toBe(400) expect(data.success).toBe(false) expect(data.error).toContain('Directory not yet created') + expect(data.output.cost).toEqual({ input: 0, output: 0, total: 0.00023456 }) expect(mockWriteWorkspaceFileByPath).not.toHaveBeenCalled() }) @@ -1500,7 +1699,7 @@ describe('Function execution request', () => { expect(mockWriteWorkspaceFileByPath).not.toHaveBeenCalled() }) - it('rejects sandboxPath outputs when the call would run in isolated-vm (E2B enabled, JS without imports)', async () => { + it('routes plain JavaScript to the remote sandbox when it declares a sandboxPath output', async () => { envFlagsMock.isRemoteSandboxEnabled = true const req = createMockRequest('POST', { @@ -1518,6 +1717,25 @@ describe('Function execution request', () => { }, }) + await POST(req) + + // Needing a sandbox filesystem selects the remote runtime the same way a + // selected sandbox image does. Refusing here instead would dead-end the + // caller: "add an import" is not a fix anyone should have to discover. + expect(mockExecuteInSandbox).toHaveBeenCalled() + expect(mockExecuteInIsolatedVM).not.toHaveBeenCalled() + }) + + it('refuses sandbox file inputs/outputs when no remote sandbox is configured', async () => { + envFlagsMock.isRemoteSandboxEnabled = false + + const req = createMockRequest('POST', { + code: 'return "content"', + language: 'javascript', + workspaceId: 'workspace-1', + contextVariables: { doc: MOUNT_REF }, + }) + const response = await POST(req) const data = await response.json() @@ -1529,6 +1747,201 @@ describe('Function execution request', () => { expect(mockWriteWorkspaceFileByPath).not.toHaveBeenCalled() }) + it('refuses sandbox file inputs/outputs for a custom tool, which always runs in isolated-vm', async () => { + envFlagsMock.isRemoteSandboxEnabled = true + + const req = createMockRequest('POST', { + code: 'return "content"', + language: 'javascript', + workspaceId: 'workspace-1', + isCustomTool: true, + contextVariables: { doc: MOUNT_REF }, + }) + + const response = await POST(req) + const data = await response.json() + + expect(response.status).toBe(422) + expect(data.success).toBe(false) + expect(data.error).toContain('custom tools always run in the isolated JavaScript VM') + expect(mockExecuteInSandbox).not.toHaveBeenCalled() + }) + + it('reports a harvest the sandbox refused as a 400 carrying its reason', async () => { + envFlagsMock.isRemoteSandboxEnabled = true + mockExecuteInSandbox.mockRejectedValueOnce( + Object.assign(new Error('Sandbox produced 21 files in /tmp/sim/outputs'), { + code: 'sandbox_output_not_exportable', + }) + ) + + const req = createMockRequest('POST', { + code: 'x', + language: 'python', + workspaceId: 'workspace-1', + }) + + const response = await POST(req) + const data = await response.json() + + // Writing too many files is the caller's to fix, so it must not surface + // as an opaque 500 that hides the count and the remedy. + expect(response.status).toBe(400) + expect(data.error).toContain('21 files') + }) + + it('scans a harvested plaintext secret even under a binary file name', async () => { + envFlagsMock.isRemoteSandboxEnabled = true + mockExecuteInSandbox.mockResolvedValueOnce({ + result: null, + stdout: '', + sandboxId: 'sbx', + collectedFiles: [ + { + path: '/tmp/sim/outputs/leak.png', + relativePath: 'leak.png', + // Valid UTF-8 carrying the resolved secret, named as an image. + contentBase64: Buffer.from('token=super-secret-value').toString('base64'), + byteLength: 24, + }, + ], + }) + + const req = createMockRequest('POST', { + // The placeholder has to be in the code: compiling it is what puts the + // resolved value in scope for the output scan. + code: 'token = {{MY_SECRET}}', + language: 'python', + workspaceId: 'workspace-1', + workflowId: 'workflow-1', + executionId: 'execution-1', + envVars: { MY_SECRET: 'super-secret-value' }, + }) + + const response = await POST(req) + const data = await response.json() + + // Classifying by file name let a secret written as plaintext under a + // binary extension skip the only provenance guard and be returned with a + // downloadable URL. Content decides now, so the name cannot dodge it. + expect(response.status).toBe(400) + expect(data.error).toContain('leak.png') + expect(data.error).toContain('resolved secret') + }) + + it('scans a harvested secret even when one invalid byte makes it non-UTF-8', async () => { + envFlagsMock.isRemoteSandboxEnabled = true + mockExecuteInSandbox.mockResolvedValueOnce({ + result: null, + stdout: '', + sandboxId: 'sbx', + collectedFiles: [ + { + path: '/tmp/sim/outputs/mixed.bin', + relativePath: 'mixed.bin', + // Literal secret plus one invalid byte, so the buffer is not valid + // UTF-8 — which used to be enough to skip the scan entirely. + contentBase64: Buffer.concat([ + Buffer.from('token=super-secret-value'), + Buffer.from([0xff]), + ]).toString('base64'), + byteLength: 25, + }, + ], + }) + + const req = createMockRequest('POST', { + code: 'token = {{MY_SECRET}}', + language: 'python', + workspaceId: 'workspace-1', + workflowId: 'workflow-1', + executionId: 'execution-1', + envVars: { MY_SECRET: 'super-secret-value' }, + }) + + const response = await POST(req) + const data = await response.json() + + // A lossy UTF-8 decode keeps ASCII runs intact, so the literal is still + // there to find — appending a byte must not buy an exemption. + expect(response.status).toBe(400) + expect(data.error).toContain('mixed.bin') + expect(data.error).toContain('resolved secret') + }) + + it('mounts a reference and hands the code its path', async () => { + envFlagsMock.isRemoteSandboxEnabled = true + + const req = createMockRequest('POST', { + code: 'x', + language: 'python', + workspaceId: 'workspace-1', + workflowId: 'workflow-1', + executionId: 'execution-1', + contextVariables: { doc: MOUNT_REF }, + }) + + await POST(req) + + const call = mockExecuteInSandbox.mock.calls[0]?.[0] + expect(call.sandboxFiles).toEqual([ + { type: 'url', path: '/tmp/sim/inputs/doc.pdf', url: 'https://presigned.example/object' }, + ]) + // The marker must not survive into the code's view of the variable — the + // whole point is that every language sees a plain path string. + const runtimePayload = call.privateInputs + .map((input: { content: string }) => input.content) + .find((content: string) => content.includes('contextVariables')) + expect(runtimePayload).toContain('/tmp/sim/inputs/doc.pdf') + expect(runtimePayload).not.toContain('__simSandboxFileMount') + }) + + it('harvests the output directory on every remote run, with no toggle', async () => { + envFlagsMock.isRemoteSandboxEnabled = true + + const req = createMockRequest('POST', { + code: 'x', + language: 'python', + workspaceId: 'workspace-1', + }) + + await POST(req) + + expect(mockExecuteInSandbox.mock.calls[0]?.[0].outputSandboxDir).toBe('/tmp/sim/outputs') + }) + + it('does not ask for an output directory on an isolate run', async () => { + envFlagsMock.isRemoteSandboxEnabled = true + + const req = createMockRequest('POST', { + code: 'return 1', + language: 'javascript', + workspaceId: 'workspace-1', + }) + + await POST(req) + + // Harvesting is free only because it rides an existing sandbox; an + // isolate run must not gain one just to look for files. + expect(mockExecuteInSandbox).not.toHaveBeenCalled() + expect(mockExecuteInIsolatedVM).toHaveBeenCalled() + }) + + it('leaves a plain JavaScript call with no file inputs or outputs in isolated-vm', async () => { + envFlagsMock.isRemoteSandboxEnabled = true + + const req = createMockRequest('POST', { + code: 'return "content"', + language: 'javascript', + workspaceId: 'workspace-1', + }) + + await POST(req) + + expect(mockExecuteInIsolatedVM).toHaveBeenCalled() + expect(mockExecuteInSandbox).not.toHaveBeenCalled() + }) + it('rejects sandbox file mounts when the call would run in isolated-vm', async () => { const req = createMockRequest('POST', { code: 'return 1', @@ -1725,6 +2138,7 @@ describe('Function execution request', () => { result: null, stdout: 'generated 1 preview', sandboxId: 'sandbox-123', + cost: { input: 0, output: 0, total: 0.00034567 }, exportedFiles: { '/tmp/fellows-previews.zip': archiveBase64 }, }) @@ -1733,6 +2147,8 @@ describe('Function execution request', () => { code: source, language: 'python', workspaceId: 'workspace-1', + workflowId: 'workflow-1', + executionId: 'execution-1', sandboxId: 'fellows-sandbox', envVars: { AIRTABLE_PAT: 'stub-airtable-token', @@ -1754,6 +2170,9 @@ describe('Function execution request', () => { ) expect(response.status).toBe(200) + await expect(response.clone().json()).resolves.toMatchObject({ + output: { cost: { input: 0, output: 0, total: 0.00034567 } }, + }) const sandboxRequest = mockExecuteInSandbox.mock.calls[0][0] expect(sandboxRequest.code).toContain("['bq', 'query'") expect(sandboxRequest.code).toContain('__sim_exec_globals__["__name__"] = "__main__"') diff --git a/apps/sim/lib/function-execution/execute-request.ts b/apps/sim/lib/function-execution/execute-request.ts index 443a3fdc0e1..1392221fbd0 100644 --- a/apps/sim/lib/function-execution/execute-request.ts +++ b/apps/sim/lib/function-execution/execute-request.ts @@ -58,6 +58,10 @@ import { readUserFileContent, unavailableLargeValueError, } from '@/lib/execution/payloads/materialization.server' +import { + collectSandboxFileMountRefs, + replaceSandboxFileMountRefs, +} from '@/lib/execution/payloads/sandbox-file-mount-ref' import { compactExecutionPayload } from '@/lib/execution/payloads/serializer' import { materializeLargeValueRef } from '@/lib/execution/payloads/store' import { @@ -76,20 +80,32 @@ import { import { isSandboxOutputFileError, isSandboxOutputLimitError, + isSandboxOutputNotExportableError, MAX_SANDBOX_OUTPUT_BYTES, + readTrustedSandboxOutputCost, } from '@/lib/execution/remote-sandbox/output-limits' +import { + MAX_BLOCK_MOUNTED_FILES, + SANDBOX_OUTPUT_DIR, +} from '@/lib/execution/remote-sandbox/sandbox-paths' +import type { SandboxCollectedFile, SandboxFile } from '@/lib/execution/remote-sandbox/types' import { isExecutionResourceLimitError } from '@/lib/execution/resource-errors' +import { planUserFileMounts, resolveUserFileMounts } from '@/lib/function-execution/sandbox-mounts' +import { uploadExecutionFile } from '@/lib/uploads/contexts/execution/execution-file-manager' import { EXACT_EMPTY_WORKSPACE_FILE_SECRET_PROVENANCE, mergeWorkspaceFileSecretProvenance, type WorkspaceFileSecretProvenance, } from '@/lib/uploads/contexts/workspace/workspace-file-secret-provenance' +import { deleteFiles } from '@/lib/uploads/core/storage-service' +import { getFileExtension, getMimeTypeFromExtension } from '@/lib/uploads/utils/file-utils' import { getWorkflowById } from '@/lib/workflows/utils' import { rebindWorkspaceFileDelegatedPrincipal } from '@/lib/workspace-files/application/delegated-principal' import { fileOperations } from '@/lib/workspace-files/application/operations' import { readWorkspaceFileContent } from '@/lib/workspace-files/application/read-workspace-file-content' import { resolveWorkspaceFileReference } from '@/lib/workspace-files/application/resolve-workspace-file-reference' -import { escapeRegExp, normalizeName, REFERENCE } from '@/executor/constants' +import { escapeRegExp, normalizeName, REFERENCE, sanitizeFileName } from '@/executor/constants' +import type { UserFile } from '@/executor/types' import { type OutputSchema, resolveBlockReference } from '@/executor/utils/block-reference' import { createReferencePattern, @@ -100,6 +116,7 @@ import { type ResolvedSecretMatcher, scanResolvedSecretString, } from '@/executor/utils/resolved-secret-content-projection' +import { isNonIdentifyingSecretLiteral } from '@/executor/utils/resolved-secret-match-policy' import type { ResolvedSecretTraceProvenanceV1 } from '@/executor/utils/resolved-secret-trace-registry' const logger = createLogger('FunctionExecuteAPI') @@ -112,6 +129,12 @@ const MAX_SANDBOX_OUTPUT_FILES = 20 const MAX_PRIVATE_FILE_SECRET_MATCH_EVENTS = 1_000_000 const SANDBOX_RUNTIME_PAYLOAD_PATH_ENV = '__SIM_RUNTIME_PAYLOAD_PATH' +interface FunctionExecutionCost { + input: number + output: number + total: number +} + interface SandboxRuntimePayload { params: Record environmentVariables: Record @@ -1202,11 +1225,25 @@ function activateReferencedSecretProvenance(context: FunctionRouteExecutionConte } } -/** Compiled secret names that still demand redaction — the exempt ones don't count. */ +/** + * Compiled secret names that still demand redaction, and whose value a scan could + * actually find. Exempt names don't count. + * + * Non-identifying literals are excluded on the same predicate + * {@link createResolvedSecretMatcher} uses to drop them, because the two decisions + * have to agree. When every in-scope value is shorter than the substitutable-literal + * minimum, the matcher builds nothing and returns `undefined`; a counter that still + * reported those names would send + * {@link getOutputFileSecretProvenance} down its no-matcher branch and classify + * every output as `unknown` — failing an export while claiming it contains a + * secret that, by that very policy, is too short to be attributed to anything. + */ function countProtectedOutputSecretNames(context: FunctionRouteExecutionContext): number { let count = 0 - for (const name of context.outputSecretPlaintextsByName.keys()) { - if (!context.unredactedSecretNames.has(name)) count += 1 + for (const [name, plaintext] of context.outputSecretPlaintextsByName) { + if (context.unredactedSecretNames.has(name)) continue + if (isNonIdentifyingSecretLiteral(plaintext)) continue + count += 1 } return count } @@ -1402,10 +1439,20 @@ function exportFailure( error: string, status: number, stdout: string, - executionTime: number + executionTime: number, + cost: FunctionExecutionCost | undefined ): NextResponse { return NextResponse.json( - { success: false, error, output: { result: null, stdout: cleanStdout(stdout), executionTime } }, + { + success: false, + error, + output: { + result: null, + stdout: cleanStdout(stdout), + executionTime, + ...(cost ? { cost } : {}), + }, + }, { status } ) } @@ -1428,6 +1475,7 @@ async function maybeExportSandboxFileToWorkspace(args: { exportedFileContent?: string stdout: string executionTime: number + cost?: FunctionExecutionCost }) { const { routeContext, @@ -1443,6 +1491,7 @@ async function maybeExportSandboxFileToWorkspace(args: { exportedFileContent, stdout, executionTime, + cost, } = args if (!outputSandboxPath) return null @@ -1452,7 +1501,8 @@ async function maybeExportSandboxFileToWorkspace(args: { 'outputSandboxPath requires outputPath. Set outputPath to the destination workspace file, e.g. "files/result.csv".', 400, stdout, - executionTime + executionTime, + cost ) } @@ -1464,7 +1514,8 @@ async function maybeExportSandboxFileToWorkspace(args: { 'Workspace context required to save sandbox file to workspace', 400, stdout, - executionTime + executionTime, + cost ) } @@ -1473,7 +1524,8 @@ async function maybeExportSandboxFileToWorkspace(args: { `Sandbox file "${outputSandboxPath}" was not found or could not be read`, 500, stdout, - executionTime + executionTime, + cost ) } @@ -1491,7 +1543,8 @@ async function maybeExportSandboxFileToWorkspace(args: { `Sandbox output files exceed ${MAX_SANDBOX_OUTPUT_BYTES} bytes total`, 400, stdout, - executionTime + executionTime, + cost ) } const fileBuffer = isBinary @@ -1565,6 +1618,7 @@ async function maybeExportSandboxFileToWorkspace(args: { }, stdout: cleanStdout(stdout), executionTime, + ...(cost ? { cost } : {}), }, resources: [{ type: 'file', id: written.id, title: written.name, path: written.vfsPath }], }) @@ -1573,7 +1627,8 @@ async function maybeExportSandboxFileToWorkspace(args: { getErrorMessage(error, 'Failed to export sandbox file'), workspaceFileExportErrorStatus(error), stdout, - executionTime + executionTime, + cost ) } } @@ -1588,6 +1643,7 @@ async function maybeExportSandboxFilesToWorkspace(args: { exportedFileContent?: string stdout: string executionTime: number + cost?: FunctionExecutionCost }) { const sandboxFiles = args.outputFiles.filter((file) => file.sandboxPath) if (sandboxFiles.length === 0) return null @@ -1596,7 +1652,8 @@ async function maybeExportSandboxFilesToWorkspace(args: { `Too many sandbox output files requested (${sandboxFiles.length}). Maximum is ${MAX_SANDBOX_OUTPUT_FILES}.`, 400, args.stdout, - args.executionTime + args.executionTime, + args.cost ) } @@ -1617,6 +1674,7 @@ async function maybeExportSandboxFilesToWorkspace(args: { args.exportedFileContent, stdout: args.stdout, executionTime: args.executionTime, + cost: args.cost, }) } @@ -1628,7 +1686,8 @@ async function maybeExportSandboxFilesToWorkspace(args: { 'Workspace context required to save sandbox files to workspace', 400, args.stdout, - args.executionTime + args.executionTime, + args.cost ) } @@ -1642,7 +1701,8 @@ async function maybeExportSandboxFilesToWorkspace(args: { `Sandbox file "${sandboxPath}" was not found or could not be read`, 500, args.stdout, - args.executionTime + args.executionTime, + args.cost ) } const outputPath = file.formatPath ?? file.path @@ -1659,7 +1719,8 @@ async function maybeExportSandboxFilesToWorkspace(args: { `Sandbox output files exceed ${MAX_SANDBOX_OUTPUT_BYTES} bytes total`, 400, args.stdout, - args.executionTime + args.executionTime, + args.cost ) } const scanBuffer = isBinary ? Buffer.from(content, 'base64') : Buffer.from(content, 'utf-8') @@ -1708,7 +1769,8 @@ async function maybeExportSandboxFilesToWorkspace(args: { getErrorMessage(error, 'Invalid sandbox output destination'), workspaceFileExportErrorStatus(error), args.stdout, - args.executionTime + args.executionTime, + args.cost ) } const duplicateDestination = validationPaths.find( @@ -1719,7 +1781,8 @@ async function maybeExportSandboxFilesToWorkspace(args: { `Duplicate sandbox output destination: ${duplicateDestination}`, 400, args.stdout, - args.executionTime + args.executionTime, + args.cost ) } @@ -1775,7 +1838,8 @@ async function maybeExportSandboxFilesToWorkspace(args: { getErrorMessage(error, 'Failed to export sandbox files'), workspaceFileExportErrorStatus(error), args.stdout, - args.executionTime + args.executionTime, + args.cost ) } @@ -1814,6 +1878,7 @@ async function maybeExportSandboxFilesToWorkspace(args: { }, stdout: cleanStdout(args.stdout), executionTime: args.executionTime, + ...(args.cost ? { cost: args.cost } : {}), }, resources: writtenFiles.map((file) => ({ type: 'file', @@ -1824,6 +1889,182 @@ async function maybeExportSandboxFilesToWorkspace(args: { }) } +/** + * Combines caller-supplied mounts — Copilot resolves its own workspace paths — + * with those resolved from platform file objects. + * + * A duplicate destination is rejected rather than settled by order: + * `writeSandboxInputs` materializes in sequence, so the later entry would + * silently overwrite the earlier one and the code would find something other + * than what it asked for at that path. + */ +function mergeSandboxFileMounts( + callerFiles: SandboxFile[] | undefined, + resolvedFiles: SandboxFile[] +): SandboxFile[] | undefined { + if (!callerFiles?.length) return resolvedFiles.length > 0 ? resolvedFiles : undefined + if (resolvedFiles.length === 0) return callerFiles + + const merged = [...callerFiles, ...resolvedFiles] + const seen = new Set() + for (const file of merged) { + if (seen.has(file.path)) { + throw new Error(`Duplicate sandbox mount path: ${file.path}`) + } + seen.add(file.path) + } + return merged +} + +/** + * A harvested file's name, derived from its path relative to the output + * directory. Subdirectories are folded into the name rather than dropped, so + * `reports/q4.csv` and `q4.csv` stay distinguishable — and a `/` never survives + * into a name that later reaches an email attachment or an upload filename. + */ +function collectedFileName(relativePath: string): string { + return sanitizeFileName(relativePath.split('/').filter(Boolean).join('-')) || 'file' +} + +/** + * Persists files harvested from the sandbox output directory as platform file + * objects, so any downstream tool that accepts a file can consume them. + * + * Uploaded here, one at a time, rather than handed to the declarative + * file-output pipeline as bytes: that path would carry the whole export budget + * as base64 through `JSON.stringify`, a response buffer, and a re-parse, so + * several multiples of the payload would be live at once for a value that is a + * couple of hundred bytes per file once stored. + */ +/** + * Removes files already uploaded when a later one in the same harvest is refused. + * + * The route answers with a failure and hands back no references, so anything + * uploaded before the refusal is unreachable — but it still occupies storage, + * and the harvest is all-or-nothing by design. Best-effort on purpose: the + * caller needs to hear why its export was refused, not that the tidy-up failed. + */ +async function discardUploadedExecutionFiles(files: readonly UserFile[]): Promise { + if (files.length === 0) return + try { + await deleteFiles( + files.map((file) => file.key), + 'execution' + ) + } catch (error) { + logger.warn('Could not remove partially uploaded sandbox output files', { + fileCount: files.length, + error: getErrorMessage(error), + }) + } +} + +async function collectExecutionOutputFiles(args: { + routeContext: FunctionRouteExecutionContext + authUserId: string + workflowId?: string + workspaceId?: string + executionId?: string + collectedFiles: SandboxCollectedFile[] + stdout: string + executionTime: number + cost?: FunctionExecutionCost +}): Promise<{ files: UserFile[] } | { response: NextResponse }> { + const { routeContext, collectedFiles } = args + if (collectedFiles.length === 0) return { files: [] } + + const resolvedWorkspaceId = + args.workspaceId || + (args.workflowId ? (await getWorkflowById(args.workflowId))?.workspaceId : undefined) + + // Fails rather than returning an empty list: the code did produce files, and + // reporting success without them would read as "your script wrote nothing". + if (!resolvedWorkspaceId || !args.workflowId || !args.executionId) { + return { + response: exportFailure( + 'Workspace, workflow, and execution context are required to return files from the sandbox.', + 400, + args.stdout, + args.executionTime, + args.cost + ), + } + } + + const files: UserFile[] = [] + // The harvest is all-or-nothing, so a throw partway through has to take the + // uploads that already succeeded with it. Without this they linger in storage + // with nothing referencing them, since the failure response carries no keys. + try { + for (const collected of args.collectedFiles) { + const buffer = Buffer.from(collected.contentBase64, 'base64') + const name = collectedFileName(collected.relativePath) + const mimeType = getMimeTypeFromExtension(getFileExtension(name)) + + // Scanned unconditionally — never gated on whether the bytes look textual. + // Both a filename check and a UTF-8 round-trip were trivially defeated: name + // the file `.png`, or append one invalid byte, and a plaintext secret sailed + // past. A lossy UTF-8 decode preserves ASCII runs, so a literal secret is + // findable in any buffer, textual or not. + // + // What stays out of reach is a secret carried in transformed form — deflated + // inside a PDF, re-encoded — which no substring scan can see. That is an + // inherent limit of scanning, not a hole in the gate, and it is why these + // files are execution-scoped rather than durable workspace files. + { + const provenance = await getOutputFileSecretProvenance(buffer, false, routeContext, { + userId: args.authUserId, + workspaceId: resolvedWorkspaceId, + }) + // An execution-scoped file has nowhere to record a provenance envelope, so + // one carrying a resolved secret cannot ship under a lock the way a + // workspace file can — it is refused instead. + if (provenance.status !== 'exact' || provenance.entries.length > 0) { + await discardUploadedExecutionFiles(files) + return { + response: exportFailure( + `Sandbox output file "${name}" contains a resolved secret value and was not returned. Write the file without embedding secret values, or export it to a workspace file where its provenance can be recorded.`, + 400, + args.stdout, + args.executionTime, + args.cost + ), + } + } + } + + const userFile = await uploadExecutionFile( + { + workspaceId: resolvedWorkspaceId, + workflowId: args.workflowId, + executionId: args.executionId, + }, + buffer, + name, + mimeType, + args.authUserId + ) + files.push(userFile) + } + } catch (error) { + await discardUploadedExecutionFiles(files) + throw error + } + + // Registers the new keys on the execution so downstream blocks are authorized + // to read them back. + routeContext.fileKeys = [ + ...new Set([...(routeContext.fileKeys ?? []), ...files.map((file) => file.key)]), + ] + + logger.info('Returned sandbox output files', { + fileCount: files.length, + totalBytes: files.reduce((total, file) => total + file.size, 0), + }) + + return { files } +} + export interface TrustedFunctionExecutionAuth { attributedUserId: string fileAccessUserId?: string @@ -1924,9 +2165,12 @@ export async function executeFunctionRequest( allowLargeValueWorkflowScope = false, workspaceId, isCustomTool = false, + files: mountedUserFiles, _sandboxFiles, } = body + const meterRemoteSandboxUsage = Boolean(workflowId && !isCustomTool && !usesMothershipSandbox) + if (selectedSandboxId && !isRemoteSandboxEnabled) { return NextResponse.json( { success: false, error: 'The Function code sandbox is not configured' }, @@ -1985,6 +2229,27 @@ export async function executeFunctionRequest( privateResolvedSecretNamesMetadataType ) } + try { + for (const file of outputFiles) { + normalizeOutputWorkspaceFileName(file.formatPath ?? file.path) + } + } catch (error) { + return appendPrivateResolvedSecretNames( + NextResponse.json( + { + success: false, + error: getErrorMessage(error, 'Invalid sandbox output destination'), + }, + { status: 400 } + ), + includePrivateResolvedSecretNames ? [] : null, + privateResolvedSecretNamesMetadataType + ) + } + + // Planned before the runtime is chosen because it is pure: it decides whether + // this execution needs a sandbox filesystem at all, without spending a presign + // or a byte of transfer on a request the guard below may still refuse. const executionParams = { ...params } executionParams._context = undefined @@ -2040,6 +2305,34 @@ export async function executeFunctionRequest( ...codeResolution.contextVariables, ...preResolvedContextVariables, } + + /** + * Files this run must place on the sandbox filesystem: those a caller passed + * explicitly — how an agent supplies one, since a model cannot write a block + * reference — plus every file the code asked for with ``, + * which arrives as a marker inside the resolved context variables. + */ + const plannedFileMounts = planUserFileMounts([ + ...((mountedUserFiles ?? []) as UserFile[]), + ...collectSandboxFileMountRefs(contextVariables), + ]) + if (plannedFileMounts.length > MAX_BLOCK_MOUNTED_FILES) { + return functionJsonResponse( + { + success: false, + error: `Too many files mounted into the sandbox (${plannedFileMounts.length}). Maximum is ${MAX_BLOCK_MOUNTED_FILES}.`, + output: { result: null, stdout: '', executionTime: Date.now() - startTime }, + }, + routeContext, + { status: 400 } + ) + } + const requestsSandboxFilesystem = + plannedFileMounts.length > 0 || + Boolean(_sandboxFiles?.length) || + outputSandboxPaths.length > 0 || + Boolean(outputSandboxPath) + const compilation = await compileCodePlaceholders({ code: codeResolution.resolvedCode, language: lang, @@ -2104,13 +2397,143 @@ export async function executeFunctionRequest( hasImports = jsImports.trim().length > 0 || extractionResult.hasRequireCalls } - if (lang === CodeLanguage.Shell) { - if (!remoteSandboxEnabled) { - throw new Error( - 'Shell execution requires a remote code sandbox to be enabled. Please contact your administrator to enable it.' - ) - } + if (lang === CodeLanguage.Shell && !remoteSandboxEnabled) { + throw new Error( + 'Shell execution requires a remote code sandbox to be enabled. Please contact your administrator to enable it.' + ) + } + + if (lang === CodeLanguage.Python && !remoteSandboxEnabled) { + throw new Error( + 'Python execution requires a remote code sandbox to be enabled. Please contact your administrator to enable it, or use JavaScript instead.' + ) + } + if (lang === CodeLanguage.JavaScript && hasImports && !remoteSandboxEnabled) { + throw new Error( + 'JavaScript code with import statements requires a remote code sandbox to be enabled. Please remove the import statements, or contact your administrator to enable it.' + ) + } + + /** + * Mounting files or harvesting outputs needs a real filesystem, so it selects + * the remote sandbox the same way a selected sandbox image does. Without this + * a plain-JavaScript block that merely attaches a file would land in + * isolated-vm and be refused by the guard below — a dead end, since "add an + * import" is not a fix a caller should have to discover. + */ + const useRemoteSandbox = + usesMothershipSandbox || + (remoteSandboxEnabled && + !isCustomTool && + (lang === CodeLanguage.Shell || + lang === CodeLanguage.Python || + (lang === CodeLanguage.JavaScript && + (hasImports || Boolean(selectedSandboxId) || requestsSandboxFilesystem)))) + + if (useRemoteSandbox && containsLargeValueRef(contextVariables)) { + throw new Error( + 'Large execution values require the JavaScript isolated-vm runtime. Remove imports, select a nested field, or read the value in a JavaScript function without a remote sandbox.' + ) + } + + // Sandbox file mounts and file exports only exist in the remote sandbox + // runtime; isolated-vm has no filesystem. Silently dropping a declared + // sandbox input/output here produced "export succeeded" responses with zero + // bytes written, so refuse the call instead. Widening `useRemoteSandbox` + // above means the only ways to arrive here are a deployment with no remote + // sandbox at all, or a custom tool — which is why neither remediation + // suggests switching language. + if (!useRemoteSandbox && requestsSandboxFilesystem) { + const remediation = !remoteSandboxEnabled + ? "No remote code sandbox is enabled on this deployment, so there is no sandbox filesystem for any language. Pass input data via params and return output as the code's return value with outputs.files[].path (no sandboxPath)." + : "custom tools always run in the isolated JavaScript VM, which has no sandbox filesystem. Pass input data via params and return output as the code's return value." + return functionJsonResponse( + { + success: false, + error: `Sandbox file inputs/outputs are unavailable for this call: ${remediation}`, + output: { result: null, stdout: '', executionTime: Date.now() - startTime }, + }, + routeContext, + { status: 422 } + ) + } + + // Resolved only after the guard: a request about to be refused must not mint + // presigned URLs or buffer bytes on its way out. + let resolvedMounts: Awaited> + try { + resolvedMounts = await resolveUserFileMounts({ + planned: plannedFileMounts, + context: { + principal: auth.principal, + workflowId, + workspaceId, + executionId, + largeValueExecutionIds, + largeValueKeys, + fileKeys, + allowLargeValueWorkflowScope, + userId: auth.fileAccessUserId, + requestId, + logger, + }, + }) + } catch (error) { + // Everything this can raise is about the files the caller named — a mount + // it may not read, one over a size ceiling, a set over the aggregate. The + // messages already say which file and what to do, so they are the response + // rather than a 500 that reads like the platform broke. Matches the + // too-many-files refusal above. + logger.warn(`[${requestId}] Could not resolve sandbox file mounts`, { + error: getErrorMessage(error), + }) + return functionJsonResponse( + { + success: false, + error: getErrorMessage(error, 'Could not mount the requested files into the sandbox.'), + output: { result: null, stdout: '', executionTime: Date.now() - startTime }, + }, + routeContext, + { status: 400 } + ) + } + const { sandboxFiles: userFileMounts, manifest: mountManifest } = resolvedMounts + const sandboxFiles = mergeSandboxFileMounts(_sandboxFiles, userFileMounts) + + // Every `` marker becomes the path its file was mounted at, + // so the code reads a plain string in whichever language it is written in. + const mountPathsByKey = new Map( + plannedFileMounts.map(({ userFile, mountPath }) => [userFile.key, mountPath]) + ) + for (const [name, value] of Object.entries(contextVariables)) { + contextVariables[name] = replaceSandboxFileMountRefs( + value, + (file) => mountPathsByKey.get(file.key) ?? file.name + ) + } + + // Harvested on every remote run rather than behind a switch: the directory is + // Sim's own, so nothing lands there unless the code put it there, and the cost + // is one listing on a run that already paid for a sandbox. Isolate runs never + // reach here, so they stay as fast as they were. + // + // Declared sandbox outputs opt out. That request names exactly which paths to + // export and answers with that export's own result, so harvesting alongside it + // would collect files the response has no shape to carry — they would be read, + // scanned, uploaded, and then dropped. Making the exclusion explicit here keeps + // it from resting on which branch happens to return first. + const declaresSandboxOutputs = outputFiles.some((file) => file.sandboxPath) + const outputSandboxDir = + useRemoteSandbox && !declaresSandboxOutputs ? SANDBOX_OUTPUT_DIR : undefined + + if (mountManifest.length > 0) { + logger.info(`[${requestId}] Mounted files into sandbox`, { + mountCount: mountManifest.length, + }) + } + + if (lang === CodeLanguage.Shell) { const shellEnvs: Record = {} for (const [k, v] of Object.entries(envVars)) { shellEnvs[k] = serializeForShellEnv(v) @@ -2133,20 +2556,24 @@ export async function executeFunctionRequest( error: shellError, exportedFileContent, exportedFiles, + collectedFiles: shellCollectedFiles, + cost: shellCost, } = await executeShellInSandbox({ code: resolvedCode, envs: shellEnvs, timeoutMs: timeout, - sandboxFiles: _sandboxFiles, + sandboxFiles, privateInputs: compilerPrivateInputs, outputSandboxPath, outputSandboxPaths, + outputSandboxDir, workspaceId, sandboxId: selectedSandboxId, ...(usesMothershipSandbox && !selectedSandboxId ? { sandboxKind: 'mothership' as const } : {}), signal: executionSignal, + meterUsage: meterRemoteSandboxUsage, }) const executionTime = Date.now() - execStart @@ -2161,7 +2588,12 @@ export async function executeFunctionRequest( { success: false, error: scrubInternalIdentifiers(shellError, compilerInternalIdentifiers), - output: { result: null, stdout: cleanStdout(shellStdout), executionTime }, + output: { + result: null, + stdout: cleanStdout(shellStdout), + executionTime, + ...(shellCost ? { cost: shellCost } : {}), + }, }, routeContext, { status: 422 } @@ -2179,72 +2611,43 @@ export async function executeFunctionRequest( exportedFileContent, stdout: shellStdout, executionTime, + cost: shellCost, }) if (fileExportResponse) { return appendResolvedSecretNames(fileExportResponse, routeContext) } } + const shellOutputFiles = await collectExecutionOutputFiles({ + routeContext, + authUserId: auth.attributedUserId, + workflowId, + workspaceId, + executionId, + collectedFiles: shellCollectedFiles ?? [], + stdout: shellStdout, + executionTime, + cost: shellCost, + }) + if ('response' in shellOutputFiles) { + return appendResolvedSecretNames(shellOutputFiles.response, routeContext) + } + return functionJsonResponse( { success: true, - output: { result: shellResult ?? null, stdout: cleanStdout(shellStdout), executionTime }, + output: { + result: shellResult ?? null, + stdout: cleanStdout(shellStdout), + executionTime, + files: shellOutputFiles.files, + ...(shellCost ? { cost: shellCost } : {}), + }, }, routeContext ) } - if (lang === CodeLanguage.Python && !remoteSandboxEnabled) { - throw new Error( - 'Python execution requires a remote code sandbox to be enabled. Please contact your administrator to enable it, or use JavaScript instead.' - ) - } - - if (lang === CodeLanguage.JavaScript && hasImports && !remoteSandboxEnabled) { - throw new Error( - 'JavaScript code with import statements requires a remote code sandbox to be enabled. Please remove the import statements, or contact your administrator to enable it.' - ) - } - - const useRemoteSandbox = - usesMothershipSandbox || - (remoteSandboxEnabled && - !isCustomTool && - (lang === CodeLanguage.Python || - (lang === CodeLanguage.JavaScript && (hasImports || Boolean(selectedSandboxId))))) - - if (useRemoteSandbox && containsLargeValueRef(contextVariables)) { - throw new Error( - 'Large execution values require the JavaScript isolated-vm runtime. Remove imports, select a nested field, or read the value in a JavaScript function without a remote sandbox.' - ) - } - - // Sandbox file mounts and sandboxPath exports only exist in the remote - // sandbox runtime; isolated-vm has no filesystem. Silently dropping a declared - // sandbox input/output here produced "export succeeded" responses with - // zero bytes written, so refuse the call instead. The remediation depends - // on WHY this call runs in isolated-vm — "switch to python" is a dead end - // when no remote sandbox is enabled or the call is a custom tool. - if ( - !useRemoteSandbox && - (outputSandboxPaths.length > 0 || outputSandboxPath || _sandboxFiles?.length) - ) { - const remediation = !remoteSandboxEnabled - ? "No remote code sandbox is enabled on this deployment, so there is no sandbox filesystem for any language. Pass input data via params and return output as the code's return value with outputs.files[].path (no sandboxPath)." - : isCustomTool - ? "custom tools always run in the isolated JavaScript VM, which has no sandbox filesystem. Pass input data via params and return output as the code's return value." - : 'plain JavaScript runs in the isolated VM, which has no sandbox filesystem. Use language "python" so the code runs in the remote sandbox, or drop sandboxPath and return the file content as the code\'s return value with outputs.files[].path.' - return functionJsonResponse( - { - success: false, - error: `Sandbox file inputs/outputs are unavailable for this call: ${remediation}`, - output: { result: null, stdout: '', executionTime: Date.now() - startTime }, - }, - routeContext, - { status: 422 } - ) - } - if (useRemoteSandbox) { logger.info(`[${requestId}] E2B status`, { enabled: remoteSandboxEnabled, @@ -2300,21 +2703,25 @@ export async function executeFunctionRequest( error: e2bError, exportedFileContent, exportedFiles, + collectedFiles: jsCollectedFiles, + cost: sandboxCost, } = await executeInSandbox({ code: codeForE2B, language: CodeLanguage.JavaScript, timeoutMs: timeout, - sandboxFiles: _sandboxFiles, + sandboxFiles, privateInputs: [...compilerPrivateInputs, runtimePrivateInput], runtimeBindings: compilerRuntimeBindings, outputSandboxPath, outputSandboxPaths, + outputSandboxDir, workspaceId, sandboxId: selectedSandboxId, ...(usesMothershipSandbox && !selectedSandboxId ? { sandboxKind: 'mothership' as const } : {}), signal: executionSignal, + meterUsage: meterRemoteSandboxUsage, }) const executionTime = Date.now() - execStart stdout += e2bStdout @@ -2340,7 +2747,12 @@ export async function executeFunctionRequest( { success: false, error: formattedError, - output: { result: null, stdout: cleanedOutput, executionTime }, + output: { + result: null, + stdout: cleanedOutput, + executionTime, + ...(sandboxCost ? { cost: sandboxCost } : {}), + }, }, routeContext, { status: 422 } @@ -2358,16 +2770,38 @@ export async function executeFunctionRequest( exportedFileContent, stdout, executionTime, + cost: sandboxCost, }) if (fileExportResponse) { return appendResolvedSecretNames(fileExportResponse, routeContext) } } + const jsOutputFiles = await collectExecutionOutputFiles({ + routeContext, + authUserId: auth.attributedUserId, + workflowId, + workspaceId, + executionId, + collectedFiles: jsCollectedFiles ?? [], + stdout, + executionTime, + cost: sandboxCost, + }) + if ('response' in jsOutputFiles) { + return appendResolvedSecretNames(jsOutputFiles.response, routeContext) + } + return functionJsonResponse( { success: true, - output: { result: e2bResult ?? null, stdout: cleanStdout(stdout), executionTime }, + output: { + result: e2bResult ?? null, + stdout: cleanStdout(stdout), + executionTime, + files: jsOutputFiles.files, + ...(sandboxCost ? { cost: sandboxCost } : {}), + }, }, routeContext ) @@ -2391,20 +2825,24 @@ export async function executeFunctionRequest( error: e2bError, exportedFileContent, exportedFiles, + collectedFiles: pythonCollectedFiles, + cost: sandboxCost, } = await executeInSandbox({ code: codeForE2B, language: CodeLanguage.Python, timeoutMs: timeout, - sandboxFiles: _sandboxFiles, + sandboxFiles, privateInputs: [...compilerPrivateInputs, runtimePrivateInput], outputSandboxPath, outputSandboxPaths, + outputSandboxDir, workspaceId, sandboxId: selectedSandboxId, ...(usesMothershipSandbox && !selectedSandboxId ? { sandboxKind: 'mothership' as const } : {}), signal: executionSignal, + meterUsage: meterRemoteSandboxUsage, }) const executionTime = Date.now() - execStart stdout += e2bStdout @@ -2430,7 +2868,12 @@ export async function executeFunctionRequest( { success: false, error: formattedError, - output: { result: null, stdout: cleanedOutput, executionTime }, + output: { + result: null, + stdout: cleanedOutput, + executionTime, + ...(sandboxCost ? { cost: sandboxCost } : {}), + }, }, routeContext, { status: 422 } @@ -2448,16 +2891,38 @@ export async function executeFunctionRequest( exportedFileContent, stdout, executionTime, + cost: sandboxCost, }) if (fileExportResponse) { return appendResolvedSecretNames(fileExportResponse, routeContext) } } + const pythonOutputFiles = await collectExecutionOutputFiles({ + routeContext, + authUserId: auth.attributedUserId, + workflowId, + workspaceId, + executionId, + collectedFiles: pythonCollectedFiles ?? [], + stdout, + executionTime, + cost: sandboxCost, + }) + if ('response' in pythonOutputFiles) { + return appendResolvedSecretNames(pythonOutputFiles.response, routeContext) + } + return functionJsonResponse( { success: true, - output: { result: e2bResult ?? null, stdout: cleanStdout(stdout), executionTime }, + output: { + result: e2bResult ?? null, + stdout: cleanStdout(stdout), + executionTime, + files: pythonOutputFiles.files, + ...(sandboxCost ? { cost: sandboxCost } : {}), + }, }, routeContext ) @@ -2636,11 +3101,21 @@ export async function executeFunctionRequest( privateResolvedSecretNamesMetadataType ) } - if (isSandboxOutputLimitError(error) || isSandboxOutputFileError(error)) { + if ( + isSandboxOutputLimitError(error) || + isSandboxOutputFileError(error) || + isSandboxOutputNotExportableError(error) + ) { + const cost = readTrustedSandboxOutputCost(error) const outputLimitResponse = { success: false, error: error.message, - output: { result: null, stdout: cleanStdout(stdout), executionTime }, + output: { + result: null, + stdout: cleanStdout(stdout), + executionTime, + ...(cost ? { cost } : {}), + }, } return routeContext ? functionJsonResponse(outputLimitResponse, routeContext, { status: 400 }) diff --git a/apps/sim/lib/function-execution/sandbox-mounts.test.ts b/apps/sim/lib/function-execution/sandbox-mounts.test.ts new file mode 100644 index 00000000000..2f49c989885 --- /dev/null +++ b/apps/sim/lib/function-execution/sandbox-mounts.test.ts @@ -0,0 +1,272 @@ +/** + * @vitest-environment node + * + * Mount resolution for platform file objects. The authorization assertions run + * against the real `assertUserFileContentAccess` rather than a stub: which files + * a Function block may mount is the security-relevant part of this module, and + * mocking it away would leave exactly that untested. + */ +import { beforeEach, describe, expect, it, vi } from 'vitest' +import type { UserFile } from '@/executor/types' + +const { + mockHasCloudStorage, + mockGeneratePresignedDownloadUrl, + mockDownloadServableFileFromStorage, + mockReadWorkspaceFileRecordByKey, +} = vi.hoisted(() => ({ + mockHasCloudStorage: vi.fn(), + mockGeneratePresignedDownloadUrl: vi.fn(), + mockDownloadServableFileFromStorage: vi.fn(), + mockReadWorkspaceFileRecordByKey: vi.fn(), +})) + +vi.mock('@/lib/uploads/core/storage-service', () => ({ + hasCloudStorage: mockHasCloudStorage, + generatePresignedDownloadUrl: mockGeneratePresignedDownloadUrl, +})) + +vi.mock('@/lib/uploads/utils/file-utils.server', () => ({ + downloadServableFileFromStorage: mockDownloadServableFileFromStorage, +})) + +vi.mock('@/lib/workspace-files/application/read-workspace-file-content-by-key', () => ({ + readWorkspaceFileRecordByKey: { execute: mockReadWorkspaceFileRecordByKey }, +})) + +import { + MOUNT_URL_TTL_SECONDS, + planUserFileMounts, + resolveUserFileMounts, +} from '@/lib/function-execution/sandbox-mounts' + +const WORKSPACE_ID = 'ws-1' +const WORKFLOW_ID = 'wf-1' +const EXECUTION_ID = 'exec-1' + +function executionFile(overrides: Partial = {}): UserFile { + return { + id: 'file_1', + name: 'report.csv', + url: 'https://storage.example/report.csv', + size: 32, + type: 'text/csv', + key: `execution/${WORKSPACE_ID}/${WORKFLOW_ID}/${EXECUTION_ID}/abc/report.csv`, + context: 'execution', + ...overrides, + } +} + +function workspaceFile(overrides: Partial = {}): UserFile { + return { + id: 'wf_1', + name: 'brief.pdf', + url: 'https://storage.example/brief.pdf', + size: 64, + type: 'application/pdf', + key: `workspace/${WORKSPACE_ID}/brief.pdf`, + context: 'workspace', + ...overrides, + } +} + +const executionContext = { + workspaceId: WORKSPACE_ID, + workflowId: WORKFLOW_ID, + executionId: EXECUTION_ID, + userId: 'user-1', + requestId: 'req-1', +} + +describe('planUserFileMounts', () => { + it('sanitizes names into a single safe path segment', () => { + const planned = planUserFileMounts([executionFile({ name: 'Q4 Sales (Final).csv' })]) + + expect(planned[0].mountPath).toBe('/tmp/sim/inputs/Q4-Sales-_Final_.csv') + }) + + it('cannot be escaped by a traversal in the file name', () => { + const planned = planUserFileMounts([ + executionFile({ name: '../../etc/passwd' }), + executionFile({ id: 'file_2', key: 'execution/other', name: '..' }), + ]) + + for (const { mountPath } of planned) { + expect(mountPath.startsWith('/tmp/sim/inputs/')).toBe(true) + expect(mountPath).not.toContain('/../') + expect(mountPath.endsWith('/..')).toBe(false) + } + }) + + it('suffixes colliding names so neither file is silently overwritten', () => { + const planned = planUserFileMounts([ + executionFile({ id: 'file_1', key: 'execution/a/report.csv', name: 'report.csv' }), + executionFile({ id: 'file_2', key: 'execution/b/report.csv', name: 'report.csv' }), + executionFile({ id: 'file_3', key: 'execution/c/report.csv', name: 'report.csv' }), + ]) + + expect(planned.map((entry) => entry.mountPath)).toEqual([ + '/tmp/sim/inputs/report.csv', + '/tmp/sim/inputs/report-2.csv', + '/tmp/sim/inputs/report-3.csv', + ]) + }) + + it('mounts one storage key once however many sources named it', () => { + // A caller listing the same file twice, and a `` marker for + // a file the caller also passed explicitly, both land in one list here. A + // second copy of identical bytes costs a presign and a duplicate transfer, + // and charges the byte budget and the 20-file ceiling twice over. + const planned = planUserFileMounts([ + executionFile({ id: 'file_1', name: 'report.csv' }), + executionFile({ id: 'file_1_again', name: 'report.csv' }), + executionFile({ id: 'file_2', name: 'renamed.csv' }), + workspaceFile(), + ]) + + expect(planned.map((entry) => entry.mountPath)).toEqual([ + '/tmp/sim/inputs/report.csv', + '/tmp/sim/inputs/brief.pdf', + ]) + }) +}) + +describe('resolveUserFileMounts', () => { + beforeEach(() => { + vi.clearAllMocks() + mockHasCloudStorage.mockReturnValue(true) + mockGeneratePresignedDownloadUrl.mockResolvedValue('https://presigned.example/object') + mockReadWorkspaceFileRecordByKey.mockResolvedValue({ file: { id: 'wf_1' } }) + // Sized from the file being read: the aggregate budget counts bytes actually + // buffered, so a fixed-size stub would never let the total ceiling trip. + mockDownloadServableFileFromStorage.mockImplementation(async (file: UserFile) => ({ + buffer: file.size > 16 ? Buffer.alloc(file.size) : Buffer.from('a,b\n1,2'), + contentType: file.type, + })) + }) + + it('mounts by presigned URL when cloud storage is configured', async () => { + const planned = planUserFileMounts([executionFile()]) + + const { sandboxFiles, manifest } = await resolveUserFileMounts({ + planned, + context: executionContext, + }) + + // The sandbox fetches the bytes itself, so nothing transits the web process. + expect(sandboxFiles).toEqual([ + { + type: 'url', + path: '/tmp/sim/inputs/report.csv', + url: 'https://presigned.example/object', + // Granted exactly what the mount was charged against the aggregate, so + // an understated size is refused rather than silently overrunning it. + maxBytes: 32, + }, + ]) + expect(mockGeneratePresignedDownloadUrl).toHaveBeenCalledWith( + planned[0].userFile.key, + 'execution', + MOUNT_URL_TTL_SECONDS + ) + expect(mockDownloadServableFileFromStorage).not.toHaveBeenCalled() + expect(manifest).toEqual([ + { name: 'report.csv', path: '/tmp/sim/inputs/report.csv', size: 32, type: 'text/csv' }, + ]) + }) + + it('buffers bytes inline when there is no cloud storage to presign from', async () => { + mockHasCloudStorage.mockReturnValue(false) + + const { sandboxFiles } = await resolveUserFileMounts({ + planned: planUserFileMounts([executionFile()]), + context: executionContext, + }) + + // A presigned URL under local storage is an app-internal serve path the + // remote sandbox cannot reach. + expect(mockGeneratePresignedDownloadUrl).not.toHaveBeenCalled() + expect(sandboxFiles).toEqual([ + { + path: '/tmp/sim/inputs/report.csv', + content: Buffer.alloc(32).toString('base64'), + encoding: 'base64', + }, + ]) + }) + + it('rejects a file over the per-file mount ceiling before presigning it', async () => { + await expect( + resolveUserFileMounts({ + planned: planUserFileMounts([executionFile({ size: 600 * 1024 * 1024 })]), + context: executionContext, + }) + ).rejects.toThrow(/per-file mount limit/) + + expect(mockGeneratePresignedDownloadUrl).not.toHaveBeenCalled() + }) + + it('rejects a batch over the total inline budget', async () => { + mockHasCloudStorage.mockReturnValue(false) + + await expect( + resolveUserFileMounts({ + planned: planUserFileMounts( + ['a', 'b', 'c', 'd', 'e', 'f'].map((id) => + executionFile({ + id, + key: `execution/${WORKSPACE_ID}/${WORKFLOW_ID}/${EXECUTION_ID}/${id}/${id}.bin`, + name: `${id}.bin`, + size: 9 * 1024 * 1024, + }) + ) + ), + context: executionContext, + }) + ).rejects.toThrow(/total mount limit/) + }) + + it('authorizes a design-time workspace upload through its workspace record', async () => { + const { sandboxFiles } = await resolveUserFileMounts({ + planned: planUserFileMounts([workspaceFile()]), + context: { ...executionContext, principal: { kind: 'sim_user' } as never }, + }) + + // The common case for a Function block: a file pinned in the block config is + // a workspace key, which never touches the execution-scope check at all. + expect(mockReadWorkspaceFileRecordByKey).toHaveBeenCalledWith( + expect.objectContaining({ + input: expect.objectContaining({ key: `workspace/${WORKSPACE_ID}/brief.pdf` }), + }) + ) + expect(sandboxFiles).toHaveLength(1) + }) + + it('refuses an execution file belonging to a different workflow', async () => { + const foreign = executionFile({ + key: `execution/${WORKSPACE_ID}/other-workflow/other-exec/xyz/secrets.csv`, + }) + + await expect( + resolveUserFileMounts({ + planned: planUserFileMounts([foreign]), + context: executionContext, + }) + ).rejects.toThrow(/not available in this execution/) + + expect(mockGeneratePresignedDownloadUrl).not.toHaveBeenCalled() + }) + + it('admits an execution file from another run when its key is in the allowlist', async () => { + const priorRun = executionFile({ + key: `execution/${WORKSPACE_ID}/${WORKFLOW_ID}/earlier-exec/xyz/prior.csv`, + }) + + const { sandboxFiles } = await resolveUserFileMounts({ + planned: planUserFileMounts([priorRun]), + context: { ...executionContext, fileKeys: [priorRun.key] }, + }) + + expect(sandboxFiles).toHaveLength(1) + }) +}) diff --git a/apps/sim/lib/function-execution/sandbox-mounts.ts b/apps/sim/lib/function-execution/sandbox-mounts.ts new file mode 100644 index 00000000000..cbad1fa2859 --- /dev/null +++ b/apps/sim/lib/function-execution/sandbox-mounts.ts @@ -0,0 +1,325 @@ +import { createLogger } from '@sim/logger' +import { + assertUserFileContentAccess, + type ExecutionMaterializationContext, + readUserFileContentWithContributors, +} from '@/lib/execution/payloads/materialization.server' +import { MAX_SANDBOX_URL_MOUNT_BYTES } from '@/lib/execution/remote-sandbox/output-limits' +import { SANDBOX_INPUT_DIR } from '@/lib/execution/remote-sandbox/sandbox-paths' +import type { SandboxFile } from '@/lib/execution/remote-sandbox/types' +import { buildStorageKeySegment } from '@/lib/uploads/core/storage-key' +import { generatePresignedDownloadUrl, hasCloudStorage } from '@/lib/uploads/core/storage-service' +import type { StorageContext } from '@/lib/uploads/shared/types' +import { + isGeneratedDocumentSourceType, + resolveTrustedFileContext, +} from '@/lib/uploads/utils/file-utils' +import type { UserFile } from '@/executor/types' + +const logger = createLogger('SandboxMounts') + +/** + * Lifetime of a presigned URL handed to the sandbox to fetch a mounted object. + * The URL grants read to exactly that one object and dies with the sandbox. + * + * Sized well past the worst provisioning path rather than the typical one: a + * runtime-strategy sandbox can spend up to RUNTIME_INSTALL_TIMEOUT_MS installing + * dependencies, and only then does the in-sandbox `curl` start its own 300s + * window. At the previous 600s the URL could expire mid-download and surface as + * an opaque "failed to fetch mounted file". + */ +export const MOUNT_URL_TTL_SECONDS = 1800 + +/** + * Per-file ceiling for URL-mounted files, shared with the sandbox layer that + * enforces it on the transferred bytes so the pre-check and the backstop can + * never drift apart. + */ +export const MOUNT_URL_MAX_BYTES = MAX_SANDBOX_URL_MOUNT_BYTES + +/** + * Aggregate ceiling across all URL mounts in one request. Rejects an oversized + * request up front instead of filling the sandbox disk one slow fetch at a time. + */ +export const MAX_TOTAL_URL_BYTES = 2 * 1024 * 1024 * 1024 + +/** Per-file ceiling when bytes must pass through the web process. */ +export const MAX_INLINE_MOUNT_FILE_BYTES = 10 * 1024 * 1024 + +/** Aggregate ceiling for buffered mounts, bounding web heap rather than disk. */ +export const MAX_INLINE_MOUNT_TOTAL_BYTES = 50 * 1024 * 1024 + +/** + * Running byte totals for one resolve pass. `buffered` bytes pass through the web + * process; `url` bytes are fetched straight into the sandbox. Tracked separately + * because the two ceilings protect different resources — web heap vs sandbox disk. + */ +export interface SandboxMountBudget { + buffered: number + url: number +} + +export function createSandboxMountBudget(): SandboxMountBudget { + return { buffered: 0, url: 0 } +} + +/** One object to mount, independent of how the caller located it. */ +export interface SandboxMountSource { + mountPath: string + key: string + storageContext: StorageContext + /** Size recorded for the stored object, used for the pre-read ceilings. */ + declaredSize: number + /** + * True when `key` holds generator source rather than the servable bytes. Such + * an object must never be presigned: the sandbox would receive source text + * under a `.docx` name and the caller's script would fail on a file that looks + * fine. It also means {@link declaredSize} describes the generator, not the + * document, so the pre-read ceilings say nothing and the read is capped instead. + */ + rendersFromSource: boolean + /** + * Bounded read producing the inline payload. Only called on the buffered + * branch, so a URL mount never reads bytes into the web process. + */ + readInline(maxBytes: number): Promise +} + +export interface SandboxInlineMountPayload { + content: string + encoding?: 'base64' + /** Decoded length, which is what the buffered budget counts. */ + byteLength: number +} + +/** + * Mounts one stored object into the sandbox and records its bytes against the + * running totals. + * + * With cloud storage the sandbox fetches the bytes itself from a presigned URL; + * with local storage a presigned URL is an app-internal serve path a remote + * sandbox cannot reach, so the bytes are buffered through the web process under + * the tighter inline ceilings. + */ +export async function pushSandboxFileMount( + sandboxFiles: SandboxFile[], + source: SandboxMountSource, + budget: SandboxMountBudget +): Promise { + if (hasCloudStorage() && !source.rendersFromSource) { + /** + * The number this mount is both admitted on and later held to. + * + * Resolved once, before any comparison, because a non-finite size makes every + * `>` test false — an aggregate check reading `budget.url + NaN` would pass + * silently while the mount still consumed real budget. A missing or + * nonsensical size therefore costs the per-file maximum rather than nothing, + * and a zero takes a one-byte floor, since zero reads as "unlimited" to curl. + */ + const grantedBytes = + Number.isFinite(source.declaredSize) && source.declaredSize >= 0 + ? Math.max(1, source.declaredSize) + : MOUNT_URL_MAX_BYTES + + if (grantedBytes > MOUNT_URL_MAX_BYTES) { + throw new Error( + `Input file "${source.mountPath}" is ${Math.round(grantedBytes / 1024 / 1024)}MB, over the ${MOUNT_URL_MAX_BYTES / 1024 / 1024}MB per-file mount limit.` + ) + } + if (budget.url + grantedBytes > MAX_TOTAL_URL_BYTES) { + throw new Error( + `Mounting "${source.mountPath}" would exceed the ${MAX_TOTAL_URL_BYTES / 1024 / 1024 / 1024}GB total mount limit. Mount fewer or smaller files.` + ) + } + const url = await generatePresignedDownloadUrl( + source.key, + source.storageContext, + MOUNT_URL_TTL_SECONDS + ) + /** + * Granted exactly what it was charged, so the aggregate stays honest without a + * stat round-trip per file. Charging the recorded size while permitting the + * global per-file maximum would let understated sizes accumulate far past the + * ceiling — twenty mounts each claiming a byte and each allowed 500MB. + */ + sandboxFiles.push({ + type: 'url', + path: source.mountPath, + url, + maxBytes: grantedBytes, + }) + budget.url += grantedBytes + return + } + + const remainingBudget = Math.max(0, MAX_INLINE_MOUNT_TOTAL_BYTES - budget.buffered) + + if (!source.rendersFromSource) { + if (source.declaredSize > MAX_INLINE_MOUNT_FILE_BYTES) { + throw new Error( + `Input file "${source.mountPath}" is ${Math.round(source.declaredSize / 1024 / 1024)}MB, over the ${MAX_INLINE_MOUNT_FILE_BYTES / 1024 / 1024}MB per-file mount limit.` + ) + } + if (source.declaredSize > remainingBudget) { + throw new Error( + `Mounting "${source.mountPath}" would exceed the ${MAX_INLINE_MOUNT_TOTAL_BYTES / 1024 / 1024}MB total mount limit. Mount fewer or smaller files.` + ) + } + } + + const inline = await source.readInline(Math.min(MAX_INLINE_MOUNT_FILE_BYTES, remainingBudget)) + sandboxFiles.push({ + path: source.mountPath, + content: inline.content, + ...(inline.encoding ? { encoding: inline.encoding } : {}), + }) + budget.buffered += inline.byteLength +} + +export interface PlannedUserFileMount { + userFile: UserFile + mountPath: string +} + +/** What the running code is told about its mounts, so it never guesses a path. */ +export interface SandboxMountManifestEntry { + name: string + path: string + size: number + type: string +} + +/** + * Derives a mount file name that is safe as a path segment. + * + * `sanitizeFileName` (via {@link buildStorageKeySegment}) already maps `/` and + * `\` to `_`, so no traversal survives it; the explicit guards cover the + * degenerate remainders it does leave intact, since `.` and `-` are permitted + * characters and `..` would otherwise pass through unchanged. + */ +function safeMountFileName(name: string): string { + const segment = buildStorageKeySegment('', name) + if (!segment || segment === '.' || segment === '..') return 'file' + return segment +} + +function uniqueMountFileName(name: string, used: Set): string { + const safe = safeMountFileName(name) + if (!used.has(safe)) { + used.add(safe) + return safe + } + // Two upstream blocks each producing `report.csv` must both survive: without a + // suffix the second write silently overwrites the first and the code sees one file. + const dot = safe.lastIndexOf('.') + const stem = dot > 0 ? safe.slice(0, dot) : safe + const extension = dot > 0 ? safe.slice(dot) : '' + for (let attempt = 2; ; attempt += 1) { + const candidate = `${stem}-${attempt}${extension}` + if (!used.has(candidate)) { + used.add(candidate) + return candidate + } + } +} + +/** + * Assigns each file a deterministic mount path. Pure and I/O-free, so a caller + * can decide whether an execution needs a sandbox filesystem before spending a + * presign or a byte of transfer on a request that may still be refused. + * + * A storage key mounts once. The same object arrives from independent sources — + * a caller listing it twice, or listing one the code also asked for with + * `` — and a second copy of identical bytes costs a presign, a + * duplicate transfer, and a second charge against both the byte budget and the + * per-request file ceiling. First occurrence wins, so the name listed first is + * the one the code sees. + */ +export function planUserFileMounts( + files: readonly UserFile[], + mountDir: string = SANDBOX_INPUT_DIR +): PlannedUserFileMount[] { + const used = new Set() + const mountedKeys = new Set() + const planned: PlannedUserFileMount[] = [] + + for (const userFile of files) { + if (mountedKeys.has(userFile.key)) continue + mountedKeys.add(userFile.key) + planned.push({ + userFile, + mountPath: `${mountDir}/${uniqueMountFileName(userFile.name, used)}`, + }) + } + + return planned +} + +/** + * Resolves planned platform file objects into sandbox mounts. + * + * Authorization runs through {@link assertUserFileContentAccess} rather than the + * tool-file check used by ordinary integrations. For an `execution/` key the + * latter grants on workspace membership alone, which would let a Function block + * mount any execution file from any past run of any workflow in the workspace; + * this one additionally requires the workflow to match and the key to be in the + * execution's allowlist. It is asserted before the transport branches, because + * the URL path never reads the bytes and so never reaches the check embedded in + * the reader. + */ +export async function resolveUserFileMounts(args: { + planned: readonly PlannedUserFileMount[] + context: ExecutionMaterializationContext +}): Promise<{ sandboxFiles: SandboxFile[]; manifest: SandboxMountManifestEntry[] }> { + const sandboxFiles: SandboxFile[] = [] + const manifest: SandboxMountManifestEntry[] = [] + const budget = createSandboxMountBudget() + + for (const { userFile, mountPath } of args.planned) { + const storageContext = resolveTrustedFileContext(userFile.key, userFile.context) + await assertUserFileContentAccess(userFile, args.context) + + await pushSandboxFileMount( + sandboxFiles, + { + mountPath, + key: userFile.key, + storageContext, + declaredSize: userFile.size, + rendersFromSource: isGeneratedDocumentSourceType(userFile.type), + readInline: async (maxBytes) => { + // Base64 regardless of content type: the payload is reproduced exactly + // for any byte sequence, and picking utf8 for a mistyped binary would + // substitute U+FFFD and hand the code a corrupted file. + const { content } = await readUserFileContentWithContributors(userFile, { + ...args.context, + encoding: 'base64', + maxBytes, + maxSourceBytes: maxBytes, + }) + return { + content, + encoding: 'base64' as const, + byteLength: Buffer.byteLength(content, 'base64'), + } + }, + }, + budget + ) + + manifest.push({ + name: userFile.name, + path: mountPath, + size: userFile.size, + type: userFile.type, + }) + } + + logger.info('Resolved sandbox file mounts', { + mountCount: sandboxFiles.length, + bufferedBytes: budget.buffered, + urlBytes: budget.url, + }) + + return { sandboxFiles, manifest } +} diff --git a/apps/sim/lib/internal/file/execute-tool.test.ts b/apps/sim/lib/internal/file/execute-tool.test.ts index cadc082eacf..29304cb3047 100644 --- a/apps/sim/lib/internal/file/execute-tool.test.ts +++ b/apps/sim/lib/internal/file/execute-tool.test.ts @@ -9,6 +9,8 @@ const mocks = vi.hoisted(() => ({ createPrincipal: vi.fn(), executeManage: vi.fn(), executeParser: vi.fn(), + searchContent: vi.fn(), + getProvenance: vi.fn(), })) vi.mock('@/lib/internal/principals/executor', () => ({ @@ -17,12 +19,27 @@ vi.mock('@/lib/internal/principals/executor', () => ({ vi.mock('@/lib/internal/file/operations', () => ({ executeFileManageOperation: mocks.executeManage, + getFileContentProvenance: mocks.getProvenance, + fileContentJsonResponse: ( + body: Record, + includePrivateProvenance: boolean, + init?: ResponseInit, + provenance?: Record + ) => + Response.json( + includePrivateProvenance ? { ...body, __resolvedSecretTraceProvenance: provenance } : body, + init + ), })) vi.mock('@/lib/internal/file/parser', () => ({ executeFileParserOperation: mocks.executeParser, })) +vi.mock('@/lib/workspace-files/application/search-workspace-file-content', () => ({ + searchWorkspaceFileContent: { execute: mocks.searchContent }, +})) + import { executeFileTool } from '@/lib/internal/file/execute-tool' import type { InternalToolOperationCall } from '@/lib/internal/tool-operations/types' import { WORKSPACE_FILES_DELEGATION_AUDIENCE } from '@/lib/workspace-files/application/authorization' @@ -53,6 +70,26 @@ const BILLING_ATTRIBUTION = { payerSubscription: null, } satisfies BillingAttributionSnapshot +const SEARCH_RESULT = { + results: [{ fileId: 'file-1', lineNumber: 2, text: 'needle' }], + count: 1, + truncated: false, + complete: true, + indexStatus: { + readyFiles: 1, + pendingFiles: 0, + failedFiles: 0, + skippedFiles: 0, + partialFiles: 0, + }, + sources: [ + { + identity: { fileId: 'file-1', key: 'workspace/workspace-1/file.txt' }, + ownerUserId: 'user-1', + }, + ], +} + function request( toolId: string, input: unknown, @@ -92,6 +129,127 @@ describe('executeFileTool', () => { }) mocks.executeManage.mockResolvedValue(Response.json({ success: true })) mocks.executeParser.mockResolvedValue(Response.json({ success: true })) + mocks.searchContent.mockResolvedValue(SEARCH_RESULT) + mocks.getProvenance.mockResolvedValue({ version: 1, complete: true, entries: [] }) + }) + + it('searches with the trusted workspace and delegated executor principal', async () => { + const response = await executeFileTool( + request('file_search', { query: 'needle', maxResults: 25 }) + ) + + expect(response.status).toBe(200) + await expect(response.json()).resolves.toMatchObject({ + success: true, + data: { + results: [{ fileId: 'file-1', lineNumber: 2, text: 'needle' }], + count: 1, + }, + }) + expect(mocks.searchContent).toHaveBeenCalledWith({ + principal: expect.objectContaining({ serviceId: 'executor' }), + input: { workspaceId: 'workspace-1', query: 'needle', maxResults: 25, signal: undefined }, + }) + expect(mocks.executeManage).not.toHaveBeenCalled() + }) + + it('uses the default search cap and aggregates provenance for every matched file', async () => { + const response = await executeFileTool( + request( + 'file_search', + { query: 'needle' }, + { + headers: new Headers({ + 'x-sim-request-private-tool-metadata': 'resolved-secret-provenance-v1', + }), + } + ) + ) + + expect(mocks.searchContent).toHaveBeenCalledWith( + expect.objectContaining({ + input: { workspaceId: 'workspace-1', query: 'needle', maxResults: 50 }, + }) + ) + expect(mocks.getProvenance).toHaveBeenCalledWith( + expect.objectContaining({ serviceId: 'executor' }), + 'workspace-1', + expect.arrayContaining([ + expect.objectContaining({ + identity: expect.objectContaining({ fileId: 'file-1' }), + }), + ]), + undefined + ) + const body = await response.json() + expect(body.data.sources).toBeUndefined() + expect(body.__resolvedSecretTraceProvenance).toMatchObject({ complete: true }) + }) + + it.each([ + [{ query: 'ab', maxResults: 50 }, 400], + [{ query: 'abc\0def', maxResults: 50 }, 400], + [{ query: 'needle', maxResults: 201 }, 400], + [{ query: 'needle', maxResults: 0 }, 400], + ])('rejects invalid search input before authorization', async (input, status) => { + const response = await executeFileTool(request('file_search', input)) + + expect(response.status).toBe(status) + expect(mocks.createPrincipal).not.toHaveBeenCalled() + expect(mocks.searchContent).not.toHaveBeenCalled() + }) + + it('does not expose unexpected search infrastructure errors', async () => { + mocks.searchContent.mockRejectedValueOnce(new Error('database host and query details')) + + const response = await executeFileTool(request('file_search', { query: 'needle' })) + + expect(response.status).toBe(500) + await expect(response.json()).resolves.toEqual({ + success: false, + error: 'Failed to search workspace files', + }) + }) + + it('propagates cancellation that arrives while search work is running', async () => { + const controller = new AbortController() + mocks.searchContent.mockImplementationOnce(async () => { + controller.abort(new DOMException('cancelled', 'AbortError')) + return SEARCH_RESULT + }) + + await expect( + executeFileTool(request('file_search', { query: 'needle' }, { signal: controller.signal })) + ).rejects.toMatchObject({ name: 'AbortError' }) + expect(mocks.searchContent).toHaveBeenCalledWith( + expect.objectContaining({ + input: expect.objectContaining({ signal: controller.signal }), + }) + ) + expect(mocks.getProvenance).not.toHaveBeenCalled() + }) + + it('propagates cancellation that arrives while search provenance is loading', async () => { + const controller = new AbortController() + mocks.getProvenance.mockImplementationOnce(async () => { + controller.abort(new DOMException('cancelled', 'AbortError')) + return { version: 1, complete: true, entries: [] } + }) + + await expect( + executeFileTool( + request( + 'file_search', + { query: 'needle' }, + { + signal: controller.signal, + headers: new Headers({ + 'x-sim-request-private-tool-metadata': 'resolved-secret-provenance-v1', + }), + } + ) + ) + ).rejects.toMatchObject({ name: 'AbortError' }) }) it.each(Object.entries(MANAGE_INPUTS))('validates and dispatches %s', async (toolId, input) => { diff --git a/apps/sim/lib/internal/file/execute-tool.ts b/apps/sim/lib/internal/file/execute-tool.ts index 1d70d1f5a0f..9b2ebddf507 100644 --- a/apps/sim/lib/internal/file/execute-tool.ts +++ b/apps/sim/lib/internal/file/execute-tool.ts @@ -1,9 +1,19 @@ import { resolvePrincipalAttribution, resolvePrincipalSubject } from '@sim/auth/principal' import { createLogger } from '@sim/logger' -import { getErrorMessage } from '@sim/utils/errors' +import { getErrorMessage, toError } from '@sim/utils/errors' +import { z } from 'zod' import { fileParseContract } from '@/lib/api/contracts/storage-transfer' import { fileManageContract } from '@/lib/api/contracts/tools/file' -import { executeFileManageOperation } from '@/lib/internal/file/operations' +import { asOrchestrationError, statusForOrchestrationError } from '@/lib/core/orchestration/types' +import { + RESOLVED_SECRET_PROVENANCE_METADATA_V1, + requestsPrivateToolMetadata, +} from '@/lib/execution/private-tool-metadata' +import { + executeFileManageOperation, + fileContentJsonResponse, + getFileContentProvenance, +} from '@/lib/internal/file/operations' import { executeFileParserOperation } from '@/lib/internal/file/parser' import { createExecutorPrincipalFromExecutionContext } from '@/lib/internal/principals/executor' import { @@ -14,6 +24,13 @@ import { import { parseInternalToolInput } from '@/lib/internal/tool-operations/parse-input' import type { InternalToolOperationHandler } from '@/lib/internal/tool-operations/types' import { WORKSPACE_FILES_DELEGATION_AUDIENCE } from '@/lib/workspace-files/application/authorization' +import { searchWorkspaceFileContent } from '@/lib/workspace-files/application/search-workspace-file-content' +import { + FILE_SEARCH_DEFAULT_MAX_RESULTS, + FILE_SEARCH_MAX_QUERY_LENGTH, + FILE_SEARCH_MAX_RESULTS, + FILE_SEARCH_MIN_QUERY_LENGTH, +} from '@/lib/workspace-files/search/constants' const logger = createLogger('FileToolExecution') @@ -29,9 +46,26 @@ const FILE_MANAGE_TOOL_IDS = new Set([ 'file_parser_v2', 'file_parser_v3', 'file_read', + 'file_search', 'file_write', ]) +const fileSearchInputSchema = z + .object({ + query: z + .string() + .min(FILE_SEARCH_MIN_QUERY_LENGTH) + .max(FILE_SEARCH_MAX_QUERY_LENGTH) + .refine((query) => !query.includes('\0'), 'Search query cannot contain NUL characters'), + maxResults: z + .number() + .int() + .min(1) + .max(FILE_SEARCH_MAX_RESULTS) + .default(FILE_SEARCH_DEFAULT_MAX_RESULTS), + }) + .strict() + export const executeFileTool: InternalToolOperationHandler = async (request) => { request.signal?.throwIfAborted() if (!FILE_MANAGE_TOOL_IDS.has(request.toolId)) { @@ -46,6 +80,14 @@ export const executeFileTool: InternalToolOperationHandler = async (request) => return Response.json({ success: false, error: 'Authentication required' }, { status: 401 }) } + const isSearchTool = request.toolId === 'file_search' + const searchInput = isSearchTool ? fileSearchInputSchema.safeParse(request.input) : null + if (searchInput && !searchInput.success) { + return Response.json( + { success: false, error: searchInput.error.issues[0]?.message ?? 'Invalid search input' }, + { status: 400 } + ) + } const isParserTool = request.toolId === 'file_fetch' || request.toolId === 'file_parser' || @@ -53,15 +95,42 @@ export const executeFileTool: InternalToolOperationHandler = async (request) => request.toolId === 'file_parser_v3' const parserInput = isParserTool ? parseInternalToolInput(fileParseContract, request.input) : null if (parserInput && !parserInput.success) return parserInput.response - const manageInput = isParserTool - ? null - : parseInternalToolInput(fileManageContract, request.input) + const manageInput = + isParserTool || isSearchTool ? null : parseInternalToolInput(fileManageContract, request.input) if (manageInput && !manageInput.success) return manageInput.response try { const principal = await createExecutorPrincipalFromExecutionContext({ context: request.context, audience: WORKSPACE_FILES_DELEGATION_AUDIENCE, }) + if (searchInput) { + request.signal?.throwIfAborted() + const result = await searchWorkspaceFileContent.execute({ + principal, + input: { + workspaceId, + query: searchInput.data.query, + maxResults: searchInput.data.maxResults, + signal: request.signal, + }, + }) + request.signal?.throwIfAborted() + const { sources, ...data } = result + const includePrivateProvenance = requestsPrivateToolMetadata( + request.headers, + RESOLVED_SECRET_PROVENANCE_METADATA_V1 + ) + const provenance = includePrivateProvenance + ? await getFileContentProvenance(principal, workspaceId, sources, request.signal) + : undefined + request.signal?.throwIfAborted() + return fileContentJsonResponse( + { success: true, data }, + includePrivateProvenance, + undefined, + provenance + ) + } const { attributedUserId } = resolvePrincipalAttribution(principal, { workspaceBillingOwnerUserId: request.context.billingAttribution?.billedAccountUserId, }) @@ -111,12 +180,24 @@ export const executeFileTool: InternalToolOperationHandler = async (request) => { status: internalToolIdentityFaultStatus(identityFault) } ) } + const orchestrationError = request.toolId === 'file_search' ? asOrchestrationError(error) : null + if (orchestrationError) { + return Response.json( + { success: false, error: orchestrationError.message }, + { status: statusForOrchestrationError(orchestrationError.code) } + ) + } + const isSearchFailure = request.toolId === 'file_search' const message = getErrorMessage(error, 'Unknown error') logger.error('File operation dispatch failed', { - error: message, + error: isSearchFailure ? 'Workspace file search failed' : message, + errorType: isSearchFailure ? toError(error).name : undefined, requestId: request.requestId, toolId: request.toolId, }) - return Response.json({ success: false, error: message }, { status: 500 }) + return Response.json( + { success: false, error: isSearchFailure ? 'Failed to search workspace files' : message }, + { status: 500 } + ) } } diff --git a/apps/sim/lib/internal/file/operations.test.ts b/apps/sim/lib/internal/file/operations.test.ts index 57413837212..deb2f78593f 100644 --- a/apps/sim/lib/internal/file/operations.test.ts +++ b/apps/sim/lib/internal/file/operations.test.ts @@ -164,6 +164,7 @@ vi.mock('@/app/api/files/authorization', () => ({ import { fileManageBodySchema } from '@/lib/api/contracts/tools/file' import { executeFileManageOperation } from '@/lib/internal/file/operations' +import { FileConflictError } from '@/lib/uploads/contexts/workspace' import { createWorkspaceFileDelegatedPrincipal } from '@/lib/workspace-files/application/delegated-principal' async function POST(request: Request): Promise { @@ -333,6 +334,47 @@ describe('file manage operations', () => { scope: { userId: 'user-1', workspaceId: 'workspace-1' }, }, }) + expect(mockGetBoundWorkspaceFileSecretProvenance).toHaveBeenNthCalledWith( + 1, + 'workspace-1', + expect.objectContaining({ fileId: 'file-1', contentUpdatedAt: CONTENT_UPDATED_AT }) + ) + expect(mockGetBoundWorkspaceFileSecretProvenance).toHaveBeenNthCalledWith( + 2, + 'workspace-1', + expect.objectContaining({ fileId: 'file-2', contentUpdatedAt: CONTENT_UPDATED_AT }) + ) + }) + + it('pins resolved file-input provenance to the captured content revision', async () => { + mockResolveWorkspaceFileReference.mockResolvedValue(workspaceFile('file-1')) + mockGetBoundWorkspaceFileSecretProvenance.mockResolvedValue({ + status: 'exact', + entries: [], + }) + + const response = await POST( + createMockRequest( + 'POST', + { + operation: 'content', + workspaceId: 'workspace-1', + fileInput: { + key: 'workspace/workspace-1/file-1.txt', + name: 'file-1.txt', + type: 'text/plain', + size: 6, + }, + }, + PRIVATE_REQUEST_HEADER + ) + ) + + expect(response.status).toBe(200) + expect(mockGetBoundWorkspaceFileSecretProvenance).toHaveBeenCalledWith( + 'workspace-1', + expect.objectContaining({ fileId: 'file-1', contentUpdatedAt: CONTENT_UPDATED_AT }) + ) }) it('stores exact causal provenance from a different user in the actor workspace', async () => { @@ -580,6 +622,210 @@ describe('file manage operations', () => { ) }) + it('replaces the existing file at the target path when overwrite is on', async () => { + const existing = workspaceFile('report') + mockResolveWorkspaceFileReference.mockResolvedValue(existing) + mockUpdateWorkspaceFileContent.mockResolvedValue(existing) + + const response = await POST( + createMockRequest('POST', { + operation: 'write', + workspaceId: 'workspace-1', + fileName: 'report.txt', + content: 'fresh', + overwrite: true, + }) + ) + + expect(response.status).toBe(200) + expect(mockUploadWorkspaceFile).not.toHaveBeenCalled() + expect(mockUpdateWorkspaceFileContent).toHaveBeenCalledWith( + 'workspace-1', + 'report', + 'user-1', + Buffer.from('fresh'), + 'text/plain', + { + expectedUpdatedAt: CONTENT_UPDATED_AT, + secretProvenancePolicy: { mode: 'replace', provenance: { status: 'exact', entries: [] } }, + } + ) + await expect(response.json()).resolves.toMatchObject({ + success: true, + data: { id: 'report', name: 'report.txt' }, + }) + }) + + it('creates the file when overwrite finds nothing at the target path', async () => { + mockResolveWorkspaceFileReference.mockResolvedValue(null) + + const response = await POST( + createMockRequest('POST', { + operation: 'write', + workspaceId: 'workspace-1', + fileName: 'report.txt', + content: 'fresh', + overwrite: true, + }) + ) + + expect(response.status).toBe(200) + expect(mockUpdateWorkspaceFileContent).not.toHaveBeenCalled() + expect(mockUploadWorkspaceFile).toHaveBeenCalledWith( + 'workspace-1', + 'user-1', + Buffer.from('fresh'), + 'report.txt', + 'text/plain', + // Exact, so a path created by a concurrent write conflicts instead of being suffixed. + expect.objectContaining({ exactName: true, folderId: null }) + ) + }) + + it('surfaces a conflict when a concurrent write claims the overwrite path', async () => { + mockResolveWorkspaceFileReference.mockResolvedValue(null) + mockUploadWorkspaceFile.mockRejectedValue(new FileConflictError('report.txt')) + + const response = await POST( + createMockRequest('POST', { + operation: 'write', + workspaceId: 'workspace-1', + fileName: 'report.txt', + content: 'fresh', + overwrite: true, + }) + ) + + expect(response.status).toBe(409) + await expect(response.json()).resolves.toMatchObject({ success: false }) + }) + + it('never overwrites a same-named file resolved outside the target folder', async () => { + mockResolveWorkspaceFileReference.mockResolvedValue({ + ...workspaceFile('report'), + folderId: 'folder-9', + }) + + const response = await POST( + createMockRequest('POST', { + operation: 'write', + workspaceId: 'workspace-1', + fileName: 'report.txt', + content: 'fresh', + overwrite: true, + }) + ) + + expect(response.status).toBe(200) + expect(mockUpdateWorkspaceFileContent).not.toHaveBeenCalled() + expect(mockUploadWorkspaceFile).toHaveBeenCalled() + }) + + it('keeps the suffixing create path when overwrite is off', async () => { + mockResolveWorkspaceFileReference.mockResolvedValue(workspaceFile('report')) + + const response = await POST( + createMockRequest('POST', { + operation: 'write', + workspaceId: 'workspace-1', + fileName: 'report.txt', + content: 'fresh', + }) + ) + + expect(response.status).toBe(200) + expect(mockUpdateWorkspaceFileContent).not.toHaveBeenCalled() + expect(mockUploadWorkspaceFile).toHaveBeenCalledWith( + 'workspace-1', + 'user-1', + Buffer.from('fresh'), + 'report.txt', + 'text/plain', + expect.objectContaining({ exactName: false }) + ) + }) + + it('overwrites an existing file with the bytes of a stored file input', async () => { + const existing = workspaceFile('report') + mockResolveWorkspaceFileReference.mockResolvedValue(existing) + mockUpdateWorkspaceFileContent.mockResolvedValue(existing) + mockGetBoundWorkspaceFileSecretProvenance.mockResolvedValue({ status: 'exact', entries: [] }) + + const response = await POST( + createMockRequest('POST', { + operation: 'write', + workspaceId: 'workspace-1', + fileName: 'report.txt', + fileInput: { + key: 'workspace/workspace-1/source.txt', + name: 'source.txt', + type: 'text/plain', + size: 6, + }, + overwrite: true, + }) + ) + + expect(response.status).toBe(200) + expect(mockUploadWorkspaceFile).not.toHaveBeenCalled() + expect(mockUpdateWorkspaceFileContent).toHaveBeenCalledWith( + 'workspace-1', + 'report', + 'user-1', + Buffer.from('content:source.txt'), + 'text/plain', + expect.objectContaining({ expectedUpdatedAt: CONTENT_UPDATED_AT }) + ) + }) + + it('downgrades provenance when overwriting a file owned by another user', async () => { + const existing = workspaceFile('report', 'other-user') + mockResolveWorkspaceFileReference.mockResolvedValue(existing) + mockUpdateWorkspaceFileContent.mockResolvedValue(existing) + + const response = await POST( + createMockRequest( + 'POST', + { + operation: 'write', + workspaceId: 'workspace-1', + fileName: 'report.txt', + content: 'secret-value', + overwrite: true, + __privateSecretProvenance: { + version: 1, + complete: true, + selections: [ + { + key: 'content', + provenance: { + version: 1, + complete: true, + entries: [{ name: 'TOKEN', encryptedValue: 'encrypted-token' }], + scope: { userId: 'user-1', workspaceId: 'workspace-1' }, + }, + }, + ], + }, + }, + PRIVATE_SECRET_PROVENANCE_HEADER + ) + ) + + expect(response.status).toBe(200) + expect(mockUpdateWorkspaceFileContent).toHaveBeenCalledWith( + 'workspace-1', + 'report', + 'user-1', + Buffer.from('secret-value'), + 'text/plain', + { + expectedUpdatedAt: CONTENT_UPDATED_AT, + secretProvenancePolicy: { mode: 'replace', provenance: { status: 'unknown' } }, + } + ) + }) + it('atomically binds append provenance to the exact predecessor version', async () => { const existing = workspaceFile('file-1') mockResolveWorkspaceFileReference.mockResolvedValue(existing) diff --git a/apps/sim/lib/internal/file/operations.ts b/apps/sim/lib/internal/file/operations.ts index 190592076f8..1fa6e79ff26 100644 --- a/apps/sim/lib/internal/file/operations.ts +++ b/apps/sim/lib/internal/file/operations.ts @@ -12,6 +12,7 @@ import { acquireLock, releaseLock } from '@/lib/core/config/redis' import { OrchestrationError } from '@/lib/core/orchestration/types' import { isPayloadSizeLimitError } from '@/lib/core/utils/stream-limits' import { ensureAbsoluteUrl } from '@/lib/core/utils/urls' +import { isUserFile } from '@/lib/core/utils/user-file' import { durableSecretProvenanceFromPrivateBundle } from '@/lib/execution/durable-secret-provenance' import { inspectPrivateSecretProvenanceRequest, @@ -43,7 +44,7 @@ import { import { getFileExtension, getMimeTypeFromExtension, - inferContextFromKey, + tryInferContextFromKey, } from '@/lib/uploads/utils/file-utils' import { downloadFileFromStorage, @@ -158,6 +159,12 @@ const fileInputToUserFile = (fileInput: unknown) => { if (!fileUrl && !key) return null + // A key this normalizer cannot classify is request input we cannot use, which + // is what `null` already means here — the throwing form would turn a malformed + // client value into a 500 from every operation that normalizes a file input. + const context = key ? tryInferContextFromKey(key) : null + if (key && !context) return null + return { id: key || fileUrl, name: @@ -169,7 +176,9 @@ const fileInputToUserFile = (fileInput: unknown) => { ? record.type.trim() : 'application/octet-stream', key, - context: inferContextFromKey(key), + // Only absent when there is no key at all — an unclassifiable one returned + // above rather than reaching here. + context: context ?? undefined, } } @@ -220,6 +229,14 @@ const MAX_GET_CONTENT_FILE_BYTES = 64 * 1024 * 1024 /** Combined extracted-text cap so the content array stays within the large-value-ref ceiling. */ const MAX_GET_CONTENT_TOTAL_BYTES = 64 * 1024 * 1024 +/** + * Cap on a file stored through `write`'s `fileInput`, pinned to the destination's + * own ceiling. A larger cap here would let a 50–100MB file be downloaded and + * base64-encoded in this process only for `createWorkspaceFile` to reject it, so + * the expensive transfer is refused up front instead. + */ +const MAX_WRITE_FILE_INPUT_BYTES = MAX_WORKSPACE_FILE_CONTENT_BYTES + /** Per-file download cap for the compress operation. */ const MAX_COMPRESS_FILE_BYTES = 100 * 1024 * 1024 /** Combined input cap for the compress operation to bound in-memory archiving. */ @@ -288,12 +305,15 @@ const extractUserFileTextContent = async ( return `[Binary file: ${userFile.name} (${userFile.type || 'application/octet-stream'}, ${buffer.length} bytes). Cannot extract text content.]` } -interface FileContentSource { - file: UserFile +export interface FileContentProvenanceSource { identity?: WorkspaceFileSecretProvenanceIdentity ownerUserId?: string } +interface FileContentSource extends FileContentProvenanceSource { + file: UserFile +} + async function bindSelectedContentFile( principal: Principal, workspaceId: string, @@ -317,16 +337,23 @@ async function bindSelectedContentFile( return { file, - identity: { fileId: metadata.id, key: metadata.key, context: 'workspace' }, + identity: { + fileId: metadata.id, + key: metadata.key, + context: 'workspace', + contentUpdatedAt: metadata.contentUpdatedAt ?? undefined, + }, ownerUserId: metadata.uploadedBy, } } -async function getFileContentProvenance( +export async function getFileContentProvenance( principal: Principal, workspaceId: string, - sources: readonly FileContentSource[] + sources: readonly FileContentProvenanceSource[], + signal?: AbortSignal ): Promise { + signal?.throwIfAborted() const ownerIds = new Set( sources .map((source) => source.ownerUserId) @@ -339,14 +366,20 @@ async function getFileContentProvenance( const accumulator = new ResolvedSecretTraceProvenanceAccumulator(scope) for (const source of sources) { + signal?.throwIfAborted() if (!source.identity || !source.ownerUserId) { accumulator.markIncomplete('file-source-unidentified') continue } const { provenance } = await readWorkspaceFileSecretProvenance.execute({ principal, - input: { fileId: source.identity.fileId, assertedWorkspaceId: workspaceId }, + input: { + fileId: source.identity.fileId, + assertedWorkspaceId: workspaceId, + expectedContentUpdatedAt: source.identity.contentUpdatedAt, + }, }) + signal?.throwIfAborted() /** * `unrecorded` is a more specific `unknown`, and this accumulator has not opted into the * workspace file surface's policy, so it latches exactly as it did before. @@ -451,6 +484,35 @@ function resolveFileWriteSecretProvenance(options: { return { success: true, contentProvenance: content } } +/** + * Resolves the file an overwriting write should replace, or null when nothing exists at the + * target path. The shared reference resolver falls back to a workspace-wide name match, so the + * result is accepted only when it sits at exactly the folder and leaf name being written. + */ +async function resolveWriteOverwriteTarget(options: { + principal: Principal + workspaceId: string + folderId: string | null + folderSegments: string[] + leafName: string +}) { + const { principal, workspaceId, folderId, folderSegments, leafName } = options + let existing: Awaited> + try { + existing = await resolveWorkspaceFileReference({ + principal, + operation: fileOperations.updateContent, + workspaceId, + reference: [...folderSegments, leafName].join('/'), + }) + } catch (error) { + if (error instanceof OrchestrationError && error.code === 'not_found') return null + throw error + } + if ((existing.folderId ?? null) !== folderId || existing.name !== leafName) return null + return existing +} + async function deriveWorkspaceFileSecretProvenance(options: { principal: Principal workspaceId: string @@ -476,7 +538,7 @@ async function deriveWorkspaceFileSecretProvenance(options: { return mergeWorkspaceFileSecretProvenance(...provenances) } -function fileContentJsonResponse( +export function fileContentJsonResponse( body: Record, includePrivateProvenance: boolean, init?: ResponseInit, @@ -698,7 +760,12 @@ export async function executeFileManageOperation( return [ { file: userFile, - identity: { fileId: file.id, key: file.key, context: 'workspace' }, + identity: { + fileId: file.id, + key: file.key, + context: 'workspace', + contentUpdatedAt: file.contentUpdatedAt ?? undefined, + }, ownerUserId: file.uploadedBy, }, ] @@ -742,14 +809,14 @@ export async function executeFileManageOperation( logger.info('File content extracted', { count: contents.length }) const provenance = includePrivateContentProvenance - ? await getFileContentProvenance(principal, workspaceId, sources) + ? await getFileContentProvenance(principal, workspaceId, sources, signal) : undefined return contentResponse({ success: true, data: { contents } }, undefined, provenance) } case 'write': { - const { fileName, content, contentType } = body + const { fileName, content, fileInput, contentType, overwrite } = body signal?.throwIfAborted() const provenanceResolution = resolveFileWriteSecretProvenance({ headers, @@ -763,33 +830,162 @@ export async function executeFileManageOperation( { status: 400 } ) } - const { folderSegments, leafName } = splitWorkspaceFilePath(fileName) + + // Storing an existing file object rather than text: read its bytes under + // the caller's own authorization, then write them unchanged. Base64 so a + // binary payload survives — decoding it as UTF-8 would corrupt it. + let sourceEncoding: 'utf-8' | 'base64' = 'utf-8' + let sourceContent = content ?? '' + let sourceName = fileName + let sourceContentType = contentType + /** + * Copying bytes carries the source's secret lineage, exactly as archiving + * does. Without this the copy would land with no provenance row — the + * "safe" state — and a file the platform had locked as secret-derived + * would be readable again under its new id. + * + * A source with no workspace row resolves to `unknown` rather than empty, + * because nothing durable records what went into it. + */ + let inputProvenance: WorkspaceFileSecretProvenance | undefined + if (fileInput !== undefined && fileInput !== null) { + /** + * Two shapes reach here and only one already identifies a file. A block + * reference, or an id the tool layer resolved through the execution + * index or workspace metadata, arrives carrying `id`/`key`/`url`/`name`. + * The file picker instead stores `{name, path, key, size, type}` with no + * `id` or `url`, which the shared normalizer turns into one — the same + * conversion every other operation in this file applies to its input. + * + * Identity is all that is demanded, deliberately. `size` is never read + * before the download and the download reports the real content type, so + * requiring them would reject an otherwise usable reference over two + * fields nothing depends on. + */ + const sourceFile: UserFile | null = isUserFile(fileInput) + ? { + ...fileInput, + size: fileInput.size ?? 0, + type: fileInput.type ?? 'application/octet-stream', + } + : fileInputToUserFile(fileInput) + if (!sourceFile) { + return Response.json( + { success: false, error: 'fileInput must be a file object' }, + { status: 400 } + ) + } + const denied = await assertOperationFileAccess(sourceFile, context) + if (denied) return denied + + inputProvenance = await deriveWorkspaceFileSecretProvenance({ + principal, + workspaceId, + targetOwnerUserId: userId, + sources: [await bindSelectedContentFile(principal, workspaceId, sourceFile)], + }) + + const downloaded = await downloadServableFileFromStorage(sourceFile, requestId, logger, { + maxBytes: MAX_WRITE_FILE_INPUT_BYTES, + signal, + // A generated document that references other files needs a principal + // to resolve them; without one the resolver can only serve an + // already-published artifact and throws when there is none. + filePrincipal: principal, + }) + sourceEncoding = 'base64' + sourceContent = downloaded.buffer.toString('base64') + sourceName = fileName?.trim() || sourceFile.name + sourceContentType = contentType || downloaded.contentType || sourceFile.type + } + const writeProvenanceSources = [ + provenanceResolution.contentProvenance, + inputProvenance, + ].filter((entry): entry is WorkspaceFileSecretProvenance => entry !== undefined) + // Left undefined when neither side recorded anything, so a plain text + // write still stores no provenance row rather than an empty one. + const writeProvenance = writeProvenanceSources.length + ? mergeWorkspaceFileSecretProvenance(...writeProvenanceSources) + : undefined + + const { folderSegments, leafName } = splitWorkspaceFilePath(sourceName ?? '') await admitCreateWorkspaceFile(principal, workspaceId) const { folderId } = await ensureWorkspaceFileFolderPathOperation.execute({ principal, input: { workspaceId, pathSegments: folderSegments }, }) - const mimeType = contentType || getMimeTypeFromExtension(getFileExtension(leafName)) + const mimeType = sourceContentType || getMimeTypeFromExtension(getFileExtension(leafName)) + + if (overwrite) { + const existing = await resolveWriteOverwriteTarget({ + principal, + workspaceId, + folderId: folderId ?? null, + folderSegments, + leafName, + }) + if (existing) { + // Writing into a file someone else owns must not hand its owner an + // exact, re-resolvable secret lineage, exactly as appending does. + const overwriteProvenance = + writeProvenance?.status === 'exact' && + writeProvenance.entries.length > 0 && + existing.uploadedBy !== userId + ? { status: 'unknown' as const } + : writeProvenance + const { file: overwritten } = await updateWorkspaceFileContent.execute({ + principal, + input: { + fileId: existing.id, + assertedWorkspaceId: workspaceId, + content: sourceContent, + encoding: sourceEncoding, + contentType: mimeType, + provenanceMode: 'replace_empty', + expectedUpdatedAt: existing.contentUpdatedAt ?? undefined, + ...(overwriteProvenance ? { secretProvenance: overwriteProvenance } : {}), + }, + }) + + logger.info('File overwritten', { + fileId: overwritten.id, + name: overwritten.name, + size: overwritten.size, + }) + + return Response.json({ + success: true, + data: { + id: overwritten.id, + name: overwritten.name, + size: overwritten.size, + url: ensureAbsoluteUrl(overwritten.url ?? overwritten.path), + }, + }) + } + } + const result = await createWorkspaceFile.execute({ principal, input: { workspaceId, name: leafName, contentType: mimeType, - content: content ?? '', - encoding: 'utf-8', + content: sourceContent, + encoding: sourceEncoding, folderId, - exactName: false, - ...(provenanceResolution.contentProvenance - ? { secretProvenance: provenanceResolution.contentProvenance } - : {}), + // An overwrite that found no target must land on the exact path or fail. Suffixing + // would silently satisfy the request at the wrong name when a concurrent write + // created that path in between; exactName surfaces the race as a conflict instead. + exactName: Boolean(overwrite), + ...(writeProvenance ? { secretProvenance: writeProvenance } : {}), }, }) - const fileBuffer = Buffer.from(content ?? '', 'utf-8') + const fileBuffer = Buffer.from(sourceContent, sourceEncoding) logger.info('File created', { fileId: result.file.id, - name: fileName, + name: sourceName, size: fileBuffer.length, }) diff --git a/apps/sim/lib/internal/tool-operations/registry.server.ts b/apps/sim/lib/internal/tool-operations/registry.server.ts index c84c5735af6..9a752b81578 100644 --- a/apps/sim/lib/internal/tool-operations/registry.server.ts +++ b/apps/sim/lib/internal/tool-operations/registry.server.ts @@ -828,6 +828,7 @@ const FILE_TOOL_IDS = [ 'file_write', 'file_get', 'file_read', + 'file_search', 'file_get_content', 'file_compress', 'file_decompress', diff --git a/apps/sim/lib/knowledge/connectors/sync-engine.test.ts b/apps/sim/lib/knowledge/connectors/sync-engine.test.ts index 12a5919fafd..0e459335f1f 100644 --- a/apps/sim/lib/knowledge/connectors/sync-engine.test.ts +++ b/apps/sim/lib/knowledge/connectors/sync-engine.test.ts @@ -1257,9 +1257,16 @@ describe('executeSync deferred hydration rate limits', () => { expect(dbChainMockFns.set).toHaveBeenCalledWith( expect.objectContaining({ status: 'error', - nextSyncAt: new Date(NOW.getTime() + 45 * 60 * 1000), + consecutiveFailures: 0, }) ) + const failureUpdate = dbChainMockFns.set.mock.calls.find( + ([update]) => update.status === 'error' + )?.[0] + expect(failureUpdate?.nextSyncAt.getTime()).toBeGreaterThanOrEqual( + NOW.getTime() + 45 * 60 * 1000 + ) + expect(failureUpdate?.nextSyncAt.getTime()).toBeLessThanOrEqual(NOW.getTime() + 46 * 60 * 1000) }) }) @@ -2389,6 +2396,43 @@ describe('buildSyncFailureUpdate', () => { }) }) +describe('buildSyncRateLimitUpdate', () => { + const now = new Date('2026-08-20T00:00:00.000Z') + + it('preserves the failure counter and schedules after the provider deadline', async () => { + const { buildSyncRateLimitUpdate } = await import('@/lib/knowledge/connectors/sync-engine') + const providerDelayMs = 45 * 60 * 1000 + const update = buildSyncRateLimitUpdate(now, 9, 'rate limited', providerDelayMs) + + expect(update.status).toBe('error') + expect(update.lastSyncError).toBe('rate limited') + expect(update.consecutiveFailures).toBe(9) + expect(update.nextSyncAt.getTime()).toBeGreaterThanOrEqual(now.getTime() + providerDelayMs) + expect(update.nextSyncAt.getTime()).toBeLessThanOrEqual( + now.getTime() + providerDelayMs + 60_000 + ) + }) + + it('uses a conservative fallback without consuming the breaker', async () => { + const { buildSyncRateLimitUpdate } = await import('@/lib/knowledge/connectors/sync-engine') + const update = buildSyncRateLimitUpdate(now, null, 'rate limited') + const fallbackMs = 30 * 60 * 1000 + + expect(update.consecutiveFailures).toBe(0) + expect(update.nextSyncAt.getTime()).toBeGreaterThanOrEqual(now.getTime() + fallbackMs) + expect(update.nextSyncAt.getTime()).toBeLessThanOrEqual(now.getTime() + fallbackMs + 60_000) + }) + + it('caps the provider deadline and releases the sync lease', async () => { + const { buildSyncRateLimitUpdate } = await import('@/lib/knowledge/connectors/sync-engine') + const update = buildSyncRateLimitUpdate(now, 4, 'rate limited', 30 * 24 * 60 * 60 * 1000) + + expect(update.nextSyncAt).toEqual(new Date(now.getTime() + 24 * 60 * 60 * 1000)) + expect(update.syncLockToken).toBeNull() + expect(update.syncLockLeaseAt).toBeNull() + }) +}) + describe('buildSyncCapacityUpdate', () => { it('requires operator action without consuming the transient-failure breaker', async () => { const { buildSyncCapacityUpdate } = await import('@/lib/knowledge/connectors/sync-engine') diff --git a/apps/sim/lib/knowledge/connectors/sync-engine.ts b/apps/sim/lib/knowledge/connectors/sync-engine.ts index 7c7b8d059a1..c501a529593 100644 --- a/apps/sim/lib/knowledge/connectors/sync-engine.ts +++ b/apps/sim/lib/knowledge/connectors/sync-engine.ts @@ -71,6 +71,8 @@ import { hasIndexablePayload } from '@/connectors/utils' const logger = createLogger('ConnectorSyncEngine') +const RATE_LIMIT_RETRY_JITTER_MAX_MS = 60_000 + /** * Raised when a run discovers mid-flight that it no longer holds its sync lock. * @@ -1358,6 +1360,40 @@ export function buildSyncFailureUpdate( } } +/** + * The connector row written after a provider positively identifies throttling. + * + * Structured throttling is a transient quota or availability condition, so it + * must not consume the breaker reserved for persistent connector failures. The + * provider deadline remains authoritative, with a short post-deadline jitter + * to avoid releasing every connector sharing the same quota window at once. + * When the provider omits a usable deadline, the first rung of the ordinary + * failure ladder provides a conservative fallback. + */ +export function buildSyncRateLimitUpdate( + now: Date, + previousFailures: number | null | undefined, + errorMessage: string, + retryAfterMs?: number +) { + const maximumBackoffMs = CONNECTOR_FAILURE_BACKOFF_CAP_MINUTES * 60 * 1000 + const providerBackoffMs = + typeof retryAfterMs === 'number' && Number.isFinite(retryAfterMs) && retryAfterMs > 0 + ? retryAfterMs + : connectorFailureBackoffMinutes(1) * 60 * 1000 + const jitterMs = randomInt(0, RATE_LIMIT_RETRY_JITTER_MAX_MS + 1) + + return { + status: 'error' as const, + lastSyncError: errorMessage, + nextSyncAt: new Date(now.getTime() + Math.min(providerBackoffMs + jitterMs, maximumBackoffMs)), + consecutiveFailures: previousFailures ?? 0, + syncLockToken: null, + syncLockLeaseAt: null, + updatedAt: now, + } +} + /** * A deterministic capacity rejection needs operator action, not an automatic * retry or the transient-failure circuit breaker. Keep its precise diagnostic, @@ -3181,6 +3217,7 @@ export async function executeSync( const errorMessage = toError(error).message const retryAfterMs = getRetryAfterMs(error) + const rateLimited = isRateLimitError(error) logger.error('Sync failed', { connectorId, error: errorMessage, @@ -3193,12 +3230,19 @@ export async function executeSync( const failureUpdate = error instanceof ConnectorSyncCapacityError ? buildSyncCapacityUpdate(new Date(), connector.consecutiveFailures, errorMessage) - : buildSyncFailureUpdate( - new Date(), - connector.consecutiveFailures, - errorMessage, - retryAfterMs - ) + : rateLimited + ? buildSyncRateLimitUpdate( + new Date(), + connector.consecutiveFailures, + errorMessage, + retryAfterMs + ) + : buildSyncFailureUpdate( + new Date(), + connector.consecutiveFailures, + errorMessage, + retryAfterMs + ) if (failureUpdate.status === 'disabled') { logger.warn('Connector disabled after repeated failures', { diff --git a/apps/sim/lib/logs/execution/logging-session.ts b/apps/sim/lib/logs/execution/logging-session.ts index c2fe239248b..ece15ee9196 100644 --- a/apps/sim/lib/logs/execution/logging-session.ts +++ b/apps/sim/lib/logs/execution/logging-session.ts @@ -46,6 +46,7 @@ import type { SerializableExecutionState } from '@/executor/execution/types' import type { BlockLog } from '@/executor/types' import { projectResolvedSecretDiagnosticError } from '@/executor/utils/resolved-secret-content-projection' import { + emptyResolvedSecretTraceProvenance, isResolvedSecretTraceProvenanceV1, RESOLVED_SECRET_TRACE_CHECKPOINT_VERSION, type ResolvedSecretTraceProvenanceV1, @@ -124,10 +125,6 @@ function getActiveBlockDisplayProvenance( const logger = createLogger('LoggingSession') -function emptyResolvedSecretTraceProvenance(): ResolvedSecretTraceProvenanceV1 { - return { version: 1, complete: true, entries: [] } -} - type CompletionAttempt = 'complete' | 'error' | 'cancelled' | 'paused' export interface SecretSafeDisplayContent { diff --git a/apps/sim/lib/table/__tests__/column-type-registry.test.ts b/apps/sim/lib/table/__tests__/column-type-registry.test.ts index 91ec369e31e..a63886a4a5c 100644 --- a/apps/sim/lib/table/__tests__/column-type-registry.test.ts +++ b/apps/sim/lib/table/__tests__/column-type-registry.test.ts @@ -10,6 +10,7 @@ * here. */ import { describe, expect, it } from 'vitest' +import { zonedWallClockToUtc } from '@/lib/core/utils/timezone' import type { ColumnType } from '@/lib/table/column-types' import { ALL_COLUMN_TYPES, @@ -121,6 +122,143 @@ describe('conversion write-back', () => { }) }) +describe('ttl columns', () => { + const column = { name: 'expires_at', type: 'ttl' } as ColumnDefinition + + it('stores integer epoch seconds while accepting date-shaped input', () => { + expect(COLUMN_TYPE_REGISTRY.ttl.coerce('2023-11-14T22:13:20Z', column)).toEqual({ + ok: true, + value: 1_700_000_000, + }) + expect(COLUMN_TYPE_REGISTRY.ttl.coerce(1_700_000_000, column)).toEqual({ + ok: true, + value: 1_700_000_000, + }) + expect(COLUMN_TYPE_REGISTRY.ttl.coerce('1700000000', column)).toEqual({ + ok: true, + value: 1_700_000_000, + }) + expect(COLUMN_TYPE_REGISTRY.ttl.coerce('2023-11-14T22:13:20.123Z', column)).toEqual({ + ok: true, + value: 1_700_000_001, + }) + expect(COLUMN_TYPE_REGISTRY.ttl.coerce('not-a-date', column)).toEqual({ ok: false }) + expect(COLUMN_TYPE_REGISTRY.ttl.coerce(1_700_000_000.5, column)).toEqual({ ok: false }) + }) + + it.each(['2023-02-29', '2023-02-29T12:00:00', '2023-02-29T12:00:00-05:00'])( + 'rejects a nonexistent ISO calendar input: %s', + (value) => { + expect(COLUMN_TYPE_REGISTRY.ttl.coerce(value, column, { timezone: 'UTC' })).toEqual({ + ok: false, + }) + } + ) + + it.each([ + ['2024-02-29', '2024-02-29T00:00:00Z'], + ['2024-02-29T12:00:00', '2024-02-29T12:00:00Z'], + ['2024-02-29T12:00:00-05:00', '2024-02-29T17:00:00Z'], + ])('accepts a valid leap-day ISO calendar input: %s', (value, expectedInstant) => { + expect(COLUMN_TYPE_REGISTRY.ttl.coerce(value, column, { timezone: 'UTC' })).toEqual({ + ok: true, + value: Math.floor(Date.parse(expectedInstant) / 1000), + }) + }) + + it('renders and edits epoch seconds as a date', () => { + expect(COLUMN_TYPE_REGISTRY.ttl.formatForDisplay(1_700_000_000, column)).toBe( + '11/14/2023 10:13:20 PM' + ) + expect(COLUMN_TYPE_REGISTRY.ttl.formatForInput(1_700_000_000, column)).toBe( + '2023-11-14T22:13:20Z' + ) + expect( + COLUMN_TYPE_REGISTRY.ttl.formatForInput(1_700_000_000, column, { + timezone: 'America/New_York', + }) + ).toBe('2023-11-14T17:13:20-05:00') + }) + + it('preserves the exact instant across both sides of a daylight-saving fold', () => { + expect( + COLUMN_TYPE_REGISTRY.ttl.formatForInput(1_699_162_200, column, { + timezone: 'America/New_York', + }) + ).toBe('2023-11-05T01:30:00-04:00') + expect( + COLUMN_TYPE_REGISTRY.ttl.formatForInput(1_699_165_800, column, { + timezone: 'America/New_York', + }) + ).toBe('2023-11-05T01:30:00-05:00') + }) + + it('matches the shared wall-clock resolver in every effective timezone', () => { + const wallClock = '2026-06-15T09:00:30' + for (const timezone of [ + 'UTC', + 'America/Los_Angeles', + 'America/New_York', + 'Asia/Kathmandu', + 'Australia/Lord_Howe', + ]) { + const expected = Math.floor(zonedWallClockToUtc(wallClock, timezone).getTime() / 1000) + expect(COLUMN_TYPE_REGISTRY.ttl.coerce(wallClock, column, { timezone })).toEqual({ + ok: true, + value: expected, + }) + } + }) + + it.each([ + ['Europe/Berlin', '2026-03-29T02:30'], + ['Australia/Lord_Howe', '2026-10-04T02:15'], + ])('coerces a %s spring-forward gap wall clock to the compatible epoch', (timezone, input) => { + const expected = Math.floor(zonedWallClockToUtc(input, timezone).getTime() / 1000) + expect(COLUMN_TYPE_REGISTRY.ttl.coerce(input, column, { timezone })).toEqual({ + ok: true, + value: expected, + }) + }) + + it('coerces a localized month-name gap input in the explicit workspace timezone', () => { + const timezone = 'America/New_York' + const expected = Math.floor(zonedWallClockToUtc('2026-03-08T02:30', timezone).getTime() / 1000) + expect(COLUMN_TYPE_REGISTRY.ttl.coerce('March 8, 2026 2:30 AM', column, { timezone })).toEqual({ + ok: true, + value: expected, + }) + }) + + it('rejects an impossible ISO expiration date', () => { + expect( + COLUMN_TYPE_REGISTRY.ttl.coerce('2026-02-30T12:00:00', column, { timezone: 'UTC' }) + ).toEqual({ ok: false }) + }) + + it('round-trips epoch seconds after the editor timezone changes', () => { + for (const seconds of [1_700_000_000, 1_699_162_200, 1_699_165_800]) { + for (const timezone of [ + 'UTC', + 'America/Los_Angeles', + 'America/New_York', + 'Asia/Kathmandu', + 'Australia/Lord_Howe', + ]) { + const editable = COLUMN_TYPE_REGISTRY.ttl.formatForInput(seconds, column, { timezone }) + expect(COLUMN_TYPE_REGISTRY.ttl.coerce(editable, column, { timezone })).toEqual({ + ok: true, + value: seconds, + }) + } + } + }) + + it('limits a table to one ttl column', () => { + expect(COLUMN_TYPE_REGISTRY.ttl.maxPerTable).toBe(1) + }) +}) + describe('intentional divergences from the pre-registry behavior', () => { // A differential run of the registry against the pre-refactor implementations // (55 values x 7 column shapes) found ZERO coercion differences and exactly diff --git a/apps/sim/lib/table/__tests__/service-filter-threading.test.ts b/apps/sim/lib/table/__tests__/service-filter-threading.test.ts index 52ae732e805..a7089a000f4 100644 --- a/apps/sim/lib/table/__tests__/service-filter-threading.test.ts +++ b/apps/sim/lib/table/__tests__/service-filter-threading.test.ts @@ -15,6 +15,8 @@ import { decodeCursor } from '@/lib/table/rows/cursor' import { buildFilterClause, buildSortClause } from '@/lib/table/sql' import type { ColumnDefinition, TableDefinition } from '@/lib/table/types' +const { mockFireTableTrigger } = vi.hoisted(() => ({ mockFireTableTrigger: vi.fn() })) + vi.mock('@/lib/table/sql', () => ({ buildFilterClause: vi.fn(() => sql`true`), buildSortClause: vi.fn(() => sql`true`), @@ -23,7 +25,7 @@ vi.mock('@/lib/table/sql', () => ({ })) vi.mock('@/lib/table/trigger', () => ({ - fireTableTrigger: vi.fn(), + fireTableTrigger: mockFireTableTrigger, })) vi.mock('@/lib/table/workflow-group-deps', () => ({ @@ -64,7 +66,9 @@ vi.mock('@/lib/table/validation', () => ({ })) import { + deleteRow, deleteRowsByFilter, + deleteRowsByIds, queryRows, requireTableRowIds, updateRowsByFilter, @@ -183,6 +187,104 @@ describe('service filter threading', () => { }) }) +describe('delete trigger dispatch', () => { + beforeEach(() => { + vi.clearAllMocks() + resetDbChainMock() + }) + + it('fires with the committed snapshot after deleting one row', async () => { + dbChainMockFns.returning.mockResolvedValueOnce([{ id: 'row-1', data: { name: 'Ada' } }]) + + await deleteRow(TABLE, 'row-1', 'req-delete-one') + + expect(mockFireTableTrigger).toHaveBeenCalledWith( + TABLE.id, + TABLE.workspaceId, + TABLE.name, + 'delete', + [{ id: 'row-1', data: { name: 'Ada' } }], + null, + TABLE.schema, + 'req-delete-one' + ) + }) + + it('returns after deleting one row without waiting for trigger dispatch', async () => { + dbChainMockFns.returning.mockResolvedValueOnce([{ id: 'row-1', data: { name: 'Ada' } }]) + let releaseTrigger: (() => void) | undefined + const triggerPending = new Promise((resolve) => { + releaseTrigger = resolve + }) + mockFireTableTrigger.mockReturnValueOnce(triggerPending) + const deletion = deleteRow(TABLE, 'row-1', 'req-delete-one') + const onDeleteSettled = vi.fn() + void deletion.then(onDeleteSettled) + + await vi.waitFor(() => expect(mockFireTableTrigger).toHaveBeenCalledTimes(1)) + await Promise.resolve() + + try { + expect(onDeleteSettled).toHaveBeenCalledTimes(1) + } finally { + releaseTrigger?.() + await deletion + } + }) + + it('fires once with every committed snapshot in an ID batch', async () => { + dbChainMockFns.returning.mockResolvedValueOnce([ + { id: 'row-1', data: { name: 'Ada' } }, + { id: 'row-2', data: { name: 'Grace' } }, + ]) + + await deleteRowsByIds( + TABLE, + { tableId: TABLE.id, workspaceId: TABLE.workspaceId, rowIds: ['row-1', 'row-2'] }, + 'req-delete-many' + ) + + expect(mockFireTableTrigger).toHaveBeenCalledWith( + TABLE.id, + TABLE.workspaceId, + TABLE.name, + 'delete', + [ + { id: 'row-1', data: { name: 'Ada' } }, + { id: 'row-2', data: { name: 'Grace' } }, + ], + null, + TABLE.schema, + 'req-delete-many' + ) + }) + + it('dispatches byte-bounded ID-delete snapshots before loading the next batch', async () => { + setEnv({ + TABLE_MAX_ROW_SIZE_BYTES: TABLE_LIMITS.DELETE_SNAPSHOT_BATCH_MAX_BYTES * 2, + }) + dbChainMockFns.returning + .mockResolvedValueOnce([{ id: 'row-1', data: { name: 'Ada' } }]) + .mockResolvedValueOnce([{ id: 'row-2', data: { name: 'Grace' } }]) + + try { + await deleteRowsByIds( + TABLE, + { tableId: TABLE.id, workspaceId: TABLE.workspaceId, rowIds: ['row-1', 'row-2'] }, + 'req-delete-bounded' + ) + } finally { + setEnv({ TABLE_MAX_ROW_SIZE_BYTES: undefined }) + } + + expect(mockFireTableTrigger).toHaveBeenCalledTimes(2) + expect(mockFireTableTrigger.mock.calls[0][4]).toEqual([{ id: 'row-1', data: { name: 'Ada' } }]) + expect(mockFireTableTrigger.mock.calls[1][4]).toEqual([ + { id: 'row-2', data: { name: 'Grace' } }, + ]) + }) +}) + describe('bulk update/delete limited-subset ordering', () => { beforeEach(() => { vi.clearAllMocks() diff --git a/apps/sim/lib/table/__tests__/trigger.test.ts b/apps/sim/lib/table/__tests__/trigger.test.ts index 64b5c4afc5e..fc04df054f1 100644 --- a/apps/sim/lib/table/__tests__/trigger.test.ts +++ b/apps/sim/lib/table/__tests__/trigger.test.ts @@ -60,7 +60,7 @@ interface Payload { function webhookEntry(config: Record = {}) { return { webhook: { id: 'wh_1', providerConfig: { tableId: 'tbl_1', eventType: 'insert', ...config } }, - workflow: { id: 'wf_1' }, + workflow: { id: 'wf_1', workspaceId: 'ws_1' }, } } @@ -69,12 +69,13 @@ function firedPayloads(): Payload[] { } async function fire( - eventType: 'insert' | 'update', + eventType: 'insert' | 'update' | 'delete', data: RowData, oldRows: Map | null = null ) { await fireTableTrigger( 'tbl_1', + 'ws_1', 'Issues', eventType, [{ id: 'row_1', data } as never], @@ -166,6 +167,19 @@ describe('fireTableTrigger — payload shape', () => { const [payload] = firedPayloads() expect(payload.changedColumns).toEqual(['Status']) }) + + it('emits the deleted row snapshot as the event row and previous row', async () => { + mockFetchActiveWebhooks.mockResolvedValue([webhookEntry({ eventType: 'delete' })]) + + await fire('delete', { col_title: 'Removed issue', col_status: 'opt_closed' }) + + const [payload] = firedPayloads() + const deletedRow = { Title: 'Removed issue', Status: 'Closed' } + expect(payload.rawRow).toEqual(deletedRow) + expect(payload.row).toEqual({ ...deletedRow, Tags: null }) + expect(payload.previousRow).toEqual(deletedRow) + expect(payload.changedColumns).toEqual([]) + }) }) describe('fireTableTrigger — gating', () => { @@ -180,6 +194,14 @@ describe('fireTableTrigger — gating', () => { expect(mockProcessPolledWebhookEvent).not.toHaveBeenCalled() }) + it('fires nothing for a workflow in a different workspace', async () => { + mockFetchActiveWebhooks.mockResolvedValue([ + { ...webhookEntry(), workflow: { id: 'wf_other', workspaceId: 'ws_other' } }, + ]) + await fire('insert', { col_title: 'x' }) + expect(mockProcessPolledWebhookEvent).not.toHaveBeenCalled() + }) + it('fires nothing when the event type does not match', async () => { mockFetchActiveWebhooks.mockResolvedValue([webhookEntry({ eventType: 'update' })]) await fire('insert', { col_title: 'x' }) diff --git a/apps/sim/lib/table/__tests__/validation.test.ts b/apps/sim/lib/table/__tests__/validation.test.ts index 9d698c9d96b..04b70ce4af9 100644 --- a/apps/sim/lib/table/__tests__/validation.test.ts +++ b/apps/sim/lib/table/__tests__/validation.test.ts @@ -195,6 +195,18 @@ describe('Validation', () => { expect(result.errors).toContain('Duplicate column names found') }) + it('rejects more than one TTL column', () => { + const result = validateTableSchema({ + columns: [ + { name: 'expires_at', type: 'ttl' }, + { name: 'delete_at', type: 'ttl' }, + ], + } as TableSchema) + + expect(result.valid).toBe(false) + expect(result.errors).toContain('A table can have at most 1 Expiration column') + }) + it('should reject null schema', () => { const result = validateTableSchema(null as unknown as TableSchema) expect(result.valid).toBe(false) diff --git a/apps/sim/lib/table/application/groups.test.ts b/apps/sim/lib/table/application/groups.test.ts index 6eb36a21c5c..59f3135caa7 100644 --- a/apps/sim/lib/table/application/groups.test.ts +++ b/apps/sim/lib/table/application/groups.test.ts @@ -72,7 +72,7 @@ vi.mock('@/lib/workflows/application/context', () => ({ resolveActiveWorkflowApplicationContext: mocks.resolveWorkflowContext, })) vi.mock('@/lib/workflows/application/resolve-workflow-outputs', () => ({ - loadResolvedWorkflowOutputs: mocks.loadWorkflowOutputs, + loadResolvedDeployedWorkflowOutputs: mocks.loadWorkflowOutputs, })) import { v2WorkflowGroupSchema } from '@/lib/api/contracts/v2/tables' diff --git a/apps/sim/lib/table/application/groups.ts b/apps/sim/lib/table/application/groups.ts index f8eef13eaca..485c960b78e 100644 --- a/apps/sim/lib/table/application/groups.ts +++ b/apps/sim/lib/table/application/groups.ts @@ -35,7 +35,7 @@ import { } from '@/lib/table/workflow-groups/service' import { resolveActiveWorkflowApplicationContext } from '@/lib/workflows/application/context' import type { ResolveWorkflowOutputsResult } from '@/lib/workflows/application/resolve-workflow-outputs' -import { loadResolvedWorkflowOutputs } from '@/lib/workflows/application/resolve-workflow-outputs' +import { loadResolvedDeployedWorkflowOutputs } from '@/lib/workflows/application/resolve-workflow-outputs' import { getEnrichment } from '@/enrichments/registry' import type { EnrichmentConfig } from '@/enrichments/types' @@ -64,7 +64,7 @@ async function resolveWorkflowForAuthorizedTableCommand( workflowId, assertedWorkspaceId: workspaceId, }) - return loadResolvedWorkflowOutputs(workflowContext) + return loadResolvedDeployedWorkflowOutputs(workflowContext) } async function resolveRelatedWorkflowForTableRoute( @@ -1079,6 +1079,11 @@ export const addWorkflowTableGroupOutput = defineAuthorizedTableUseCase({ context.workspaceId ) const outputs = requireWorkflowOutputs(resolvedWorkflow, group.workflowId) + validateRequestedOutputs( + [...group.outputs, { blockId: input.blockId, path: input.path }], + resolvedWorkflow, + group.workflowId + ) const output = outputs.find( (candidate) => candidate.blockId === input.blockId && candidate.path === input.path ) diff --git a/apps/sim/lib/table/column-types/extension-points.test.ts b/apps/sim/lib/table/column-types/extension-points.test.ts new file mode 100644 index 00000000000..84484a2ae6d --- /dev/null +++ b/apps/sim/lib/table/column-types/extension-points.test.ts @@ -0,0 +1,79 @@ +/** + * @vitest-environment node + */ +import { afterEach, describe, expect, it } from 'vitest' +import { + COLUMN_TYPE_REGISTRY, + validateColumnTypeLimits, + valueForTypeConversion, + wouldExceedColumnTypeLimit, +} from '@/lib/table/column-types' +import type { ColumnDefinition } from '@/lib/table/types' + +const definition = COLUMN_TYPE_REGISTRY.string +const originalMaxPerTable = definition.maxPerTable +const originalValueForConversion = definition.valueForConversion + +function restoreOptionalProperty(key: 'maxPerTable' | 'valueForConversion', value: unknown) { + if (value === undefined) { + Reflect.deleteProperty(definition, key) + return + } + Object.assign(definition, { [key]: value }) +} + +afterEach(() => { + restoreOptionalProperty('maxPerTable', originalMaxPerTable) + restoreOptionalProperty('valueForConversion', originalValueForConversion) +}) + +describe('column type extension points', () => { + it('enforces registry-declared per-table limits', () => { + Object.assign(definition, { maxPerTable: 1 }) + const columns: ColumnDefinition[] = [ + { name: 'first', type: 'string' }, + { name: 'second', type: 'string' }, + ] + + expect(wouldExceedColumnTypeLimit(columns.slice(0, 1), 'string', 1)).toBe(true) + expect(validateColumnTypeLimits(columns)).toEqual([ + `A table can have at most 1 ${definition.label} column`, + ]) + }) + + it('lets the source type normalize a value before conversion', () => { + Object.assign(definition, { + valueForConversion: (_value: unknown, target: ColumnDefinition) => + target.type === 'number' ? 42 : 'unchanged', + }) + + expect( + valueForTypeConversion( + 'stored-value', + { name: 'source', type: 'string' }, + { name: 'target', type: 'number' } + ) + ).toBe(42) + expect( + valueForTypeConversion( + 'stored-value', + { name: 'source', type: 'number' }, + { name: 'target', type: 'string' } + ) + ).toBe('stored-value') + }) + + it('preserves an intentional null from source normalization', () => { + Object.assign(definition, { + valueForConversion: () => null, + }) + + expect( + valueForTypeConversion( + 'stored-value', + { name: 'source', type: 'string' }, + { name: 'target', type: 'number' } + ) + ).toBeNull() + }) +}) diff --git a/apps/sim/lib/table/column-types/import-coercion.ts b/apps/sim/lib/table/column-types/import-coercion.ts new file mode 100644 index 00000000000..9bc0768c61e --- /dev/null +++ b/apps/sim/lib/table/column-types/import-coercion.ts @@ -0,0 +1,20 @@ +import { parseTtlEpochSeconds } from '@/lib/table/column-types/ttl' +import type { ColumnType } from '@/lib/table/column-types/types' +import type { NormalizeDateCellOptions } from '@/lib/table/dates' +import type { JsonValue } from '@/lib/table/types' + +type ImportValue = Exclude +type ImportCoercer = (value: unknown, options?: NormalizeDateCellOptions) => ImportValue + +const IMPORT_COERCERS: Partial> = { + ttl: (value, options) => parseTtlEpochSeconds(value, options), +} + +/** Applies lightweight type-specific CSV coercion without loading the full column registry. */ +export function coerceColumnTypeImportValue( + type: ColumnType, + value: unknown, + options?: NormalizeDateCellOptions +): ImportValue | undefined { + return IMPORT_COERCERS[type]?.(value, options) +} diff --git a/apps/sim/lib/table/column-types/registry.server.ts b/apps/sim/lib/table/column-types/registry.server.ts index 6ba27f5c616..5a6e23791ea 100644 --- a/apps/sim/lib/table/column-types/registry.server.ts +++ b/apps/sim/lib/table/column-types/registry.server.ts @@ -273,6 +273,7 @@ export const COLUMN_TYPE_SERVER_REGISTRY: Record = { number: numberColumnType, boolean: booleanColumnType, date: dateColumnType, + ttl: ttlColumnType, json: jsonColumnType, select: selectColumnType, currency: currencyColumnType, @@ -90,6 +92,16 @@ export function isValueCompatible(value: unknown, target: ColumnDefinition): boo return definition.coerce(value as JsonValue, target).ok } +/** Applies source-owned normalization before a value is converted to another type. */ +export function valueForTypeConversion( + value: JsonValue, + source: ColumnDefinition, + target: ColumnDefinition +): JsonValue { + const normalized = columnTypeOf(source).valueForConversion?.(value, target) + return normalized === undefined ? value : normalized +} + /** This type's own metadata errors; types carrying no metadata report none. */ export function validateTypeMetadata(column: ColumnDefinition): string[] { return columnTypeOf(column).validateDefinition?.(column) ?? [] @@ -115,3 +127,31 @@ export function typeMetadataOf(column: ColumnDefinition): Partial | null { return columnTypeOf(column).filterOperatorsFor?.(column) ?? null } + +/** Schema-level cardinality errors declared by column type definitions. */ +export function validateColumnTypeLimits(columns: readonly ColumnDefinition[]): string[] { + const errors: string[] = [] + for (const definition of ALL_COLUMN_TYPES) { + if (definition.maxPerTable === undefined) continue + if (wouldExceedColumnTypeLimit(columns, definition.id)) { + errors.push(`A table can have at most ${definition.maxPerTable} ${definition.label} column`) + } + } + return errors +} + +/** Whether adding columns of a type would exceed its registry-declared table limit. */ +export function wouldExceedColumnTypeLimit( + columns: readonly ColumnDefinition[], + type: ColumnType, + additionalColumns = 0 +): boolean { + const definition = COLUMN_TYPE_REGISTRY[type] + if (definition.maxPerTable === undefined) return false + + const count = columns.reduce( + (total, column) => total + (column.type === type ? 1 : 0), + additionalColumns + ) + return count > definition.maxPerTable +} diff --git a/apps/sim/lib/table/column-types/ttl.test.ts b/apps/sim/lib/table/column-types/ttl.test.ts new file mode 100644 index 00000000000..717e96ebb8a --- /dev/null +++ b/apps/sim/lib/table/column-types/ttl.test.ts @@ -0,0 +1,189 @@ +/** + * @vitest-environment node + */ + +import { describe, expect, it } from 'vitest' +import { + formatInstantInTimeZone, + getSupportedTimezones, + zonedWallClockToUtc, +} from '@/lib/core/utils/timezone' +import { parseTtlEpochSeconds, ttlColumnType } from '@/lib/table/column-types/ttl' +import { retypeCellRewrite } from '@/lib/table/columns/service' +import type { ColumnDefinition, JsonValue } from '@/lib/table/types' + +const column = (over: Partial): ColumnDefinition => + ({ name: 'col', type: 'string', ...over }) as ColumnDefinition + +describe('TTL column type', () => { + it('converts epoch seconds to an ISO date before retyping', () => { + expect( + retypeCellRewrite(1_700_000_000, column({ type: 'date' }), column({ type: 'ttl' })) + ).toEqual({ value: '2023-11-14T22:13:20Z' }) + }) + + it('keeps blank and malformed TTL values out of the epoch-zero formatter', () => { + const cases: Array<[unknown, string]> = [ + [null, ''], + [undefined, ''], + ['', ''], + [' ', ' '], + [false, 'false'], + [[], ''], + ] + for (const [value, fallback] of cases) { + expect(ttlColumnType.formatForDisplay(value, column({ type: 'ttl' }))).toBe(fallback) + expect(ttlColumnType.formatForInput(value, column({ type: 'ttl' }))).toBe(fallback) + } + }) + + it('preserves blank and malformed TTL values when converting to a date', () => { + const target = column({ type: 'date' }) + const values: JsonValue[] = [null, '', ' ', false, []] + + for (const value of values) { + expect(ttlColumnType.valueForConversion?.(value, target)).toEqual(value) + } + }) + + it.each([ + ['UTC', '2026-06-15T09:00:30', '2026-06-15T09:00:30.000Z'], + ['America/New_York', '2026-06-15T09:00:30', '2026-06-15T13:00:30.000Z'], + ['America/New_York', '2026-01-15T09:00:30', '2026-01-15T14:00:30.000Z'], + ['Asia/Kathmandu', '2026-06-15T09:00:30', '2026-06-15T03:15:30.000Z'], + ['Australia/Lord_Howe', '2026-06-15T09:00:30', '2026-06-14T22:30:30.000Z'], + ])('stores %s wall-clock input as the expected epoch second', (timezone, input, iso) => { + expect(parseTtlEpochSeconds(input, { timezone })).toBe(Date.parse(iso) / 1000) + }) + + it.each([ + ['America/New_York', '2026-11-01T01:30', '2026-11-01T06:30:00.000Z'], + ['Europe/Berlin', '2026-10-25T02:30', '2026-10-25T01:30:00.000Z'], + ['Australia/Lord_Howe', '2026-04-05T01:45', '2026-04-04T15:15:00.000Z'], + ])( + 'chooses the later expiration when %s repeats a wall-clock time', + (timezone, input, laterInstant) => { + expect(parseTtlEpochSeconds(input, { timezone })).toBe(Date.parse(laterInstant) / 1000) + } + ) + + it.each([ + ['America/New_York', '2026-03-08T02:30', '2026-03-08T07:30:00.000Z'], + ['Europe/Berlin', '2026-03-29T02:30', '2026-03-29T01:30:00.000Z'], + ['Australia/Lord_Howe', '2026-10-04T02:15', '2026-10-03T15:45:00.000Z'], + ])( + 'moves a nonexistent %s wall-clock expiration forward across the gap', + (timezone, input, compatibleInstant) => { + expect(parseTtlEpochSeconds(input, { timezone })).toBe(Date.parse(compatibleInstant) / 1000) + } + ) + + it('rounds fractional instants up so expiration is never stored early', () => { + expect(parseTtlEpochSeconds('2023-11-14T22:13:20.001Z')).toBe(1_700_000_001) + expect(parseTtlEpochSeconds('2023-11-14T22:13:20.999Z')).toBe(1_700_000_001) + expect(parseTtlEpochSeconds('2023-11-14T22:13:20.0001Z')).toBe(1_700_000_001) + expect(parseTtlEpochSeconds('2023-11-14t22:13:20.001Z')).toBe(1_700_000_001) + expect(parseTtlEpochSeconds('2023-11-14t17:13:20.001', { timezone: 'America/New_York' })).toBe( + 1_700_000_001 + ) + expect(parseTtlEpochSeconds(new Date('2023-11-14T22:13:20.001Z'))).toBe(1_700_000_001) + expect(parseTtlEpochSeconds('2023-11-14T22:13:20.000Z')).toBe(1_700_000_000) + }) + + it('rounds historical sub-minute timezone offsets toward a later expiration', () => { + const timezone = 'Africa/Monrovia' + const exactInstant = Date.parse('1970-01-01T00:44:30Z') / 1000 + + expect(parseTtlEpochSeconds('1970-01-01T00:00:00', { timezone })).toBeGreaterThanOrEqual( + exactInstant + ) + + const editable = ttlColumnType.formatForInput(exactInstant, column({ type: 'ttl' }), { + timezone, + }) + expect(editable).toBe('1970-01-01T00:00:00-00:45') + expect(parseTtlEpochSeconds(editable, { timezone })).toBeGreaterThanOrEqual(exactInstant) + }) + + it('never resolves representative wall clocks early in any supported timezone', () => { + for (const timezone of getSupportedTimezones()) { + for (const wallClock of ['1970-01-01T00:00:00', '2026-06-15T09:00:30']) { + const exactSecond = Math.ceil( + zonedWallClockToUtc(wallClock, timezone, { ambiguousTime: 'later' }).getTime() / 1000 + ) + expect( + parseTtlEpochSeconds(wallClock, { timezone }), + `${timezone} ${wallClock}` + ).toBeGreaterThanOrEqual(exactSecond) + } + } + }) + + it('never moves stored epoch seconds earlier when formatted in any supported timezone', () => { + for (const timezone of getSupportedTimezones()) { + for (const seconds of [0, Date.parse('2026-11-01T06:30:00Z') / 1000]) { + const editable = ttlColumnType.formatForInput(seconds, column({ type: 'ttl' }), { + timezone, + }) + expect( + parseTtlEpochSeconds(editable, { timezone }), + `${timezone} ${editable}` + ).toBeGreaterThanOrEqual(seconds) + } + } + }) + + it('uses the timezone supplied for each call rather than a previous setting', () => { + const input = '2026-06-15T09:00:30' + + expect(parseTtlEpochSeconds(input, { timezone: 'America/New_York' })).toBe( + Date.parse('2026-06-15T13:00:30Z') / 1000 + ) + expect(parseTtlEpochSeconds(input, { timezone: 'Asia/Kathmandu' })).toBe( + Date.parse('2026-06-15T03:15:30Z') / 1000 + ) + expect(parseTtlEpochSeconds(input, { timezone: 'America/New_York' })).toBe( + Date.parse('2026-06-15T13:00:30Z') / 1000 + ) + }) + + it('round-trips the same epoch after the editor timezone changes', () => { + const seconds = Date.parse('2026-11-01T06:30:00Z') / 1000 + + for (const timezone of [ + 'UTC', + 'America/Los_Angeles', + 'America/New_York', + 'Asia/Kathmandu', + 'Australia/Lord_Howe', + ]) { + const editable = ttlColumnType.formatForInput(seconds, column({ type: 'ttl' }), { timezone }) + expect(editable).toBe(formatInstantInTimeZone(new Date(seconds * 1000), timezone)) + expect(parseTtlEpochSeconds(editable, { timezone })).toBe(seconds) + } + }) + + it('round-trips a low-year expiration through the editor', () => { + const input = '0050-01-15T12:00:00' + const seconds = parseTtlEpochSeconds(input, { timezone: 'UTC' }) + + expect(seconds).toBe(Date.parse(`${input}Z`) / 1000) + const editable = ttlColumnType.formatForInput(seconds, column({ type: 'ttl' }), { + timezone: 'UTC', + }) + expect(editable).toBe(`${input}Z`) + expect(parseTtlEpochSeconds(editable, { timezone: 'UTC' })).toBe(seconds) + }) + + it('keeps the TTL repeated-hour policy separate from ordinary date behavior', () => { + const input = '2026-11-01T01:30' + const timezone = 'America/New_York' + + expect(zonedWallClockToUtc(input, timezone, { ambiguousTime: 'earlier' }).toISOString()).toBe( + '2026-11-01T05:30:00.000Z' + ) + expect(parseTtlEpochSeconds(input, { timezone })).toBe( + Date.parse('2026-11-01T06:30:00Z') / 1000 + ) + }) +}) diff --git a/apps/sim/lib/table/column-types/ttl.ts b/apps/sim/lib/table/column-types/ttl.ts new file mode 100644 index 00000000000..5b04023e05c --- /dev/null +++ b/apps/sim/lib/table/column-types/ttl.ts @@ -0,0 +1,128 @@ +import { TypeTtl } from '@sim/emcn/icons' +import { formatInstantInTimeZone } from '@/lib/core/utils/timezone' +import type { ColumnTypeDefinition } from '@/lib/table/column-types/types' +import { + formatDateCellDisplay, + type NormalizeDateCellOptions, + normalizeDateCellValue, +} from '@/lib/table/dates' +import type { ColumnDefinition } from '@/lib/table/types' + +const NUMERIC_VALUE_PATTERN = /^-?\d+(?:\.\d+)?$/ +const ISO_DATE_PREFIX_PATTERN = /^(\d{4}-\d{2}-\d{2})(?:$|[T ])/i +const FRACTIONAL_SECONDS_PATTERN = /[T ]\d{1,2}:\d{2}:\d{2}\.(\d+)/i + +function isRepresentableEpochSeconds(value: number): boolean { + return Number.isSafeInteger(value) && !Number.isNaN(new Date(value * 1000).getTime()) +} + +/** Rounds toward the future so integer-second storage can never expire an instant early. */ +function epochSecondAtOrAfter(milliseconds: number): number { + return Math.ceil(milliseconds / 1000) +} + +/** Whether an ISO-shaped input names any instant after its whole second. */ +function hasFractionalSecond(value: string): boolean { + const digits = value.match(FRACTIONAL_SECONDS_PATTERN)?.[1] + return digits ? /[1-9]/.test(digits) : false +} + +/** Converts a TTL cell input to integer Unix epoch seconds. */ +export function parseTtlEpochSeconds( + value: unknown, + options?: NormalizeDateCellOptions +): number | null { + if (typeof value === 'number') return isRepresentableEpochSeconds(value) ? value : null + + if (value instanceof Date) { + const milliseconds = value.getTime() + return Number.isNaN(milliseconds) ? null : epochSecondAtOrAfter(milliseconds) + } + + if (typeof value !== 'string') return null + const trimmed = value.trim() + if (!trimmed) return null + + if (NUMERIC_VALUE_PATTERN.test(trimmed)) { + const numeric = Number(trimmed) + return isRepresentableEpochSeconds(numeric) ? numeric : null + } + + const ttlOptions: NormalizeDateCellOptions = { + ...options, + ambiguousTime: 'later', + offsetMinuteRounding: 'floor', + } + const normalized = normalizeDateCellValue(trimmed, ttlOptions) + if (normalized === null) return null + const instant = /^\d{4}-\d{2}-\d{2}$/.test(normalized) + ? normalizeDateCellValue(`${normalized}T00:00:00`, ttlOptions) + : normalized + if (instant === null) return null + const inputIsoDate = trimmed.match(ISO_DATE_PREFIX_PATTERN)?.[1] + if (inputIsoDate && instant.slice(0, 10) !== inputIsoDate) return null + const milliseconds = Date.parse(instant) + (hasFractionalSecond(trimmed) ? 1 : 0) + if (Number.isNaN(milliseconds)) return null + const seconds = epochSecondAtOrAfter(milliseconds) + return isRepresentableEpochSeconds(seconds) ? seconds : null +} + +function epochSecondsToIso(value: unknown): string | null { + if ( + typeof value !== 'number' && + (typeof value !== 'string' || !NUMERIC_VALUE_PATTERN.test(value.trim())) + ) { + return null + } + const seconds = typeof value === 'number' ? value : Number(value) + if (!isRepresentableEpochSeconds(seconds)) return null + return new Date(seconds * 1000).toISOString().replace('.000Z', 'Z') +} + +function epochSecondsToEditable(value: unknown, timeZone?: string): string | null { + const iso = epochSecondsToIso(value) + if (!iso || !timeZone) return iso + return formatInstantInTimeZone(new Date(iso), timeZone, { offsetMinuteRounding: 'floor' }) +} + +export const ttlColumnType: ColumnTypeDefinition = { + id: 'ttl', + label: 'Expiration', + maxPerTable: 1, + icon: TypeTtl, + jsonbCast: 'numeric', + storesOpaqueIds: false, + supportsUnique: true, + sampleValue: 1_706_659_200, + ownedMetadata: [], + workflowInputType: 'number', + editor: 'date', + expandable: false, + typeaheadPattern: /[\d\-/]/, + parseErrorMessage: 'Invalid expiration date', + + coerce(value, _column, context) { + const seconds = parseTtlEpochSeconds(value, context) + return seconds === null ? { ok: false } : { ok: true, value: seconds } + }, + + valueForConversion(value, target: ColumnDefinition) { + if (target.type !== 'date') return value + return epochSecondsToIso(value) ?? value + }, + + validateCell(value, column) { + return typeof value === 'number' && isRepresentableEpochSeconds(value) + ? null + : `${column.name} must be valid epoch seconds` + }, + + formatForDisplay(value) { + const iso = epochSecondsToIso(value) + return iso === null ? String(value ?? '') : formatDateCellDisplay(iso, { seconds: true }) + }, + + formatForInput(value, _column, context) { + return epochSecondsToEditable(value, context?.timezone) ?? String(value ?? '') + }, +} diff --git a/apps/sim/lib/table/column-types/types.ts b/apps/sim/lib/table/column-types/types.ts index f481cea0a28..72edeead0d0 100644 --- a/apps/sim/lib/table/column-types/types.ts +++ b/apps/sim/lib/table/column-types/types.ts @@ -20,6 +20,7 @@ */ import type React from 'react' +import type { NormalizeDateCellOptions } from '@/lib/table/dates' import type { ColumnDefinition, JsonValue } from '@/lib/table/types' /** @@ -36,6 +37,7 @@ export const COLUMN_TYPES = [ 'currency', 'boolean', 'date', + 'ttl', 'json', 'select', ] as const @@ -72,6 +74,8 @@ export interface ColumnTypeDefinition { /** Human label in the type picker, column header menu, and docs. */ readonly label: string + /** Maximum columns of this type a table may contain. Omitted when unlimited. */ + readonly maxPerTable?: number /** Type icon. A component reference only — never invoked server-side. */ readonly icon: React.ComponentType<{ className?: string }> /** @@ -157,7 +161,14 @@ export interface ColumnTypeDefinition { * implementation — the server calls it before persisting and the grid calls * it to fill the optimistic cache, so the two can no longer disagree. */ - coerce(value: JsonValue, column: ColumnDefinition): CoerceResult + coerce( + value: JsonValue, + column: ColumnDefinition, + context?: NormalizeDateCellOptions + ): CoerceResult + + /** Source-owned normalization applied before checking or rewriting a type conversion. */ + valueForConversion?(value: JsonValue, target: ColumnDefinition): JsonValue /** Validates a stored cell's shape. Returns an error message, or null when valid. */ validateCell(value: JsonValue, column: ColumnDefinition): string | null @@ -203,7 +214,11 @@ export interface ColumnTypeDefinition { formatForDisplay(value: unknown, column: ColumnDefinition): string /** Stored value → the text an editor input starts with. */ - formatForInput(value: unknown, column: ColumnDefinition): string + formatForInput( + value: unknown, + column: ColumnDefinition, + context?: NormalizeDateCellOptions + ): string /** * Metadata stamped onto a newly created column of this type, so the schema diff --git a/apps/sim/lib/table/columns/retype-cell.test.ts b/apps/sim/lib/table/columns/retype-cell.test.ts index 479563fc74b..b1b6a74d888 100644 --- a/apps/sim/lib/table/columns/retype-cell.test.ts +++ b/apps/sim/lib/table/columns/retype-cell.test.ts @@ -2,13 +2,25 @@ * @vitest-environment node */ -import { describe, expect, it } from 'vitest' +import { afterEach, describe, expect, it } from 'vitest' +import { COLUMN_TYPE_REGISTRY } from '@/lib/table/column-types' import { retypeCellRewrite } from '@/lib/table/columns/service' import type { ColumnDefinition } from '@/lib/table/types' const column = (over: Partial): ColumnDefinition => ({ name: 'col', type: 'string', ...over }) as ColumnDefinition +const sourceDefinition = COLUMN_TYPE_REGISTRY.string +const originalValueForConversion = sourceDefinition.valueForConversion + +afterEach(() => { + if (originalValueForConversion === undefined) { + Reflect.deleteProperty(sourceDefinition, 'valueForConversion') + return + } + Object.assign(sourceDefinition, { valueForConversion: originalValueForConversion }) +}) + describe('retypeCellRewrite', () => { it('preserves an empty string the target type can hold', () => { // `''` is a real stored value: `coerceRowValues` keeps it for `string`, and @@ -32,6 +44,29 @@ describe('retypeCellRewrite', () => { expect(retypeCellRewrite('true', column({ type: 'boolean' }))).toEqual({ value: true }) }) + it('writes back null produced by source normalization', () => { + Object.assign(sourceDefinition, { valueForConversion: () => null }) + + expect( + retypeCellRewrite('stored-value', column({ type: 'number' }), column({ type: 'string' })) + ).toEqual({ value: null }) + }) + + it('coerces source-normalized values into select storage', () => { + Object.assign(sourceDefinition, { valueForConversion: () => 'Choice' }) + + expect( + retypeCellRewrite( + 'stored-value', + column({ + type: 'select', + options: [{ id: 'opt_choice', name: 'Choice' }], + }), + column({ type: 'string' }) + ) + ).toEqual({ value: 'opt_choice' }) + }) + it('skips a cell whose stored value already matches the coercion', () => { expect(retypeCellRewrite('kept', column({ type: 'json' }))).toBeNull() expect(retypeCellRewrite(3, column({ type: 'json' }))).toBeNull() diff --git a/apps/sim/lib/table/columns/service.ts b/apps/sim/lib/table/columns/service.ts index b68ce8393ac..fa0a274d146 100644 --- a/apps/sim/lib/table/columns/service.ts +++ b/apps/sim/lib/table/columns/service.ts @@ -27,6 +27,7 @@ import { columnTypeOf, isValueCompatible, TYPE_SPECIFIC_COLUMN_KEYS, + valueForTypeConversion, } from '@/lib/table/column-types' import { migrationFrom, @@ -42,6 +43,7 @@ import { updateTableRowsWithDerivedSecretProvenance } from '@/lib/table/rows/sec import { assertValidSchema } from '@/lib/table/schema-invariants' import { selectValueToNames } from '@/lib/table/select-values' import { withLockedTable } from '@/lib/table/service' +import { assertTableRowTtlEnabled } from '@/lib/table/ttl-availability' import { scaledStatementTimeoutMs, setTableTxTimeouts } from '@/lib/table/tx' import type { ColumnDefinition, @@ -130,6 +132,8 @@ export async function addTableColumn( requestId: string, options?: ColumnMutationOptions ): Promise { + if (column.type === 'ttl') await assertTableRowTtlEnabled() + return withLockedTable( tableId, async (table, trx) => { @@ -767,17 +771,24 @@ export function applyPendingRename( */ export function retypeCellRewrite( value: unknown, - target: ColumnDefinition + target: ColumnDefinition, + source?: ColumnDefinition ): { value: JsonValue } | null { if (value === null || value === undefined) return null - if (!isValueCompatibleWithColumn(value, target)) { + const effective = source + ? valueForTypeConversion(value as JsonValue, source, target) + : (value as JsonValue) + + if (effective === null) return { value: null } + + if (!isValueCompatibleWithColumn(effective, target)) { // Incompatible non-blanks never reach here: the compatibility scan already // refused the whole conversion for them. - return value === '' ? { value: null } : null + return effective === '' ? { value: null } : null } - const coerced = columnTypeById(target.type).coerce(value as JsonValue, target) + const coerced = columnTypeById(target.type).coerce(effective, target) if (coerced.ok && !Object.is(coerced.value, value)) return { value: coerced.value } return null } @@ -849,6 +860,8 @@ export async function updateColumnType( requestId: string, options?: ColumnMutationOptions ): Promise { + if (data.newType === 'ttl') await assertTableRowTtlEnabled() + return withLockedTable( data.tableId, async (table, trx) => { @@ -913,6 +926,7 @@ export async function updateColumnType( const isSelectType = data.newType === 'select' const targetOptions = data.options ?? column.options ?? [] const targetMultiple = data.multiple ?? column.multiple + const sourceNormalizesConversion = columnTypeOf(column).valueForConversion !== undefined // Leaving `select` behind: stored cells hold option ids, which mean nothing // once the column is text/number/etc. Check compatibility against the option // NAME — that's what the cell will actually become (migrated below). @@ -944,6 +958,12 @@ export async function updateColumnType( isSelectType, targetMultiple: !!targetMultiple, }) + const renamedColumns = schema.columns.map((c, i) => (i === columnIndex ? convertedColumn : c)) + const updatedColumns = renamedColumns.map((c, i) => + i === columnIndex ? applyPendingRename(renamedColumns, columnIndex, data.newName) : c + ) + const updatedSchema: TableSchema = { ...schema, columns: updatedColumns } + assertValidSchema(updatedSchema, table.metadata?.columnOrder) let incompatibleCount = 0 let blankCount = 0 @@ -972,7 +992,7 @@ export async function updateColumnType( const effective = convertingAwayFromSelect ? selectValueForConversion(column, value) - : value + : valueForTypeConversion(value as JsonValue, column, convertedColumn) if (!isValueCompatibleWithColumn(effective, convertedColumn)) { if (effective === null || effective === '') { @@ -1000,11 +1020,6 @@ export async function updateColumnType( ) } - const renamedColumns = schema.columns.map((c, i) => (i === columnIndex ? convertedColumn : c)) - const updatedColumns = renamedColumns.map((c, i) => - i === columnIndex ? applyPendingRename(renamedColumns, columnIndex, data.newName) : c - ) - const columnValidation = validateColumnDefinition(updatedColumns[columnIndex]) if (!columnValidation.valid) { throw new OrchestrationError( @@ -1013,7 +1028,6 @@ export async function updateColumnType( ) } - const updatedSchema: TableSchema = { ...schema, columns: updatedColumns } const now = new Date() // Cell rewrites are owned by the column-type registry, keyed by direction. @@ -1029,9 +1043,7 @@ export async function updateColumnType( resolved: new Map(), } await migrationFrom(column.type)?.(migrationContext) - if (isSelectType) { - await migrationTo(data.newType)?.(migrationContext) - } else { + if (!isSelectType || sourceNormalizesConversion) { let rewriteAfterId: string | undefined while (true) { const rows = await readColumnRetypePage( @@ -1045,7 +1057,7 @@ export async function updateColumnType( if (rows.length === 0) break const coercedByRowId = new Map() for (const row of rows) { - const rewrite = retypeCellRewrite(row.value, convertedColumn) + const rewrite = retypeCellRewrite(row.value, convertedColumn, column) if (rewrite) coercedByRowId.set(row.id, rewrite.value) } await writeBackCoercedCells( @@ -1059,6 +1071,9 @@ export async function updateColumnType( if (rows.length < retypeScanBatchSize) break } } + if (isSelectType) { + await migrationTo(data.newType)?.(migrationContext) + } // A `unique` arriving with this retype is validated HERE, against the values // the conversion just wrote — not by the separate constraint write that diff --git a/apps/sim/lib/table/columns/ttl-limit.test.ts b/apps/sim/lib/table/columns/ttl-limit.test.ts new file mode 100644 index 00000000000..8738e7eee49 --- /dev/null +++ b/apps/sim/lib/table/columns/ttl-limit.test.ts @@ -0,0 +1,99 @@ +/** + * @vitest-environment node + */ +import { beforeEach, describe, expect, it, vi } from 'vitest' +import type { TableDefinition, TableLocks } from '@/lib/table/types' + +const { mockAssertTableRowTtlEnabled, mockTimeoutExecute, mockWithLockedTable } = vi.hoisted( + () => ({ + mockAssertTableRowTtlEnabled: vi.fn(), + mockTimeoutExecute: vi.fn(), + mockWithLockedTable: vi.fn(), + }) +) + +vi.mock('@/lib/table/service', () => ({ withLockedTable: mockWithLockedTable })) +vi.mock('@/lib/table/ttl-availability', () => ({ + assertTableRowTtlEnabled: mockAssertTableRowTtlEnabled, +})) + +import { addTableColumn, updateColumnType } from '@/lib/table/columns/service' + +const UNLOCKED: TableLocks = { + schemaLocked: false, + insertLocked: false, + updateLocked: false, + deleteLocked: false, +} + +function makeTable(): TableDefinition { + return { + id: 'table-1', + name: 'Tasks', + schema: { + columns: [ + { id: 'col-name', name: 'name', type: 'string' }, + { id: 'col-ttl', name: 'expires_at', type: 'ttl' }, + ], + }, + rowCount: 0, + maxRows: 100, + workspaceId: 'workspace-1', + createdBy: 'user-1', + locks: UNLOCKED, + createdAt: new Date(), + updatedAt: new Date(), + } +} + +const transaction = new Proxy( + { execute: mockTimeoutExecute }, + { + get(target, property) { + if (property in target) return target[property as keyof typeof target] + throw new Error(`Unexpected transaction method: ${String(property)}`) + }, + } +) + +describe('TTL column mutation limit', () => { + beforeEach(() => { + vi.clearAllMocks() + mockAssertTableRowTtlEnabled.mockResolvedValue(undefined) + mockTimeoutExecute.mockResolvedValue([]) + mockWithLockedTable.mockImplementation(async (_tableId, mutate) => + mutate(makeTable(), transaction) + ) + }) + + it('rejects adding a TTL column before locking when the feature is disabled', async () => { + mockAssertTableRowTtlEnabled.mockRejectedValue(new Error('Expiration columns are not enabled')) + + await expect( + addTableColumn('table-1', { name: 'expiry', type: 'ttl' }, 'request-1') + ).rejects.toThrow('Expiration columns are not enabled') + expect(mockWithLockedTable).not.toHaveBeenCalled() + }) + + it('rejects retyping to TTL before locking when the feature is disabled', async () => { + mockAssertTableRowTtlEnabled.mockRejectedValue(new Error('Expiration columns are not enabled')) + + await expect( + updateColumnType({ tableId: 'table-1', columnName: 'name', newType: 'ttl' }, 'request-1') + ).rejects.toThrow('Expiration columns are not enabled') + expect(mockWithLockedTable).not.toHaveBeenCalled() + }) + + it('rejects adding a second TTL column before persistence', async () => { + await expect( + addTableColumn('table-1', { name: 'another_expiry', type: 'ttl' }, 'request-1') + ).rejects.toThrow('A table can have at most 1 Expiration column') + }) + + it('rejects retyping another column to TTL before scanning cells', async () => { + await expect( + updateColumnType({ tableId: 'table-1', columnName: 'name', newType: 'ttl' }, 'request-1') + ).rejects.toThrow('A table can have at most 1 Expiration column') + expect(mockTimeoutExecute).toHaveBeenCalled() + }) +}) diff --git a/apps/sim/lib/table/constants.test.ts b/apps/sim/lib/table/constants.test.ts index d3bd1ef4848..050246dddb2 100644 --- a/apps/sim/lib/table/constants.test.ts +++ b/apps/sim/lib/table/constants.test.ts @@ -39,7 +39,9 @@ declare module '@/lib/table/constants?constants-test' { import { getBillingDisabledTableLimits, + getDeleteSnapshotBatchSize, getMaxPageBytes, + getMaxRowSizeBytes, TABLE_LIMITS, } from '@/lib/table/constants?constants-test' @@ -86,3 +88,35 @@ describe('getMaxPageBytes', () => { expect(getMaxPageBytes()).toBe(2 * 1024 * 1024) }) }) + +describe('getMaxRowSizeBytes', () => { + beforeEach(() => { + for (const key of Object.keys(mockEnv)) delete mockEnv[key] + }) + + it('caps overrides at the delete snapshot byte budget', () => { + mockEnv.TABLE_MAX_ROW_SIZE_BYTES = String(TABLE_LIMITS.DELETE_SNAPSHOT_BATCH_MAX_BYTES * 2) + + expect(getMaxRowSizeBytes()).toBe(TABLE_LIMITS.DELETE_SNAPSHOT_BATCH_MAX_BYTES) + }) +}) + +describe('getDeleteSnapshotBatchSize', () => { + beforeEach(() => { + for (const key of Object.keys(mockEnv)) delete mockEnv[key] + }) + + it('derives a worst-case row cap from the delete snapshot byte budget', () => { + expect(getDeleteSnapshotBatchSize()).toBe( + Math.floor(TABLE_LIMITS.DELETE_SNAPSHOT_BATCH_MAX_BYTES / TABLE_LIMITS.MAX_ROW_SIZE_BYTES) + ) + }) + + it('always processes one row and never exceeds the delete row-count cap', () => { + mockEnv.TABLE_MAX_ROW_SIZE_BYTES = String(TABLE_LIMITS.DELETE_SNAPSHOT_BATCH_MAX_BYTES * 2) + expect(getDeleteSnapshotBatchSize()).toBe(1) + + mockEnv.TABLE_MAX_ROW_SIZE_BYTES = '1' + expect(getDeleteSnapshotBatchSize()).toBe(TABLE_LIMITS.DELETE_BATCH_SIZE) + }) +}) diff --git a/apps/sim/lib/table/constants.ts b/apps/sim/lib/table/constants.ts index 2da9bf6b58a..56e1f25c05b 100644 --- a/apps/sim/lib/table/constants.ts +++ b/apps/sim/lib/table/constants.ts @@ -39,6 +39,14 @@ export const TABLE_LIMITS = { UPDATE_BATCH_SIZE: 100, /** Batch size for bulk delete operations */ DELETE_BATCH_SIZE: 1000, + /** + * Serialized row-data budget for one committed delete snapshot batch. Batch + * deletes measure stored JSONB bytes while holding row locks and stop at this + * budget. A historical row already larger than the budget is deleted alone + * and logged; current writes cannot create another because row admission is + * capped at the same value. + */ + DELETE_SNAPSHOT_BATCH_MAX_BYTES: 32 * 1024 * 1024, /** Maximum rows per batch insert */ MAX_BATCH_INSERT_SIZE: 1000, /** Maximum rows per bulk update/delete operation */ @@ -140,13 +148,33 @@ export function getMaxPageBytes(): number { /** * Maximum serialized size in bytes of a single row. Defaults to * `TABLE_LIMITS.MAX_ROW_SIZE_BYTES`; overridable via the - * `TABLE_MAX_ROW_SIZE_BYTES` env var (server-only, read at call time). + * `TABLE_MAX_ROW_SIZE_BYTES` env var (server-only, read at call time), capped + * at the delete snapshot budget so every accepted row fits in one batch. */ export function getMaxRowSizeBytes(): number { - return envNumber(env.TABLE_MAX_ROW_SIZE_BYTES, TABLE_LIMITS.MAX_ROW_SIZE_BYTES, { - min: 1, - integer: true, - }) + return Math.min( + envNumber(env.TABLE_MAX_ROW_SIZE_BYTES, TABLE_LIMITS.MAX_ROW_SIZE_BYTES, { + min: 1, + integer: true, + }), + TABLE_LIMITS.DELETE_SNAPSHOT_BATCH_MAX_BYTES + ) +} + +/** + * Initial row-count cap for a delete snapshot batch. Delete paths additionally + * measure the selected rows as stored and shorten each transaction to the byte + * budget; this count avoids scanning more candidate ids than current writes can + * possibly fit. + */ +export function getDeleteSnapshotBatchSize(): number { + return Math.max( + 1, + Math.min( + TABLE_LIMITS.DELETE_BATCH_SIZE, + Math.floor(TABLE_LIMITS.DELETE_SNAPSHOT_BATCH_MAX_BYTES / getMaxRowSizeBytes()) + ) + ) } export type PlanName = keyof typeof DEFAULT_TABLE_PLAN_LIMITS diff --git a/apps/sim/lib/table/dates.test.ts b/apps/sim/lib/table/dates.test.ts index 3ff51410e15..36df539d5c3 100644 --- a/apps/sim/lib/table/dates.test.ts +++ b/apps/sim/lib/table/dates.test.ts @@ -22,6 +22,8 @@ function localOffsetSuffix(local: Date): string { describe('isCalendarDateString', () => { it('accepts YYYY-MM-DD and rejects everything else', () => { expect(isCalendarDateString('2026-07-06')).toBe(true) + expect(isCalendarDateString('2024-02-29')).toBe(true) + expect(isCalendarDateString('2026-02-30')).toBe(false) expect(isCalendarDateString('2026-13-45')).toBe(false) expect(isCalendarDateString('2026-07-06T00:00:00Z')).toBe(false) expect(isCalendarDateString('07/06/2026')).toBe(false) @@ -80,6 +82,71 @@ describe('normalizeDateCellValue', () => { ) }) + it('uses the requested low year when applying IANA timezone rules', () => { + const normalized = normalizeDateCellValue('0050-01-15T12:00:00', { + timezone: 'America/New_York', + }) + + expect(normalized).toBe('0050-01-15T12:00:00-04:56') + expect(storedDateToEditable(normalized ?? '')).toBe('0050-01-15T12:00:00-04:56') + }) + + it('reads localized numeric wall clocks before applying the provided IANA zone', () => { + expect(normalizeDateCellValue('3/8/2026 2:30 AM', { timezone: 'America/New_York' })).toBe( + '2026-03-08T02:30:00-05:00' + ) + expect(normalizeDateCellValue('7/6/2026, 16:04:55', { timezone: 'Asia/Tokyo' })).toBe( + '2026-07-06T16:04:55+09:00' + ) + }) + + it('reads month-name wall clocks independently of the runtime timezone', () => { + expect(normalizeDateCellValue('March 8, 2026 2:30 AM', { timezone: 'America/New_York' })).toBe( + '2026-03-08T02:30:00-05:00' + ) + }) + + it('rejects impossible month-name calendar dates', () => { + expect( + normalizeDateCellValue('February 29, 2025 2:30 AM', { timezone: 'America/New_York' }) + ).toBeNull() + expect( + normalizeDateCellValue('April 31, 2026 4:04 PM', { timezone: 'America/New_York' }) + ).toBeNull() + }) + + it('accepts valid leap-day month-name wall clocks in either date order', () => { + expect( + normalizeDateCellValue('February 29, 2024 4:04 PM', { timezone: 'America/New_York' }) + ).toBe('2024-02-29T16:04:00-05:00') + expect(normalizeDateCellValue('29 Feb 2024 4:04 PM', { timezone: 'America/New_York' })).toBe( + '2024-02-29T16:04:00-05:00' + ) + }) + + it.each([ + ['America/New_York', '2026-11-01 01:30:00', '2026-11-01T01:30:00-04:00'], + ['America/New_York', '2026-03-08 02:30:00', '2026-03-08T02:30:00-05:00'], + ['Asia/Kathmandu', '2026-06-15 09:00:00', '2026-06-15T09:00:00+05:45'], + ['Australia/Lord_Howe', '2026-06-15 09:00:00', '2026-06-15T09:00:00+10:30'], + ])('uses the shared timezone rules for %s', (timezone, input, expected) => { + expect(normalizeDateCellValue(input, { timezone })).toBe(expected) + }) + + it('uses each provided timezone independently when the setting changes', () => { + const input = '2026-06-15 09:00:30' + + expect(normalizeDateCellValue(input, { timezone: 'America/New_York' })).toBe( + '2026-06-15T09:00:30-04:00' + ) + expect(normalizeDateCellValue(input, { timezone: 'Asia/Kathmandu' })).toBe( + '2026-06-15T09:00:30+05:45' + ) + expect(normalizeDateCellValue(input, { timezone: 'America/New_York' })).toBe( + '2026-06-15T09:00:30-04:00' + ) + }) + it('ignores the zone option when the input carries an explicit offset', () => { expect( normalizeDateCellValue('2026-07-06T23:04:55.000Z', { timezone: 'America/New_York' }) @@ -107,6 +174,28 @@ describe('normalizeDateCellValue', () => { expect(normalizeDateCellValue('2026-13-45')).toBeNull() expect(normalizeDateCellValue('13/06/2026')).toBeNull() }) + + it('rejects impossible ISO calendar and time fields', () => { + expect(normalizeDateCellValue('2026-02-30')).toBeNull() + expect(normalizeDateCellValue('2025-02-29T12:00:00Z')).toBeNull() + expect(normalizeDateCellValue('2026-02-30 12:00', { timezone: 'UTC' })).toBeNull() + expect(normalizeDateCellValue('2026-02-30 12:00 PDT')).toBeNull() + expect(normalizeDateCellValue('2026-07-06T24:00', { timezone: 'UTC' })).toBeNull() + expect(normalizeDateCellValue('2026-07-06 24:00+00')).toBeNull() + expect(normalizeDateCellValue('2026-07-06T12:60:00-04:00')).toBeNull() + expect(normalizeDateCellValue('02/30/2026')).toBeNull() + expect(normalizeDateCellValue('February 29, 2025')).toBeNull() + expect(normalizeDateCellValue('February 29, 2025 12:00')).toBeNull() + expect(normalizeDateCellValue('February 29, 2025 12:00', { timezone: 'UTC' })).toBeNull() + }) + + it('accepts leap days and valid daylight-saving gap wall clocks', () => { + expect(normalizeDateCellValue('2024-02-29')).toBe('2024-02-29') + expect(normalizeDateCellValue('2024-02-29T12:00:00Z')).toBe('2024-02-29T12:00:00Z') + expect(normalizeDateCellValue('2026-03-08T02:30:00', { timezone: 'America/New_York' })).toBe( + '2026-03-08T02:30:00-05:00' + ) + }) }) describe('formatDateCellDisplay', () => { diff --git a/apps/sim/lib/table/dates.ts b/apps/sim/lib/table/dates.ts index a38eca7f21e..f112ad2f242 100644 --- a/apps/sim/lib/table/dates.ts +++ b/apps/sim/lib/table/dates.ts @@ -23,7 +23,15 @@ * barrel (the barrel is server-tainted). */ -const CALENDAR_DATE_PATTERN = /^\d{4}-\d{2}-\d{2}$/ +import { + formatIsoYear, + formatUtcOffsetSuffix, + type ZonedWallClockOptions, + zonedWallClockWithOffset, +} from '@/lib/core/utils/timezone' + +const CALENDAR_DATE_PATTERN = /^(\d{4})-(\d{2})-(\d{2})$/ +const LOCALIZED_CALENDAR_DATE_PATTERN = /^(\d{1,2})\/(\d{1,2})\/(\d{4})$/ /** * Canonical (or canonical-enough legacy) instant: a literal wall time with an @@ -31,7 +39,35 @@ const CALENDAR_DATE_PATTERN = /^\d{4}-\d{2}-\d{2}$/ * groups are the wall-time fields display renders verbatim. */ const WALL_INSTANT_PATTERN = - /^(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2})(?::(\d{2}))?(?:\.\d+)?(?:Z|[+-]\d{2}:?\d{2})?$/ + /^(\d{4})-(\d{2})-(\d{2})[T ](\d{2}):(\d{2})(?::(\d{2}))?(?:\.\d+)?(?:\s*(?:Z|UTC?|GMT|[ECMP][SD]T)|[+-]\d{1,2}(?::?\d{2})?)?$/i + +const LOCALIZED_WALL_CLOCK_PATTERN = + /^(\d{1,2})\/(\d{1,2})\/(\d{4})[ ,]+(\d{1,2}):(\d{2})(?::(\d{2}))?(?:\s*(AM|PM))?$/i + +const MONTH_NAME_PATTERN = + 'Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:t(?:ember)?)?|Oct(?:ober)?|Nov(?:ember)?|Dec(?:ember)?' +const MONTH_FIRST_DATE_PATTERN = new RegExp( + `\\b(${MONTH_NAME_PATTERN})\\s+(\\d{1,2})(?:,)?\\s+(\\d{4})\\b`, + 'i' +) +const DAY_FIRST_DATE_PATTERN = new RegExp( + `\\b(\\d{1,2})\\s+(${MONTH_NAME_PATTERN})(?:,)?\\s+(\\d{4})\\b`, + 'i' +) +const MONTH_BY_ABBREVIATION: Record = { + JAN: 1, + FEB: 2, + MAR: 3, + APR: 4, + MAY: 5, + JUN: 6, + JUL: 7, + AUG: 8, + SEP: 9, + OCT: 10, + NOV: 11, + DEC: 12, +} /** * Legacy shape: old CSV imports stored date-only columns as UTC-midnight @@ -67,81 +103,10 @@ const US_ABBREVIATION_OFFSET_MINUTES: Record = { /** True when `value` is a canonical timezone-free calendar date. */ export function isCalendarDateString(value: string): boolean { - return CALENDAR_DATE_PATTERN.test(value) && !Number.isNaN(Date.parse(value)) -} - -/** A wall-clock reading of an instant in some timezone. */ -export interface WallClockParts { - year: number - /** 1-based month. */ - month: number - day: number - hour: number - minute: number - second: number -} - -/** - * The wall-clock reading of `date` in `timeZone` — or in the runtime's local - * zone when omitted. Throws a RangeError on an invalid IANA zone — callers - * validate at the boundary. - */ -export function getWallClockParts(date: Date, timeZone?: string): WallClockParts { - if (!timeZone) { - return { - year: date.getFullYear(), - month: date.getMonth() + 1, - day: date.getDate(), - hour: date.getHours(), - minute: date.getMinutes(), - second: date.getSeconds(), - } - } - const parts = new Intl.DateTimeFormat('en-US', { - timeZone, - hourCycle: 'h23', - year: 'numeric', - month: '2-digit', - day: '2-digit', - hour: '2-digit', - minute: '2-digit', - second: '2-digit', - }).formatToParts(date) - const get = (type: string) => Number(parts.find((p) => p.type === type)?.value) - return { - year: get('year'), - month: get('month'), - day: get('day'), - hour: get('hour'), - minute: get('minute'), - second: get('second'), - } -} - -/** Offset of `timeZone` from UTC (ms east) at the moment `at`. */ -function zoneOffsetMs(timeZone: string, at: Date): number { - const wall = getWallClockParts(at, timeZone) - const asUtc = Date.UTC(wall.year, wall.month - 1, wall.day, wall.hour, wall.minute, wall.second) - return asUtc - at.getTime() -} - -/** - * Converts a wall-clock reading in `timeZone` to the UTC instant it denotes. - * Two-pass so readings near a DST transition resolve with the offset in - * force at that wall time. - */ -function wallTimeInZoneToUtc(wall: Date, timeZone: string): Date { - const guess = Date.UTC( - wall.getFullYear(), - wall.getMonth(), - wall.getDate(), - wall.getHours(), - wall.getMinutes(), - wall.getSeconds(), - wall.getMilliseconds() + const calendar = value.match(CALENDAR_DATE_PATTERN) + return Boolean( + calendar && isValidCalendarDay(Number(calendar[1]), Number(calendar[2]), Number(calendar[3])) ) - const adjusted = guess - zoneOffsetMs(timeZone, new Date(guess)) - return new Date(guess - zoneOffsetMs(timeZone, new Date(adjusted))) } function pad(n: number): string { @@ -149,19 +114,11 @@ function pad(n: number): string { } function toLocalCalendarDate(date: Date): string { - return `${date.getFullYear()}-${pad(date.getMonth() + 1)}-${pad(date.getDate())}` + return `${formatIsoYear(date.getFullYear())}-${pad(date.getMonth() + 1)}-${pad(date.getDate())}` } function toUtcCalendarDate(date: Date): string { - return `${date.getUTCFullYear()}-${pad(date.getUTCMonth() + 1)}-${pad(date.getUTCDate())}` -} - -/** `Z` for zero, else `±HH:MM`. */ -function formatOffsetSuffix(offsetMinutes: number): string { - if (offsetMinutes === 0) return 'Z' - const sign = offsetMinutes > 0 ? '+' : '-' - const abs = Math.abs(offsetMinutes) - return `${sign}${pad(Math.floor(abs / 60))}:${pad(abs % 60)}` + return `${formatIsoYear(date.getUTCFullYear())}-${pad(date.getUTCMonth() + 1)}-${pad(date.getUTCDate())}` } /** @@ -186,14 +143,118 @@ function extractExplicitOffsetMinutes(value: string): number | null { function formatUtcFieldsAsWall(shifted: Date, offsetMinutes: number): string { return `${toUtcCalendarDate(shifted)}T${pad(shifted.getUTCHours())}:${pad( shifted.getUTCMinutes() - )}:${pad(shifted.getUTCSeconds())}${formatOffsetSuffix(offsetMinutes)}` + )}:${pad(shifted.getUTCSeconds())}${formatUtcOffsetSuffix(offsetMinutes)}` } /** Serializes local-read fields of `parsed` as a wall time with `offset`. */ function formatLocalFieldsAsWall(parsed: Date, offsetMinutes: number): string { return `${toLocalCalendarDate(parsed)}T${pad(parsed.getHours())}:${pad( parsed.getMinutes() - )}:${pad(parsed.getSeconds())}${formatOffsetSuffix(offsetMinutes)}` + )}:${pad(parsed.getSeconds())}${formatUtcOffsetSuffix(offsetMinutes)}` +} + +/** True when numeric year, month, and day fields describe a real calendar day. */ +function isValidCalendarDay(year: number, month: number, day: number): boolean { + if (month < 1 || month > 12 || day < 1) return false + const leapYear = year % 4 === 0 && (year % 100 !== 0 || year % 400 === 0) + const daysInMonth = [31, leapYear ? 29 : 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31] + return day <= daysInMonth[month - 1] +} + +/** Validates and formats numeric wall-clock fields as naive ISO. */ +function formatValidatedWallClock( + year: number, + month: number, + day: number, + hour: number, + minute: number, + second: number +): string | null { + if ( + !isValidCalendarDay(year, month, day) || + hour < 0 || + hour > 23 || + minute < 0 || + minute > 59 || + second < 0 || + second > 59 + ) { + return null + } + return `${String(year).padStart(4, '0')}-${pad(month)}-${pad(day)}T${pad(hour)}:${pad(minute)}:${pad(second)}` +} + +/** Reads an ISO-shaped wall clock literally, before runtime timezone normalization. */ +function parseIsoWallClock(match: RegExpMatchArray): string | null { + return formatValidatedWallClock( + Number(match[1]), + Number(match[2]), + Number(match[3]), + Number(match[4]), + Number(match[5]), + Number(match[6] ?? 0) + ) +} + +/** Reads a supported US numeric wall clock literally, including 12-hour input. */ +function parseLocalizedWallClock(match: RegExpMatchArray): string | null { + const meridiem = match[7]?.toUpperCase() + let hour = Number(match[4]) + if (meridiem) { + if (hour < 1 || hour > 12) return null + hour = (hour % 12) + (meridiem === 'PM' ? 12 : 0) + } + return formatValidatedWallClock( + Number(match[3]), + Number(match[1]), + Number(match[2]), + hour, + Number(match[5]), + Number(match[6] ?? 0) + ) +} + +interface CalendarFields { + year: number + month: number + day: number +} + +/** Extracts literal calendar fields from supported month-name date forms. */ +function extractMonthNameCalendar(value: string): CalendarFields | null { + const monthFirst = value.match(MONTH_FIRST_DATE_PATTERN) + if (monthFirst) { + return { + year: Number(monthFirst[3]), + month: MONTH_BY_ABBREVIATION[monthFirst[1].slice(0, 3).toUpperCase()], + day: Number(monthFirst[2]), + } + } + const dayFirst = value.match(DAY_FIRST_DATE_PATTERN) + if (!dayFirst) return null + return { + year: Number(dayFirst[3]), + month: MONTH_BY_ABBREVIATION[dayFirst[2].slice(0, 3).toUpperCase()], + day: Number(dayFirst[1]), + } +} + +/** Recovers broader naive `Date.parse` inputs without consulting the runtime timezone. */ +function parseNaiveWallClockAsUtc(value: string): string | null { + const calendar = extractMonthNameCalendar(value) + if (calendar && !isValidCalendarDay(calendar.year, calendar.month, calendar.day)) return null + const ms = Date.parse(`${value} UTC`) + if (Number.isNaN(ms)) return null + const parsed = new Date(ms) + if ( + calendar && + (parsed.getUTCFullYear() !== calendar.year || + parsed.getUTCMonth() + 1 !== calendar.month || + parsed.getUTCDate() !== calendar.day) + ) { + return null + } + return `${toUtcCalendarDate(parsed)}T${pad(parsed.getUTCHours())}:${pad(parsed.getUTCMinutes())}:${pad(parsed.getUTCSeconds())}` } export interface NormalizeDateCellOptions { @@ -205,6 +266,14 @@ export interface NormalizeDateCellOptions { * zone. */ timezone?: string + /** + * Which instant to use when a naive wall time occurs twice during a DST + * fall-back. Ordinary date cells preserve their historical earlier-instant + * behavior; instant-like callers may explicitly choose `later`. + */ + ambiguousTime?: ZonedWallClockOptions['ambiguousTime'] + /** How sub-minute historical offsets are serialized to RFC 3339 minutes. */ + offsetMinuteRounding?: ZonedWallClockOptions['offsetMinuteRounding'] } /** @@ -220,12 +289,37 @@ export function normalizeDateCellValue( ): string | null { const trimmed = raw.trim() if (!trimmed) return null - if (CALENDAR_DATE_PATTERN.test(trimmed)) { - return Number.isNaN(Date.parse(trimmed)) ? null : trimmed + const calendar = trimmed.match(CALENDAR_DATE_PATTERN) + if (calendar) { + return isValidCalendarDay(Number(calendar[1]), Number(calendar[2]), Number(calendar[3])) + ? trimmed + : null + } + const localizedCalendar = trimmed.match(LOCALIZED_CALENDAR_DATE_PATTERN) + if (localizedCalendar) { + const month = Number(localizedCalendar[1]) + const day = Number(localizedCalendar[2]) + const year = Number(localizedCalendar[3]) + return isValidCalendarDay(year, month, day) + ? `${String(year).padStart(4, '0')}-${pad(month)}-${pad(day)}` + : null } + const isoMatch = trimmed.match(WALL_INSTANT_PATTERN) + const isoWallClock = isoMatch ? parseIsoWallClock(isoMatch) : undefined + if (isoWallClock === null) return null + const localizedMatch = trimmed.match(LOCALIZED_WALL_CLOCK_PATTERN) + const localizedWallClock = localizedMatch ? parseLocalizedWallClock(localizedMatch) : undefined + if (localizedWallClock === null) return null const ms = Date.parse(trimmed) if (Number.isNaN(ms)) return null const parsed = new Date(ms) + const monthNameCalendar = extractMonthNameCalendar(trimmed) + if ( + monthNameCalendar && + !isValidCalendarDay(monthNameCalendar.year, monthNameCalendar.month, monthNameCalendar.day) + ) { + return null + } if (!TIME_COMPONENT_PATTERN.test(trimmed)) { return ISO_REDUCED_DATE_PATTERN.test(trimmed) ? toUtcCalendarDate(parsed) @@ -238,11 +332,12 @@ export function normalizeDateCellValue( return formatUtcFieldsAsWall(new Date(ms + explicitOffset * 60_000), explicitOffset) } if (options?.timezone) { - // `parsed`'s local getters recover the wall-clock fields V8 read from the - // naive string; stamp them with the requested zone's offset at that time. - const instant = wallTimeInZoneToUtc(parsed, options.timezone) - const offsetMinutes = Math.round(zoneOffsetMs(options.timezone, instant) / 60_000) - return formatLocalFieldsAsWall(parsed, offsetMinutes) + const wallClock = isoWallClock ?? localizedWallClock ?? parseNaiveWallClockAsUtc(trimmed) + if (!wallClock) return null + return zonedWallClockWithOffset(wallClock, options.timezone, { + ambiguousTime: options.ambiguousTime ?? 'earlier', + offsetMinuteRounding: options.offsetMinuteRounding, + }) } return formatLocalFieldsAsWall(parsed, -parsed.getTimezoneOffset()) } diff --git a/apps/sim/lib/table/delete-runner.test.ts b/apps/sim/lib/table/delete-runner.test.ts index aa19a0faac4..8eb163aeed0 100644 --- a/apps/sim/lib/table/delete-runner.test.ts +++ b/apps/sim/lib/table/delete-runner.test.ts @@ -16,6 +16,7 @@ const { mockAppendTableEvent, mockSignalTableRowsChanged, mockBuildFilterClause, + mockFireTableTrigger, } = vi.hoisted(() => ({ mockGetTableById: vi.fn(), mockGetJobProgress: vi.fn(), @@ -28,6 +29,7 @@ const { mockAppendTableEvent: vi.fn(), mockSignalTableRowsChanged: vi.fn(), mockBuildFilterClause: vi.fn(), + mockFireTableTrigger: vi.fn(), })) vi.mock('@/lib/table/service', () => ({ @@ -49,6 +51,7 @@ vi.mock('@/lib/table/events', () => ({ signalTableRowsChanged: mockSignalTableRowsChanged, })) vi.mock('@/lib/table/sql', () => ({ buildFilterClause: mockBuildFilterClause })) +vi.mock('@/lib/table/trigger', () => ({ fireTableTrigger: mockFireTableTrigger })) vi.mock('@/lib/table/constants', () => ({ TABLE_LIMITS: { DELETE_PAGE_SIZE: 2 }, USER_TABLE_ROWS_SQL_NAME: 'user_table_rows', @@ -62,7 +65,13 @@ const UNLOCKED = { updateLocked: false, deleteLocked: false, } -const table = { id: 'tbl_1', workspaceId: 'ws_1', schema: { columns: [] }, locks: UNLOCKED } +const table = { + id: 'tbl_1', + name: 'Issues', + workspaceId: 'ws_1', + schema: { columns: [] }, + locks: UNLOCKED, +} const cutoff = new Date('2026-06-05T00:00:00Z') function basePayload(overrides = {}) { @@ -77,7 +86,23 @@ describe('runTableDelete', () => { mockUpdateJobProgress.mockResolvedValue(true) mockMarkJobReady.mockResolvedValue(true) mockMarkJobFailed.mockResolvedValue(undefined) - mockDeletePageByIds.mockImplementation((_t, _w, ids: string[]) => Promise.resolve(ids.length)) + mockDeletePageByIds.mockImplementation( + async ( + _t, + _w, + ids: string[], + _proof, + _revalidate, + onDeleted?: ( + rows: Array<{ id: string; data: Record }>, + table?: typeof table + ) => void | Promise + ) => { + const rows = ids.map((id) => ({ id, data: { title: id } })) + await onDeleted?.(rows) + return rows.length + } + ) mockBuildFilterClause.mockReturnValue({}) }) @@ -115,6 +140,7 @@ describe('runTableDelete', () => { 'ws_1', ['a', 'b'], expect.anything(), + expect.any(Function), expect.any(Function) ) expect(mockMarkJobCanceled).toHaveBeenCalledWith('tbl_1', 'job_1') @@ -150,6 +176,7 @@ describe('runTableDelete', () => { 'ws_1', ['a', 'b'], expect.anything(), + expect.any(Function), expect.any(Function) ) expect(mockDeletePageByIds).toHaveBeenNthCalledWith( @@ -158,17 +185,62 @@ describe('runTableDelete', () => { 'ws_1', ['c'], expect.anything(), + expect.any(Function), expect.any(Function) ) expect(mockMarkJobReady).toHaveBeenCalledWith('tbl_1', 'job_1') expect(mockAppendTableEvent).toHaveBeenCalledWith( expect.objectContaining({ kind: 'job', type: 'delete', status: 'ready', progress: 3 }) ) + expect(mockFireTableTrigger).toHaveBeenCalledTimes(2) + expect(mockFireTableTrigger).toHaveBeenNthCalledWith( + 1, + 'tbl_1', + 'ws_1', + 'Issues', + 'delete', + [ + { id: 'a', data: { title: 'a' } }, + { id: 'b', data: { title: 'b' } }, + ], + null, + table.schema, + expect.any(String) + ) // The live grid must be told rows changed so deleted rows drop out of every open editor — // the `job` progress event only drives the delete meter, not the rows query. expect(mockSignalTableRowsChanged).toHaveBeenCalledWith('tbl_1') }) + it('uses the table definition revalidated with each committed delete batch', async () => { + const renamedTable = { + ...table, + name: 'Renamed issues', + schema: { columns: [{ id: 'col-title', name: 'Renamed title', type: 'string' }] }, + } + mockSelectRowIdPage.mockResolvedValueOnce(['a']).mockResolvedValueOnce([]) + mockDeletePageByIds.mockImplementationOnce( + async (_t, _w, ids: string[], _proof, _revalidate, onDeleted) => { + const rows = ids.map((id) => ({ id, data: { 'col-title': id } })) + await onDeleted?.(rows, renamedTable) + return rows.length + } + ) + + await runTableDelete(basePayload()) + + expect(mockFireTableTrigger).toHaveBeenCalledWith( + renamedTable.id, + renamedTable.workspaceId, + renamedTable.name, + 'delete', + [{ id: 'a', data: { 'col-title': 'a' } }], + null, + renamedTable.schema, + expect.any(String) + ) + }) + it('stops once maxRows is reached and caps the final page fetch to the remaining budget', async () => { // budget 3 with page size 2: first page fills 2, the second is capped to the remaining 1. mockSelectRowIdPage.mockResolvedValueOnce(['a', 'b']).mockResolvedValueOnce(['c']) @@ -196,6 +268,7 @@ describe('runTableDelete', () => { 'ws_1', ['x'], expect.anything(), + expect.any(Function), expect.any(Function) ) // Second page is queried after the last id of the first page (cursor advanced past 'keep'). diff --git a/apps/sim/lib/table/delete-runner.ts b/apps/sim/lib/table/delete-runner.ts index a1c14302bed..6b010f02518 100644 --- a/apps/sim/lib/table/delete-runner.ts +++ b/apps/sim/lib/table/delete-runner.ts @@ -14,9 +14,10 @@ import { } from '@/lib/table/jobs/service' import { assertRowDelete, type MutationProof, TableLockedError } from '@/lib/table/mutation-locks' import type { DbTransaction } from '@/lib/table/planner' -import { deletePageByIds, selectRowIdPage } from '@/lib/table/rows/ordering' +import { type DeletedTableRow, deletePageByIds, selectRowIdPage } from '@/lib/table/rows/ordering' import { getTableById } from '@/lib/table/service' import { buildFilterClause } from '@/lib/table/sql' +import { fireTableTrigger } from '@/lib/table/trigger' const logger = createLogger('TableDeleteRunner') @@ -122,6 +123,22 @@ export async function runTableDelete(payload: TableDeletePayload): Promise // an absent filter is still legitimate (delete-all is an explicit caller mode). if (filter && !filterClause) throw new Error('Filter is required for bulk delete') const excluded = new Set(excludeRowIds ?? []) + const dispatchDeleteTriggers = async ( + rows: DeletedTableRow[], + committedTable?: TableDefinition + ) => { + const triggerTable = committedTable ?? table + await fireTableTrigger( + triggerTable.id, + triggerTable.workspaceId, + triggerTable.name, + 'delete', + rows, + null, + triggerTable.schema, + requestId + ) + } // Resume the persisted count: a retried attempt's earlier batches are already committed, // so starting at zero would overwrite cumulative progress with this attempt's smaller @@ -170,7 +187,14 @@ export async function runTableDelete(payload: TableDeletePayload): Promise // returns or throws. (An attempt that ends up committing nothing only over-refetches — harmless.) deletedAny = true try { - processed += await deletePageByIds(tableId, workspaceId, toDelete, pageProof, revalidate) + processed += await deletePageByIds( + tableId, + workspaceId, + toDelete, + pageProof, + revalidate, + dispatchDeleteTriggers + ) } catch (err) { if (!(err instanceof TableLockedError)) throw err // A lock landed between batches. Batches already committed stay diff --git a/apps/sim/lib/table/import.test.ts b/apps/sim/lib/table/import.test.ts index d0aa04500e1..f8259213a45 100644 --- a/apps/sim/lib/table/import.test.ts +++ b/apps/sim/lib/table/import.test.ts @@ -172,6 +172,27 @@ describe('import', () => { ) expect(coerceValue('not-a-date', 'date')).toBe('not-a-date') }) + + it('coerces TTL imports to epoch seconds and rejects invalid input', () => { + expect(coerceValue('2023-11-14T22:13:20Z', 'ttl')).toBe(1_700_000_000) + expect(coerceValue('1700000000', 'ttl')).toBe(1_700_000_000) + expect(coerceValue('2023-11-14 17:13:20', 'ttl', { timezone: 'America/New_York' })).toBe( + 1_700_000_000 + ) + expect(coerceValue('not-a-date', 'ttl')).toBeNull() + }) + + it('applies the timezone supplied to each TTL import independently', () => { + const input = '2026-06-15 09:00:30' + + expect(coerceValue(input, 'ttl', { timezone: 'America/New_York' })).toBe( + Date.parse('2026-06-15T13:00:30Z') / 1000 + ) + expect(coerceValue(input, 'ttl', { timezone: 'Asia/Kathmandu' })).toBe( + Date.parse('2026-06-15T03:15:30Z') / 1000 + ) + expect(coerceValue('2023-11-14T22:13:20.001Z', 'ttl')).toBe(1_700_000_001) + }) }) describe('buildAutoMapping', () => { diff --git a/apps/sim/lib/table/import.ts b/apps/sim/lib/table/import.ts index 2ad05fa5fa5..50ea7cfc3b2 100644 --- a/apps/sim/lib/table/import.ts +++ b/apps/sim/lib/table/import.ts @@ -15,6 +15,7 @@ import type { Options as CsvParseOptions } from 'csv-parse' import { OrchestrationError } from '@/lib/core/orchestration/types' import { getColumnId } from '@/lib/table/column-keys' import type { ColumnType } from '@/lib/table/column-types' +import { coerceColumnTypeImportValue } from '@/lib/table/column-types/import-coercion' import { parseCurrencyInput } from '@/lib/table/currency' import { type NormalizeDateCellOptions, normalizeDateCellValue } from '@/lib/table/dates' import type { ColumnDefinition, RowData, TableSchema } from '@/lib/table/types' @@ -468,12 +469,10 @@ export function inferSchemaFromCsv( * back to the original string when unparseable so that schema validation can * reject it with context rather than silently inserting `null`. * - * Deliberately NOT routed through the column-type registry's `coerce`, despite - * covering the same types. The registry's contract is "coerced or rejected", - * which the write path turns into `null`; an import instead wants an - * unparseable date or JSON blob to survive as its raw string so the row-level - * validation error names the offending value. Unifying the two would silently - * swap a descriptive import error for a blanked cell. + * Deliberately not routed through the column-type registry: its contract is + * "coerced or rejected", while an import needs invalid raw text to survive so + * row-level validation can name it. Type-specific import behavior uses a + * lightweight capability map so CSV clients do not load the full registry. */ export function coerceValue( value: unknown, @@ -481,6 +480,10 @@ export function coerceValue( options?: NormalizeDateCellOptions & { currencyCode?: string } ): string | number | boolean | null | Record | unknown[] { if (value === null || value === undefined || value === '') return null + + const typeSpecificValue = coerceColumnTypeImportValue(colType, value, options) + if (typeSpecificValue !== undefined) return typeSpecificValue + switch (colType) { case 'number': { const n = Number(value) diff --git a/apps/sim/lib/table/orchestration/import.test.ts b/apps/sim/lib/table/orchestration/import.test.ts index 405613b0de1..dfac5d90e8d 100644 --- a/apps/sim/lib/table/orchestration/import.test.ts +++ b/apps/sim/lib/table/orchestration/import.test.ts @@ -271,6 +271,39 @@ describe('performTableCsvImport', () => { }) }) + it('counts invalid TTL cells that the import blanks', async () => { + const result = await performTableCsvImport( + importParams({ + table: { + ...TABLE, + schema: { + columns: [ + { + id: 'col_expires_at', + name: 'expires_at', + type: 'ttl', + required: false, + unique: false, + }, + ], + }, + }, + fileStream: csvStream('expires_at\n2023-11-14T22:13:20Z\nnot-a-date\n'), + }) + ) + + expect(result.success).toBe(true) + expect(result.data?.rejections).toEqual({ + rowsRejected: 0, + cellsRejected: 1, + rejectedSamples: [], + }) + expect(mockImportAppendRows.mock.calls[0][2]).toEqual([ + { col_expires_at: 1_700_000_000 }, + { col_expires_at: null }, + ]) + }) + it('omits the accounting entirely from a clean import', async () => { const result = await performTableCsvImport(importParams()) diff --git a/apps/sim/lib/table/rows/__tests__/ordering-delete.test.ts b/apps/sim/lib/table/rows/__tests__/ordering-delete.test.ts new file mode 100644 index 00000000000..c16b325af9d --- /dev/null +++ b/apps/sim/lib/table/rows/__tests__/ordering-delete.test.ts @@ -0,0 +1,209 @@ +/** + * @vitest-environment node + */ +import { databaseMock } from '@sim/testing' +import { beforeEach, describe, expect, it, vi } from 'vitest' +import type { MutationProof } from '@/lib/table/mutation-locks' +import type { DbTransaction } from '@/lib/table/planner' + +const { mockGetDeleteSnapshotBatchSize } = vi.hoisted(() => ({ + mockGetDeleteSnapshotBatchSize: vi.fn(() => 1), +})) + +vi.mock('@/lib/table/constants', () => ({ + getDeleteSnapshotBatchSize: mockGetDeleteSnapshotBatchSize, + TABLE_LIMITS: { DELETE_SNAPSHOT_BATCH_MAX_BYTES: 100, UPDATE_BATCH_SIZE: 100 }, +})) +vi.mock('@/lib/table/tx', () => ({ setTableTxTimeouts: vi.fn() })) + +import { + type DeletedRowsHandler, + deleteOrderedRowsByIds, + deletePageByIds, + planDeleteSnapshotBatch, +} from '@/lib/table/rows/ordering' + +const mockTransaction = databaseMock.db.transaction as ReturnType +const proof = {} as MutationProof<'delete'> + +type DeleteRunner = (onDeleted: DeletedRowsHandler) => Promise + +describe('ordered row delete trigger handoff', () => { + beforeEach(() => { + vi.clearAllMocks() + mockGetDeleteSnapshotBatchSize.mockReturnValue(1) + }) + + it.each([ + [ + 'direct deletes', + (onDeleted: DeletedRowsHandler) => + deleteOrderedRowsByIds({ + tableId: 'table-1', + workspaceId: 'workspace-1', + rowIds: ['row-1', 'row-2'], + proof, + onDeleted, + }), + ], + [ + 'background delete pages', + (onDeleted: DeletedRowsHandler) => + deletePageByIds('table-1', 'workspace-1', ['row-1', 'row-2'], proof, undefined, onDeleted), + ], + ])( + 'runs %s handlers after commit and before the next batch', + async (_label, run: DeleteRunner) => { + const events: string[] = [] + let batchIndex = 0 + let releaseFirstHandler: (() => void) | undefined + const firstHandlerGate = new Promise((resolve) => { + releaseFirstHandler = resolve + }) + const trx = { + select: () => ({ + from: () => ({ + where: () => ({ + orderBy: () => ({ + for: async () => [{ id: `row-${batchIndex + 1}`, snapshotBytes: 20 }], + }), + }), + }), + }), + delete: () => ({ + where: () => ({ + returning: async () => { + const id = `row-${batchIndex + 1}` + batchIndex++ + return [{ id, data: { title: id } }] + }, + }), + }), + } as unknown as DbTransaction + + mockTransaction.mockImplementation( + async (callback: (transaction: DbTransaction) => Promise) => { + const result = await callback(trx) + events.push(`commit-${mockTransaction.mock.calls.length}`) + return result + } + ) + + const onDeleted = vi.fn(async (rows: Array<{ id: string }>) => { + events.push(`trigger-${rows[0]?.id}`) + if (rows[0]?.id === 'row-1') await firstHandlerGate + }) + const pending = run(onDeleted) + + await vi.waitFor(() => { + expect(events).toEqual(['commit-1', 'trigger-row-1']) + }) + expect(mockTransaction).toHaveBeenCalledTimes(1) + + releaseFirstHandler?.() + await pending + + expect(events).toEqual(['commit-1', 'trigger-row-1', 'commit-2', 'trigger-row-2']) + expect(onDeleted.mock.calls.map(([rows]) => rows)).toEqual([ + [{ id: 'row-1', data: { title: 'row-1' } }], + [{ id: 'row-2', data: { title: 'row-2' } }], + ]) + } + ) + + it('splits one count-sized candidate batch at the snapshot byte budget', async () => { + mockGetDeleteSnapshotBatchSize.mockReturnValue(3) + const snapshots = [ + [ + { id: 'row-1', snapshotBytes: 60 }, + { id: 'row-2', snapshotBytes: 60 }, + { id: 'row-3', snapshotBytes: 10 }, + ], + [ + { id: 'row-2', snapshotBytes: 60 }, + { id: 'row-3', snapshotBytes: 10 }, + ], + ] + const deletedBatches = [ + [{ id: 'row-1', data: { title: 'row-1' } }], + [ + { id: 'row-2', data: { title: 'row-2' } }, + { id: 'row-3', data: { title: 'row-3' } }, + ], + ] + let transactionIndex = 0 + + mockTransaction.mockImplementation( + async (callback: (transaction: DbTransaction) => Promise) => { + const currentIndex = transactionIndex++ + const trx = { + select: () => ({ + from: () => ({ + where: () => ({ + orderBy: () => ({ + for: async () => snapshots[currentIndex], + }), + }), + }), + }), + delete: () => ({ + where: () => ({ + returning: async () => deletedBatches[currentIndex], + }), + }), + } as unknown as DbTransaction + return callback(trx) + } + ) + const onDeleted = vi.fn() + + await expect( + deleteOrderedRowsByIds({ + tableId: 'table-1', + workspaceId: 'workspace-1', + rowIds: ['row-1', 'row-2', 'row-3'], + proof, + onDeleted, + }) + ).resolves.toEqual(['row-1', 'row-2', 'row-3']) + + expect(mockTransaction).toHaveBeenCalledTimes(2) + expect(onDeleted.mock.calls.map(([rows]) => rows)).toEqual(deletedBatches) + }) +}) + +describe('delete snapshot byte planning', () => { + it('stops before an existing row would exceed the byte budget', () => { + expect( + planDeleteSnapshotBatch( + ['missing-row', 'row-1', 'row-2'], + [ + { id: 'row-1', snapshotBytes: 60 }, + { id: 'row-2', snapshotBytes: 60 }, + ], + 100 + ) + ).toEqual({ + rowIds: ['missing-row', 'row-1'], + consumedCount: 2, + oversizedRow: undefined, + }) + }) + + it('isolates an oversized legacy row so no other snapshot joins it', () => { + expect( + planDeleteSnapshotBatch( + ['legacy-row', 'row-2'], + [ + { id: 'legacy-row', snapshotBytes: 150 }, + { id: 'row-2', snapshotBytes: 10 }, + ], + 100 + ) + ).toEqual({ + rowIds: ['legacy-row'], + consumedCount: 1, + oversizedRow: { id: 'legacy-row', snapshotBytes: 150 }, + }) + }) +}) diff --git a/apps/sim/lib/table/rows/ordering.ts b/apps/sim/lib/table/rows/ordering.ts index c776cc6e075..f20ebee0ef0 100644 --- a/apps/sim/lib/table/rows/ordering.ts +++ b/apps/sim/lib/table/rows/ordering.ts @@ -8,9 +8,10 @@ import { db } from '@sim/db' import { userTableRows } from '@sim/db/schema' +import { createLogger } from '@sim/logger' import { and, asc, desc, eq, gt, inArray, lt, lte, type SQL, sql } from 'drizzle-orm' import type { DbOrTx } from '@/lib/db/types' -import { TABLE_LIMITS } from '@/lib/table/constants' +import { getDeleteSnapshotBatchSize, TABLE_LIMITS } from '@/lib/table/constants' import type { MutationProof } from '@/lib/table/mutation-locks' import { keyBetween, nKeysBetween } from '@/lib/table/order-key' import { type DbExecutor, type DbTransaction, withSeqscanOff } from '@/lib/table/planner' @@ -19,6 +20,106 @@ import { mutateTableRowsWithSecretProvenance } from '@/lib/table/rows/secret-pro import { setTableTxTimeouts } from '@/lib/table/tx' import type { RowData, TableDefinition, TableRowSecretProvenanceWrite } from '@/lib/table/types' +const logger = createLogger('TableRowOrdering') + +export interface DeletedTableRow { + id: string + data: RowData +} + +export type DeletedRowsHandler = ( + rows: DeletedTableRow[], + table?: TableDefinition +) => void | Promise + +interface DeleteSnapshotSize { + id: string + snapshotBytes: number +} + +interface DeleteSnapshotBatchPlan { + rowIds: string[] + consumedCount: number + oversizedRow?: DeleteSnapshotSize +} + +/** + * Selects the largest input-order prefix whose existing rows fit the snapshot + * byte budget. Missing ids are consumed without cost. A legacy row that already + * exceeds the budget is isolated as the only existing row in its transaction so + * deleting historical data remains possible without combining it with another + * snapshot. + */ +export function planDeleteSnapshotBatch( + candidateRowIds: readonly string[], + snapshotSizes: readonly DeleteSnapshotSize[], + maxBytes = TABLE_LIMITS.DELETE_SNAPSHOT_BATCH_MAX_BYTES +): DeleteSnapshotBatchPlan { + const bytesById = new Map(snapshotSizes.map((row) => [row.id, row.snapshotBytes])) + let consumedCount = 0 + let batchBytes = 0 + let existingRows = 0 + let oversizedRow: DeleteSnapshotSize | undefined + + for (const id of candidateRowIds) { + const measuredBytes = bytesById.get(id) + if (measuredBytes === undefined) { + consumedCount++ + continue + } + const snapshotBytes = + Number.isFinite(measuredBytes) && measuredBytes >= 0 ? measuredBytes : maxBytes + 1 + if (existingRows > 0 && batchBytes + snapshotBytes > maxBytes) break + + consumedCount++ + existingRows++ + batchBytes += snapshotBytes + if (snapshotBytes > maxBytes) { + oversizedRow = { id, snapshotBytes } + break + } + } + + return { + rowIds: candidateRowIds.slice(0, consumedCount), + consumedCount, + oversizedRow, + } +} + +async function planLockedDeleteSnapshotBatch( + trx: DbTransaction, + tableId: string, + workspaceId: string, + candidateRowIds: readonly string[] +): Promise { + const snapshotSizes = await trx + .select({ + id: userTableRows.id, + snapshotBytes: sql`octet_length(${userTableRows.data}::text)`.mapWith(Number), + }) + .from(userTableRows) + .where( + and( + eq(userTableRows.tableId, tableId), + eq(userTableRows.workspaceId, workspaceId), + inArray(userTableRows.id, [...candidateRowIds]) + ) + ) + .orderBy(asc(userTableRows.id)) + .for('update') + return planDeleteSnapshotBatch(candidateRowIds, snapshotSizes) +} + +function warnForOversizedLegacySnapshot(oversizedRow: DeleteSnapshotSize | undefined): void { + if (!oversizedRow) return + logger.warn('Deleting oversized legacy row in an isolated snapshot batch', { + rowId: oversizedRow.id, + snapshotBytes: oversizedRow.snapshotBytes, + maxBytes: TABLE_LIMITS.DELETE_SNAPSHOT_BATCH_MAX_BYTES, + }) +} + /** * Starting `position` for an append import — `max(position) + 1`, or 0 when empty. Read once, * unlocked, before streaming: the import worker is the table's sole writer, so it can assign @@ -274,8 +375,8 @@ export async function insertOrderedRow(params: { /** * Deletes a single row by id in its own transaction. Deleting a row never changes - * another row's `order_key`, so no positional reshift is needed. Returns `false` - * when no row matched. + * another row's `order_key`, so no positional reshift is needed. Returns the + * deleted row snapshot, or `null` when no row matched. */ export async function deleteOrderedRow(params: { tableId: string @@ -283,9 +384,9 @@ export async function deleteOrderedRow(params: { workspaceId: string /** Proof the caller asserted the delete lock (see `mutation-locks.ts`). */ proof: MutationProof<'delete'> -}): Promise { +}): Promise { const { tableId, rowId, workspaceId } = params - return db.transaction(async (trx) => { + const deletedRow = await db.transaction(async (trx) => { await setTableTxTimeouts(trx) const [deleted] = await trx .delete(userTableRows) @@ -296,16 +397,27 @@ export async function deleteOrderedRow(params: { eq(userTableRows.workspaceId, workspaceId) ) ) - .returning({ id: userTableRows.id }) - return Boolean(deleted) + .returning({ id: userTableRows.id, data: userTableRows.data }) + return deleted ? { id: deleted.id, data: deleted.data as RowData } : null }) + if (deletedRow) { + const snapshotBytes = Buffer.byteLength(JSON.stringify(deletedRow.data), 'utf8') + warnForOversizedLegacySnapshot( + snapshotBytes > TABLE_LIMITS.DELETE_SNAPSHOT_BATCH_MAX_BYTES + ? { id: deletedRow.id, snapshotBytes } + : undefined + ) + } + return deletedRow } /** - * Deletes the given row ids in batches within one transaction. Deletes leave - * `order_key` untouched, so no positional recompaction is needed. Returns the - * deleted row ids. The caller resolves which ids to delete (used by both - * delete-by-ids and delete-by-filter). + * Deletes the given row ids in byte-bounded, independently committed batches. + * Deletes leave `order_key` untouched, so no positional recompaction is needed. + * The post-commit handler is awaited before the next batch so deleted JSON + * snapshots cannot accumulate in memory. Returns only the compact deleted ids; + * the caller resolves which ids to delete (used by both delete-by-ids and + * delete-by-filter). */ export async function deleteOrderedRowsByIds(params: { tableId: string @@ -313,28 +425,38 @@ export async function deleteOrderedRowsByIds(params: { rowIds: string[] /** Proof the caller asserted the delete lock (see `mutation-locks.ts`). */ proof: MutationProof<'delete'> -}): Promise<{ id: string }[]> { - const { tableId, workspaceId, rowIds } = params + /** Handles each bounded snapshot batch after its transaction commits. */ + onDeleted?: DeletedRowsHandler +}): Promise { + const { tableId, workspaceId, rowIds, onDeleted } = params if (rowIds.length === 0) return [] - return db.transaction(async (trx) => { - await setTableTxTimeouts(trx, { statementMs: 60_000 }) - const deleted: { id: string }[] = [] - for (let i = 0; i < rowIds.length; i += TABLE_LIMITS.DELETE_BATCH_SIZE) { - const batch = rowIds.slice(i, i + TABLE_LIMITS.DELETE_BATCH_SIZE) + const batchSize = getDeleteSnapshotBatchSize() + const deletedIds: string[] = [] + let index = 0 + while (index < rowIds.length) { + const candidates = rowIds.slice(index, index + batchSize) + const { rows, plan } = await db.transaction(async (trx) => { + await setTableTxTimeouts(trx, { statementMs: 60_000 }) + const plan = await planLockedDeleteSnapshotBatch(trx, tableId, workspaceId, candidates) const rows = await trx .delete(userTableRows) .where( and( eq(userTableRows.tableId, tableId), eq(userTableRows.workspaceId, workspaceId), - inArray(userTableRows.id, batch) + inArray(userTableRows.id, plan.rowIds) ) ) - .returning({ id: userTableRows.id }) - deleted.push(...rows) - } - return deleted - }) + .returning({ id: userTableRows.id, data: userTableRows.data }) + return { rows, plan } + }) + index += plan.consumedCount + warnForOversizedLegacySnapshot(plan.oversizedRow) + const deletedRows = rows.map((row) => ({ id: row.id, data: row.data as RowData })) + deletedIds.push(...deletedRows.map((row) => row.id)) + await onDeleted?.(deletedRows) + } + return deletedIds } /** @@ -467,26 +589,36 @@ export async function deletePageByIds( /** Proof the caller asserted the delete lock (see `mutation-locks.ts`). */ _proof: MutationProof<'delete'>, /** Re-asserts the lock inside each batch transaction. See {@link guardBatch}. */ - revalidate?: MutationRevalidator + revalidate?: MutationRevalidator, + /** Called after each batch commits, with snapshots suitable for delete triggers. */ + onDeleted?: DeletedRowsHandler ): Promise { let deleted = 0 - for (let i = 0; i < rowIds.length; i += TABLE_LIMITS.DELETE_BATCH_SIZE) { - const batch = rowIds.slice(i, i + TABLE_LIMITS.DELETE_BATCH_SIZE) - const rows = await db.transaction(async (trx) => { + const batchSize = getDeleteSnapshotBatchSize() + let index = 0 + while (index < rowIds.length) { + const candidates = rowIds.slice(index, index + batchSize) + const { rows, table, plan } = await db.transaction(async (trx) => { await setTableTxTimeouts(trx, { statementMs: 60_000 }) - await guardBatch(trx, tableId, revalidate) - return trx + const table = await guardBatch(trx, tableId, revalidate) + const plan = await planLockedDeleteSnapshotBatch(trx, tableId, workspaceId, candidates) + const rows = await trx .delete(userTableRows) .where( and( eq(userTableRows.tableId, tableId), eq(userTableRows.workspaceId, workspaceId), - inArray(userTableRows.id, batch) + inArray(userTableRows.id, plan.rowIds) ) ) - .returning({ id: userTableRows.id }) + .returning({ id: userTableRows.id, data: userTableRows.data }) + return { rows, table, plan } }) - deleted += rows.length + index += plan.consumedCount + warnForOversizedLegacySnapshot(plan.oversizedRow) + const deletedRows = rows.map((row) => ({ id: row.id, data: row.data as RowData })) + deleted += deletedRows.length + await onDeleted?.(deletedRows, table) } return deleted } diff --git a/apps/sim/lib/table/rows/service.ts b/apps/sim/lib/table/rows/service.ts index f8933c05cdf..f036f1efce0 100644 --- a/apps/sim/lib/table/rows/service.ts +++ b/apps/sim/lib/table/rows/service.ts @@ -51,6 +51,7 @@ import { } from '@/lib/table/rows/executions' import { acquireRowOrderLock, + type DeletedTableRow, deleteOrderedRow, deleteOrderedRowsByIds, insertOrderedRow, @@ -110,6 +111,24 @@ import { cancelWorkflowGroupRuns, runWorkflowColumn } from '@/lib/table/workflow const logger = createLogger('TableRowsService') +async function dispatchDeleteTriggers( + table: TableDefinition, + deletedRows: DeletedTableRow[], + requestId: string +): Promise { + if (deletedRows.length === 0) return + await fireTableTrigger( + table.id, + table.workspaceId, + table.name, + 'delete', + deletedRows, + null, + table.schema, + requestId + ) +} + /** * Inserts a single row into a table. * @@ -207,6 +226,7 @@ export async function insertRow( void fireTableTrigger( data.tableId, + table.workspaceId, table.name, 'insert', [insertedRow], @@ -383,7 +403,16 @@ export function dispatchAfterBatchInsert( requestId: string, actorUserId?: string | null ): void { - void fireTableTrigger(table.id, table.name, 'insert', result, null, table.schema, requestId) + void fireTableTrigger( + table.id, + table.workspaceId, + table.name, + 'insert', + result, + null, + table.schema, + requestId + ) // Scope to the newly-inserted row ids so the dispatcher doesn't walk every // row in the table. After the sidecar migration, all existing rows have // zero entries → `mode:'new'`'s `NOT EXISTS` filter would otherwise include @@ -865,6 +894,7 @@ export async function upsertRow( }) void fireTableTrigger( data.tableId, + table.workspaceId, table.name, 'insert', [result.row], @@ -876,6 +906,7 @@ export async function upsertRow( const oldRows = new Map([[result.row.id, result.previousData]]) void fireTableTrigger( data.tableId, + table.workspaceId, table.name, 'update', [result.row], @@ -1801,6 +1832,7 @@ export async function updateRow( const oldRows = new Map([[data.rowId, existingRow.data as RowData]]) void fireTableTrigger( data.tableId, + table.workspaceId, table.name, 'update', [updatedRow], @@ -1886,6 +1918,7 @@ export async function deleteRow( if (!deleted) throw new OrchestrationError('not_found', 'Row not found') logger.info(`[${requestId}] Deleted row ${rowId} from table ${table.id}`) + void dispatchDeleteTriggers(table, [deleted], requestId) } type BulkUpdateMatch = { id: string; data: RowData } @@ -2061,6 +2094,7 @@ function dispatchBulkUpdateEffects( })) void fireTableTrigger( table.id, + table.workspaceId, table.name, 'update', updatedRows, @@ -2483,6 +2517,7 @@ export async function batchUpdateRows( if (updatedRowsForTrigger.length > 0) { void fireTableTrigger( data.tableId, + table.workspaceId, table.name, 'update', updatedRowsForTrigger, @@ -2573,7 +2608,7 @@ export async function deleteRowsByFilter( ) const limit = data.limit - const deletedRows: { id: string }[] = [] + const deletedRowIds: string[] = [] if (limit === undefined) { const cutoff = new Date() let afterId: string | undefined @@ -2589,14 +2624,14 @@ export async function deleteRowsByFilter( if (page.length === 0) break const nextAfterId = page[page.length - 1] for (let index = 0; index < page.length; index += TABLE_LIMITS.DELETE_BATCH_SIZE) { - deletedRows.push( - ...(await deleteOrderedRowsByIds({ - tableId: table.id, - workspaceId: table.workspaceId, - rowIds: page.slice(index, index + TABLE_LIMITS.DELETE_BATCH_SIZE), - proof, - })) - ) + const deletedIds = await deleteOrderedRowsByIds({ + tableId: table.id, + workspaceId: table.workspaceId, + rowIds: page.slice(index, index + TABLE_LIMITS.DELETE_BATCH_SIZE), + proof, + onDeleted: (rows) => dispatchDeleteTriggers(table, rows, requestId), + }) + deletedRowIds.push(...deletedIds) } afterId = nextAfterId if (page.length < TABLE_LIMITS.DELETE_PAGE_SIZE) break @@ -2612,19 +2647,18 @@ export async function deleteRowsByFilter( ) const rowIds = matchingRows.map((row) => row.id) if (rowIds.length > 0) { - deletedRows.push( - ...(await deleteOrderedRowsByIds({ - tableId: table.id, - workspaceId: table.workspaceId, - rowIds, - proof, - })) - ) + const deletedIds = await deleteOrderedRowsByIds({ + tableId: table.id, + workspaceId: table.workspaceId, + rowIds, + proof, + onDeleted: (rows) => dispatchDeleteTriggers(table, rows, requestId), + }) + deletedRowIds.push(...deletedIds) } } - if (deletedRows.length === 0) return { affectedCount: 0, affectedRowIds: [] } - const deletedRowIds = deletedRows.map((row) => row.id) + if (deletedRowIds.length === 0) return { affectedCount: 0, affectedRowIds: [] } logger.info(`[${requestId}] Deleted ${deletedRowIds.length} rows from table ${table.id}`) @@ -2650,19 +2684,18 @@ export async function deleteRowsByIds( const uniqueRequestedRowIds = Array.from(new Set(data.rowIds)) - const deletedRows = await deleteOrderedRowsByIds({ + const deletedIds = await deleteOrderedRowsByIds({ tableId: data.tableId, workspaceId: data.workspaceId, rowIds: uniqueRequestedRowIds, proof, + onDeleted: (rows) => dispatchDeleteTriggers(table, rows, requestId), }) - const deletedIds = deletedRows.map((r) => r.id) const deletedIdSet = new Set(deletedIds) const missingRowIds = uniqueRequestedRowIds.filter((id) => !deletedIdSet.has(id)) logger.info(`[${requestId}] Deleted ${deletedIds.length} rows by ID from table ${data.tableId}`) - return { deletedCount: deletedIds.length, deletedRowIds: deletedIds, diff --git a/apps/sim/lib/table/schema-invariants.ts b/apps/sim/lib/table/schema-invariants.ts index 132b343dddc..ffb7407f1b5 100644 --- a/apps/sim/lib/table/schema-invariants.ts +++ b/apps/sim/lib/table/schema-invariants.ts @@ -11,6 +11,7 @@ import { OrchestrationError } from '@/lib/core/orchestration/types' import { getColumnId } from '@/lib/table/column-keys' +import { validateColumnTypeLimits } from '@/lib/table/column-types' import type { TableSchema, WorkflowGroup } from '@/lib/table/types' /** @@ -19,7 +20,7 @@ import type { TableSchema, WorkflowGroup } from '@/lib/table/types' * etc. Returns a list of human-readable errors (empty if valid). */ export function validateSchema(schema: TableSchema, columnOrder: string[] | undefined): string[] { - const errors: string[] = [] + const errors = validateColumnTypeLimits(schema.columns) // Group refs and columnOrder hold stable column ids (not display names). const columnsById = new Map(schema.columns.map((c) => [getColumnId(c), c])) const groups = schema.workflowGroups ?? [] diff --git a/apps/sim/lib/table/service.test.ts b/apps/sim/lib/table/service.test.ts index 3bde50fa497..d3141b6394b 100644 --- a/apps/sim/lib/table/service.test.ts +++ b/apps/sim/lib/table/service.test.ts @@ -12,6 +12,10 @@ import { beforeEach, describe, expect, it, vi } from 'vitest' import type { DbOrTx } from '@/lib/db/types' import type { TableSchema } from '@/lib/table/types' +const { mockAssertTableRowTtlEnabled } = vi.hoisted(() => ({ + mockAssertTableRowTtlEnabled: vi.fn(), +})) + vi.mock('@/lib/realtime/notify', () => ({ notifyWorkspaceTablesChanged: vi.fn().mockResolvedValue(undefined), })) @@ -21,6 +25,10 @@ vi.mock('@/lib/table/billing', () => ({ notifyTableRowUsage: vi.fn(), })) +vi.mock('@/lib/table/ttl-availability', () => ({ + assertTableRowTtlEnabled: mockAssertTableRowTtlEnabled, +})) + import { createTable, getTableById } from '@/lib/table/service' const WORKSPACE_ID = '6fc7631d-88cd-46f8-9f0a-d4764daef7f8' @@ -58,6 +66,16 @@ describe('createTable schema invariants', () => { beforeEach(() => { vi.clearAllMocks() resetDbChainMock() + mockAssertTableRowTtlEnabled.mockResolvedValue(undefined) + }) + + it('rejects a TTL schema before persistence when the feature is disabled', async () => { + mockAssertTableRowTtlEnabled.mockRejectedValue(new Error('Expiration columns are not enabled')) + + await expect( + create({ columns: [{ name: 'expires_at', type: 'ttl' }] } as TableSchema) + ).rejects.toThrow('Expiration columns are not enabled') + expect(dbChainMockFns.insert).not.toHaveBeenCalled() }) /** diff --git a/apps/sim/lib/table/service.ts b/apps/sim/lib/table/service.ts index cc35089085c..36328223e2a 100644 --- a/apps/sim/lib/table/service.ts +++ b/apps/sim/lib/table/service.ts @@ -57,6 +57,7 @@ import { mutateTableRowsWithSecretProvenance, } from '@/lib/table/rows/secret-provenance' import { assertValidSchema } from '@/lib/table/schema-invariants' +import { assertTableRowTtlEnabled } from '@/lib/table/ttl-availability' import { setTableTxTimeouts } from '@/lib/table/tx' import { type CreateTableData, @@ -560,6 +561,10 @@ export async function createTable( ) } + if (data.schema.columns.some((column) => column.type === 'ttl')) { + await assertTableRowTtlEnabled() + } + const tableId = `tbl_${generateId().replace(/-/g, '')}` const now = new Date() @@ -828,6 +833,7 @@ export async function addTableColumnsWithTx( ...table.schema, columns: [...table.schema.columns, ...additions], } + assertValidSchema(updatedSchema, table.metadata?.columnOrder) const now = new Date() await trx diff --git a/apps/sim/lib/table/trigger.ts b/apps/sim/lib/table/trigger.ts index a08ebc093a0..53623598ac0 100644 --- a/apps/sim/lib/table/trigger.ts +++ b/apps/sim/lib/table/trigger.ts @@ -1,7 +1,7 @@ /** * Direct trigger firing for table row events. * - * When rows are inserted or updated in a table, this module looks up any + * When rows are inserted, updated, or deleted in a table, this module looks up any * active webhook triggers watching that table and fires workflow executions * immediately - no polling or cron involved. */ @@ -15,7 +15,8 @@ import { readCanonicalTriggerValue } from '@/lib/webhooks/polling/canonical' const logger = createLogger('TableTrigger') -type EventType = 'insert' | 'update' +type EventType = 'insert' | 'update' | 'delete' +type TableTriggerRow = Pick interface TableTriggerPayload { row: Record | null @@ -44,16 +45,17 @@ interface WebhookConfig { * This is fire-and-forget - errors are logged but never thrown. * Call with `void fireTableTrigger(...)` to avoid blocking the caller. * - * @param eventType - 'insert' for new rows, 'update' for changed rows - * @param oldRows - Map of row ID to previous data. Pass null for inserts. + * @param workspaceId - Canonical workspace that owns the mutated table. + * @param eventType - The committed row mutation that should trigger workflows. + * @param rows - Committed row snapshots; only the ID and data are needed, including for deletes. + * @param oldRows - Map of row ID to previous data. Pass null for inserts and deletes. */ export async function fireTableTrigger( tableId: string, + workspaceId: string, tableName: string, eventType: EventType, - // Accepts a row without its executions sidecar: the payload projects id and - // data only, and the upsert path deliberately does not load one. - rows: Array>, + rows: TableTriggerRow[], oldRows: Map | null, schema: TableSchema, requestId: string @@ -75,6 +77,7 @@ export async function fireTableTrigger( // Filter to webhooks watching this table with a matching event type const matching = webhooks.filter((entry) => { + if (entry.workflow.workspaceId !== workspaceId) return false const config = entry.webhook.providerConfig as WebhookConfig | null // Canonical key `tableId` first; `tableSelector`/`manualTableId` are a transitional // basic-first fallback for configs deployed before the canonical key was written. @@ -103,7 +106,7 @@ export async function fireTableTrigger( const includeHeaders = config?.includeHeaders !== false for (const row of rows) { - const previousIdData = oldRows?.get(row.id) ?? null + const previousIdData = eventType === 'delete' ? row.data : (oldRows?.get(row.id) ?? null) const rawRow = toNamedRow(row.data) const previousRow = previousIdData ? toNamedRow(previousIdData) : null const changedColumns = previousIdData diff --git a/apps/sim/lib/table/ttl-availability.test.ts b/apps/sim/lib/table/ttl-availability.test.ts new file mode 100644 index 00000000000..4710ce7d0b9 --- /dev/null +++ b/apps/sim/lib/table/ttl-availability.test.ts @@ -0,0 +1,34 @@ +/** + * @vitest-environment node + */ +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const { mockIsFeatureEnabled } = vi.hoisted(() => ({ mockIsFeatureEnabled: vi.fn() })) + +vi.mock('@/lib/core/config/feature-flags', () => ({ + isFeatureEnabled: mockIsFeatureEnabled, +})) + +import { assertTableRowTtlEnabled, isTableRowTtlEnabled } from '@/lib/table/ttl-availability' + +describe('table row TTL availability', () => { + beforeEach(() => { + vi.clearAllMocks() + }) + + it('resolves the global table-row-ttl flag without rollout context', async () => { + mockIsFeatureEnabled.mockResolvedValue(true) + + await expect(isTableRowTtlEnabled()).resolves.toBe(true) + expect(mockIsFeatureEnabled).toHaveBeenCalledWith('table-row-ttl') + }) + + it('rejects TTL column creation while the flag is disabled', async () => { + mockIsFeatureEnabled.mockResolvedValue(false) + + await expect(assertTableRowTtlEnabled()).rejects.toMatchObject({ + code: 'validation', + message: 'Expiration columns are not enabled', + }) + }) +}) diff --git a/apps/sim/lib/table/ttl-availability.ts b/apps/sim/lib/table/ttl-availability.ts new file mode 100644 index 00000000000..f5442b35975 --- /dev/null +++ b/apps/sim/lib/table/ttl-availability.ts @@ -0,0 +1,13 @@ +import { isFeatureEnabled } from '@/lib/core/config/feature-flags' +import { OrchestrationError } from '@/lib/core/orchestration/types' + +/** Whether TTL columns and their cleanup behavior are enabled globally. */ +export function isTableRowTtlEnabled(): Promise { + return isFeatureEnabled('table-row-ttl') +} + +/** Rejects attempts to introduce a TTL column while the feature is disabled. */ +export async function assertTableRowTtlEnabled(): Promise { + if (await isTableRowTtlEnabled()) return + throw new OrchestrationError('validation', 'Expiration columns are not enabled') +} diff --git a/apps/sim/lib/table/validation.ts b/apps/sim/lib/table/validation.ts index aa9a918beb8..e5fd89c3d2d 100644 --- a/apps/sim/lib/table/validation.ts +++ b/apps/sim/lib/table/validation.ts @@ -14,6 +14,7 @@ import { columnTypeOf, isColumnType, TYPE_SPECIFIC_COLUMN_KEYS, + validateColumnTypeLimits, validateTypeMetadata, } from '@/lib/table/column-types' import { @@ -244,6 +245,8 @@ export function validateTableSchema(schema: TableSchema): ValidationResult { errors.push('Duplicate column names found') } + errors.push(...validateColumnTypeLimits(schema.columns)) + return { valid: errors.length === 0, errors } } diff --git a/apps/sim/lib/table/workflow-groups/service.test.ts b/apps/sim/lib/table/workflow-groups/service.test.ts index 5d8f164409f..179736c2b55 100644 --- a/apps/sim/lib/table/workflow-groups/service.test.ts +++ b/apps/sim/lib/table/workflow-groups/service.test.ts @@ -4,9 +4,10 @@ import { beforeEach, describe, expect, it, vi } from 'vitest' import type { TableDefinition, WorkflowGroup } from '@/lib/table/types' -const { mockWithLockedTable, mockGetTableById } = vi.hoisted(() => ({ +const { mockWithLockedTable, mockGetTableById, mockAssertTableRowTtlEnabled } = vi.hoisted(() => ({ mockWithLockedTable: vi.fn(), mockGetTableById: vi.fn(), + mockAssertTableRowTtlEnabled: vi.fn(), })) vi.mock('@/lib/table/service', () => ({ @@ -20,6 +21,9 @@ vi.mock('@/lib/table/mutation-locks', () => ({ vi.mock('@/lib/table/rows/secret-provenance', () => ({ updateTableRowsWithDerivedSecretProvenance: vi.fn(), })) +vi.mock('@/lib/table/ttl-availability', () => ({ + assertTableRowTtlEnabled: mockAssertTableRowTtlEnabled, +})) vi.mock('@/lib/table/workflow-columns', () => ({ runWorkflowColumn: vi.fn().mockResolvedValue(undefined), stripGroupDeps: (schema: unknown) => schema, @@ -34,7 +38,11 @@ vi.mock('@/lib/table/schema-invariants', () => ({ })) import { TABLE_LIMITS } from '@/lib/table/constants' -import { addWorkflowGroup } from '@/lib/table/workflow-groups/service' +import { + addWorkflowGroup, + addWorkflowGroupOutput, + updateWorkflowGroup, +} from '@/lib/table/workflow-groups/service' function groupAt(index: number): WorkflowGroup { return { @@ -73,6 +81,7 @@ function tableWithGroups(count: number): TableDefinition { describe('addWorkflowGroup group ceiling', () => { beforeEach(() => { vi.clearAllMocks() + mockAssertTableRowTtlEnabled.mockResolvedValue(undefined) }) function add(existingGroups: number) { @@ -108,3 +117,57 @@ describe('addWorkflowGroup group ceiling', () => { await expect(add(TABLE_LIMITS.MAX_WORKFLOW_GROUPS_PER_TABLE - 1)).resolves.toBeDefined() }) }) + +describe('workflow group TTL availability', () => { + beforeEach(() => { + vi.clearAllMocks() + mockAssertTableRowTtlEnabled.mockRejectedValue(new Error('Expiration columns are not enabled')) + }) + + it.each([ + [ + 'group creation', + () => + addWorkflowGroup( + { + tableId: 'table-1', + workspaceId: 'workspace-1', + group: groupAt(1), + outputColumns: [{ name: 'expires_at', type: 'ttl' }], + } as Parameters[0], + 'request-1' + ), + ], + [ + 'group update', + () => + updateWorkflowGroup( + { + tableId: 'table-1', + workspaceId: 'workspace-1', + groupId: 'group-1', + newOutputColumns: [{ name: 'expires_at', type: 'ttl' }], + } as Parameters[0], + 'request-1' + ), + ], + [ + 'single output addition', + () => + addWorkflowGroupOutput( + { + tableId: 'table-1', + workspaceId: 'workspace-1', + groupId: 'group-1', + blockId: 'block-1', + path: 'expiresAt', + resolvedOutput: { workflowId: 'workflow-1', columnType: 'ttl', order: [] }, + }, + 'request-1' + ), + ], + ])('rejects TTL introduction through %s while disabled', async (_label, introduceTtl) => { + await expect(introduceTtl()).rejects.toThrow('Expiration columns are not enabled') + expect(mockWithLockedTable).not.toHaveBeenCalled() + }) +}) diff --git a/apps/sim/lib/table/workflow-groups/service.ts b/apps/sim/lib/table/workflow-groups/service.ts index 7f526593b5a..056f10dd316 100644 --- a/apps/sim/lib/table/workflow-groups/service.ts +++ b/apps/sim/lib/table/workflow-groups/service.ts @@ -26,6 +26,7 @@ import { stripGroupExecutions } from '@/lib/table/rows/executions' import { updateTableRowsWithDerivedSecretProvenance } from '@/lib/table/rows/secret-provenance' import { assertValidSchema } from '@/lib/table/schema-invariants' import { getTableById, withLockedTable } from '@/lib/table/service' +import { assertTableRowTtlEnabled } from '@/lib/table/ttl-availability' import { setTableTxTimeouts } from '@/lib/table/tx' import type { AddWorkflowGroupData, @@ -132,6 +133,10 @@ export async function addWorkflowGroup( data: AddWorkflowGroupData, requestId: string ): Promise { + if (data.outputColumns.some((column) => column.type === 'ttl')) { + await assertTableRowTtlEnabled() + } + const updatedTable = await withLockedTable( data.tableId, async (table, trx) => { @@ -258,6 +263,10 @@ export async function updateWorkflowGroup( requestId: string ): Promise { const mappingUpdates = data.mappingUpdates ?? [] + const introducesTtl = + data.newOutputColumns?.some((column) => column.type === 'ttl') === true || + data.resolvedMappingTypes?.columns.some((column) => column.type === 'ttl') === true + if (introducesTtl) await assertTableRowTtlEnabled() // Phase 1 (no lock): consume the output types resolved and authorized by the // application command. Resolution stays outside the advisory-lock critical @@ -640,6 +649,8 @@ export async function addWorkflowGroupOutput( }, requestId: string ): Promise { + if (data.resolvedOutput.columnType === 'ttl') await assertTableRowTtlEnabled() + // Phase 1 (no lock): validate the authorized workflow metadata against the // group's current workflow. Phase 2 re-validates the same binding under the // table lock before applying the mutation. @@ -741,6 +752,15 @@ export async function addWorkflowGroupOutput( const [db, ib] = orderKey(b) return da !== db ? da - db : ia - ib }) + const invalidOutput = allGroupOutputs.find( + (output) => !resolvedOrder.has(`${output.blockId}::${output.path}`) + ) + if (invalidOutput) { + throw new OrchestrationError( + 'conflict', + `Workflow group "${data.groupId}" mappings changed concurrently; retry the add.` + ) + } const orderedGroupColIds = allGroupOutputs.map((o) => o.columnName) const updatedGroup: WorkflowGroup = { ...group, diff --git a/apps/sim/lib/uploads/contexts/workspace/workspace-file-manager-errors.test.ts b/apps/sim/lib/uploads/contexts/workspace/workspace-file-manager-errors.test.ts index 04300f1765a..5fd1cfddf48 100644 --- a/apps/sim/lib/uploads/contexts/workspace/workspace-file-manager-errors.test.ts +++ b/apps/sim/lib/uploads/contexts/workspace/workspace-file-manager-errors.test.ts @@ -1,7 +1,7 @@ /** * @vitest-environment node */ -import { dbChainMockFns, resetDbChainMock } from '@sim/testing' +import { dbChainMockFns, queueTableRows, resetDbChainMock, schemaMock } from '@sim/testing' import { afterAll, beforeEach, describe, expect, it, vi } from 'vitest' import { listWorkspaceFiles, loadActiveWorkspaceFileContext } from './workspace-file-manager' @@ -23,6 +23,56 @@ describe('listWorkspaceFiles error handling', () => { 'database unavailable' ) }) + + it('contains asynchronous record-mapping failures by default', async () => { + dbChainMockFns.orderBy.mockReset() + queueTableRows(schemaMock.workspaceFiles, [ + { + id: 'file-1', + key: 'workspace/workspace-1/file-1.md', + userId: 'user-1', + workspaceId: 'workspace-1', + folderId: null, + originalName: 'file-1.md', + contentType: 'text/markdown', + sizeBytes: null, + width: null, + height: null, + deletedAt: null, + uploadedAt: new Date('2026-01-01T00:00:00Z'), + updatedAt: new Date('2026-01-01T00:00:00Z'), + contentUpdatedAt: new Date('2026-01-01T00:00:00Z'), + }, + ]) + + await expect(listWorkspaceFiles('workspace-1')).resolves.toEqual([]) + }) + + it('propagates asynchronous record-mapping failures for authoritative callers', async () => { + dbChainMockFns.orderBy.mockReset() + queueTableRows(schemaMock.workspaceFiles, [ + { + id: 'file-1', + key: 'workspace/workspace-1/file-1.md', + userId: 'user-1', + workspaceId: 'workspace-1', + folderId: null, + originalName: 'file-1.md', + contentType: 'text/markdown', + sizeBytes: null, + width: null, + height: null, + deletedAt: null, + uploadedAt: new Date('2026-01-01T00:00:00Z'), + updatedAt: new Date('2026-01-01T00:00:00Z'), + contentUpdatedAt: new Date('2026-01-01T00:00:00Z'), + }, + ]) + + await expect(listWorkspaceFiles('workspace-1', { throwOnError: true })).rejects.toThrow( + 'Workspace file is missing canonical size_bytes metadata' + ) + }) }) describe('loadActiveWorkspaceFileContext', () => { diff --git a/apps/sim/lib/uploads/contexts/workspace/workspace-file-manager.ts b/apps/sim/lib/uploads/contexts/workspace/workspace-file-manager.ts index 61e45f59c1c..8abdc85a040 100644 --- a/apps/sim/lib/uploads/contexts/workspace/workspace-file-manager.ts +++ b/apps/sim/lib/uploads/contexts/workspace/workspace-file-manager.ts @@ -1296,7 +1296,7 @@ export async function listWorkspaceFiles( .orderBy(workspaceFiles.uploadedAt) const files = await (limit === undefined ? query : query.limit(limit)) - return hydrateWorkspaceFilePaths(files, workspaceId, options) + return await hydrateWorkspaceFilePaths(files, workspaceId, options) } catch (error) { logger.error(`Failed to list workspace files for ${workspaceId}:`, error) if (options?.throwOnError) throw error diff --git a/apps/sim/lib/uploads/utils/context-prefix.test.ts b/apps/sim/lib/uploads/utils/context-prefix.test.ts new file mode 100644 index 00000000000..c8cc8a10747 --- /dev/null +++ b/apps/sim/lib/uploads/utils/context-prefix.test.ts @@ -0,0 +1,17 @@ +import { describe, expect, it } from 'vitest' +import { inferContextFromKey, tryInferContextFromKey } from '@/lib/uploads/utils/file-utils' + +describe('tryInferContextFromKey', () => { + it('classifies a known prefix the same way the throwing form does', () => { + for (const key of ['workspace/a/b.txt', 'execution/a/b/c/d.bin', 'kb/x', 'logs/y']) { + expect(tryInferContextFromKey(key)).toBe(inferContextFromKey(key)) + } + }) + + it('answers null where the throwing form raises, so caller input cannot 500', () => { + for (const key of ['', 'garbage', 'not-a-prefix/x.txt', '../escape']) { + expect(tryInferContextFromKey(key)).toBeNull() + expect(() => inferContextFromKey(key)).toThrow() + } + }) +}) diff --git a/apps/sim/lib/uploads/utils/file-utils.ts b/apps/sim/lib/uploads/utils/file-utils.ts index 39cfdc2a6aa..3f0df1c73c8 100644 --- a/apps/sim/lib/uploads/utils/file-utils.ts +++ b/apps/sim/lib/uploads/utils/file-utils.ts @@ -757,9 +757,30 @@ export function isInternalFileUrl(fileUrl: string): boolean { * row — see `resolveStoredFileContext` — never this prefix. */ export function inferContextFromKey(key: string): StorageContext { - if (!key) { - throw new Error('Cannot infer context from empty key') + const context = tryInferContextFromKey(key) + if (!context) { + throw new Error( + key + ? `File key must start with a context prefix (kb/, knowledge-base/, chat/, copilot/, execution/, workspace/, profile-pictures/, og-images/, workspace-logos/, or logs/). Got: ${key}` + : 'Cannot infer context from empty key' + ) } + return context +} + +/** + * {@link inferContextFromKey} for a key that came from a caller rather than from + * our own storage, answering `null` instead of throwing. + * + * The throwing form is right where an unclassifiable key means the platform + * built one wrong — that is a bug and should be loud. It is wrong where the key + * is request input being normalized, because there an unrecognized prefix just + * means "this is not a file we can use", and a throw turns a malformed request + * into a 500. Both share this one list so a new context cannot be added to only + * half of them. + */ +export function tryInferContextFromKey(key: string): StorageContext | null { + if (!key) return null if (key.startsWith('kb/') || key.startsWith('knowledge-base/')) return 'knowledge-base' if (key.startsWith('chat/')) return 'chat' @@ -771,9 +792,7 @@ export function inferContextFromKey(key: string): StorageContext { if (key.startsWith('workspace-logos/')) return 'workspace-logos' if (key.startsWith('logs/')) return 'logs' - throw new Error( - `File key must start with a context prefix (kb/, knowledge-base/, chat/, copilot/, execution/, workspace/, profile-pictures/, og-images/, workspace-logos/, or logs/). Got: ${key}` - ) + return null } /** diff --git a/apps/sim/lib/workflows/application/resolve-workflow-outputs.test.ts b/apps/sim/lib/workflows/application/resolve-workflow-outputs.test.ts index 6ccf06f983a..f750feea8b8 100644 --- a/apps/sim/lib/workflows/application/resolve-workflow-outputs.test.ts +++ b/apps/sim/lib/workflows/application/resolve-workflow-outputs.test.ts @@ -7,7 +7,9 @@ import { OrchestrationError } from '@/lib/core/orchestration/types' const mocks = vi.hoisted(() => ({ flatten: vi.fn(), + loadDeployed: vi.fn(), load: vi.fn(), + NoActiveDeploymentError: class NoActiveDeploymentError extends Error {}, order: vi.fn(), resolveContext: vi.fn(), resolvePermission: vi.fn(), @@ -33,10 +35,15 @@ vi.mock('@/lib/workflows/blocks/flatten-outputs', () => ({ })) vi.mock('@/lib/workflows/persistence/utils', () => ({ + NoActiveDeploymentError: mocks.NoActiveDeploymentError, + loadDeployedWorkflowState: mocks.loadDeployed, loadWorkflowFromNormalizedTables: mocks.load, })) -import { resolveWorkflowOutputs } from '@/lib/workflows/application/resolve-workflow-outputs' +import { + loadResolvedDeployedWorkflowOutputs, + resolveWorkflowOutputs, +} from '@/lib/workflows/application/resolve-workflow-outputs' const principal = { kind: 'delegated' as const, @@ -59,12 +66,16 @@ describe('resolveWorkflowOutputs', () => { workspaceOrganizationId: null, allowPersonalApiKeys: true, billedAccountUserId: 'billing-owner-1', - workflow: { id: 'workflow-1' }, + workflow: { id: 'workflow-1', isDeployed: true }, }) mocks.load.mockResolvedValue({ blocks: { block1: { id: 'block-1', type: 'agent', name: 'Agent', subBlocks: {} } }, edges: [], }) + mocks.loadDeployed.mockResolvedValue({ + blocks: { block1: { id: 'block-1', type: 'agent', name: 'Agent', subBlocks: {} } }, + edges: [], + }) mocks.flatten.mockReturnValue([ { blockId: 'block-1', @@ -110,6 +121,42 @@ describe('resolveWorkflowOutputs', () => { expect(mocks.load).not.toHaveBeenCalled() }) + it('resolves table mappings from the active deployment state', async () => { + const context = await mocks.resolveContext() + + await expect(loadResolvedDeployedWorkflowOutputs(context)).resolves.toMatchObject({ + workflowId: 'workflow-1', + outputs: [{ blockId: 'block-1', path: 'content' }], + }) + + expect(mocks.loadDeployed).toHaveBeenCalledWith('workflow-1', 'workspace-1') + expect(mocks.load).not.toHaveBeenCalled() + }) + + it('rejects a workflow without an active deployment before resolving mappings', async () => { + const context = { + ...(await mocks.resolveContext()), + workflow: { id: 'workflow-1', isDeployed: false }, + } + + await expect(loadResolvedDeployedWorkflowOutputs(context)).rejects.toMatchObject({ + code: 'validation', + message: 'Workflow must have an active deployment', + }) + expect(mocks.loadDeployed).not.toHaveBeenCalled() + }) + + it('rejects inconsistent deployment metadata without returning draft mappings', async () => { + const context = await mocks.resolveContext() + mocks.loadDeployed.mockRejectedValueOnce(new mocks.NoActiveDeploymentError()) + + await expect(loadResolvedDeployedWorkflowOutputs(context)).rejects.toMatchObject({ + code: 'validation', + message: 'Workflow must have an active deployment', + }) + expect(mocks.load).not.toHaveBeenCalled() + }) + it('rejects expired delegated scope before loading workflow state', async () => { await expect( resolveWorkflowOutputs.execute({ diff --git a/apps/sim/lib/workflows/application/resolve-workflow-outputs.ts b/apps/sim/lib/workflows/application/resolve-workflow-outputs.ts index 209d26107f6..cb60823c006 100644 --- a/apps/sim/lib/workflows/application/resolve-workflow-outputs.ts +++ b/apps/sim/lib/workflows/application/resolve-workflow-outputs.ts @@ -1,3 +1,4 @@ +import { OrchestrationError } from '@/lib/core/orchestration/types' import { defineAuthorizedWorkflowUseCase } from '@/lib/workflows/application/authorized-workflow-use-case' import { type ActiveWorkflowApplicationContext, @@ -9,7 +10,11 @@ import { flattenWorkflowOutputs, getBlockExecutionOrder, } from '@/lib/workflows/blocks/flatten-outputs' -import { loadWorkflowFromNormalizedTables } from '@/lib/workflows/persistence/utils' +import { + loadDeployedWorkflowState, + loadWorkflowFromNormalizedTables, + NoActiveDeploymentError, +} from '@/lib/workflows/persistence/utils' export interface ResolveWorkflowOutputsInput { workflowId: string @@ -22,14 +27,14 @@ export interface ResolveWorkflowOutputsResult { executionOrderByBlockId: Record } -/** Loads output metadata after a top-level application command has authorized this workflow context. */ -export async function loadResolvedWorkflowOutputs( - context: ActiveWorkflowApplicationContext -): Promise { - const normalized = await loadWorkflowFromNormalizedTables(context.workflowId) - if (!normalized) { - return { workflowId: context.workflowId, outputs: null, executionOrderByBlockId: {} } - } +type ResolvableWorkflowState = + | NonNullable>> + | Awaited> + +function resolveWorkflowOutputsFromState( + workflowId: string, + normalized: ResolvableWorkflowState +): ResolveWorkflowOutputsResult { const blocks = Object.values(normalized.blocks ?? {}).map((block) => ({ id: block.id, type: block.type, @@ -38,12 +43,41 @@ export async function loadResolvedWorkflowOutputs( subBlocks: block.subBlocks as Record | undefined, })) return { - workflowId: context.workflowId, + workflowId, outputs: flattenWorkflowOutputs(blocks, normalized.edges ?? []), executionOrderByBlockId: getBlockExecutionOrder(blocks, normalized.edges ?? []), } } +/** Loads output metadata after a top-level application command has authorized this workflow context. */ +export async function loadResolvedWorkflowOutputs( + context: ActiveWorkflowApplicationContext +): Promise { + const normalized = await loadWorkflowFromNormalizedTables(context.workflowId) + if (!normalized) { + return { workflowId: context.workflowId, outputs: null, executionOrderByBlockId: {} } + } + return resolveWorkflowOutputsFromState(context.workflowId, normalized) +} + +/** Loads output metadata from the active deployment after workflow authorization. */ +export async function loadResolvedDeployedWorkflowOutputs( + context: ActiveWorkflowApplicationContext +): Promise { + if (!context.workflow.isDeployed) { + throw new OrchestrationError('validation', 'Workflow must have an active deployment') + } + try { + const normalized = await loadDeployedWorkflowState(context.workflowId, context.workspaceId) + return resolveWorkflowOutputsFromState(context.workflowId, normalized) + } catch (error) { + if (error instanceof NoActiveDeploymentError) { + throw new OrchestrationError('validation', 'Workflow must have an active deployment') + } + throw error + } +} + export const resolveWorkflowOutputs = defineAuthorizedWorkflowUseCase({ operation: workflowOperations.read, resolveContext: ({ input }: { input: ResolveWorkflowOutputsInput }) => diff --git a/apps/sim/lib/workflows/application/run-workflow-from-copilot.test.ts b/apps/sim/lib/workflows/application/run-workflow-from-copilot.test.ts index 593516912b4..d8976c7264e 100644 --- a/apps/sim/lib/workflows/application/run-workflow-from-copilot.test.ts +++ b/apps/sim/lib/workflows/application/run-workflow-from-copilot.test.ts @@ -391,4 +391,133 @@ describe('Copilot workflow run application commands', () => { expect(readAttemptedExecutionId(error)).toBeUndefined() }) }) + + describe('failed-run provenance crossing', () => { + function trackingLifecycle() { + const importCrossingProvenance = vi.fn().mockResolvedValue(true) + return { + importCrossingProvenance, + lifecycle: { + resolvedSecretTraceRegistry: { + exportProvenanceForValue: vi.fn(() => undefined), + beginPendingActivation: vi.fn(() => vi.fn()), + importCrossingProvenance, + }, + }, + } + } + + async function runExpectingFailure(input: { lifecycle: unknown }) { + await expect( + runWorkflowFromCopilot.execute({ + principal, + input: { + workflowId: 'workflow-1', + useDraftState: true, + lifecycle: input.lifecycle, + hasWorkflowInput: false, + useMockPayload: true, + }, + }) + ).rejects.toThrow() + } + + /** + * The executor attaches its result to every throw, so a failure without one never reached a + * block. Nothing crossed, and saying so keeps the caller's tool result — and the reason its + * run could not start — instead of reducing it to "result unavailable". + */ + it('vouches for a failure that never reached the engine', async () => { + const { importCrossingProvenance, lifecycle: tracked } = trackingLifecycle() + mocks.executeWorkflow.mockRejectedValueOnce(new Error('workflow is not deployed')) + + await runExpectingFailure({ lifecycle: tracked }) + + expect(importCrossingProvenance).toHaveBeenCalledWith( + { version: 1, complete: true, entries: [] }, + expect.objectContaining({ thrownMessage: 'workflow is not deployed' }), + expect.objectContaining({ origin: 'copilotWorkflowMutation.failedRunCrossing' }) + ) + }) + + /** + * The post-run crossing is inside the same try, so its failure reaches the catch with no + * execution result — the same evidence a never-started run leaves. An execution exists and + * its provenance was never imported, so this must not be vouched for. + */ + it('does not vouch when the crossing threw after the run returned', async () => { + const importCrossingProvenance = vi + .fn() + .mockImplementationOnce(() => { + throw new Error('crossing import failed') + }) + .mockResolvedValue(true) + + await runExpectingFailure({ + lifecycle: { + resolvedSecretTraceRegistry: { + exportProvenanceForValue: vi.fn(() => undefined), + beginPendingActivation: vi.fn(() => vi.fn()), + importCrossingProvenance, + }, + }, + }) + + expect(importCrossingProvenance).toHaveBeenNthCalledWith( + 2, + undefined, + expect.objectContaining({ thrownMessage: 'crossing import failed' }), + expect.objectContaining({ origin: 'copilotWorkflowMutation.failedRunCrossing' }) + ) + }) + + /** + * The executor's post-execution work can throw after a run has already produced a result. + * `executeWorkflow` carries it on that throw, so this reaches the catch with a result and + * must not be claimed as never-started. + */ + it('does not vouch when post-execution work threw after the engine ran', async () => { + const { importCrossingProvenance, lifecycle: tracked } = trackingLifecycle() + const incomplete = { version: 1 as const, complete: false, entries: [] } + mocks.executeWorkflow.mockRejectedValueOnce( + Object.assign(new Error('post-execution persistence failed'), { + executionResult: { + success: true, + output: { ran: true }, + executionState: { resolvedSecretTraceProvenance: incomplete }, + }, + }) + ) + + await runExpectingFailure({ lifecycle: tracked }) + + expect(importCrossingProvenance).toHaveBeenCalledWith( + incomplete, + expect.objectContaining({ output: { ran: true } }), + expect.objectContaining({ origin: 'copilotWorkflowMutation.failedRunCrossing' }) + ) + }) + + /** A run that did execute and could not vouch still hands back its incomplete envelope. */ + it('passes through an incomplete envelope from a run that did execute', async () => { + const { importCrossingProvenance, lifecycle: tracked } = trackingLifecycle() + const incomplete = { version: 1 as const, complete: false, entries: [] } + const failure = Object.assign(new Error('block failed'), { + executionResult: { + success: false, + output: { partial: true }, + executionState: { resolvedSecretTraceProvenance: incomplete }, + }, + }) + mocks.executeWorkflow.mockRejectedValueOnce(failure) + + await runExpectingFailure({ lifecycle: tracked }) + + expect(importCrossingProvenance).toHaveBeenCalledWith( + incomplete, + expect.objectContaining({ output: { partial: true } }), + expect.objectContaining({ origin: 'copilotWorkflowMutation.failedRunCrossing' }) + ) + }) + }) }) diff --git a/apps/sim/lib/workflows/application/run-workflow-from-copilot.ts b/apps/sim/lib/workflows/application/run-workflow-from-copilot.ts index 6467156f445..31ed03811fc 100644 --- a/apps/sim/lib/workflows/application/run-workflow-from-copilot.ts +++ b/apps/sim/lib/workflows/application/run-workflow-from-copilot.ts @@ -29,11 +29,14 @@ import { } from '@/lib/workflows/triggers/run-options' import type { SerializableExecutionState } from '@/executor/execution/types' import type { ExecutionResult } from '@/executor/types' -import { attachAttemptedExecutionId } from '@/executor/utils/errors' +import { attachAttemptedExecutionId, hasExecutionResult } from '@/executor/utils/errors' const logger = createLogger('CopilotWorkflowRun') -import type { ResolvedSecretTraceRegistry } from '@/executor/utils/resolved-secret-trace-registry' +import { + emptyResolvedSecretTraceProvenance, + type ResolvedSecretTraceRegistry, +} from '@/executor/utils/resolved-secret-trace-registry' export interface CopilotWorkflowRunLifecycle { billingAttribution?: BillingAttributionSnapshot @@ -250,6 +253,13 @@ async function executeCopilotRun(params: { params.executionInput ) const completePendingActivation = registry?.beginPendingActivation() + /** + * The run's own result, once the executor returns it. The post-run crossing below is inside the + * same `try`, so its failure reaches the catch carrying nothing — and on that evidence alone it + * is indistinguishable from a run that never started. Holding the result here keeps the real + * envelope available to describe content that certainly exists. + */ + let runResult: ExecutionResult | undefined /** * The executor call is the first statement of this `try`, so everything caught below is * post-dispatch by construction, while authorization, admission and provenance export all @@ -302,6 +312,7 @@ async function executeCopilotRun(params: { }, childExecutionId ) + runResult = result if (registry) { await registry.importCrossingProvenance( result.executionState?.resolvedSecretTraceProvenance, @@ -325,16 +336,23 @@ async function executeCopilotRun(params: { * as never started and invite the duplicate this id exists to prevent. */ if (registry) { - const executionResult = - typeof error === 'object' && - error !== null && - 'executionResult' in error && - typeof error.executionResult === 'object' - ? (error.executionResult as ExecutionResult) - : undefined + /** + * Either source counts as proof a run exists: the error carries the result when the run or + * its post-execution work threw, and `runResult` holds it when the failure came later still + * — from the crossing below, after the executor had already returned. + */ + const executionResult = hasExecutionResult(error) ? error.executionResult : runResult try { + /** + * Only a failure with no result from either source can claim nothing ran, and saying so + * keeps the caller's failure reason instead of reducing the tool result to "result + * unavailable" for a message that named no secret because none had been resolved yet. + * Every other failure hands back the envelope it has, and an incomplete one still latches. + */ await registry.importCrossingProvenance( - executionResult?.executionState?.resolvedSecretTraceProvenance, + executionResult + ? executionResult.executionState?.resolvedSecretTraceProvenance + : emptyResolvedSecretTraceProvenance(), { output: executionResult?.output, logs: executionResult?.logs, diff --git a/apps/sim/lib/workflows/custom-blocks/operations.ts b/apps/sim/lib/workflows/custom-blocks/operations.ts index 4b9018dfa2e..a27909089f9 100644 --- a/apps/sim/lib/workflows/custom-blocks/operations.ts +++ b/apps/sim/lib/workflows/custom-blocks/operations.ts @@ -8,10 +8,12 @@ import { } from '@sim/db/schema' import { createLogger } from '@sim/logger' import { generateId, generateShortId } from '@sim/utils/id' -import { and, eq, isNull, sql } from 'drizzle-orm' +import { and, eq, isNull, ne, sql } from 'drizzle-orm' import { isOrganizationFeatureEntitled } from '@/lib/billing/core/subscription' +import { acquireOrganizationMutationLock } from '@/lib/billing/organizations/membership' import { isBillingEnabled, isCustomBlocksEnabled } from '@/lib/core/config/env-flags' import { mapWithConcurrency } from '@/lib/core/utils/concurrency' +import type { DbOrTx } from '@/lib/db/types' import { extractInputFieldsFromBlocks, type WorkflowInputField } from '@/lib/workflows/input-format' import { loadDeployedWorkflowState } from '@/lib/workflows/persistence/utils' import { getWorkspaceWithOwner } from '@/lib/workspaces/permissions/utils' @@ -495,41 +497,59 @@ export async function publishCustomBlock(params: { throw new CustomBlockValidationError('You can only publish a workflow from its own workspace') } - const ws = wf.workspaceId ? await getWorkspaceWithOwner(wf.workspaceId) : null - if (!ws?.organizationId || ws.organizationId !== organizationId) { - throw new CustomBlockValidationError('Workflow does not belong to this organization') - } - - // One block per workflow: the (org, type) unique index doesn't prevent the same - // workflow being published under a fresh `custom_block_*` type, so guard here. - const [existing] = await db - .select({ id: customBlock.id }) - .from(customBlock) - .where(eq(customBlock.workflowId, workflowId)) - .limit(1) - if (existing) { - throw new CustomBlockValidationError('This workflow is already published as a block') - } - const id = generateId() const type = `${CUSTOM_BLOCK_TYPE_PREFIX}${generateShortId(10).toLowerCase()}` const now = new Date() - await db.insert(customBlock).values({ - id, - organizationId, - workflowId, - type, - name, - description, - iconUrl: iconUrl ?? null, - inputs: inputs ?? [], - outputs: exposedOutputs ?? [], - enabled: true, - traceChildRuns, - createdBy: userId, - createdAt: now, - updatedAt: now, + /** + * The org-belongs check and the insert run under the organization mutation + * lock, together, because an admin workspace move holds that same lock while + * it re-homes a workspace and unpublishes the blocks bound to its workflows. + * Reading the workspace's organization outside the lock lets a publish that + * validated against the OLD organization commit after the move's cleanup + * scan, leaving a source-organization block bound to a workflow that now + * lives in another tenant — which `getCustomBlockAuthority` would resolve and + * execute under the wrong owner's credentials and billing. + */ + const ws = await db.transaction(async (tx) => { + await acquireOrganizationMutationLock(tx, organizationId) + + const workspaceRow = wf.workspaceId + ? await getWorkspaceWithOwner(wf.workspaceId, { executor: tx }) + : null + if (!workspaceRow?.organizationId || workspaceRow.organizationId !== organizationId) { + throw new CustomBlockValidationError('Workflow does not belong to this organization') + } + + // One block per workflow: the (org, type) unique index doesn't prevent the same + // workflow being published under a fresh `custom_block_*` type, so guard here. + const [existing] = await tx + .select({ id: customBlock.id }) + .from(customBlock) + .where(eq(customBlock.workflowId, workflowId)) + .limit(1) + if (existing) { + throw new CustomBlockValidationError('This workflow is already published as a block') + } + + await tx.insert(customBlock).values({ + id, + organizationId, + workflowId, + type, + name, + description, + iconUrl: iconUrl ?? null, + inputs: inputs ?? [], + outputs: exposedOutputs ?? [], + enabled: true, + traceChildRuns, + createdBy: userId, + createdAt: now, + updatedAt: now, + }) + + return workspaceRow }) logger.info('Published custom block', { id, type, organizationId, workflowId }) @@ -588,9 +608,16 @@ export async function updateCustomBlock( await db.update(customBlock).set(patch).where(eq(customBlock.id, id)) } -/** Unpublish (hard-delete) a custom block. */ -export async function deleteCustomBlock(id: string): Promise { - await db.delete(customBlock).where(eq(customBlock.id, id)) +/** + * Unpublish (hard-delete) a custom block. + * + * Accepts an executor so a caller that must unpublish atomically with something + * else can enlist it — the admin workspace move unpublishes blocks in the same + * transaction that re-homes their bound workflow, keeping a block and its + * workflow from ever being visible in two different organizations. + */ +export async function deleteCustomBlock(id: string, executor: DbOrTx = db): Promise { + await executor.delete(customBlock).where(eq(customBlock.id, id)) } /** @@ -603,11 +630,14 @@ export async function deleteCustomBlock(id: string): Promise { */ export async function getCustomBlockUsageCounts( organizationId: string, - blockType: string + blockType: string, + scope?: { onlyWorkspaceId?: string; excludeWorkspaceId?: string } ): Promise<{ usageCount: number; deployedUsageCount: number }> { const orgActiveWorkflow = and( eq(workspace.organizationId, organizationId), - isNull(workflow.archivedAt) + isNull(workflow.archivedAt), + scope?.onlyWorkspaceId ? eq(workflow.workspaceId, scope.onlyWorkspaceId) : undefined, + scope?.excludeWorkspaceId ? ne(workflow.workspaceId, scope.excludeWorkspaceId) : undefined ) // Escape LIKE wildcards — the `_`s in `custom_block_` would otherwise match // any character and let unrelated states through to the jsonb parse. diff --git a/apps/sim/lib/workflows/executor/execute-workflow.test.ts b/apps/sim/lib/workflows/executor/execute-workflow.test.ts index 4240058f7d3..9f6b48317fe 100644 --- a/apps/sim/lib/workflows/executor/execute-workflow.test.ts +++ b/apps/sim/lib/workflows/executor/execute-workflow.test.ts @@ -56,6 +56,7 @@ vi.mock('@/lib/workflows/executor/pause-persistence', () => ({ })) import { executeWorkflow } from '@/lib/workflows/executor/execute-workflow' +import { hasExecutionResult } from '@/executor/utils/errors' const workflowExecutionLoggerCallIndex = loggerMock.createLogger.mock.calls.findIndex( ([name]) => name === 'WorkflowExecution' @@ -234,6 +235,32 @@ describe('executeWorkflow', () => { ) }) + it('forwards a trusted immutable workflow state to the execution snapshot', async () => { + const workflowStateOverride = { + blocks: { 'block-1': { id: 'block-1', type: 'start_trigger' } }, + edges: [], + loops: {}, + parallels: {}, + variables: { + 'variable-1': { id: 'variable-1', name: 'deployed', value: 'frozen' }, + }, + deploymentVersionId: 'deployment-version-1', + } + + await executeWorkflow(workflow, 'request-1', { prompt: 'hello' }, 'actor-1', { + enabled: true, + principal, + billingAttribution, + workflowStateOverride, + }) + + const coreParams = executeWorkflowCoreMock.mock.calls[0]?.[0] as { + snapshot: ExecutionSnapshot + } + expect(coreParams.snapshot.metadata.workflowStateOverride).toEqual(workflowStateOverride) + expect(coreParams.snapshot.workflowVariables).toEqual(workflowStateOverride.variables) + }) + it('waits for post-execution persistence before resolving', async () => { let resolvePostExecution!: () => void waitForPostExecutionMock.mockReturnValueOnce( @@ -296,6 +323,44 @@ describe('executeWorkflow', () => { expect(executionSettled).toBe(true) }) + /** + * Post-execution work runs after the core has produced a result and the executor never sees + * its failure, so this layer is the only one that can carry the result onto it. Callers read a + * missing result as proof that no block ran — a Copilot run would report an executed workflow + * as never started and vouch for content it cannot describe. + */ + it('carries the execution result onto a post-execution failure', async () => { + const result = { success: true, output: { ran: true }, logs: [] } + executeWorkflowCoreMock.mockResolvedValueOnce(result) + handlePostExecutionPauseStateMock.mockRejectedValueOnce(new Error('pause persistence failed')) + + const thrown = await executeWorkflow(workflow, 'request-1', undefined, 'actor-1', { + enabled: true, + principal, + billingAttribution, + }).catch((error: unknown) => error) + + expect(hasExecutionResult(thrown)).toBe(true) + expect((thrown as { executionResult?: unknown }).executionResult).toBe(result) + }) + + /** A non-Error cannot carry the result, so it is normalized before anything reads it. */ + it('normalizes a non-Error post-execution failure so it can carry the result', async () => { + const result = { success: true, output: { ran: true }, logs: [] } + executeWorkflowCoreMock.mockResolvedValueOnce(result) + handlePostExecutionPauseStateMock.mockRejectedValueOnce('pause persistence exploded') + + const thrown = await executeWorkflow(workflow, 'request-1', undefined, 'actor-1', { + enabled: true, + principal, + billingAttribution, + }).catch((error: unknown) => error) + + expect(thrown).toBeInstanceOf(Error) + expect(hasExecutionResult(thrown)).toBe(true) + expect((thrown as { executionResult?: unknown }).executionResult).toBe(result) + }) + it('transfers post-execution ownership with successful streaming metadata', async () => { const result = await executeWorkflow(workflow, 'request-1', undefined, 'actor-1', { enabled: true, diff --git a/apps/sim/lib/workflows/executor/execute-workflow.ts b/apps/sim/lib/workflows/executor/execute-workflow.ts index 8336ea332d2..39cdad0c07a 100644 --- a/apps/sim/lib/workflows/executor/execute-workflow.ts +++ b/apps/sim/lib/workflows/executor/execute-workflow.ts @@ -1,5 +1,6 @@ import type { WorkflowExecutionPrincipal } from '@sim/auth/principal' import { createLogger } from '@sim/logger' +import { toError } from '@sim/utils/errors' import { generateId } from '@sim/utils/id' import { assertBillingAttributionSnapshot, @@ -13,6 +14,7 @@ import { handlePostExecutionPauseState } from '@/lib/workflows/executor/pause-pe import { ExecutionSnapshot } from '@/executor/execution/snapshot' import type { ExecutionMetadata, SerializableExecutionState } from '@/executor/execution/types' import type { ExecutionResult, StreamingExecution } from '@/executor/types' +import { attachExecutionResult, hasExecutionResult } from '@/executor/utils/errors' import type { ResolvedSecretTraceProvenanceV1 } from '@/executor/utils/resolved-secret-trace-registry' import type { CoreTriggerType } from '@/stores/logs/filters/types' @@ -48,6 +50,8 @@ export interface ExecuteWorkflowOptions { abortSignal?: AbortSignal /** Use the live/draft workflow state instead of the deployed state. Used by copilot. */ useDraftState?: boolean + /** Immutable workflow state selected by a trusted server-side trigger boundary. */ + workflowStateOverride?: NonNullable /** Stop execution after this block completes. Used for "run until block" feature. */ stopAfterBlockId?: string /** Run-from-block configuration using a prior execution snapshot. */ @@ -128,6 +132,12 @@ export async function executeWorkflow( loggingSession.setTrustedExecutionCorrelation(streamConfig.trustedExecutionCorrelation) } let postExecutionOwnershipTransferred = false + /** + * Held outside the `try` so the catch can carry it. The executor attaches its result when the + * run itself throws, but the post-execution work below can throw after a run has already + * produced one — and callers read a missing result as proof that no block ran. + */ + let executionResult: ExecutionResult | undefined try { const metadata: ExecutionMetadata = { @@ -142,6 +152,7 @@ export async function executeWorkflow( triggerType, triggerBlockId: streamConfig?.triggerBlockId, useDraftState: streamConfig?.useDraftState ?? false, + workflowStateOverride: streamConfig?.workflowStateOverride, startTime: new Date().toISOString(), isClientSession: false, enforceCredentialAccess: streamConfig?.enforceCredentialAccess ?? false, @@ -163,13 +174,13 @@ export async function executeWorkflow( metadata, workflow, input, - workflow.variables || {}, + streamConfig?.workflowStateOverride?.variables ?? workflow.variables ?? {}, streamConfig?.selectedOutputs || [] ) const executionStartMs = Date.now() - const result = await executeWorkflowCore({ + const result = (executionResult = await executeWorkflowCore({ snapshot, callbacks: { onStream: streamConfig?.onStream, @@ -197,7 +208,7 @@ export async function executeWorkflow( trustedInitialResolvedSecretTraceProvenance: streamConfig?.trustedInitialResolvedSecretTraceProvenance, runFromBlock: streamConfig?.runFromBlock, - }) + })) const blockTypes = [ ...new Set( @@ -240,7 +251,22 @@ export async function executeWorkflow( } return result - } catch (error: unknown) { + } catch (caught: unknown) { + /** + * Normalized before anything reads it, for the reason the executor normalizes its own throw: + * a value that cannot carry the result would otherwise reach callers bare, and they read a + * missing result as proof that no block ran. `toError` returns an `Error` unchanged, so a + * custom error class keeps its identity and every ordinary failure is untouched. + */ + const error = toError(caught) + /** + * Carries the run's result on a failure raised after it produced one — the post-execution + * work below the executor call can throw, and the executor never saw it. Skipped when the + * executor already attached its own, which is the more specific record. + */ + if (executionResult && !hasExecutionResult(error)) { + attachExecutionResult(error, executionResult) + } const errorDiagnostic = loggingSession.projectDiagnosticError(error) logger.error(`[${requestId}] Workflow execution failed`, errorDiagnostic) diff --git a/apps/sim/lib/workflows/orchestration/deploy.test.ts b/apps/sim/lib/workflows/orchestration/deploy.test.ts index 262a45c30d7..08e4f63fce8 100644 --- a/apps/sim/lib/workflows/orchestration/deploy.test.ts +++ b/apps/sim/lib/workflows/orchestration/deploy.test.ts @@ -896,6 +896,7 @@ describe('mutation lock on the orchestration entry points', () => { expect(result.success).toBe(false) expect(result.error).toContain('locked') + expect(result.errorCode).toBe('locked') expect(mockRecordAudit).not.toHaveBeenCalled() }) diff --git a/apps/sim/lib/workflows/orchestration/deploy.ts b/apps/sim/lib/workflows/orchestration/deploy.ts index 5584a52412f..9290143972e 100644 --- a/apps/sim/lib/workflows/orchestration/deploy.ts +++ b/apps/sim/lib/workflows/orchestration/deploy.ts @@ -151,7 +151,7 @@ export async function performFullDeploy( // Backstop for every caller — routes may assert first to render their own 423, // but the copilot deploy tools call this directly. const lockDenial = await workflowLockDenial(workflowId) - if (lockDenial) return { success: false, error: lockDenial, errorCode: 'validation' } + if (lockDenial) return { success: false, error: lockDenial, errorCode: 'locked' } const [workflowRecord] = await db .select() @@ -509,6 +509,7 @@ export interface PerformFullUndeployParams { export interface PerformFullUndeployResult { success: boolean error?: string + errorCode?: OrchestrationErrorCode warnings?: string[] } @@ -526,7 +527,7 @@ export async function performFullUndeploy( const requestId = params.requestId ?? generateRequestId() const lockDenial = await workflowLockDenial(workflowId) - if (lockDenial) return { success: false, error: lockDenial } + if (lockDenial) return { success: false, error: lockDenial, errorCode: 'locked' } const [workflowRecord] = await db .select() @@ -661,7 +662,7 @@ export async function performActivateVersion( const idempotencyKey = params.idempotencyKey ?? generateId() const lockDenial = await workflowLockDenial(workflowId) - if (lockDenial) return { success: false, error: lockDenial, errorCode: 'validation' } + if (lockDenial) return { success: false, error: lockDenial, errorCode: 'locked' } const [versionRow] = await db .select({ diff --git a/apps/sim/lib/workflows/types.ts b/apps/sim/lib/workflows/types.ts index 9e51d7ff1a7..28a68489132 100644 --- a/apps/sim/lib/workflows/types.ts +++ b/apps/sim/lib/workflows/types.ts @@ -12,6 +12,15 @@ export const USER_FILE_ACCESSIBLE_PROPERTIES = [ 'size', 'type', 'base64', + /** + * Path to the file on the sandbox filesystem, mounted on demand. + * + * The counterpart to `base64`: that one inlines the bytes and is JavaScript- + * only, while this one hands any language a real path to open — which is what + * a CLI or a library like pandas or ffmpeg actually needs. Referencing it runs + * the block in the remote sandbox, since the isolated VM has no filesystem. + */ + 'path', ] as const export type UserFileAccessibleProperty = (typeof USER_FILE_ACCESSIBLE_PROPERTIES)[number] @@ -23,6 +32,7 @@ export const USER_FILE_PROPERTY_TYPES: Record { expect(executorOperationIds).toEqual([ 'files.read_metadata', 'files.read_content', + 'files.search_content', 'files.download', 'files.create', 'files.update_content', diff --git a/apps/sim/lib/workspace-files/application/operations.ts b/apps/sim/lib/workspace-files/application/operations.ts index 2752a576521..9c4c6174435 100644 --- a/apps/sim/lib/workspace-files/application/operations.ts +++ b/apps/sim/lib/workspace-files/application/operations.ts @@ -35,6 +35,12 @@ export const fileOperations = { workspaceApiKey: 'allow', ...ALL_FILE_TOOL_PRINCIPAL_POLICY, }), + searchContent: defineWorkspaceOperation({ + id: 'files.search_content', + minimumRole: 'read', + workspaceApiKey: 'allow', + ...ALL_FILE_TOOL_PRINCIPAL_POLICY, + }), download: defineWorkspaceOperation({ id: 'files.download', minimumRole: 'read', diff --git a/apps/sim/lib/workspace-files/application/read-workspace-file-secret-provenance.ts b/apps/sim/lib/workspace-files/application/read-workspace-file-secret-provenance.ts index 45075640fa7..27c8a695108 100644 --- a/apps/sim/lib/workspace-files/application/read-workspace-file-secret-provenance.ts +++ b/apps/sim/lib/workspace-files/application/read-workspace-file-secret-provenance.ts @@ -11,6 +11,8 @@ import { resolveActiveWorkspaceFileContext } from '@/lib/workspace-files/applica export interface ReadWorkspaceFileSecretProvenanceInput { fileId: string assertedWorkspaceId?: string + /** Fails closed when the caller's derived content no longer matches the canonical file revision. */ + expectedContentUpdatedAt?: Date } export const readWorkspaceFileSecretProvenance = defineAuthorizedWorkspaceFileUseCase({ @@ -28,6 +30,7 @@ export const readWorkspaceFileSecretProvenance = defineAuthorizedWorkspaceFileUs fileId: file.id, key: file.key, context: 'workspace', + contentUpdatedAt: input.expectedContentUpdatedAt, }), ownerUserId: file.uploadedBy, } diff --git a/apps/sim/lib/workspace-files/application/search-workspace-file-content.ts b/apps/sim/lib/workspace-files/application/search-workspace-file-content.ts new file mode 100644 index 00000000000..d8eb4d701eb --- /dev/null +++ b/apps/sim/lib/workspace-files/application/search-workspace-file-content.ts @@ -0,0 +1,35 @@ +import { OrchestrationError } from '@/lib/core/orchestration/types' +import { loadActiveWorkspaceContext } from '@/lib/uploads/contexts/workspace' +import { defineAuthorizedWorkspaceFileUseCase } from '@/lib/workspace-files/application/authorized-workspace-file-use-case' +import { fileOperations } from '@/lib/workspace-files/application/operations' +import { searchWorkspaceFileIndex } from '@/lib/workspace-files/search/repository' +import { isFileSearchCaseSensitive } from '@/lib/workspace-files/search/text' + +export interface SearchWorkspaceFileContentInput { + workspaceId: string + query: string + maxResults: number + signal?: AbortSignal +} + +async function resolveSearchWorkspaceFileContext(input: SearchWorkspaceFileContentInput) { + input.signal?.throwIfAborted() + const workspace = await loadActiveWorkspaceContext(input.workspaceId) + input.signal?.throwIfAborted() + if (!workspace) throw new OrchestrationError('not_found', 'Workspace not found') + return workspace +} + +export const searchWorkspaceFileContent = defineAuthorizedWorkspaceFileUseCase({ + operation: fileOperations.searchContent, + resolveContext: ({ input }: { input: SearchWorkspaceFileContentInput }) => + resolveSearchWorkspaceFileContext(input), + execute: ({ input, context }) => + searchWorkspaceFileIndex({ + workspaceId: context.workspaceId, + query: input.query, + maxResults: input.maxResults, + caseSensitive: isFileSearchCaseSensitive(input.query), + signal: input.signal, + }), +}) diff --git a/apps/sim/lib/workspace-files/search/constants.ts b/apps/sim/lib/workspace-files/search/constants.ts new file mode 100644 index 00000000000..1f2ba412d72 --- /dev/null +++ b/apps/sim/lib/workspace-files/search/constants.ts @@ -0,0 +1,24 @@ +export const FILE_SEARCH_MIN_QUERY_LENGTH = 3 +export const FILE_SEARCH_MAX_QUERY_LENGTH = 512 +export const FILE_SEARCH_DEFAULT_MAX_RESULTS = 50 +export const FILE_SEARCH_MAX_RESULTS = 200 + +export const FILE_SEARCH_MAX_SOURCE_BYTES = 25 * 1024 * 1024 +export const FILE_SEARCH_MAX_EXTRACTED_BYTES = 25 * 1024 * 1024 +export const FILE_SEARCH_MAX_PREVIEW_BYTES = 2 * 1024 +export const FILE_SEARCH_SEGMENT_CHARS = 16 * 1024 +export const FILE_SEARCH_SEGMENT_OVERLAP_CHARS = + FILE_SEARCH_MAX_QUERY_LENGTH + FILE_SEARCH_MAX_PREVIEW_BYTES +export const FILE_SEARCH_INSERT_BATCH_ROWS = 250 +export const FILE_SEARCH_INSERT_BATCH_BYTES = 1024 * 1024 + +export const FILE_SEARCH_INDEX_GLOBAL_CONCURRENCY = 10 +export const FILE_SEARCH_INDEX_WORKSPACE_OUTSTANDING = 2 +export const FILE_SEARCH_INDEX_MAX_OUTSTANDING = 100 +export const FILE_SEARCH_INDEX_DISPATCH_WORKSPACES = 100 +export const FILE_SEARCH_DISPATCH_INTERVAL_MS = 60 * 1000 +export const FILE_SEARCH_DISPATCH_MAX_DURATION_SECONDS = 60 +export const FILE_SEARCH_INDEX_MAX_DURATION_SECONDS = 15 * 60 +export const FILE_SEARCH_INDEX_STALE_DISPATCH_MS = 6 * 60 * 60 * 1000 +export const FILE_SEARCH_INDEX_STALE_REAP_LIMIT = 100 +export const FILE_SEARCH_BACKFILL_PAGE_SIZE = 1000 diff --git a/apps/sim/lib/workspace-files/search/dispatcher.test.ts b/apps/sim/lib/workspace-files/search/dispatcher.test.ts new file mode 100644 index 00000000000..98845cb5eff --- /dev/null +++ b/apps/sim/lib/workspace-files/search/dispatcher.test.ts @@ -0,0 +1,36 @@ +/** + * @vitest-environment node + */ +import { describe, expect, it } from 'vitest' +import { + buildWorkspaceFileSearchTriggerItems, + shouldUseWorkspaceFileSearchTrigger, +} from '@/lib/workspace-files/search/dispatcher' + +describe('workspace file search dispatch policy', () => { + it('uses Trigger.dev from inside a task even when the deployment flag is absent', () => { + expect(shouldUseWorkspaceFileSearchTrigger(false, true)).toBe(true) + expect(shouldUseWorkspaceFileSearchTrigger(true, false)).toBe(true) + expect(shouldUseWorkspaceFileSearchTrigger(false, false)).toBe(false) + }) + + it('deduplicates each immutable revision without including file contents', () => { + const payload = { + workspaceId: 'workspace-1', + fileId: 'file-1', + sourceContentUpdatedAt: '2026-08-29T12:00:00.000Z', + } + + expect(buildWorkspaceFileSearchTriggerItems([payload], 'us-east-1')).toEqual([ + { + payload, + options: { + idempotencyKey: 'workspace-file-search:file-1:2026-08-29T12:00:00.000Z', + idempotencyKeyTTL: '1h', + tags: ['workspaceId:workspace-1', 'fileId:file-1'], + region: 'us-east-1', + }, + }, + ]) + }) +}) diff --git a/apps/sim/lib/workspace-files/search/dispatcher.ts b/apps/sim/lib/workspace-files/search/dispatcher.ts new file mode 100644 index 00000000000..3c9518a9fe1 --- /dev/null +++ b/apps/sim/lib/workspace-files/search/dispatcher.ts @@ -0,0 +1,515 @@ +import { db } from '@sim/db' +import { + workspaceFileSearchBackfill, + workspaceFileSearchDispatchQueue, + workspaceFileSearchIndex, + workspaceFileSearchSegment, + workspaceFiles, +} from '@sim/db/schema' +import { createLogger } from '@sim/logger' +import { getErrorMessage } from '@sim/utils/errors' +import { + and, + asc, + count, + eq, + exists, + gt, + inArray, + isNotNull, + isNull, + lt, + notExists, + or, + type SQL, + sql, +} from 'drizzle-orm' +import { isTriggerDevEnabled } from '@/lib/core/config/env-flags' +import { isInsideTriggerRun } from '@/lib/core/config/trigger-runtime' +import { runDetached } from '@/lib/core/utils/background' +import type { DbTransaction } from '@/lib/db/types' +import { + FILE_SEARCH_BACKFILL_PAGE_SIZE, + FILE_SEARCH_INDEX_DISPATCH_WORKSPACES, + FILE_SEARCH_INDEX_MAX_OUTSTANDING, + FILE_SEARCH_INDEX_STALE_DISPATCH_MS, + FILE_SEARCH_INDEX_STALE_REAP_LIMIT, + FILE_SEARCH_INDEX_WORKSPACE_OUTSTANDING, +} from '@/lib/workspace-files/search/constants' +import { + indexWorkspaceFileForSearch, + markWorkspaceFileSearchIndexFailed, + type WorkspaceFileSearchIndexPayload, +} from '@/lib/workspace-files/search/indexing' +import type { workspaceFileSearchIndexTask } from '@/background/workspace-file-search-index' + +const logger = createLogger('WorkspaceFileSearchDispatcher') +const DISPATCH_LOCK_NAME = 'workspace-file-search-dispatch' +const BACKFILL_CURSOR_ID = 'workspace-file-search-v1' + +interface RevisionIdentity { + fileId: string + sourceContentUpdatedAt: Date +} + +interface PreparedDispatch { + payloads: WorkspaceFileSearchIndexPayload[] + backfilledFiles: number + reapedClaims: number + lockAcquired: boolean +} + +export interface WorkspaceFileSearchDispatchResult { + dispatchedFiles: number + backfilledFiles: number + reapedClaims: number + lockAcquired: boolean +} + +export function shouldUseWorkspaceFileSearchTrigger( + triggerDevEnabled: boolean, + insideTriggerRun: boolean +): boolean { + return triggerDevEnabled || insideTriggerRun +} + +export function buildWorkspaceFileSearchTriggerItems( + payloads: readonly WorkspaceFileSearchIndexPayload[], + region: string +) { + return payloads.map((payload) => ({ + payload, + options: { + idempotencyKey: `workspace-file-search:${payload.fileId}:${payload.sourceContentUpdatedAt}`, + idempotencyKeyTTL: '1h' as const, + tags: [`workspaceId:${payload.workspaceId}`, `fileId:${payload.fileId}`], + region, + }, + })) +} + +function revisionFilter(rows: readonly RevisionIdentity[]): SQL | undefined { + return or( + ...rows.map((row) => + and( + eq(workspaceFileSearchIndex.fileId, row.fileId), + eq(workspaceFileSearchIndex.sourceContentUpdatedAt, row.sourceContentUpdatedAt) + ) + ) + ) +} + +async function enqueueWorkspaces( + tx: DbTransaction, + workspaceIds: readonly string[], + now: Date +): Promise { + const uniqueWorkspaceIds = [...new Set(workspaceIds)] + if (uniqueWorkspaceIds.length === 0) return + await tx + .insert(workspaceFileSearchDispatchQueue) + .values( + uniqueWorkspaceIds.map((workspaceId) => ({ + workspaceId, + enqueuedAt: now, + updatedAt: now, + })) + ) + .onConflictDoUpdate({ + target: workspaceFileSearchDispatchQueue.workspaceId, + set: { updatedAt: now }, + }) +} + +async function seedBackfillPage(tx: DbTransaction, now: Date): Promise { + await tx + .insert(workspaceFileSearchBackfill) + .values({ id: BACKFILL_CURSOR_ID, updatedAt: now }) + .onConflictDoNothing() + + const [cursor] = await tx + .select() + .from(workspaceFileSearchBackfill) + .where(eq(workspaceFileSearchBackfill.id, BACKFILL_CURSOR_ID)) + .for('update') + .limit(1) + if (!cursor || cursor.completedAt) return 0 + + const rows = await tx + .select({ + workspaceId: workspaceFiles.workspaceId, + fileId: workspaceFiles.id, + sourceContentUpdatedAt: workspaceFiles.contentUpdatedAt, + }) + .from(workspaceFiles) + .where( + and( + eq(workspaceFiles.context, 'workspace'), + isNull(workspaceFiles.deletedAt), + isNotNull(workspaceFiles.workspaceId), + cursor.afterWorkspaceId && cursor.afterFileId + ? or( + gt(workspaceFiles.workspaceId, cursor.afterWorkspaceId), + and( + eq(workspaceFiles.workspaceId, cursor.afterWorkspaceId), + gt(workspaceFiles.id, cursor.afterFileId) + ) + ) + : undefined + ) + ) + .orderBy(asc(workspaceFiles.workspaceId), asc(workspaceFiles.id)) + .limit(FILE_SEARCH_BACKFILL_PAGE_SIZE) + .for('share', { of: workspaceFiles }) + + const files = rows.filter( + (row): row is typeof row & { workspaceId: string } => row.workspaceId !== null + ) + if (files.length > 0) { + await tx + .insert(workspaceFileSearchIndex) + .values( + files.map((file) => ({ + workspaceId: file.workspaceId, + fileId: file.fileId, + sourceContentUpdatedAt: file.sourceContentUpdatedAt, + status: 'pending' as const, + updatedAt: now, + })) + ) + .onConflictDoNothing() + await enqueueWorkspaces( + tx, + files.map((file) => file.workspaceId), + now + ) + } + + const last = files.at(-1) + await tx + .update(workspaceFileSearchBackfill) + .set({ + afterWorkspaceId: last?.workspaceId ?? cursor.afterWorkspaceId, + afterFileId: last?.fileId ?? cursor.afterFileId, + completedAt: rows.length < FILE_SEARCH_BACKFILL_PAGE_SIZE ? now : null, + updatedAt: now, + }) + .where(eq(workspaceFileSearchBackfill.id, BACKFILL_CURSOR_ID)) + return files.length +} + +async function reapStaleClaims(tx: DbTransaction, now: Date): Promise { + const staleBefore = new Date(now.getTime() - FILE_SEARCH_INDEX_STALE_DISPATCH_MS) + const rows = await tx + .select({ + workspaceId: workspaceFileSearchIndex.workspaceId, + fileId: workspaceFileSearchIndex.fileId, + sourceContentUpdatedAt: workspaceFileSearchIndex.sourceContentUpdatedAt, + currentFileId: workspaceFiles.id, + }) + .from(workspaceFileSearchIndex) + .leftJoin( + workspaceFiles, + and( + eq(workspaceFiles.id, workspaceFileSearchIndex.fileId), + eq(workspaceFiles.workspaceId, workspaceFileSearchIndex.workspaceId), + eq(workspaceFiles.context, 'workspace'), + isNull(workspaceFiles.deletedAt), + eq(workspaceFiles.contentUpdatedAt, workspaceFileSearchIndex.sourceContentUpdatedAt) + ) + ) + .where( + and( + eq(workspaceFileSearchIndex.status, 'pending'), + isNotNull(workspaceFileSearchIndex.dispatchedAt), + lt(workspaceFileSearchIndex.dispatchedAt, staleBefore) + ) + ) + .orderBy(asc(workspaceFileSearchIndex.dispatchedAt), asc(workspaceFileSearchIndex.fileId)) + .limit(FILE_SEARCH_INDEX_STALE_REAP_LIMIT) + .for('update', { of: workspaceFileSearchIndex, skipLocked: true }) + + const current = rows.filter((row) => row.currentFileId !== null) + const obsolete = rows.filter((row) => row.currentFileId === null) + const currentFilter = revisionFilter(current) + if (currentFilter) { + await tx + .update(workspaceFileSearchIndex) + .set({ dispatchedAt: null, updatedAt: now }) + .where(currentFilter) + await enqueueWorkspaces( + tx, + current.map((row) => row.workspaceId), + now + ) + } + const obsoleteFilter = revisionFilter(obsolete) + if (obsoleteFilter) { + await tx + .delete(workspaceFileSearchSegment) + .where( + or( + ...obsolete.map((row) => + and( + eq(workspaceFileSearchSegment.fileId, row.fileId), + eq(workspaceFileSearchSegment.sourceContentUpdatedAt, row.sourceContentUpdatedAt) + ) + ) + ) + ) + await tx.delete(workspaceFileSearchIndex).where(obsoleteFilter) + } + return rows.length +} + +async function claimQueuedWorkspaceJobs( + tx: DbTransaction, + workspaceIds: readonly string[], + remainingGlobalCapacity: number, + now: Date +): Promise { + if (workspaceIds.length === 0 || remainingGlobalCapacity <= 0) return [] + const workspaceValues = sql.join( + workspaceIds.map((workspaceId) => sql`(${workspaceId})`), + sql`, ` + ) + const rows = await tx.execute<{ + workspaceId: string + fileId: string + sourceContentUpdatedAt: Date + }>(sql` + WITH selected_workspace(workspace_id) AS ( + VALUES ${workspaceValues} + ), + workspace_active AS ( + SELECT search_index.workspace_id, count(*)::int AS active_count + FROM workspace_file_search_index AS search_index + INNER JOIN selected_workspace AS selected + ON selected.workspace_id = search_index.workspace_id + WHERE search_index.status = 'pending' + AND search_index.dispatched_at IS NOT NULL + GROUP BY search_index.workspace_id + ), + ranked AS ( + SELECT + search_index.workspace_id, + search_index.file_id, + search_index.source_content_updated_at, + search_index.updated_at, + coalesce(workspace_active.active_count, 0) AS active_count, + row_number() OVER ( + PARTITION BY search_index.workspace_id + ORDER BY + search_index.updated_at, + search_index.file_id, + search_index.source_content_updated_at + ) AS workspace_rank + FROM workspace_file_search_index AS search_index + INNER JOIN selected_workspace AS selected + ON selected.workspace_id = search_index.workspace_id + INNER JOIN workspace_files AS file + ON file.id = search_index.file_id + AND file.workspace_id = search_index.workspace_id + AND file.context = 'workspace' + AND file.deleted_at IS NULL + AND file.content_updated_at = search_index.source_content_updated_at + LEFT JOIN workspace_active + ON workspace_active.workspace_id = search_index.workspace_id + WHERE search_index.status = 'pending' + AND search_index.dispatched_at IS NULL + ), + candidates AS ( + SELECT workspace_id, file_id, source_content_updated_at + FROM ranked + WHERE workspace_rank <= ${FILE_SEARCH_INDEX_WORKSPACE_OUTSTANDING} - active_count + ORDER BY updated_at, workspace_id, file_id, source_content_updated_at + LIMIT ${remainingGlobalCapacity} + ) + UPDATE workspace_file_search_index AS search_index + SET dispatched_at = ${now.toISOString()}::timestamp + FROM candidates + WHERE search_index.file_id = candidates.file_id + AND search_index.source_content_updated_at = candidates.source_content_updated_at + AND search_index.status = 'pending' + AND search_index.dispatched_at IS NULL + RETURNING + search_index.workspace_id AS "workspaceId", + search_index.file_id AS "fileId", + search_index.source_content_updated_at AS "sourceContentUpdatedAt" + `) + + const remainingForWorkspace = tx + .select({ fileId: workspaceFileSearchIndex.fileId }) + .from(workspaceFileSearchIndex) + .innerJoin( + workspaceFiles, + and( + eq(workspaceFiles.id, workspaceFileSearchIndex.fileId), + eq(workspaceFiles.workspaceId, workspaceFileSearchDispatchQueue.workspaceId), + eq(workspaceFiles.context, 'workspace'), + isNull(workspaceFiles.deletedAt), + eq(workspaceFiles.contentUpdatedAt, workspaceFileSearchIndex.sourceContentUpdatedAt) + ) + ) + .where( + and( + eq(workspaceFileSearchIndex.workspaceId, workspaceFileSearchDispatchQueue.workspaceId), + eq(workspaceFileSearchIndex.status, 'pending'), + isNull(workspaceFileSearchIndex.dispatchedAt) + ) + ) + await tx + .update(workspaceFileSearchDispatchQueue) + .set({ lastDispatchedAt: now, updatedAt: now }) + .where( + and( + inArray(workspaceFileSearchDispatchQueue.workspaceId, workspaceIds), + exists(remainingForWorkspace) + ) + ) + await tx + .delete(workspaceFileSearchDispatchQueue) + .where( + and( + inArray(workspaceFileSearchDispatchQueue.workspaceId, workspaceIds), + notExists(remainingForWorkspace) + ) + ) + + return rows.map((row) => ({ + workspaceId: row.workspaceId, + fileId: row.fileId, + sourceContentUpdatedAt: new Date(row.sourceContentUpdatedAt).toISOString(), + })) +} + +export async function prepareWorkspaceFileSearchDispatch(): Promise { + return db.transaction(async (tx) => { + const [lock] = await tx.execute<{ acquired: boolean }>( + sql`SELECT pg_try_advisory_xact_lock(hashtextextended(${DISPATCH_LOCK_NAME}, 0)) AS acquired` + ) + if (!lock?.acquired) { + return { payloads: [], backfilledFiles: 0, reapedClaims: 0, lockAcquired: false } + } + + const now = new Date() + const backfilledFiles = await seedBackfillPage(tx, now) + const reapedClaims = await reapStaleClaims(tx, now) + const [{ active }] = await tx + .select({ active: count() }) + .from(workspaceFileSearchIndex) + .where( + and( + eq(workspaceFileSearchIndex.status, 'pending'), + isNotNull(workspaceFileSearchIndex.dispatchedAt) + ) + ) + const remainingGlobalCapacity = Math.max(0, FILE_SEARCH_INDEX_MAX_OUTSTANDING - Number(active)) + if (remainingGlobalCapacity === 0) { + return { payloads: [], backfilledFiles, reapedClaims, lockAcquired: true } + } + + const workspaces = await tx + .select({ workspaceId: workspaceFileSearchDispatchQueue.workspaceId }) + .from(workspaceFileSearchDispatchQueue) + .orderBy( + sql`${workspaceFileSearchDispatchQueue.lastDispatchedAt} ASC NULLS FIRST`, + asc(workspaceFileSearchDispatchQueue.enqueuedAt), + asc(workspaceFileSearchDispatchQueue.workspaceId) + ) + .limit(Math.min(FILE_SEARCH_INDEX_DISPATCH_WORKSPACES, remainingGlobalCapacity)) + .for('update', { skipLocked: true }) + + const payloads = await claimQueuedWorkspaceJobs( + tx, + workspaces.map((workspace) => workspace.workspaceId), + remainingGlobalCapacity, + now + ) + return { payloads, backfilledFiles, reapedClaims, lockAcquired: true } + }) +} + +async function releaseDispatchClaims(payloads: readonly WorkspaceFileSearchIndexPayload[]) { + if (payloads.length === 0) return + const rows = payloads.map((payload) => ({ + workspaceId: payload.workspaceId, + fileId: payload.fileId, + sourceContentUpdatedAt: new Date(payload.sourceContentUpdatedAt), + })) + await db.transaction(async (tx) => { + const filter = revisionFilter(rows) + if (filter) { + await tx + .update(workspaceFileSearchIndex) + .set({ dispatchedAt: null, updatedAt: new Date() }) + .where(and(filter, eq(workspaceFileSearchIndex.status, 'pending'))) + } + await enqueueWorkspaces( + tx, + rows.map((row) => row.workspaceId), + new Date() + ) + }) +} + +async function dispatchPreparedJobs( + payloads: readonly WorkspaceFileSearchIndexPayload[] +): Promise { + if (payloads.length === 0) return 0 + if (!shouldUseWorkspaceFileSearchTrigger(isTriggerDevEnabled, isInsideTriggerRun())) { + runDetached('workspace-file-search-index', async () => { + for (const payload of payloads) { + try { + await indexWorkspaceFileForSearch(payload, new AbortController().signal) + } catch { + await markWorkspaceFileSearchIndexFailed(payload) + } + } + }) + return payloads.length + } + + const [{ tasks }, { resolveTriggerRegion }] = await Promise.all([ + import('@trigger.dev/sdk'), + import('@/lib/core/async-jobs/region'), + ]) + const region = await resolveTriggerRegion() + const result = await tasks.batchTrigger( + 'workspace-file-search-index', + buildWorkspaceFileSearchTriggerItems(payloads, region) + ) + logger.info('Dispatched workspace file search indexing batch', { + batchId: result.batchId, + files: payloads.length, + }) + return payloads.length +} + +export async function dispatchWorkspaceFileSearchIndexJobs(): Promise { + const prepared = await prepareWorkspaceFileSearchDispatch() + if (!prepared.lockAcquired || prepared.payloads.length === 0) { + return { + dispatchedFiles: 0, + backfilledFiles: prepared.backfilledFiles, + reapedClaims: prepared.reapedClaims, + lockAcquired: prepared.lockAcquired, + } + } + try { + const dispatchedFiles = await dispatchPreparedJobs(prepared.payloads) + return { + dispatchedFiles, + backfilledFiles: prepared.backfilledFiles, + reapedClaims: prepared.reapedClaims, + lockAcquired: prepared.lockAcquired, + } + } catch (error) { + await releaseDispatchClaims(prepared.payloads) + logger.error('Failed to dispatch workspace file search indexing batch', { + files: prepared.payloads.length, + error: getErrorMessage(error), + }) + throw error + } +} diff --git a/apps/sim/lib/workspace-files/search/enqueue-dispatch.test.ts b/apps/sim/lib/workspace-files/search/enqueue-dispatch.test.ts new file mode 100644 index 00000000000..f6435c8bdd3 --- /dev/null +++ b/apps/sim/lib/workspace-files/search/enqueue-dispatch.test.ts @@ -0,0 +1,55 @@ +/** + * @vitest-environment node + */ +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' + +const mocks = vi.hoisted(() => ({ + dispatch: vi.fn(), + resolveRegion: vi.fn(), + runDetached: vi.fn(), + trigger: vi.fn(), +})) + +vi.mock('@trigger.dev/sdk', () => ({ tasks: { trigger: mocks.trigger } })) +vi.mock('@/background/workspace-file-search-dispatch', () => ({ + workspaceFileSearchDispatchTask: {}, +})) +vi.mock('@/lib/core/async-jobs/region', () => ({ resolveTriggerRegion: mocks.resolveRegion })) +vi.mock('@/lib/core/config/env-flags', () => ({ isTriggerDevEnabled: true })) +vi.mock('@/lib/core/utils/background', () => ({ runDetached: mocks.runDetached })) +vi.mock('@/lib/workspace-files/search/dispatcher', () => ({ + dispatchWorkspaceFileSearchIndexJobs: mocks.dispatch, +})) + +import { enqueueWorkspaceFileSearchDispatch } from '@/lib/workspace-files/search/enqueue-dispatch' + +describe('workspace file search dispatcher enqueue', () => { + beforeEach(() => { + vi.clearAllMocks() + vi.useFakeTimers() + vi.setSystemTime(new Date('2026-08-29T12:34:45.000Z')) + mocks.resolveRegion.mockResolvedValue('us-east-1') + mocks.trigger.mockResolvedValue({ id: 'run-1' }) + }) + + afterEach(() => { + vi.useRealTimers() + }) + + it('waits only for durable Trigger.dev acceptance and does not run the dispatcher inline', async () => { + await expect(enqueueWorkspaceFileSearchDispatch()).resolves.toEqual({ + backend: 'trigger-dev', + jobId: 'run-1', + }) + + expect(mocks.trigger).toHaveBeenCalledWith('workspace-file-search-dispatch', undefined, { + idempotencyKey: 'workspace-file-search-dispatch:29800114', + idempotencyKeyTTL: '5m', + maxDuration: 60, + region: 'us-east-1', + ttl: '5m', + }) + expect(mocks.dispatch).not.toHaveBeenCalled() + expect(mocks.runDetached).not.toHaveBeenCalled() + }) +}) diff --git a/apps/sim/lib/workspace-files/search/enqueue-dispatch.ts b/apps/sim/lib/workspace-files/search/enqueue-dispatch.ts new file mode 100644 index 00000000000..750fd96d56a --- /dev/null +++ b/apps/sim/lib/workspace-files/search/enqueue-dispatch.ts @@ -0,0 +1,43 @@ +import { isTriggerDevEnabled } from '@/lib/core/config/env-flags' +import { runDetached } from '@/lib/core/utils/background' +import { + FILE_SEARCH_DISPATCH_INTERVAL_MS, + FILE_SEARCH_DISPATCH_MAX_DURATION_SECONDS, +} from '@/lib/workspace-files/search/constants' +import { dispatchWorkspaceFileSearchIndexJobs } from '@/lib/workspace-files/search/dispatcher' + +export interface WorkspaceFileSearchDispatchEnqueueResult { + backend: 'trigger-dev' | 'inline' + jobId: string | null +} + +/** + * Durably hands a dispatcher run to Trigger.dev and returns after acceptance. The inline branch is + * development-only and detaches from the HTTP response because the local server is long-lived. + */ +export async function enqueueWorkspaceFileSearchDispatch(): Promise { + if (!isTriggerDevEnabled) { + runDetached('workspace-file-search-dispatch', dispatchWorkspaceFileSearchIndexJobs) + return { backend: 'inline', jobId: null } + } + + const [{ tasks }, { workspaceFileSearchDispatchTask }, { resolveTriggerRegion }] = + await Promise.all([ + import('@trigger.dev/sdk'), + import('@/background/workspace-file-search-dispatch'), + import('@/lib/core/async-jobs/region'), + ]) + const scheduleWindow = Math.floor(Date.now() / FILE_SEARCH_DISPATCH_INTERVAL_MS) + const handle = await tasks.trigger( + 'workspace-file-search-dispatch', + undefined, + { + idempotencyKey: `workspace-file-search-dispatch:${scheduleWindow}`, + idempotencyKeyTTL: '5m', + maxDuration: FILE_SEARCH_DISPATCH_MAX_DURATION_SECONDS, + region: await resolveTriggerRegion(), + ttl: '5m', + } + ) + return { backend: 'trigger-dev', jobId: handle.id } +} diff --git a/apps/sim/lib/workspace-files/search/indexing.ts b/apps/sim/lib/workspace-files/search/indexing.ts new file mode 100644 index 00000000000..e09978e5bee --- /dev/null +++ b/apps/sim/lib/workspace-files/search/indexing.ts @@ -0,0 +1,391 @@ +import { Buffer, isUtf8 } from 'node:buffer' +import { db } from '@sim/db' +import { + workspaceFileSearchIndex, + workspaceFileSearchSegment, + workspaceFiles, +} from '@sim/db/schema' +import { createLogger } from '@sim/logger' +import { toError } from '@sim/utils/errors' +import { and, eq, isNull, ne, or } from 'drizzle-orm' +import { isPayloadSizeLimitError } from '@/lib/core/utils/stream-limits' +import { isSupportedFileType, parseBuffer } from '@/lib/file-parsers' +import { + fetchServableWorkspaceFileBuffer, + getWorkspaceFile, +} from '@/lib/uploads/contexts/workspace' +import { getFileExtension } from '@/lib/uploads/utils/file-utils' +import { + FILE_SEARCH_INSERT_BATCH_BYTES, + FILE_SEARCH_INSERT_BATCH_ROWS, + FILE_SEARCH_MAX_EXTRACTED_BYTES, + FILE_SEARCH_MAX_SOURCE_BYTES, +} from '@/lib/workspace-files/search/constants' +import { + iterateLogicalLines, + segmentLogicalLine, + truncateUtf8ToBytes, +} from '@/lib/workspace-files/search/text' + +const logger = createLogger('WorkspaceFileSearchIndexer') + +export interface WorkspaceFileSearchIndexPayload { + workspaceId: string + fileId: string + sourceContentUpdatedAt: string +} + +type SearchIndexStatus = 'ready' | 'skipped' | 'failed' + +interface ExtractedIndexText { + text: string + partial: boolean +} + +function sameRevision(left: Date | null | undefined, right: Date): boolean { + return Boolean(left && left.getTime() === right.getTime()) +} + +async function extractIndexText( + buffer: Buffer, + fileName: string +): Promise { + if (buffer.length === 0) return { text: '', partial: false } + const extension = getFileExtension(fileName) + if (extension && isSupportedFileType(extension)) { + const parsed = await parseBuffer(buffer, extension) + if (parsed.metadata?.degraded) return null + const content = parsed.content ?? '' + const bounded = truncateUtf8ToBytes(content, FILE_SEARCH_MAX_EXTRACTED_BYTES) + return { + text: bounded, + partial: parsed.metadata?.truncated === true || bounded.length < content.length, + } + } + if (!isUtf8(buffer) || buffer.includes(0)) return null + const content = buffer.toString('utf8') + return { + text: truncateUtf8ToBytes(content, FILE_SEARCH_MAX_EXTRACTED_BYTES), + partial: buffer.length > FILE_SEARCH_MAX_EXTRACTED_BYTES, + } +} + +async function clearRevision( + workspaceId: string, + fileId: string, + sourceContentUpdatedAt: Date +): Promise { + await db + .delete(workspaceFileSearchSegment) + .where( + and( + eq(workspaceFileSearchSegment.workspaceId, workspaceId), + eq(workspaceFileSearchSegment.fileId, fileId), + eq(workspaceFileSearchSegment.sourceContentUpdatedAt, sourceContentUpdatedAt) + ) + ) +} + +async function discardObsoleteRevision(options: { + workspaceId: string + fileId: string + sourceContentUpdatedAt: Date +}): Promise { + await db.transaction(async (tx) => { + await tx + .delete(workspaceFileSearchSegment) + .where( + and( + eq(workspaceFileSearchSegment.workspaceId, options.workspaceId), + eq(workspaceFileSearchSegment.fileId, options.fileId), + eq(workspaceFileSearchSegment.sourceContentUpdatedAt, options.sourceContentUpdatedAt) + ) + ) + await tx + .delete(workspaceFileSearchIndex) + .where( + and( + eq(workspaceFileSearchIndex.workspaceId, options.workspaceId), + eq(workspaceFileSearchIndex.fileId, options.fileId), + eq(workspaceFileSearchIndex.sourceContentUpdatedAt, options.sourceContentUpdatedAt) + ) + ) + }) +} + +async function markTerminal(options: { + workspaceId: string + fileId: string + sourceContentUpdatedAt: Date + status: SearchIndexStatus + partial?: boolean + failureReason?: string + lineCount?: number + indexedBytes?: number +}): Promise { + return db.transaction(async (tx) => { + const [current] = await tx + .select({ + contentUpdatedAt: workspaceFiles.contentUpdatedAt, + deletedAt: workspaceFiles.deletedAt, + context: workspaceFiles.context, + workspaceId: workspaceFiles.workspaceId, + }) + .from(workspaceFiles) + .where(eq(workspaceFiles.id, options.fileId)) + .for('update') + .limit(1) + + const isCurrent = + current?.workspaceId === options.workspaceId && + current.context === 'workspace' && + current.deletedAt === null && + sameRevision(current.contentUpdatedAt, options.sourceContentUpdatedAt) + if (!isCurrent) { + await tx + .delete(workspaceFileSearchSegment) + .where( + and( + eq(workspaceFileSearchSegment.workspaceId, options.workspaceId), + eq(workspaceFileSearchSegment.fileId, options.fileId), + eq(workspaceFileSearchSegment.sourceContentUpdatedAt, options.sourceContentUpdatedAt) + ) + ) + await tx + .delete(workspaceFileSearchIndex) + .where( + and( + eq(workspaceFileSearchIndex.workspaceId, options.workspaceId), + eq(workspaceFileSearchIndex.fileId, options.fileId), + eq(workspaceFileSearchIndex.sourceContentUpdatedAt, options.sourceContentUpdatedAt) + ) + ) + return false + } + + await tx + .insert(workspaceFileSearchIndex) + .values({ + fileId: options.fileId, + workspaceId: options.workspaceId, + sourceContentUpdatedAt: options.sourceContentUpdatedAt, + status: options.status, + partial: options.partial ?? false, + failureReason: options.failureReason, + lineCount: options.lineCount ?? 0, + indexedBytes: options.indexedBytes ?? 0, + dispatchedAt: new Date(), + }) + .onConflictDoUpdate({ + target: [workspaceFileSearchIndex.fileId, workspaceFileSearchIndex.sourceContentUpdatedAt], + set: { + status: options.status, + partial: options.partial ?? false, + failureReason: options.failureReason, + lineCount: options.lineCount ?? 0, + indexedBytes: options.indexedBytes ?? 0, + updatedAt: new Date(), + }, + }) + await tx + .delete(workspaceFileSearchSegment) + .where( + and( + eq(workspaceFileSearchSegment.fileId, options.fileId), + ne(workspaceFileSearchSegment.sourceContentUpdatedAt, options.sourceContentUpdatedAt) + ) + ) + await tx + .delete(workspaceFileSearchIndex) + .where( + and( + eq(workspaceFileSearchIndex.fileId, options.fileId), + ne(workspaceFileSearchIndex.sourceContentUpdatedAt, options.sourceContentUpdatedAt), + or( + ne(workspaceFileSearchIndex.status, 'pending'), + isNull(workspaceFileSearchIndex.dispatchedAt) + ) + ) + ) + return true + }) +} + +async function insertSearchSegments(options: { + workspaceId: string + fileId: string + sourceContentUpdatedAt: Date + text: string + signal: AbortSignal +}): Promise { + type SegmentInsert = typeof workspaceFileSearchSegment.$inferInsert + let batch: SegmentInsert[] = [] + let batchBytes = 0 + let lineCount = 0 + + const flush = async () => { + if (batch.length === 0) return + options.signal.throwIfAborted() + await db.insert(workspaceFileSearchSegment).values(batch) + batch = [] + batchBytes = 0 + } + + for (const line of iterateLogicalLines(options.text)) { + lineCount = line.lineNumber + for (const segment of segmentLogicalLine(line)) { + const segmentBytes = Buffer.byteLength(segment.content, 'utf8') + if ( + batch.length >= FILE_SEARCH_INSERT_BATCH_ROWS || + (batch.length > 0 && batchBytes + segmentBytes > FILE_SEARCH_INSERT_BATCH_BYTES) + ) { + await flush() + } + batch.push({ + workspaceId: options.workspaceId, + fileId: options.fileId, + sourceContentUpdatedAt: options.sourceContentUpdatedAt, + ...segment, + }) + batchBytes += segmentBytes + } + } + await flush() + return lineCount +} + +export async function indexWorkspaceFileForSearch( + payload: WorkspaceFileSearchIndexPayload, + signal: AbortSignal +): Promise { + signal.throwIfAborted() + const sourceContentUpdatedAt = new Date(payload.sourceContentUpdatedAt) + if (Number.isNaN(sourceContentUpdatedAt.getTime())) { + throw new Error('Workspace file search index payload has an invalid source revision') + } + + const file = await getWorkspaceFile(payload.workspaceId, payload.fileId, { + throwOnError: true, + }) + if (!file || !sameRevision(file.contentUpdatedAt, sourceContentUpdatedAt)) { + await discardObsoleteRevision({ ...payload, sourceContentUpdatedAt }) + return + } + + const [state] = await db + .select({ + status: workspaceFileSearchIndex.status, + workspaceId: workspaceFileSearchIndex.workspaceId, + }) + .from(workspaceFileSearchIndex) + .where( + and( + eq(workspaceFileSearchIndex.fileId, payload.fileId), + eq(workspaceFileSearchIndex.sourceContentUpdatedAt, sourceContentUpdatedAt) + ) + ) + .limit(1) + if (state && state.workspaceId !== payload.workspaceId) { + await discardObsoleteRevision({ ...payload, sourceContentUpdatedAt }) + return + } + if (state?.status === 'ready' || state?.status === 'skipped') return + + await db + .insert(workspaceFileSearchIndex) + .values({ + fileId: payload.fileId, + workspaceId: payload.workspaceId, + sourceContentUpdatedAt, + status: 'pending', + dispatchedAt: new Date(), + }) + .onConflictDoUpdate({ + target: [workspaceFileSearchIndex.fileId, workspaceFileSearchIndex.sourceContentUpdatedAt], + set: { + status: 'pending', + failureReason: null, + partial: false, + lineCount: 0, + indexedBytes: 0, + updatedAt: new Date(), + }, + }) + await clearRevision(payload.workspaceId, payload.fileId, sourceContentUpdatedAt) + + if (file.size > FILE_SEARCH_MAX_SOURCE_BYTES) { + await markTerminal({ + ...payload, + sourceContentUpdatedAt, + status: 'skipped', + failureReason: 'source_too_large', + }) + return + } + + try { + const { buffer } = await fetchServableWorkspaceFileBuffer(file, { + maxBytes: FILE_SEARCH_MAX_SOURCE_BYTES, + signal, + }) + signal.throwIfAborted() + const extracted = await extractIndexText(buffer, file.name) + if (!extracted) { + await markTerminal({ + ...payload, + sourceContentUpdatedAt, + status: 'skipped', + failureReason: 'binary_or_degraded', + }) + return + } + const lineCount = await insertSearchSegments({ + ...payload, + sourceContentUpdatedAt, + text: extracted.text, + signal, + }) + await markTerminal({ + ...payload, + sourceContentUpdatedAt, + status: 'ready', + partial: extracted.partial, + lineCount, + indexedBytes: Buffer.byteLength(extracted.text, 'utf8'), + }) + } catch (error) { + if (signal.aborted) throw error + if (isPayloadSizeLimitError(error)) { + await clearRevision(payload.workspaceId, payload.fileId, sourceContentUpdatedAt) + await markTerminal({ + ...payload, + sourceContentUpdatedAt, + status: 'skipped', + failureReason: 'source_too_large', + }) + return + } + await clearRevision(payload.workspaceId, payload.fileId, sourceContentUpdatedAt) + logger.error('Workspace file search indexing failed', { + workspaceId: payload.workspaceId, + fileId: payload.fileId, + errorType: toError(error).name, + }) + throw error + } +} + +/** Marks a revision failed only after Trigger.dev has exhausted every retry attempt. */ +export async function markWorkspaceFileSearchIndexFailed( + payload: WorkspaceFileSearchIndexPayload +): Promise { + const sourceContentUpdatedAt = new Date(payload.sourceContentUpdatedAt) + if (Number.isNaN(sourceContentUpdatedAt.getTime())) return + await clearRevision(payload.workspaceId, payload.fileId, sourceContentUpdatedAt) + await markTerminal({ + ...payload, + sourceContentUpdatedAt, + status: 'failed', + failureReason: 'indexing_error', + }) +} diff --git a/apps/sim/lib/workspace-files/search/repository.ts b/apps/sim/lib/workspace-files/search/repository.ts new file mode 100644 index 00000000000..c26fa060378 --- /dev/null +++ b/apps/sim/lib/workspace-files/search/repository.ts @@ -0,0 +1,180 @@ +import { db } from '@sim/db' +import { + workspaceFileSearchIndex, + workspaceFileSearchSegment, + workspaceFiles, +} from '@sim/db/schema' +import { and, desc, eq, isNull, sql } from 'drizzle-orm' +import type { WorkspaceFileSecretProvenanceIdentity } from '@/lib/uploads/contexts/workspace/workspace-file-secret-provenance' +import { + createFileSearchPreview, + escapeFileSearchLikePattern, +} from '@/lib/workspace-files/search/text' + +export interface WorkspaceFileSearchIndexStatus { + readyFiles: number + pendingFiles: number + failedFiles: number + skippedFiles: number + partialFiles: number +} + +export interface WorkspaceFileSearchSource { + identity: WorkspaceFileSecretProvenanceIdentity + ownerUserId: string +} + +export interface WorkspaceFileSearchResult { + results: Array<{ + fileId: string + lineNumber: number + text: string + }> + count: number + truncated: boolean + complete: boolean + indexStatus: WorkspaceFileSearchIndexStatus + sources: WorkspaceFileSearchSource[] +} + +interface SearchWorkspaceFileIndexInput { + workspaceId: string + query: string + maxResults: number + caseSensitive: boolean + signal?: AbortSignal +} + +export async function searchWorkspaceFileIndex({ + workspaceId, + query, + maxResults, + caseSensitive, + signal, +}: SearchWorkspaceFileIndexInput): Promise { + signal?.throwIfAborted() + const escapedPattern = `%${escapeFileSearchLikePattern(query)}%` + const matchExpression = caseSensitive + ? sql`${workspaceFileSearchSegment.content} LIKE ${escapedPattern} ESCAPE '\\'` + : sql`${workspaceFileSearchSegment.content} ILIKE ${escapedPattern} ESCAPE '\\'` + const matchPosition = caseSensitive + ? sql`strpos(${workspaceFileSearchSegment.content}, ${query})` + : sql`strpos(lower(${workspaceFileSearchSegment.content}), lower(${query}))` + const surroundingContext = sql`least( + ${matchPosition} - 1, + char_length(${workspaceFileSearchSegment.content}) - (${matchPosition} - 1) - char_length(${query}) + )` + + const rows = await db + .selectDistinctOn( + [workspaceFiles.originalName, workspaceFiles.id, workspaceFileSearchSegment.lineNumber], + { + fileId: workspaceFiles.id, + fileName: workspaceFiles.originalName, + fileKey: workspaceFiles.key, + ownerUserId: workspaceFiles.userId, + contentUpdatedAt: workspaceFiles.contentUpdatedAt, + lineNumber: workspaceFileSearchSegment.lineNumber, + segmentNumber: workspaceFileSearchSegment.segmentNumber, + segmentStart: workspaceFileSearchSegment.segmentStart, + lineLength: workspaceFileSearchSegment.lineLength, + content: workspaceFileSearchSegment.content, + } + ) + .from(workspaceFileSearchSegment) + .innerJoin( + workspaceFileSearchIndex, + and( + eq(workspaceFileSearchIndex.fileId, workspaceFileSearchSegment.fileId), + eq( + workspaceFileSearchIndex.sourceContentUpdatedAt, + workspaceFileSearchSegment.sourceContentUpdatedAt + ), + eq(workspaceFileSearchIndex.status, 'ready') + ) + ) + .innerJoin( + workspaceFiles, + and( + eq(workspaceFiles.id, workspaceFileSearchSegment.fileId), + eq(workspaceFiles.workspaceId, workspaceId), + eq(workspaceFiles.context, 'workspace'), + isNull(workspaceFiles.deletedAt), + eq(workspaceFiles.contentUpdatedAt, workspaceFileSearchSegment.sourceContentUpdatedAt) + ) + ) + .where(and(eq(workspaceFileSearchSegment.workspaceId, workspaceId), matchExpression)) + .orderBy( + workspaceFiles.originalName, + workspaceFiles.id, + workspaceFileSearchSegment.lineNumber, + desc(surroundingContext), + workspaceFileSearchSegment.segmentNumber + ) + .limit(maxResults + 1) + + signal?.throwIfAborted() + const coverageRows = await db + .select({ + readyFiles: sql`count(*) filter (where ${workspaceFileSearchIndex.status} = 'ready')::int`, + pendingFiles: sql`count(*) filter (where ${workspaceFileSearchIndex.status} is null or ${workspaceFileSearchIndex.status} = 'pending')::int`, + failedFiles: sql`count(*) filter (where ${workspaceFileSearchIndex.status} = 'failed')::int`, + skippedFiles: sql`count(*) filter (where ${workspaceFileSearchIndex.status} = 'skipped')::int`, + partialFiles: sql`count(*) filter (where ${workspaceFileSearchIndex.partial} is true)::int`, + }) + .from(workspaceFiles) + .leftJoin( + workspaceFileSearchIndex, + and( + eq(workspaceFileSearchIndex.fileId, workspaceFiles.id), + eq(workspaceFileSearchIndex.sourceContentUpdatedAt, workspaceFiles.contentUpdatedAt) + ) + ) + .where( + and( + eq(workspaceFiles.workspaceId, workspaceId), + eq(workspaceFiles.context, 'workspace'), + isNull(workspaceFiles.deletedAt) + ) + ) + + signal?.throwIfAborted() + const resultRows = rows.slice(0, maxResults) + const indexStatus = coverageRows[0] ?? { + readyFiles: 0, + pendingFiles: 0, + failedFiles: 0, + skippedFiles: 0, + partialFiles: 0, + } + const sourcesByFileId = new Map() + for (const row of resultRows) { + sourcesByFileId.set(row.fileId, { + identity: { + fileId: row.fileId, + key: row.fileKey, + context: 'workspace', + contentUpdatedAt: row.contentUpdatedAt, + }, + ownerUserId: row.ownerUserId, + }) + } + + const results = resultRows.map((row) => ({ + fileId: row.fileId, + lineNumber: row.lineNumber, + text: createFileSearchPreview(row.content, query, caseSensitive, undefined, { + prefixOmitted: row.segmentStart > 0, + suffixOmitted: row.segmentStart + row.content.length < row.lineLength, + }), + })) + signal?.throwIfAborted() + return { + results, + count: results.length, + truncated: rows.length > maxResults, + complete: indexStatus.pendingFiles === 0 && indexStatus.failedFiles === 0, + indexStatus, + sources: [...sourcesByFileId.values()], + } +} diff --git a/apps/sim/lib/workspace-files/search/text.test.ts b/apps/sim/lib/workspace-files/search/text.test.ts new file mode 100644 index 00000000000..98a499d107d --- /dev/null +++ b/apps/sim/lib/workspace-files/search/text.test.ts @@ -0,0 +1,84 @@ +import { Buffer } from 'node:buffer' +import { describe, expect, it, vi } from 'vitest' +import { + createFileSearchPreview, + escapeFileSearchLikePattern, + isFileSearchCaseSensitive, + iterateLogicalLines, + segmentLogicalLine, + truncateUtf8ToBytes, +} from '@/lib/workspace-files/search/text' + +describe('workspace file search text utilities', () => { + it('implements Unicode smart-case and escapes LIKE metacharacters', () => { + expect(isFileSearchCaseSensitive('résumé')).toBe(false) + expect(isFileSearchCaseSensitive('Résumé')).toBe(true) + expect(isFileSearchCaseSensitive('東京A')).toBe(true) + expect(escapeFileSearchLikePattern('100%_done\\')).toBe('100\\%\\_done\\\\') + }) + + it('normalizes CRLF and preserves one-based logical line numbers', () => { + expect([...iterateLogicalLines('first\r\nsecond\n')]).toEqual([ + { lineNumber: 1, text: 'first' }, + { lineNumber: 2, text: 'second' }, + { lineNumber: 3, text: '' }, + ]) + }) + + it('creates overlapping segments that preserve boundary matches', () => { + const segments = [...segmentLogicalLine({ lineNumber: 3, text: 'abcdefghijklmnop' }, 10, 4)] + expect(segments.map(({ content }) => content)).toEqual(['abcdefghij', 'ghijklmnop']) + expect(segments[1]).toMatchObject({ lineNumber: 3, segmentNumber: 1, segmentStart: 6 }) + expect(segments[0].content).toContain('ghij') + expect(segments[1].content).toContain('ghij') + }) + + it('returns a match-centered UTF-8-safe bounded preview', () => { + const line = `${'🙂'.repeat(800)}needle${'é'.repeat(800)}` + const preview = createFileSearchPreview(line, 'needle', false) + expect(preview).toContain('needle') + expect(preview.startsWith('…')).toBe(true) + expect(preview.endsWith('…')).toBe(true) + expect(Buffer.byteLength(preview, 'utf8')).toBeLessThanOrEqual(2048) + expect(preview).not.toContain('�') + }) + + it('maps case-folded offsets back to the original line', () => { + const line = `${'İ'.repeat(1200)}needle${'x'.repeat(1200)}` + const preview = createFileSearchPreview(line, 'needle', false) + + expect(preview).toContain('needle') + expect(Buffer.byteLength(preview, 'utf8')).toBeLessThanOrEqual(2048) + }) + + it('centers previews with locale-independent case folding', () => { + const localeLowerCase = vi + .spyOn(String.prototype, 'toLocaleLowerCase') + .mockImplementation(function (this: string) { + return String(this).replaceAll('I', 'ı').toLowerCase() + }) + + try { + const line = `${'x'.repeat(1500)}I${'y'.repeat(1500)}` + const preview = createFileSearchPreview(line, 'i', false, 128) + expect(preview).toContain('I') + } finally { + localeLowerCase.mockRestore() + } + }) + + it('shows omitted logical-line content beyond the selected segment', () => { + expect( + createFileSearchPreview('needle and nearby text', 'needle', false, 2048, { + prefixOmitted: true, + suffixOmitted: true, + }) + ).toBe('…needle and nearby text…') + }) + + it('truncates extracted text on a UTF-8 boundary', () => { + const truncated = truncateUtf8ToBytes('abc🙂def', 6) + expect(truncated).toBe('abc') + expect(Buffer.byteLength(truncated, 'utf8')).toBeLessThanOrEqual(6) + }) +}) diff --git a/apps/sim/lib/workspace-files/search/text.ts b/apps/sim/lib/workspace-files/search/text.ts new file mode 100644 index 00000000000..60691b803f1 --- /dev/null +++ b/apps/sim/lib/workspace-files/search/text.ts @@ -0,0 +1,178 @@ +import { Buffer, isUtf8 } from 'node:buffer' +import { + FILE_SEARCH_MAX_PREVIEW_BYTES, + FILE_SEARCH_SEGMENT_CHARS, + FILE_SEARCH_SEGMENT_OVERLAP_CHARS, +} from '@/lib/workspace-files/search/constants' + +export interface LogicalLine { + lineNumber: number + text: string +} + +export interface SearchSegment { + lineNumber: number + segmentNumber: number + segmentStart: number + lineLength: number + content: string +} + +export function isFileSearchCaseSensitive(query: string): boolean { + return /\p{Lu}/u.test(query) +} + +export function escapeFileSearchLikePattern(query: string): string { + return query.replace(/[\\%_]/g, '\\$&') +} + +export function* iterateLogicalLines(text: string): Generator { + let lineStart = 0 + let lineNumber = 1 + for (let index = 0; index <= text.length; index += 1) { + if (index !== text.length && text.charCodeAt(index) !== 10) continue + const hasCarriageReturn = index > lineStart && text.charCodeAt(index - 1) === 13 + yield { + lineNumber, + text: text.slice(lineStart, hasCarriageReturn ? index - 1 : index), + } + lineStart = index + 1 + lineNumber += 1 + } +} + +function safeSegmentEnd(text: string, requestedEnd: number): number { + if (requestedEnd >= text.length) return text.length + const previousCodeUnit = text.charCodeAt(requestedEnd - 1) + const nextCodeUnit = text.charCodeAt(requestedEnd) + return previousCodeUnit >= 0xd800 && previousCodeUnit <= 0xdbff && nextCodeUnit >= 0xdc00 + ? requestedEnd - 1 + : requestedEnd +} + +export function* segmentLogicalLine( + line: LogicalLine, + segmentChars = FILE_SEARCH_SEGMENT_CHARS, + overlapChars = FILE_SEARCH_SEGMENT_OVERLAP_CHARS +): Generator { + if (line.text.length === 0) return + const step = Math.max(1, segmentChars - overlapChars) + let segmentNumber = 0 + for (let start = 0; start < line.text.length; start += step) { + const end = safeSegmentEnd(line.text, Math.min(line.text.length, start + segmentChars)) + yield { + lineNumber: line.lineNumber, + segmentNumber, + segmentStart: start, + lineLength: line.text.length, + content: line.text.slice(start, end), + } + segmentNumber += 1 + if (end === line.text.length) break + } +} + +function utf8PrefixWithinBudget(text: string, maxBytes: number): string { + let low = 0 + let high = text.length + while (low < high) { + const middle = Math.ceil((low + high) / 2) + if (Buffer.byteLength(text.slice(0, middle), 'utf8') <= maxBytes) low = middle + else high = middle - 1 + } + let end = low + if (end > 0 && end < text.length) { + const previousCodeUnit = text.charCodeAt(end - 1) + const nextCodeUnit = text.charCodeAt(end) + if ( + previousCodeUnit >= 0xd800 && + previousCodeUnit <= 0xdbff && + nextCodeUnit >= 0xdc00 && + nextCodeUnit <= 0xdfff + ) { + end -= 1 + } + } + return text.slice(0, end) +} + +function utf8SuffixWithinBudget(text: string, maxBytes: number): string { + const reversedCodePoints = [...text].reverse().join('') + return [...utf8PrefixWithinBudget(reversedCodePoints, maxBytes)].reverse().join('') +} + +export function truncateUtf8ToBytes(text: string, maxBytes: number): string { + const candidate = text.length > maxBytes ? text.slice(0, maxBytes) : text + const encoded = Buffer.from(candidate, 'utf8') + if (encoded.length <= maxBytes) return candidate + let end = maxBytes + while (end > 0 && !isUtf8(encoded.subarray(0, end))) end -= 1 + return encoded.subarray(0, end).toString('utf8') +} + +function findFileSearchMatchRange( + line: string, + query: string, + caseSensitive: boolean +): { start: number; end: number } { + if (caseSensitive) { + const start = Math.max(0, line.indexOf(query)) + return { start, end: Math.min(line.length, start + query.length) } + } + + const searchableLine = line.toLowerCase() + const searchableQuery = query.toLowerCase() + const foldedStart = searchableLine.indexOf(searchableQuery) + if (foldedStart < 0) return { start: 0, end: Math.min(line.length, query.length) } + + const originalStarts: number[] = [] + const originalEnds: number[] = [] + for (let offset = 0; offset < line.length; ) { + const codePoint = line.codePointAt(offset) + if (codePoint === undefined) break + const character = String.fromCodePoint(codePoint) + const foldedCharacter = character.toLowerCase() + const end = offset + character.length + for (let foldedOffset = 0; foldedOffset < foldedCharacter.length; foldedOffset += 1) { + originalStarts.push(offset) + originalEnds.push(end) + } + offset = end + } + + const foldedEnd = foldedStart + searchableQuery.length + const start = originalStarts[foldedStart] ?? 0 + const end = originalEnds[foldedEnd - 1] ?? Math.min(line.length, start + query.length) + return { start, end } +} + +export function createFileSearchPreview( + line: string, + query: string, + caseSensitive: boolean, + maxBytes = FILE_SEARCH_MAX_PREVIEW_BYTES, + boundaries: { prefixOmitted?: boolean; suffixOmitted?: boolean } = {} +): string { + const boundaryBytes = + (boundaries.prefixOmitted ? Buffer.byteLength('…', 'utf8') : 0) + + (boundaries.suffixOmitted ? Buffer.byteLength('…', 'utf8') : 0) + if (Buffer.byteLength(line, 'utf8') + boundaryBytes <= maxBytes) { + return `${boundaries.prefixOmitted ? '…' : ''}${line}${boundaries.suffixOmitted ? '…' : ''}` + } + + const { start: matchStart, end: matchEnd } = findFileSearchMatchRange(line, query, caseSensitive) + const leadingEllipsis = boundaries.prefixOmitted || matchStart > 0 ? '…' : '' + const trailingEllipsis = boundaries.suffixOmitted || matchEnd < line.length ? '…' : '' + const ellipsisBytes = Buffer.byteLength(leadingEllipsis + trailingEllipsis, 'utf8') + const match = line.slice(matchStart, matchEnd) + const matchBytes = Buffer.byteLength(match, 'utf8') + const surroundingBudget = Math.max(0, maxBytes - ellipsisBytes - matchBytes) + const beforeBudget = Math.floor(surroundingBudget / 2) + const afterBudget = surroundingBudget - beforeBudget + const before = utf8SuffixWithinBudget(line.slice(0, matchStart), beforeBudget) + const after = utf8PrefixWithinBudget(line.slice(matchEnd), afterBudget) + const preview = `${boundaries.prefixOmitted || before.length < matchStart ? '…' : ''}${ + before + }${match}${after}${boundaries.suffixOmitted || matchEnd + after.length < line.length ? '…' : ''}` + return utf8PrefixWithinBudget(preview, maxBytes) +} diff --git a/apps/sim/lib/workspaces/admin-move-source-impact.test.ts b/apps/sim/lib/workspaces/admin-move-source-impact.test.ts new file mode 100644 index 00000000000..1b55f9a86e7 --- /dev/null +++ b/apps/sim/lib/workspaces/admin-move-source-impact.test.ts @@ -0,0 +1,167 @@ +/** @vitest-environment node */ + +import { member, subscription } from '@sim/db/schema' +import { dbChainMockFns, queueTableRows, resetDbChainMock } from '@sim/testing' +import { PgDialect } from 'drizzle-orm/pg-core' +import { afterAll, beforeEach, describe, expect, it, vi } from 'vitest' +import { resolveMoveEntitlements } from '@/lib/workspaces/admin-move-source-impact' + +vi.unmock('drizzle-orm') + +const { isSubscriptionBackedEntitlement } = vi.hoisted(() => ({ + isSubscriptionBackedEntitlement: vi.fn(() => true), +})) + +vi.mock('@/lib/billing/core/subscription', () => ({ isSubscriptionBackedEntitlement })) + +const SOURCE = 'org-source' +const DESTINATION = 'org-destination' + +/** + * `resolveMoveEntitlements` decides whether a move silently strips Enterprise + * capability, which is the one blocker the admin cannot recover from after the + * fact. Every past defect in it read a benign absence as a verdict: a missing + * subscription row as "not entitled", a `past_due` row as usable, a + * billing-blocked owner as entitled. + * + * Two of those live in JavaScript and one lives in SQL, and the split decides + * how each is pinned. The chain mock returns queued rows verbatim and never + * evaluates a `WHERE`, so queueing a `past_due` row would prove nothing: the + * filter that excludes it is `inArray(subscription.status, + * USABLE_SUBSCRIPTION_STATUSES)`, and the mock would hand the row back either + * way. That guard is asserted against the rendered SQL instead. The + * plan comparison and the billing-blocked exclusion both run in JavaScript + * over the returned rows, so those are pinned with data. + */ +describe('resolveMoveEntitlements', () => { + afterAll(resetDbChainMock) + + beforeEach(() => { + vi.clearAllMocks() + resetDbChainMock() + isSubscriptionBackedEntitlement.mockReturnValue(true) + }) + + it('asks the database only for usable subscriptions', async () => { + /** + * Asserted against the SQL rather than with a `past_due` fixture, because + * the chain mock ignores `WHERE` and would return one regardless. + * + * `USABLE_...` and not `ENTITLED_...` is the whole point: the gates this + * blocker protects resolve through `getOrganizationSubscriptionUsable`, + * which accepts only `active`. A `past_due` Enterprise subscription is + * entitled but not usable, so its features are already gone, and reading + * it as Enterprise would wave a real downgrade through. + */ + queueTableRows(subscription, []) + queueTableRows(member, []) + + await resolveMoveEntitlements(SOURCE, DESTINATION) + + const dialect = new PgDialect() + const rendered = dbChainMockFns.where.mock.calls.map(([condition]) => + dialect.sqlToQuery(condition as never) + ) + const statuses = rendered.flatMap((query) => + query.params.filter((param) => param === 'active' || param === 'past_due') + ) + expect(statuses).toEqual(['active']) + }) + + it('reports no loss for a personal source, which has nothing to lose', async () => { + await expect(resolveMoveEntitlements(null, DESTINATION)).resolves.toEqual({ + sourceIsEnterprise: false, + destinationIsEnterprise: false, + capabilitiesLost: [], + }) + }) + + it('reports no loss when entitlement comes from deployment configuration', async () => { + /** + * Billing disabled, or self-hosted with access control on, grants + * entitlement with no `subscription` row anywhere. Reading that absence as + * "the destination is not Enterprise" would block every move in those + * deployments. + */ + isSubscriptionBackedEntitlement.mockReturnValue(false) + + await expect(resolveMoveEntitlements(SOURCE, DESTINATION)).resolves.toEqual({ + sourceIsEnterprise: false, + destinationIsEnterprise: false, + capabilitiesLost: [], + }) + }) + + it('names what an Enterprise to Team move would strip', async () => { + queueTableRows(subscription, [ + { referenceId: SOURCE, plan: 'enterprise' }, + { referenceId: DESTINATION, plan: 'team' }, + ]) + queueTableRows(member, []) + + const result = await resolveMoveEntitlements(SOURCE, DESTINATION) + + expect(result.sourceIsEnterprise).toBe(true) + expect(result.destinationIsEnterprise).toBe(false) + /** + * The list is derived from the organization settings section union, so it + * must cover the sections a hand-written list kept missing, not just the + * headline ones. + */ + expect(result.capabilitiesLost).toEqual( + expect.arrayContaining([ + 'permission groups', + 'organization usage monitoring', + 'audit logs', + 'data drains', + 'whitelabel branding', + 'workspace forking', + 'custom blocks', + ]) + ) + }) + + it('reports no loss when both organizations are Enterprise', async () => { + queueTableRows(subscription, [ + { referenceId: SOURCE, plan: 'enterprise' }, + { referenceId: DESTINATION, plan: 'enterprise' }, + ]) + queueTableRows(member, []) + + await expect(resolveMoveEntitlements(SOURCE, DESTINATION)).resolves.toEqual({ + sourceIsEnterprise: true, + destinationIsEnterprise: true, + capabilitiesLost: [], + }) + }) + + it('reports no loss when a non-Enterprise source moves anywhere', async () => { + queueTableRows(subscription, [{ referenceId: SOURCE, plan: 'team' }]) + queueTableRows(member, []) + + await expect(resolveMoveEntitlements(SOURCE, DESTINATION)).resolves.toEqual({ + sourceIsEnterprise: false, + destinationIsEnterprise: false, + capabilitiesLost: [], + }) + }) + + it('treats a billing-blocked Enterprise destination as a downgrade', async () => { + /** + * The gates resolve through the owner's billing state, so an Enterprise + * row behind a blocked owner buys the destination nothing. Counting the + * row alone would wave the downgrade through. + */ + queueTableRows(subscription, [ + { referenceId: SOURCE, plan: 'enterprise' }, + { referenceId: DESTINATION, plan: 'enterprise' }, + ]) + queueTableRows(member, [{ organizationId: DESTINATION }]) + + const result = await resolveMoveEntitlements(SOURCE, DESTINATION) + + expect(result.sourceIsEnterprise).toBe(true) + expect(result.destinationIsEnterprise).toBe(false) + expect(result.capabilitiesLost.length).toBeGreaterThan(0) + }) +}) diff --git a/apps/sim/lib/workspaces/admin-move-source-impact.ts b/apps/sim/lib/workspaces/admin-move-source-impact.ts new file mode 100644 index 00000000000..696e11d8aff --- /dev/null +++ b/apps/sim/lib/workspaces/admin-move-source-impact.ts @@ -0,0 +1,658 @@ +import { db } from '@sim/db' +import { + account, + credential, + credentialGroup, + customBlock, + type DataRetentionSettings, + member, + organization, + organizationMemberUsageLimit, + permissionGroup, + permissionGroupWorkspace, + permissions, + subscription, + user, + userStats, + workflow, + workspace, + workspaceBYOKKeys, + workspaceEnvironment, +} from '@sim/db/schema' +import { and, eq, inArray, isNull, ne, or } from 'drizzle-orm' +import { alias } from 'drizzle-orm/pg-core' +import type { OrganizationSettingsSection } from '@/components/settings/navigation' +import { isSubscriptionBackedEntitlement } from '@/lib/billing/core/subscription' +import { isEnterprise } from '@/lib/billing/plan-helpers' +import { USABLE_SUBSCRIPTION_STATUSES } from '@/lib/billing/subscriptions/utils' +import type { DbOrTx } from '@/lib/db/types' +import { getCustomBlockUsageCounts } from '@/lib/workflows/custom-blocks/operations' + +/** + * Everything the source organization loses when a workspace leaves it, plus the + * in-transaction cleanup that keeps the cross-org invariants documented in + * `admin-move.ts` from ever being violated. + * + * Split out of `admin-move.ts` because the move orchestration and the question + * "what does the org left behind lose?" are separate concerns with no shared + * state — the move calls in, passes a source organization id, and gets a + * reviewable summary back. + */ + +/** + * Every organization settings section, labelled for a move review, or `null` + * when the section is not gated on an Enterprise plan. + * + * Typed as a total `Record` over {@link OrganizationSettingsSection} on + * purpose. The first hand-maintained version of this list drifted and silently + * under-reported what an Enterprise to Team move would cost, which is the one + * failure mode a downgrade disclosure cannot have. Now adding a section to + * that union fails the build here until somebody decides whether it is gated. + * + * The gating mirrors `isOrganizationSettingsSectionAvailable`: on hosted every + * section except `members` and `billing` resolves to `hasEnterprisePlan`. The + * type is imported type-only so this domain module stays free of the settings + * navigation module's React and icon imports. + */ +const ENTERPRISE_GATED_SECTION_LABELS: Record = { + members: null, + billing: null, + usage: 'organization usage monitoring', + 'access-control': 'permission groups', + 'audit-logs': 'audit logs', + sso: 'SSO settings and domains', + sessions: 'session policies and organization session revocation', + 'data-retention': 'data retention policies', + 'data-drains': 'data drains', + whitelabeling: 'whitelabel branding', +} + +/** + * Enterprise-gated capabilities that are not organization settings sections, + * and so cannot be derived from the section union above. + */ +const ENTERPRISE_GATED_NON_SECTION_CAPABILITIES = ['workspace forking', 'custom blocks'] as const + +/** Capabilities gated on the owning organization holding an Enterprise plan. */ +const ENTERPRISE_GATED_CAPABILITIES: readonly string[] = [ + ...Object.values(ENTERPRISE_GATED_SECTION_LABELS).filter( + (label): label is string => label !== null + ), + ...ENTERPRISE_GATED_NON_SECTION_CAPABILITIES, +] + +export interface WorkspaceMoveSourceOrganizationRow { + id: string + name: string + ownerId: string | null + ownerName: string | null + ownerEmail: string | null +} + +/** + * The organization a workspace is leaving. + * + * Unlike `getDestinationOrganization` this uses a LEFT join on the owner: an + * organization with no owner must not block a move *out* of it — moving the + * workspace away is precisely the repair for that state. + */ +export async function getSourceOrganization( + organizationId: string, + executor: DbOrTx = db +): Promise { + const [row] = await executor + .select({ + id: organization.id, + name: organization.name, + ownerId: member.userId, + ownerName: user.name, + ownerEmail: user.email, + }) + .from(organization) + .leftJoin(member, and(eq(member.organizationId, organization.id), eq(member.role, 'owner'))) + .leftJoin(user, eq(user.id, member.userId)) + .where(eq(organization.id, organizationId)) + .limit(1) + + return row ?? null +} + +export interface CrossOrgForkEdge { + workspaceId: string + name: string + organizationId: string | null + direction: 'parent' | 'child' +} + +/** + * Fork edges that would span two organizations once the workspace lands in + * `destinationOrganizationId`, in both directions. + * + * Deliberately does NOT filter archived workspaces the way `getForkParent` / + * `getForkChildren` do. Those are read helpers for the settings UI; an archived + * workspace can be unarchived, so the invariant has to hold for it too. + */ +export async function findCrossOrgForkEdges( + workspaceId: string, + destinationOrganizationId: string, + executor: DbOrTx = db +): Promise { + const parent = alias(workspace, 'fork_parent') + const [parentRows, childRows] = await Promise.all([ + executor + .select({ + workspaceId: parent.id, + name: parent.name, + organizationId: parent.organizationId, + }) + .from(workspace) + .innerJoin(parent, eq(parent.id, workspace.forkedFromWorkspaceId)) + .where( + and( + eq(workspace.id, workspaceId), + or(isNull(parent.organizationId), ne(parent.organizationId, destinationOrganizationId)) + ) + ), + executor + .select({ + workspaceId: workspace.id, + name: workspace.name, + organizationId: workspace.organizationId, + }) + .from(workspace) + .where( + and( + eq(workspace.forkedFromWorkspaceId, workspaceId), + or( + isNull(workspace.organizationId), + ne(workspace.organizationId, destinationOrganizationId) + ) + ) + ), + ]) + + return [ + ...parentRows.map((row) => ({ ...row, direction: 'parent' as const })), + ...childRows.map((row) => ({ ...row, direction: 'child' as const })), + ] +} + +export interface UnpublishableCustomBlock { + id: string + type: string + name: string + movingWorkspaceUsage: { live: number; deployed: number } + sourceOrgElsewhereUsage: { live: number; deployed: number } +} + +/** + * Source-org custom blocks bound to a workflow inside the moving workspace. + * + * Usage is reported as two separate numbers because they mean different things + * to the admin confirming the move: placements inside the moving workspace + * leave with it, while placements elsewhere in the source org are collateral + * that stays behind and breaks. `getCustomBlockUsageCounts` counts the whole + * org, so the moving workspace's own share is measured and subtracted. + */ +export interface SourceOrgCustomBlockRow { + id: string + type: string + name: string +} + +/** + * The source-org custom blocks bound to this workspace's workflows, and nothing + * more. + * + * Separate from {@link findUnpublishableCustomBlocks} because the move runs + * inside a transaction and must stay on its executor: the usage counts that + * enrich the preflight report come from `getCustomBlockUsageCounts`, which + * reads through the global client with no executor seam. The move only needs + * the ids to delete and the names for the audit entry, so it takes this. + */ +export async function findSourceOrgCustomBlocksForWorkspace( + workspaceId: string, + sourceOrganizationId: string, + executor: DbOrTx = db +): Promise { + return executor + .select({ + id: customBlock.id, + type: customBlock.type, + name: customBlock.name, + }) + .from(customBlock) + .innerJoin(workflow, eq(workflow.id, customBlock.workflowId)) + .where( + and( + eq(workflow.workspaceId, workspaceId), + eq(customBlock.organizationId, sourceOrganizationId) + ) + ) +} + +/** + * Preflight-only: the blocks above, enriched with how much breaks. Never call + * this from inside a transaction — `getCustomBlockUsageCounts` reads through + * the global client. + */ +export async function findUnpublishableCustomBlocks( + workspaceId: string, + sourceOrganizationId: string, + executor: DbOrTx = db +): Promise<{ items: UnpublishableCustomBlock[]; total: number }> { + const rows = await findSourceOrgCustomBlocksForWorkspace( + workspaceId, + sourceOrganizationId, + executor + ) + + if (rows.length === 0) return { items: [], total: 0 } + + /** + * Cap BEFORE the fan-out. Each surviving row costs two more queries, so + * enriching an unbounded set would let one admin preflight open hundreds of + * concurrent connections and exhaust the pool. The caller bounds the list for + * the contract anyway; bounding here makes the query cost bounded too. + */ + const MAX_ENRICHED_BLOCKS = 500 + const enrichable = rows.slice(0, MAX_ENRICHED_BLOCKS) + + /** + * Both scopes are measured with the SAME predicates rather than derived by + * subtraction. `getCustomBlockUsageCounts` returns `usageCount` as the union + * of live-editor and active-deployment placements, so subtracting a + * live-only count from it misattributes a block that appears solely in the + * moving workspace's deployment to the source organization's collateral. + */ + /** + * Bounded concurrency. Each row costs two queries, so a flat `Promise.all` + * over the cap would open a thousand at once and saturate the pool for an + * admin preflight. Chunked keeps the ceiling at `ENRICHMENT_CONCURRENCY * 2`. + */ + const ENRICHMENT_CONCURRENCY = 10 + const items: UnpublishableCustomBlock[] = [] + for (let index = 0; index < enrichable.length; index += ENRICHMENT_CONCURRENCY) { + const chunk = await Promise.all( + enrichable.slice(index, index + ENRICHMENT_CONCURRENCY).map(async (row) => { + const [moving, elsewhere] = await Promise.all([ + getCustomBlockUsageCounts(sourceOrganizationId, row.type, { + onlyWorkspaceId: workspaceId, + }), + getCustomBlockUsageCounts(sourceOrganizationId, row.type, { + excludeWorkspaceId: workspaceId, + }), + ]) + return { + ...row, + movingWorkspaceUsage: { live: moving.usageCount, deployed: moving.deployedUsageCount }, + sourceOrgElsewhereUsage: { + live: elsewhere.usageCount, + deployed: elsewhere.deployedUsageCount, + }, + } + }) + ) + items.push(...chunk) + } + /** `total` is the untruncated row count so the caller can disclose the gap. */ + return { items, total: rows.length } +} + +export interface WorkspaceMoveCredentialSummaryRow { + items: Array<{ + id: string + displayName: string + type: string + backedBySourceOrgMember: boolean + }> + credentialGroupCount: number + environmentVariableKeys: string[] + byokKeyCount: number + /** Rows omitted to stay within response limits. */ + truncatedCredentials: number + truncatedEnvironmentVariableKeys: number +} + +/** + * Secrets that travel with the workspace, enumerated so the destination's + * admins can see exactly what they inherit. + * + * Reads display metadata only — never `encrypted*` columns, and only the + * *keys* of environment variables. `backedBySourceOrgMember` marks credentials + * whose backing identity belongs to someone in the source organization, + * mirroring `getOrganizationTransferCredentialDependenciesTx`'s predicate: the + * destination would be able to act as that person. + */ +export async function collectWorkspaceCredentialSummary( + workspaceId: string, + sourceOrganizationId: string | null, + executor: DbOrTx = db +): Promise { + const [credentialRows, groupRows, environmentRows, byokRows] = await Promise.all([ + executor + .select({ + id: credential.id, + displayName: credential.displayName, + type: credential.type, + oauthOwnerId: account.userId, + envOwnerUserId: credential.envOwnerUserId, + }) + .from(credential) + .leftJoin(account, eq(account.id, credential.accountId)) + .where(eq(credential.workspaceId, workspaceId)), + executor + .select({ id: credentialGroup.id }) + .from(credentialGroup) + .where(eq(credentialGroup.workspaceId, workspaceId)), + executor + .select({ variables: workspaceEnvironment.variables }) + .from(workspaceEnvironment) + .where(eq(workspaceEnvironment.workspaceId, workspaceId)) + .limit(1), + executor + .select({ id: workspaceBYOKKeys.id }) + .from(workspaceBYOKKeys) + .where(eq(workspaceBYOKKeys.workspaceId, workspaceId)), + ]) + + const sourceMemberIds = sourceOrganizationId + ? new Set( + ( + await executor + .select({ userId: member.userId }) + .from(member) + .where(eq(member.organizationId, sourceOrganizationId)) + ).map((row) => row.userId) + ) + : new Set() + + const variables = environmentRows[0]?.variables + const CREDENTIAL_LIMIT = 1_000 + const allEnvironmentKeys = + variables && typeof variables === 'object' ? Object.keys(variables).sort() : [] + return { + truncatedCredentials: Math.max(credentialRows.length - CREDENTIAL_LIMIT, 0), + truncatedEnvironmentVariableKeys: Math.max(allEnvironmentKeys.length - CREDENTIAL_LIMIT, 0), + items: credentialRows.slice(0, CREDENTIAL_LIMIT).map((row) => { + const backingUserId = row.oauthOwnerId ?? row.envOwnerUserId + return { + id: row.id, + displayName: row.displayName, + type: row.type, + backedBySourceOrgMember: backingUserId !== null && sourceMemberIds.has(backingUserId), + } + }), + credentialGroupCount: groupRows.length, + environmentVariableKeys: allEnvironmentKeys.slice(0, CREDENTIAL_LIMIT), + byokKeyCount: byokRows.length, + } +} + +export interface WorkspaceMoveEntitlementsResult { + sourceIsEnterprise: boolean + destinationIsEnterprise: boolean + capabilitiesLost: string[] +} + +/** + * Whether the destination can carry the source's entitlements. + * + * Reads through `isOrganizationOnEnterprisePlan` rather than the `subscription` + * table so this verdict can never disagree with the gates it protects. A + * personal source has no entitlements to lose. + */ +export async function resolveMoveEntitlements( + sourceOrganizationId: string | null, + destinationOrganizationId: string, + executor: DbOrTx = db +): Promise { + /** + * Compares ACTUAL Enterprise plans, not `isOrganizationOnEnterprisePlan`. + * + * That helper is really "on a paid organization plan" — `isOrgPlan = isTeam + * || isEnterprise` — so it reports a Team destination as entitled. The API + * gates it backs do accept Team, but the surfaces a user actually reaches do + * not: whitelabeling, SSO settings and access control each gate on + * `isEnterprise` directly. An Enterprise → Team move therefore does lose + * capability, and a downgrade blocker built on the looser predicate would + * wave exactly that case through. + * + * In the two deployment-configured modes there is nothing to compare — no + * subscription row need exist — so no capability can be lost. + */ + if (!sourceOrganizationId || !isSubscriptionBackedEntitlement()) { + return { sourceIsEnterprise: false, destinationIsEnterprise: false, capabilitiesLost: [] } + } + + const [entitledRows, blockedRows] = await Promise.all([ + executor + .select({ referenceId: subscription.referenceId, plan: subscription.plan }) + .from(subscription) + .where( + and( + inArray(subscription.referenceId, [sourceOrganizationId, destinationOrganizationId]), + /** + * `USABLE_...`, not `ENTITLED_...`. The gates resolve through + * `getOrganizationSubscriptionUsable`, which accepts only `active` + * — a `past_due` Enterprise subscription is entitled but not + * usable, so the features go away while an entitled-status filter + * would still call the destination Enterprise and allow the move. + */ + inArray(subscription.status, USABLE_SUBSCRIPTION_STATUSES) + ) + ), + executor + .select({ organizationId: member.organizationId }) + .from(member) + .innerJoin(userStats, eq(userStats.userId, member.userId)) + .where( + and( + inArray(member.organizationId, [sourceOrganizationId, destinationOrganizationId]), + eq(member.role, 'owner'), + eq(userStats.billingBlocked, true) + ) + ), + ]) + + const blocked = new Set(blockedRows.map((row) => row.organizationId)) + const enterpriseOrganizationIds = new Set( + entitledRows + .filter((row) => isEnterprise(row.plan)) + .map((row) => row.referenceId) + .filter((organizationId) => !blocked.has(organizationId)) + ) + + const sourceIsEnterprise = enterpriseOrganizationIds.has(sourceOrganizationId) + const destinationIsEnterprise = enterpriseOrganizationIds.has(destinationOrganizationId) + return { + sourceIsEnterprise, + destinationIsEnterprise, + capabilitiesLost: + sourceIsEnterprise && !destinationIsEnterprise ? [...ENTERPRISE_GATED_CAPABILITIES] : [], + } +} + +export interface RetainedCollaboratorCap { + userId: string + email: string + sourceOrgLimitDollars: number | null +} + +/** + * Collaborators who keep explicit workspace access after the move, together + * with the per-member usage cap that stops applying to them. + * + * The cap is looked up as `(payer organization, actor)`, so once the payer + * becomes the destination these people fall back to the destination's pooled + * limit with no individual ceiling. The source figures are reported — never + * copied — so the destination's admin can re-apply deliberately. + */ +export async function findRetainedCollaboratorCaps( + workspaceId: string, + sourceOrganizationId: string, + executor: DbOrTx = db +): Promise { + const rows = await executor + .select({ + userId: permissions.userId, + email: user.email, + usageLimit: organizationMemberUsageLimit.usageLimit, + }) + .from(permissions) + .innerJoin(user, eq(user.id, permissions.userId)) + /** + * No membership join. `setOrgMemberUsageLimit` explicitly supports targets + * that are not `member` rows — "external members are supported" — so an + * external collaborator can hold a source-organization cap. Requiring + * membership here silently dropped exactly those people from the review, + * which is the opposite of the field's purpose: disclosing every cap that + * stops applying. The cap row itself already scopes to the source org. + */ + .leftJoin( + organizationMemberUsageLimit, + and( + eq(organizationMemberUsageLimit.userId, permissions.userId), + eq(organizationMemberUsageLimit.organizationId, sourceOrganizationId) + ) + ) + .where(and(eq(permissions.entityType, 'workspace'), eq(permissions.entityId, workspaceId))) + + /** + * Cap-holders first. The caller bounds this list, and the whole point of the + * field is disclosing caps that stop applying — ordering by presence means a + * workspace with more collaborators than the bound can still never drop a + * real cap in favour of a collaborator who has none. + */ + return rows + .map((row) => ({ + userId: row.userId, + email: row.email, + sourceOrgLimitDollars: row.usageLimit === null ? null : Number(row.usageLimit), + })) + .sort((left, right) => { + if (left.sourceOrgLimitDollars === right.sourceOrgLimitDollars) return 0 + if (left.sourceOrgLimitDollars === null) return 1 + if (right.sourceOrgLimitDollars === null) return -1 + return 0 + }) +} + +/** Permission-group rows that will be detached, resolved to group names. */ +export async function findAttachedPermissionGroups( + workspaceId: string, + executor: DbOrTx = db +): Promise> { + return executor + .select({ + permissionGroupId: permissionGroupWorkspace.permissionGroupId, + name: permissionGroup.name, + }) + .from(permissionGroupWorkspace) + .innerJoin(permissionGroup, eq(permissionGroup.id, permissionGroupWorkspace.permissionGroupId)) + .where(eq(permissionGroupWorkspace.workspaceId, workspaceId)) +} + +/** Counts the source-org retention entries that name this workspace. */ +export function countRetentionRulesForWorkspace( + settings: DataRetentionSettings | null | undefined, + workspaceId: string +): { piiRedactionRules: number; retentionOverrides: number } { + return { + piiRedactionRules: (settings?.piiRedaction?.rules ?? []).filter( + (rule) => rule.workspaceId === workspaceId + ).length, + retentionOverrides: (settings?.retentionOverrides ?? []).filter( + (override) => override.workspaceId === workspaceId + ).length, + } +} + +/** Removes every entry naming `workspaceId`, or `null` when nothing changed. */ +export function stripRetentionRulesForWorkspace( + settings: DataRetentionSettings | null | undefined, + workspaceId: string +): DataRetentionSettings | null { + if (!settings) return null + const counts = countRetentionRulesForWorkspace(settings, workspaceId) + if (counts.piiRedactionRules === 0 && counts.retentionOverrides === 0) return null + + const next: DataRetentionSettings = { ...settings } + if (settings.piiRedaction?.rules) { + next.piiRedaction = { + ...settings.piiRedaction, + rules: settings.piiRedaction.rules.filter((rule) => rule.workspaceId !== workspaceId), + } + } + if (settings.retentionOverrides) { + next.retentionOverrides = settings.retentionOverrides.filter( + (override) => override.workspaceId !== workspaceId + ) + } + return next +} + +/** + * Deletes the source-org rows that cannot follow the workspace and would + * otherwise desynchronize, and strips the source org's retention entries that + * name it. Runs inside the move transaction. + * + * `permission_group_workspace` grants nothing after the move — `resolveWorkspaceGroup` + * filters by the workspace's *current* organization — but `getGroupWorkspaces` + * joins `workspace` with no organization filter, so leaving the rows would leak + * the departed workspace's name into the source org's group UI and permanently + * desync the denormalized `organization_id` column. + */ +export async function cleanupSourceOrganizationArtifactsTx( + tx: DbOrTx, + params: { workspaceId: string; sourceOrganizationId: string } +): Promise<{ detachedPermissionGroupIds: string[] }> { + const detached = await tx + .delete(permissionGroupWorkspace) + .where(eq(permissionGroupWorkspace.workspaceId, params.workspaceId)) + .returning({ permissionGroupId: permissionGroupWorkspace.permissionGroupId }) + + const [sourceOrg] = await tx + .select({ dataRetentionSettings: organization.dataRetentionSettings }) + .from(organization) + .where(eq(organization.id, params.sourceOrganizationId)) + .for('update') + .limit(1) + + const strippedSettings = stripRetentionRulesForWorkspace( + sourceOrg?.dataRetentionSettings, + params.workspaceId + ) + if (strippedSettings) { + await tx + .update(organization) + .set({ dataRetentionSettings: strippedSettings, updatedAt: new Date() }) + .where(eq(organization.id, params.sourceOrganizationId)) + } + + return { detachedPermissionGroupIds: detached.map((row) => row.permissionGroupId) } +} + +/** True when the two organizations present different whitelabel branding. */ +export async function willBrandingChange( + sourceOrganizationId: string, + destinationOrganizationId: string, + executor: DbOrTx = db +): Promise { + const rows = await executor + .select({ id: organization.id, whitelabelSettings: organization.whitelabelSettings }) + .from(organization) + .where(inArray(organization.id, [sourceOrganizationId, destinationOrganizationId])) + + /** An absent settings object and an empty one both mean default branding. */ + const normalize = (settings: unknown): string => { + if (!settings || typeof settings !== 'object') return '' + const entries = Object.entries(settings as Record) + .filter(([, value]) => value !== null && value !== undefined && value !== '') + .sort(([left], [right]) => (left < right ? -1 : left > right ? 1 : 0)) + return entries.length === 0 ? '' : JSON.stringify(entries) + } + const source = rows.find((row) => row.id === sourceOrganizationId)?.whitelabelSettings + const destination = rows.find((row) => row.id === destinationOrganizationId)?.whitelabelSettings + return normalize(source) !== normalize(destination) +} diff --git a/apps/sim/lib/workspaces/admin-move.test.ts b/apps/sim/lib/workspaces/admin-move.test.ts index ea83ac9526c..92f7d858949 100644 --- a/apps/sim/lib/workspaces/admin-move.test.ts +++ b/apps/sim/lib/workspaces/admin-move.test.ts @@ -28,6 +28,13 @@ import { WORKSPACE_MODE } from '@/lib/workspaces/policy' vi.unmock('drizzle-orm') const { + resolveMoveEntitlements, + findCrossOrgForkEdges, + findUnpublishableCustomBlocks, + findSourceOrgCustomBlocksForWorkspace, + cleanupSourceOrganizationArtifactsTx, + deleteCustomBlock, + acquireOrganizationMutationLock, recordAudit, recordAuditOnce, enqueueOrReschedulePendingOutboxEvent, @@ -39,7 +46,24 @@ const { sendInvitationEmail, countPendingSeatInvitations, resolveSeatCapacity, + collectWorkspaceCredentialSummary, + getSourceOrganization, } = vi.hoisted(() => ({ + resolveMoveEntitlements: vi.fn(() => + Promise.resolve({ + sourceIsEnterprise: false, + destinationIsEnterprise: false, + capabilitiesLost: [] as string[], + }) + ), + findCrossOrgForkEdges: vi.fn(() => Promise.resolve([])), + findUnpublishableCustomBlocks: vi.fn(() => Promise.resolve({ items: [], total: 0 })), + findSourceOrgCustomBlocksForWorkspace: vi.fn(() => Promise.resolve([])), + cleanupSourceOrganizationArtifactsTx: vi.fn(() => + Promise.resolve({ detachedPermissionGroupIds: [] }) + ), + deleteCustomBlock: vi.fn(), + acquireOrganizationMutationLock: vi.fn(), recordAudit: vi.fn(), recordAuditOnce: vi.fn(), enqueueOrReschedulePendingOutboxEvent: vi.fn(), @@ -51,16 +75,66 @@ const { sendInvitationEmail: vi.fn(), countPendingSeatInvitations: vi.fn(() => Promise.resolve(0)), resolveSeatCapacity: vi.fn(() => Promise.resolve(10)), + collectWorkspaceCredentialSummary: vi.fn(), + getSourceOrganization: vi.fn(), })) +const SOURCE_ORGANIZATION = { + id: 'org-source', + name: 'Source', + ownerId: 'source-owner', + ownerName: 'Source Owner', + ownerEmail: 'source-owner@example.com', +} + +const EMPTY_CREDENTIALS = { + items: [] as Array<{ + id: string + displayName: string + type: string + backedBySourceOrgMember: boolean + }>, + credentialGroupCount: 0, + environmentVariableKeys: [] as string[], + byokKeyCount: 0, + truncatedCredentials: 0, + truncatedEnvironmentVariableKeys: 0, +} + +const POPULATED_CREDENTIALS = { + ...EMPTY_CREDENTIALS, + items: [ + { id: 'credential-1', displayName: 'Slack', type: 'oauth', backedBySourceOrgMember: true }, + ], + credentialGroupCount: 1, + environmentVariableKeys: ['OPENAI_API_KEY'], + byokKeyCount: 2, +} + +/** A workspace whose secrets exceed the response bounds, so rows were dropped. */ +const TRUNCATED_CREDENTIALS = { + ...POPULATED_CREDENTIALS, + truncatedCredentials: 3, + truncatedEnvironmentVariableKeys: 7, +} + vi.mock('@sim/audit', () => ({ - AuditAction: { WORKSPACE_UPDATED: 'workspace.updated', INVITATION_UPDATED: 'invitation.updated' }, - AuditResourceType: { WORKSPACE: 'workspace' }, + AuditAction: { + WORKSPACE_UPDATED: 'workspace.updated', + INVITATION_UPDATED: 'invitation.updated', + ORGANIZATION_UPDATED: 'organization.updated', + CUSTOM_BLOCK_DELETED: 'custom_block.deleted', + }, + AuditResourceType: { + WORKSPACE: 'workspace', + ORGANIZATION: 'organization', + CUSTOM_BLOCK: 'custom_block', + }, recordAudit, recordAuditOnce, })) vi.mock('@/lib/billing/organizations/membership', () => ({ - acquireOrganizationMutationLock: vi.fn(), + acquireOrganizationMutationLock, })) vi.mock('@/lib/billing/storage/payer-transfer', () => ({ changeWorkspaceStoragePayerInTx })) vi.mock('@/lib/billing/validation/seat-management', () => ({ @@ -87,6 +161,23 @@ vi.mock('@/lib/invitations/send', () => ({ sendInvitationEmail, })) vi.mock('@/lib/table/billing', () => ({ invalidateWorkspaceTableLimitsCache })) +vi.mock('@/lib/workflows/custom-blocks/operations', () => ({ deleteCustomBlock })) +vi.mock('@/lib/workspaces/admin-move-source-impact', () => ({ + cleanupSourceOrganizationArtifactsTx, + collectWorkspaceCredentialSummary, + countRetentionRulesForWorkspace: vi.fn(() => ({ + piiRedactionRules: 0, + retentionOverrides: 0, + })), + findAttachedPermissionGroups: vi.fn(() => Promise.resolve([])), + findCrossOrgForkEdges, + findRetainedCollaboratorCaps: vi.fn(() => Promise.resolve([])), + findUnpublishableCustomBlocks, + findSourceOrgCustomBlocksForWorkspace, + getSourceOrganization, + resolveMoveEntitlements, + willBrandingChange: vi.fn(() => Promise.resolve(false)), +})) const movedWorkspace = { id: 'workspace-1', @@ -109,6 +200,15 @@ const personalWorkspace = { storageUsedBytes: 128, } +/** Organization-owned source, for the org-to-org path. */ +const organizationWorkspace = { + ...movedWorkspace, + name: 'Organization workspace', + workspaceMode: WORKSPACE_MODE.ORGANIZATION, + organizationId: 'org-source', + billedAccountUserId: 'source-org-owner', +} + const destination = { id: 'org-1', name: 'Destination', @@ -118,22 +218,67 @@ const destination = { } /** - * The move flow reads the workspace twice in order — the locked classification - * row and the final summary reload — so the workspace queue gets one set per - * read. All invitation/grant/permission selects resolve the queue-less empty - * default. + * The move flow reads the workspace three times in order: the optimistic + * pre-transaction organization read that decides which organizations to lock, + * the locked classification row, and the final summary reload. The workspace + * queue therefore gets one set per read, in that order. + * + * Keep this comment in step with the reads — a stale count silently shifts + * every later queue entry onto the wrong statement, which surfaces as an + * unrelated "could not be reloaded" failure rather than a queueing error. + * + * All invitation/grant/permission selects resolve the queue-less empty default. */ function queueMoveSelects(workspaceRow: Record) { + queueTableRows(workspace, [workspaceRow]) queueTableRows(workspace, [workspaceRow]) queueTableRows(workspace, [workspaceRow]) queueTableRows(organization, [destination]) } +/** + * The reload path reads the completed operation, then the workspace twice — the + * applied-state check and the summary reload — and the destination once. + */ +function queueMoveOperationSelects(audit: Record) { + queueTableRows(outboxEvent, [ + { + eventType: 'admin.workspace-move-operation', + status: 'completed', + payload: { + request: { + workspaceId: movedWorkspace.id, + destinationOrganizationId: destination.id, + expectedOwnerId: movedWorkspace.ownerId, + }, + audit, + }, + }, + ]) + queueTableRows(workspace, [movedWorkspace]) + queueTableRows(workspace, [movedWorkspace]) + queueTableRows(organization, [destination]) +} + afterAll(resetDbChainMock) beforeEach(() => { vi.clearAllMocks() resetDbChainMock() + /** + * `vi.clearAllMocks` clears call records but keeps implementations, so a + * `mockResolvedValue` set by one case would otherwise leak into every case + * after it. The entitlement resolver is the dangerous one: leaking a + * downgrade verdict turns unrelated moves into `destination-entitlement- + * downgrade` failures that depend on test order. + */ + resolveMoveEntitlements.mockResolvedValue({ + sourceIsEnterprise: false, + destinationIsEnterprise: false, + capabilitiesLost: [], + }) + collectWorkspaceCredentialSummary.mockResolvedValue(EMPTY_CREDENTIALS) + getSourceOrganization.mockResolvedValue(SOURCE_ORGANIZATION) changeWorkspaceStoragePayerInTx.mockResolvedValue({ billableBytes: 128, newPayer: { type: 'organization', id: destination.id }, @@ -156,8 +301,8 @@ describe('classifyWorkspaceMoveState', () => { ).toBe('already-moved') }) - it('continues to reject inter-organization transfers', () => { - expect(() => + it('classifies a workspace owned by a different organization as a move', () => { + expect( classifyWorkspaceMoveState( { workspaceMode: WORKSPACE_MODE.ORGANIZATION, @@ -166,6 +311,19 @@ describe('classifyWorkspaceMoveState', () => { }, 'org-2' ) + ).toBe('move') + }) + + it('rejects a drifted organization mode when no organization is assigned', () => { + expect(() => + classifyWorkspaceMoveState( + { + workspaceMode: WORKSPACE_MODE.ORGANIZATION, + organizationId: null, + archivedAt: null, + }, + 'org-destination' + ) ).toThrowError( expect.objectContaining>({ code: 'already-organization-workspace', @@ -221,6 +379,44 @@ describe('workspace move invitation bounds', () => { }) }) + it('reports a pending invitation as a blocker for an organization-owned source', async () => { + queueTableRows(workspace, [organizationWorkspace]) + queueTableRows(organization, [destination]) + queueTableRows(invitationWorkspaceGrant, [ + { + id: 'invitation-1', + email: 'invitee@example.com', + organizationId: 'org-source', + membershipIntent: 'internal', + permission: 'read', + }, + ]) + + const preflight = await getWorkspaceMovePreflight(organizationWorkspace.id, destination.id) + + expect(preflight.blockers).toEqual([expect.stringContaining('pending invitation')]) + expect(preflight.sourceOrganization).toMatchObject({ id: 'org-source' }) + }) + + it('reports no invitation blocker for a personal source', async () => { + queueTableRows(workspace, [personalWorkspace]) + queueTableRows(organization, [destination]) + queueTableRows(invitationWorkspaceGrant, [ + { + id: 'invitation-1', + email: 'invitee@example.com', + organizationId: null, + membershipIntent: 'internal', + permission: 'read', + }, + ]) + + const preflight = await getWorkspaceMovePreflight(personalWorkspace.id, destination.id) + + expect(preflight.blockers).toEqual([]) + expect(preflight.sourceOrganization).toBeNull() + }) + it('blocks a move when bounded invitation rows expand into too many workspace grants', async () => { queueTableRows(workspace, [personalWorkspace]) queueTableRows(organization, [destination]) @@ -501,6 +697,15 @@ describe('moveWorkspaceToOrganization retries', () => { previousBillingOwnerId: personalWorkspace.billedAccountUserId, newBillingOwnerId: destination.ownerId, organizationAssignedAt: expect.any(String), + /** + * Persisted so a reload of a completed operation can still name the + * organization the workspace came from — the payer transfer has + * already overwritten `workspace.organizationId` by then. + */ + sourceOrganizationId: null, + /** Persisted so the reload path can replay the source-org audit. */ + unpublishedCustomBlocks: [], + detachedPermissionGroupIds: [], }, }, }) @@ -635,6 +840,205 @@ describe('moveWorkspaceToOrganization retries', () => { ) }) + /** + * A completed move records `sourceOrganizationId` even when it is `null`, so + * a reload can tell "this workspace came from a personal source" apart from + * "this operation predates the field". Collapsing the two made every reload + * of a personal-source move claim its origin had failed to persist. + */ + it('does not warn about an unpersisted source for a move recorded as personal', async () => { + queueMoveOperationSelects({ + actor: { id: null, name: 'Admin Panel', email: 'admin@sim.ai' }, + previousBillingOwnerId: personalWorkspace.billedAccountUserId, + newBillingOwnerId: destination.ownerId, + organizationAssignedAt: '2026-08-20T00:00:00.000Z', + sourceOrganizationId: null, + }) + + const view = await getWorkspaceMoveOperation( + movedWorkspace.id, + destination.id, + movedWorkspace.ownerId, + 'operation-1' + ) + + expect(view.notices).toEqual([]) + expect(view.sourceOrganization).toBeNull() + }) + + it('still warns when the payload never recorded a source organization', async () => { + queueMoveOperationSelects({ + actor: { id: null, name: 'Admin Panel', email: 'admin@sim.ai' }, + previousBillingOwnerId: personalWorkspace.billedAccountUserId, + newBillingOwnerId: destination.ownerId, + organizationAssignedAt: '2026-08-20T00:00:00.000Z', + }) + + const view = await getWorkspaceMoveOperation( + movedWorkspace.id, + destination.id, + movedWorkspace.ownerId, + 'operation-1' + ) + + expect(view.notices).toEqual([ + 'This move was recorded before the source organization was persisted, so it cannot be reported.', + ]) + }) + + it('reports the workspace credentials when a completed operation is reloaded', async () => { + collectWorkspaceCredentialSummary.mockResolvedValueOnce(POPULATED_CREDENTIALS) + queueMoveOperationSelects({ + actor: { id: null, name: 'Admin Panel', email: 'admin@sim.ai' }, + previousBillingOwnerId: personalWorkspace.billedAccountUserId, + newBillingOwnerId: destination.ownerId, + organizationAssignedAt: '2026-08-20T00:00:00.000Z', + sourceOrganizationId: 'org-source', + }) + + const view = await getWorkspaceMoveOperation( + movedWorkspace.id, + destination.id, + movedWorkspace.ownerId, + 'operation-1' + ) + + /** Resolved against the recorded source, so `backedBySourceOrgMember` means something. */ + expect(collectWorkspaceCredentialSummary).toHaveBeenCalledWith(movedWorkspace.id, 'org-source') + expect(view.credentials).toEqual(POPULATED_CREDENTIALS) + }) + + /** + * A recorded id whose organization has since been deleted is the third state: + * the payload answered, but the answer can no longer be resolved to a name. + */ + it('distinguishes a deleted source organization from an unrecorded one', async () => { + getSourceOrganization.mockResolvedValueOnce(null) + queueMoveOperationSelects({ + actor: { id: null, name: 'Admin Panel', email: 'admin@sim.ai' }, + previousBillingOwnerId: personalWorkspace.billedAccountUserId, + newBillingOwnerId: destination.ownerId, + organizationAssignedAt: '2026-08-20T00:00:00.000Z', + sourceOrganizationId: 'org-source', + }) + + const view = await getWorkspaceMoveOperation( + movedWorkspace.id, + destination.id, + movedWorkspace.ownerId, + 'operation-1' + ) + + expect(view.sourceOrganization).toBeNull() + expect(view.notices).toEqual([ + 'The organization this workspace came from has since been deleted, so it can no longer be named.', + ]) + }) + + it('reports the workspace credentials in the applied summary', async () => { + queueMoveSelects(organizationWorkspace) + collectWorkspaceCredentialSummary.mockResolvedValueOnce(POPULATED_CREDENTIALS) + + const summary = await moveWorkspaceToOrganization({ + workspaceId: organizationWorkspace.id, + destinationOrganizationId: destination.id, + adminEmail: 'admin@sim.ai', + durableOperationId: 'operation-1', + }) + + /** The PRE-move organization: that is what `backedBySourceOrgMember` compares against. */ + expect(collectWorkspaceCredentialSummary).toHaveBeenCalledWith( + organizationWorkspace.id, + 'org-source', + expect.anything() + ) + expect(summary.credentials).toEqual(POPULATED_CREDENTIALS) + /** Nothing was dropped, so the review is complete and says nothing about truncation. */ + expect(summary.sourceOrganizationImpact.truncated).toBeNull() + }) + + /** + * The applied path used to hardcode these two counters to zero, which would + * present a truncated credential list as a complete one. + */ + it('carries dropped credential counts into the applied truncation record', async () => { + queueMoveSelects(organizationWorkspace) + collectWorkspaceCredentialSummary.mockResolvedValueOnce(TRUNCATED_CREDENTIALS) + + const summary = await moveWorkspaceToOrganization({ + workspaceId: organizationWorkspace.id, + destinationOrganizationId: destination.id, + adminEmail: 'admin@sim.ai', + durableOperationId: 'operation-1', + }) + + expect(summary.sourceOrganizationImpact.truncated).toMatchObject({ + credentials: 3, + environmentVariableKeys: 7, + }) + }) + + it('carries dropped credential counts into a reloaded truncation record', async () => { + collectWorkspaceCredentialSummary.mockResolvedValueOnce(TRUNCATED_CREDENTIALS) + queueMoveOperationSelects({ + actor: { id: null, name: 'Admin Panel', email: 'admin@sim.ai' }, + previousBillingOwnerId: personalWorkspace.billedAccountUserId, + newBillingOwnerId: destination.ownerId, + organizationAssignedAt: '2026-08-20T00:00:00.000Z', + sourceOrganizationId: 'org-source', + }) + + const view = await getWorkspaceMoveOperation( + movedWorkspace.id, + destination.id, + movedWorkspace.ownerId, + 'operation-1' + ) + + expect(view.sourceOrganizationImpact.truncated).toMatchObject({ + credentials: 3, + environmentVariableKeys: 7, + }) + }) + + it('reports the workspace credentials on a retry of a completed move', async () => { + queueMoveSelects(movedWorkspace) + queueTableRows(outboxEvent, [ + { + eventType: 'admin.workspace-move-operation', + status: 'completed', + payload: { + request: { + workspaceId: movedWorkspace.id, + destinationOrganizationId: destination.id, + expectedOwnerId: movedWorkspace.ownerId, + }, + audit: { + actor: { id: null, name: 'Admin Panel', email: 'admin@sim.ai' }, + previousBillingOwnerId: personalWorkspace.billedAccountUserId, + newBillingOwnerId: destination.ownerId, + organizationAssignedAt: '2026-08-20T00:00:00.000Z', + sourceOrganizationId: null, + }, + }, + }, + ]) + collectWorkspaceCredentialSummary.mockResolvedValueOnce(POPULATED_CREDENTIALS) + + const summary = await moveWorkspaceToOrganization({ + workspaceId: movedWorkspace.id, + destinationOrganizationId: destination.id, + adminEmail: 'admin@sim.ai', + expectedOwnerId: movedWorkspace.ownerId, + auditOperationId: 'operation-1', + operationCorrelationId: 'operation-1', + durableOperationId: 'operation-1', + }) + + expect(summary.credentials).toEqual(POPULATED_CREDENTIALS) + expect(summary.notices).toEqual([]) + }) + it('takes shared advisory locks before the workspace row lock and payer mutation', async () => { queueMoveSelects(personalWorkspace) @@ -653,6 +1057,324 @@ describe('moveWorkspaceToOrganization retries', () => { expect(payerMutation).toBeGreaterThan(firstForUpdate) }) + it('locks both organizations in ascending id order, after invitation locks and before the row lock', async () => { + queueMoveSelects(organizationWorkspace) + + await moveWorkspaceToOrganization({ + workspaceId: organizationWorkspace.id, + destinationOrganizationId: destination.id, + adminEmail: 'admin@sim.ai', + durableOperationId: 'operation-1', + }) + + const lockedOrganizationIds = acquireOrganizationMutationLock.mock.calls.map( + (call) => call[1] as string + ) + expect(lockedOrganizationIds).toEqual(['org-1', 'org-source']) + const invitationLock = acquireInvitationMutationLocks.mock.invocationCallOrder[0] + const firstOrganizationLock = acquireOrganizationMutationLock.mock.invocationCallOrder[0] + const firstForUpdate = dbChainMockFns.for.mock.invocationCallOrder[0] + expect(firstOrganizationLock).toBeGreaterThan(invitationLock) + expect(firstForUpdate).toBeGreaterThan(firstOrganizationLock) + }) + + it('fences the payer transfer on the source organization it read under the locks', async () => { + queueMoveSelects(organizationWorkspace) + + await moveWorkspaceToOrganization({ + workspaceId: organizationWorkspace.id, + destinationOrganizationId: destination.id, + adminEmail: 'admin@sim.ai', + durableOperationId: 'operation-1', + }) + + expect(changeWorkspaceStoragePayerInTx).toHaveBeenCalledWith( + expect.anything(), + expect.objectContaining({ + organizationId: destination.id, + expectedCurrentPayer: { + organizationId: 'org-source', + billedAccountUserId: organizationWorkspace.billedAccountUserId, + }, + }) + ) + }) + + it('records the loss in the source organization audit view, not the destination', async () => { + queueMoveSelects(organizationWorkspace) + findSourceOrgCustomBlocksForWorkspace.mockResolvedValueOnce([ + { id: 'block-1', type: 'custom_block_1', name: 'Reporter' }, + ] as never) + + await moveWorkspaceToOrganization({ + workspaceId: organizationWorkspace.id, + destinationOrganizationId: destination.id, + adminEmail: 'admin@sim.ai', + durableOperationId: 'operation-1', + }) + + /** + * `workspaceId: null` + `metadata.organizationId` is the org-level branch + * of `buildOrgScopeCondition`. The workspace-scoped move entry resolves to + * the destination after the move, so without these the organization that + * lost the workspace would have no record of it. + */ + const entries = recordAudit.mock.calls.map((call) => call[0]) + expect(entries).toContainEqual( + expect.objectContaining({ + workspaceId: null, + action: 'organization.updated', + resourceId: 'org-source', + metadata: expect.objectContaining({ organizationId: 'org-source' }), + }) + ) + expect(entries).toContainEqual( + expect.objectContaining({ + workspaceId: null, + action: 'custom_block.deleted', + resourceId: 'block-1', + metadata: expect.objectContaining({ organizationId: 'org-source' }), + }) + ) + }) + + it('records no source-organization entry for a personal source', async () => { + queueMoveSelects(personalWorkspace) + + await moveWorkspaceToOrganization({ + workspaceId: personalWorkspace.id, + destinationOrganizationId: destination.id, + adminEmail: 'admin@sim.ai', + }) + + expect(recordAudit.mock.calls.map((call) => call[0])).not.toContainEqual( + expect.objectContaining({ action: 'organization.updated' }) + ) + }) + + it('reports the source organization in the applied summary', async () => { + queueMoveSelects(organizationWorkspace) + + const summary = await moveWorkspaceToOrganization({ + workspaceId: organizationWorkspace.id, + destinationOrganizationId: destination.id, + adminEmail: 'admin@sim.ai', + durableOperationId: 'operation-1', + }) + + /** + * The summary reloads the workspace AFTER the payer transfer has rewritten + * `organizationId`, so the source is only reportable if it was captured + * beforehand and threaded through. + */ + expect(summary.sourceOrganization).toMatchObject({ id: 'org-source' }) + }) + + it('re-fences the payer transfer after a SourceOrganizationChangedError retry', async () => { + /** + * The optimistic pre-transaction organization read decides which + * organizations get locked. When the workspace moves between that read and + * the locked read, the attempt must abort and retry — otherwise the payer + * transfer is fenced on an organization the workspace has already left, and + * `changeWorkspaceStoragePayerInTx`'s optimistic check is the only thing + * standing between that and a corrupted storage ledger. + * + * First locked read reports a different organization than the pre-read, so + * the loop retries; the second attempt fences on the organization it + * actually observed under the locks. + */ + queueTableRows(workspace, [organizationWorkspace]) + queueTableRows(workspace, [{ ...organizationWorkspace, organizationId: 'org-moved' }]) + queueTableRows(workspace, [{ ...organizationWorkspace, organizationId: 'org-moved' }]) + queueTableRows(workspace, [{ ...organizationWorkspace, organizationId: 'org-moved' }]) + queueTableRows(organization, [destination]) + + await moveWorkspaceToOrganization({ + workspaceId: organizationWorkspace.id, + destinationOrganizationId: destination.id, + adminEmail: 'admin@sim.ai', + durableOperationId: 'operation-1', + }) + + expect(changeWorkspaceStoragePayerInTx).toHaveBeenCalledTimes(1) + expect(changeWorkspaceStoragePayerInTx).toHaveBeenCalledWith( + expect.anything(), + expect.objectContaining({ + expectedCurrentPayer: expect.objectContaining({ organizationId: 'org-moved' }), + }) + ) + }) + + it('unpublishes source-organization custom blocks bound to the moving workspace', async () => { + queueMoveSelects(organizationWorkspace) + findSourceOrgCustomBlocksForWorkspace.mockResolvedValueOnce([ + { id: 'block-1', type: 'custom_block_1', name: 'Reporter' }, + ] as never) + + await moveWorkspaceToOrganization({ + workspaceId: organizationWorkspace.id, + destinationOrganizationId: destination.id, + adminEmail: 'admin@sim.ai', + durableOperationId: 'operation-1', + }) + + expect(deleteCustomBlock).toHaveBeenCalledWith('block-1', expect.anything()) + expect(cleanupSourceOrganizationArtifactsTx).toHaveBeenCalledWith( + expect.anything(), + expect.objectContaining({ sourceOrganizationId: 'org-source' }) + ) + }) + + it('refuses a cross-organization fork edge without mutating anything', async () => { + queueMoveSelects(organizationWorkspace) + findCrossOrgForkEdges.mockResolvedValueOnce([ + { + workspaceId: 'parent-1', + name: 'Parent', + organizationId: 'org-source', + direction: 'parent', + }, + ] as never) + + await expect( + moveWorkspaceToOrganization({ + workspaceId: organizationWorkspace.id, + destinationOrganizationId: destination.id, + adminEmail: 'admin@sim.ai', + durableOperationId: 'operation-1', + }) + ).rejects.toMatchObject>({ code: 'fork-lineage-conflict' }) + + expect(changeWorkspaceStoragePayerInTx).not.toHaveBeenCalled() + expect(deleteCustomBlock).not.toHaveBeenCalled() + }) + + it('refuses a cross-organization fork edge on a PERSONAL source too', async () => { + queueMoveSelects(personalWorkspace) + findCrossOrgForkEdges.mockResolvedValueOnce([ + { workspaceId: 'parent-1', name: 'Parent', organizationId: 'org-other', direction: 'parent' }, + ] as never) + + /** + * A personal workspace whose parent has since moved into an organization + * still produces a cross-org edge. Gating the check on an organization + * source let the transaction accept a move preflight had already refused. + */ + await expect( + moveWorkspaceToOrganization({ + workspaceId: personalWorkspace.id, + destinationOrganizationId: destination.id, + adminEmail: 'admin@sim.ai', + }) + ).rejects.toMatchObject>({ code: 'fork-lineage-conflict' }) + + expect(changeWorkspaceStoragePayerInTx).not.toHaveBeenCalled() + }) + + it('does not fence a move when both organizations are equally entitled', async () => { + queueMoveSelects(organizationWorkspace) + /** + * Not `...Once`: the resolver runs twice, once before the transaction for + * preflight reporting and again under the locks as the fence. A `...Once` + * here is consumed by the pre-transaction call, leaving the fence on the + * default mock and making this assert nothing. + * + * The fence must key off `capabilitiesLost`, never off Enterprise being + * present on both sides or a `subscription` row existing. Deployment + * configuration grants entitlement with no rows at all (see + * `resolveMoveEntitlements` and its own suite), so a fence that read + * either signal directly would reject EVERY organization-to-organization + * move in those modes. + */ + resolveMoveEntitlements.mockResolvedValue({ + sourceIsEnterprise: true, + destinationIsEnterprise: true, + capabilitiesLost: [], + }) + + await moveWorkspaceToOrganization({ + workspaceId: organizationWorkspace.id, + destinationOrganizationId: destination.id, + adminEmail: 'admin@sim.ai', + durableOperationId: 'operation-1', + }) + + expect(changeWorkspaceStoragePayerInTx).toHaveBeenCalledTimes(1) + }) + + it('refuses an organization source without a durable operation id', async () => { + queueMoveSelects(organizationWorkspace) + + /** + * The source organization's audit is written after commit and cannot be + * reconstructed once the workspace has left, so the durable payload is the + * only place its id survives a crash. Unreachable in production (both + * non-durable callers select through `ownedAttachableWorkspacesWhere`, + * which requires a null `organizationId`), and pinned here so a new caller + * that forgets the id fails loudly instead of losing the record. + */ + await expect( + moveWorkspaceToOrganization({ + workspaceId: organizationWorkspace.id, + destinationOrganizationId: destination.id, + adminEmail: 'admin@sim.ai', + }) + ).rejects.toThrow(/without a durable operation id/) + + expect(changeWorkspaceStoragePayerInTx).not.toHaveBeenCalled() + }) + + it('refuses an entitlement downgrade without mutating anything', async () => { + queueMoveSelects(organizationWorkspace) + /** + * Not `...Once`: the resolver runs twice — once before the transaction for + * preflight reporting, and again under the locks as the fence. + */ + resolveMoveEntitlements.mockResolvedValue({ + sourceIsEnterprise: true, + destinationIsEnterprise: false, + capabilitiesLost: ['permission groups', 'workspace forking'], + }) + + await expect( + moveWorkspaceToOrganization({ + workspaceId: organizationWorkspace.id, + destinationOrganizationId: destination.id, + adminEmail: 'admin@sim.ai', + durableOperationId: 'operation-1', + }) + ).rejects.toMatchObject>({ + code: 'destination-entitlement-downgrade', + }) + + expect(changeWorkspaceStoragePayerInTx).not.toHaveBeenCalled() + }) + + it('leaves the personal source path untouched', async () => { + queueMoveSelects(personalWorkspace) + + await moveWorkspaceToOrganization({ + workspaceId: personalWorkspace.id, + destinationOrganizationId: destination.id, + adminEmail: 'admin@sim.ai', + }) + + expect(acquireOrganizationMutationLock.mock.calls.map((call) => call[1])).toEqual([ + destination.id, + ]) + expect(deleteCustomBlock).not.toHaveBeenCalled() + expect(cleanupSourceOrganizationArtifactsTx).not.toHaveBeenCalled() + expect(changeWorkspaceStoragePayerInTx).toHaveBeenCalledWith( + expect.anything(), + expect.objectContaining({ + expectedCurrentPayer: { + organizationId: null, + billedAccountUserId: personalWorkspace.billedAccountUserId, + }, + }) + ) + }) + it('rejects a stale batch selection when workspace ownership changed', async () => { queueMoveSelects({ ...personalWorkspace, ownerId: 'new-owner' }) diff --git a/apps/sim/lib/workspaces/admin-move.ts b/apps/sim/lib/workspaces/admin-move.ts index 401fb6ea787..2a662d07d2e 100644 --- a/apps/sim/lib/workspaces/admin-move.ts +++ b/apps/sim/lib/workspaces/admin-move.ts @@ -16,21 +16,8 @@ import { PERMISSION_RANK, type PermissionType } from '@sim/platform-authz/worksp import { getPostgresConstraintName, getPostgresErrorCode } from '@sim/utils/errors' import { generateId } from '@sim/utils/id' import { normalizeEmail } from '@sim/utils/string' -import { - and, - asc, - count, - eq, - gt, - ilike, - inArray, - isNotNull, - isNull, - lte, - ne, - or, - sql, -} from 'drizzle-orm' +import { and, asc, count, eq, gt, ilike, inArray, isNotNull, lte, ne, or, sql } from 'drizzle-orm' +import { alias } from 'drizzle-orm/pg-core' import { acquireOrganizationMutationLock } from '@/lib/billing/organizations/membership' import { changeWorkspaceStoragePayerInTx } from '@/lib/billing/storage/payer-transfer' import { @@ -54,6 +41,21 @@ import { getInvitationById, isInvitationExpired } from '@/lib/invitations/core' import { acquireInvitationMutationLocks } from '@/lib/invitations/locks' import { PENDING_INVITATION_UNIQUE_INDEX, sendInvitationEmail } from '@/lib/invitations/send' import { invalidateWorkspaceTableLimitsCache } from '@/lib/table/billing' +import { deleteCustomBlock } from '@/lib/workflows/custom-blocks/operations' +import { + type CrossOrgForkEdge, + cleanupSourceOrganizationArtifactsTx, + collectWorkspaceCredentialSummary, + countRetentionRulesForWorkspace, + findAttachedPermissionGroups, + findCrossOrgForkEdges, + findRetainedCollaboratorCaps, + findSourceOrgCustomBlocksForWorkspace, + findUnpublishableCustomBlocks, + getSourceOrganization, + resolveMoveEntitlements, + willBrandingChange, +} from '@/lib/workspaces/admin-move-source-impact' import { mergeInvitationMembershipIntent, mergeInvitationRole, @@ -62,6 +64,31 @@ import { import { WORKSPACE_MODE } from '@/lib/workspaces/policy' const logger = createLogger('AdminWorkspaceMove') + +/** Second `member` alias so one query can test membership of both organizations. */ +const sourceMember = alias(member, 'source_member') + +/** + * Moving a workspace between organizations is the only operation in the product + * capable of separating an artifact from the organization that owns it, so two + * invariants that nothing else has ever had to defend are enforced here. + * + * **A custom block and its bound workflow always share an organization.** + * `publishCustomBlock` refuses a workflow outside the target org, so the pair + * has always been co-located. `getCustomBlockAuthority` resolves by the + * *consumer's* org and `admitCustomBlockChildExecution` deliberately skips its + * concurrency reservation because "the consumer and source workspaces are always + * in the same organization" — a stranded row would run a foreign tenant's + * workflow under its owner's credentials, billed to the wrong payer. The move + * therefore unpublishes every source-org block bound to the moving workspace. + * + * **A fork parent and child always share an organization.** `assertCanFork` + * pins the child to the source's org, and `resolveForkEdge` has no org check at + * all. The move refuses to run while a cross-org edge would result; the fork + * must be disconnected first. + * + * Neither invariant tolerates a transitional or "inert" violation. + */ /** * A dashboard member add may move several grants from one invitation in * consecutive short transactions. Let that split/merge sequence settle before @@ -83,6 +110,11 @@ export class WorkspaceMoveError extends Error { | 'already-organization-workspace' | 'seat-capacity-exceeded' | 'invitation-volume-exceeded' + | 'source-equals-destination' + | 'move-operation-parameter-mismatch' + | 'destination-entitlement-downgrade' + | 'fork-lineage-conflict' + | 'pending-invitations-present' ) { super(message) this.name = 'WorkspaceMoveError' @@ -97,13 +129,104 @@ export interface WorkspaceMoveCandidate { ownerEmail: string workspaceMode: string organizationId: string | null + /** Name of the organization that currently owns the workspace, if any. */ + organizationName: string | null billedAccountUserId: string /** Archived workspaces are movable; surfaced so admin UIs can label them. */ archived: boolean + /** Non-null when the workspace cannot be moved, explaining why. */ + ineligibleReason?: string | null +} + +/** + * The organization a workspace is moving out of. Unlike a destination, an + * ownerless source must not block the move — moving out of it is the fix — so + * every owner field is nullable. + */ +export interface WorkspaceMoveSourceOrganization { + id: string + name: string + ownerId: string | null + ownerName: string | null + ownerEmail: string | null +} + +/** + * Everything the source organization loses or has cleaned up by the move, so an + * admin can review the damage before confirming. + */ +export interface WorkspaceMoveSourceImpact { + /** + * Source-org custom blocks bound to the moving workspace's workflows. These + * are unpublished by the move — see the cross-org invariant in the module + * header. Usage is split because the two halves mean different things: + * placements inside the moving workspace leave with it, while placements + * elsewhere in the source org are collateral that stays behind and breaks. + */ + unpublishedCustomBlocks: Array<{ + id: string + type: string + name: string + movingWorkspaceUsage: { live: number; deployed: number } + sourceOrgElsewhereUsage: { live: number; deployed: number } + }> + /** Fork edges crossing the org boundary. Non-empty blocks the move. */ + blockingForkEdges: Array<{ + workspaceId: string + name: string + organizationId: string | null + direction: 'parent' | 'child' + }> + detachedPermissionGroups: Array<{ permissionGroupId: string; name: string }> + strippedRetentionRules: { piiRedactionRules: number; retentionOverrides: number } + /** Retained collaborators whose source-org per-member cap stops applying. */ + retainedCollaboratorCaps: Array<{ + userId: string + email: string + sourceOrgLimitDollars: number | null + }> + /** The workspace visibly re-skins when the two orgs' whitelabel settings differ. */ + brandingChanges: boolean + /** Rows omitted to stay inside the contract's array bounds, or `null`. */ + truncated: { + customBlocks: number + permissionGroups: number + collaboratorCaps: number + forkEdges: number + credentials: number + environmentVariableKeys: number + } | null +} + +/** Workspace secrets that travel with the move. Never carries secret material. */ +export interface WorkspaceMoveCredentialSummary { + items: Array<{ + id: string + displayName: string + type: string + /** Backed by a source-org member's identity, so the destination inherits their access. */ + backedBySourceOrgMember: boolean + }> + credentialGroupCount: number + /** Variable names only — values are never read. */ + environmentVariableKeys: string[] + byokKeyCount: number + /** Rows omitted to stay within response limits. */ + truncatedCredentials: number + truncatedEnvironmentVariableKeys: number +} + +export interface WorkspaceMoveEntitlements { + sourceIsEnterprise: boolean + destinationIsEnterprise: boolean + /** Non-empty when the destination cannot carry the source's entitlements. */ + capabilitiesLost: string[] } export interface WorkspaceMovePreflight { workspace: WorkspaceMoveCandidate + /** `null` for a personal or grandfathered source. */ + sourceOrganization: WorkspaceMoveSourceOrganization | null destinationOrganization: { id: string name: string @@ -117,6 +240,7 @@ export interface WorkspaceMovePreflight { email: string permission: 'admin' | 'write' | 'read' organizationMember: boolean + sourceOrganizationMember: boolean }> invitations: Array<{ id: string @@ -125,6 +249,13 @@ export interface WorkspaceMovePreflight { permission: 'admin' | 'write' | 'read' workspaceGrantCount: number }> + sourceOrganizationImpact: WorkspaceMoveSourceImpact + credentials: WorkspaceMoveCredentialSummary + entitlements: WorkspaceMoveEntitlements + /** Non-empty means the move will throw; the UI must not offer a confirm. */ + blockers: string[] + /** Advisory consequences the admin should read but which never block. */ + notices: string[] warning: string | null } @@ -168,6 +299,12 @@ interface WorkspaceMoveDestination { interface MoveTransactionResult { performedMove: boolean + /** What the source organization lost, for its own audit entry. */ + sourceOrganizationOutcome: { + sourceOrganizationId: string + unpublishedCustomBlocks: Array<{ id: string; type: string; name: string }> + detachedPermissionGroupIds: string[] + } | null previousBillingOwnerId: string destinationOwnerId: string organizationAssignedAt: Date | null @@ -192,6 +329,22 @@ interface AdminWorkspaceMoveOperationPayload { previousBillingOwnerId: string newBillingOwnerId: string organizationAssignedAt: string + /** + * The organization the workspace came from. Persisted because the payer + * transfer overwrites `workspace.organizationId`, so a reload of a + * completed operation cannot recover it from the row — and the admin UI + * reloads exactly that way after a lost response. + * Optional: operations recorded before this field existed have no value. + */ + sourceOrganizationId?: string | null + /** + * Persisted so the reload path can replay the source organization's loss + * audit. That write is fire-and-forget after commit, so a crash in between + * would otherwise leave the organization that lost the workspace with no + * record and no way to reconstruct one. + */ + unpublishedCustomBlocks?: Array<{ id: string; type: string; name: string }> + detachedPermissionGroupIds?: string[] } } @@ -224,7 +377,10 @@ function parseAdminWorkspaceMoveOperationPayload( Array.isArray(actor) || typeof auditRecord.previousBillingOwnerId !== 'string' || typeof auditRecord.newBillingOwnerId !== 'string' || - typeof auditRecord.organizationAssignedAt !== 'string' + typeof auditRecord.organizationAssignedAt !== 'string' || + (auditRecord.sourceOrganizationId !== undefined && + auditRecord.sourceOrganizationId !== null && + typeof auditRecord.sourceOrganizationId !== 'string') ) { return null } @@ -251,10 +407,79 @@ function parseAdminWorkspaceMoveOperationPayload( previousBillingOwnerId: auditRecord.previousBillingOwnerId, newBillingOwnerId: auditRecord.newBillingOwnerId, organizationAssignedAt: auditRecord.organizationAssignedAt, + /** + * Deliberately NOT collapsed to `null`. A recorded `null` is an answer — + * the workspace came from a personal source, so there is no organization + * to name and nothing was lost. Only an absent key leaves the origin + * unknown, and merging the two made every reload of a personal-source + * move report that its source organization had failed to persist. + */ + sourceOrganizationId: auditRecord.sourceOrganizationId as string | null | undefined, + unpublishedCustomBlocks: + (auditRecord.unpublishedCustomBlocks as + | Array<{ id: string; type: string; name: string }> + | undefined) ?? [], + detachedPermissionGroupIds: (auditRecord.detachedPermissionGroupIds as string[]) ?? [], }, } } +/** + * Where a completed move came from, rebuilt from its durable payload alone. + * The payer transfer has already overwritten `workspace.organizationId` by the + * time any of these paths run, so the payload is the only surviving record. + * + * `unknown` is true only when the payload genuinely cannot answer: it predates + * {@link AdminWorkspaceMoveOperationPayload.audit.sourceOrganizationId}, or it + * names an organization that has since been deleted. A personal source is a + * recorded answer, not a gap, and must not be reported as one. + */ +async function resolveRecordedSourceOrganization( + audit: AdminWorkspaceMoveOperationPayload['audit'] | null, + executor: DbOrTx +): Promise<{ + id: string | null + organization: WorkspaceMoveSourceOrganization | null + /** False only for a payload written before the field existed. */ + recorded: boolean + unknown: boolean +}> { + const recorded = audit ? audit.sourceOrganizationId !== undefined : false + const id = audit?.sourceOrganizationId ?? null + const organization = id ? await getSourceOrganization(id, executor) : null + return { + id, + organization, + recorded, + unknown: !recorded || (id !== null && organization === null), + } +} + +/** + * The truncation record for an applied summary, merging what the move's own + * lists dropped with what the credential summary dropped — the same merge + * preflight performs, so a partial applied review is never presented as a + * complete one. + */ +function buildAppliedTruncation(params: { + unpublishedCustomBlocks: number + detachedPermissionGroups: number + credentials: WorkspaceMoveCredentialSummary +}): WorkspaceMoveSourceImpact['truncated'] { + const truncated = { + customBlocks: Math.max(params.unpublishedCustomBlocks - PREFLIGHT_LIST_LIMITS.customBlocks, 0), + permissionGroups: Math.max( + params.detachedPermissionGroups - PREFLIGHT_LIST_LIMITS.permissionGroups, + 0 + ), + collaboratorCaps: 0, + forkEdges: 0, + credentials: params.credentials.truncatedCredentials, + environmentVariableKeys: params.credentials.truncatedEnvironmentVariableKeys, + } + return Object.values(truncated).some((dropped) => dropped > 0) ? truncated : null +} + function workspaceMoveOperationMatches( payload: unknown, params: AdminWorkspaceMoveOperationRequest @@ -267,6 +492,18 @@ function workspaceMoveOperationMatches( ) } +/** + * The workspace changed organizations between the optimistic pre-transaction + * read and the locked read, so the wrong organization was locked. Handled by + * the same retry loop as {@link InvitationSetChangedError}. + */ +class SourceOrganizationChangedError extends Error { + constructor(readonly organizationId: string | null) { + super('Workspace organization changed while acquiring workspace move locks') + this.name = 'SourceOrganizationChangedError' + } +} + class InvitationSetChangedError extends Error { constructor(readonly invitationIds: string[]) { super('Pending invitation set changed while acquiring workspace move locks') @@ -281,7 +518,11 @@ function isConcurrentPendingInvitationInsert(error: unknown): boolean { ) } -/** Returns movable personal/grandfathered workspaces by case-insensitive name or exact UUID. */ +/** + * Returns movable workspaces by case-insensitive name or exact UUID, including + * organization-owned ones. `organizationName` is joined so an admin can see + * which organization a candidate would be taken *from* before selecting it. + */ export async function searchWorkspaceMoveCandidates(search: string, limit = 20, offset = 0) { const query = search.trim() if (!query) { @@ -297,19 +538,15 @@ export async function searchWorkspaceMoveCandidates(search: string, limit = 20, ownerEmail: user.email, workspaceMode: workspace.workspaceMode, organizationId: workspace.organizationId, + organizationName: organization.name, billedAccountUserId: workspace.billedAccountUserId, archivedAt: workspace.archivedAt, total: sql`count(*) over()`.mapWith(Number), }) .from(workspace) .innerJoin(user, eq(user.id, workspace.ownerId)) - .where( - and( - ne(workspace.workspaceMode, WORKSPACE_MODE.ORGANIZATION), - isNull(workspace.organizationId), - or(eq(workspace.id, query), ilike(workspace.name, `%${query}%`)) - ) - ) + .leftJoin(organization, eq(organization.id, workspace.organizationId)) + .where(and(or(eq(workspace.id, query), ilike(workspace.name, `%${query}%`)), undefined)) .orderBy(asc(workspace.name)) .limit(Math.min(Math.max(limit, 1), 50)) .offset(Math.max(offset, 0)) @@ -317,9 +554,16 @@ export async function searchWorkspaceMoveCandidates(search: string, limit = 20, const boundedLimit = Math.min(Math.max(limit, 1), 50) const total = rows[0]?.total ?? 0 return { + /** + * Ineligible rows are returned, not hidden. A support admin searching for a + * workspace by name needs to learn that it exists and why it cannot move — + * an empty result is indistinguishable from "no such workspace" and leaves + * them with no next step. + */ data: rows.map(({ archivedAt, total: _total, ...row }) => ({ ...row, archived: archivedAt !== null, + ineligibleReason: describeWorkspaceMoveIneligibility(row), })), pagination: { total, @@ -342,6 +586,14 @@ export async function getWorkspaceMovePreflight( } assertWorkspaceMovable(workspaceRow) + const sourceOrganizationId = workspaceRow.organizationId + if (sourceOrganizationId === destinationOrganizationId) { + throw new WorkspaceMoveError( + 'Workspace already belongs to this organization', + 'source-equals-destination' + ) + } + const destination = await getDestinationOrganization(destinationOrganizationId) if (!destination) { throw new WorkspaceMoveError('Destination organization not found', 'organization-not-found') @@ -355,6 +607,7 @@ export async function getWorkspaceMovePreflight( email: user.email, permission: permissions.permissionType, memberId: member.id, + sourceMemberId: sourceMember.id, }) .from(permissions) .innerJoin(user, eq(user.id, permissions.userId)) @@ -365,6 +618,13 @@ export async function getWorkspaceMovePreflight( eq(member.organizationId, destinationOrganizationId) ) ) + .leftJoin( + sourceMember, + and( + eq(sourceMember.userId, permissions.userId), + sourceOrganizationId ? eq(sourceMember.organizationId, sourceOrganizationId) : sql`false` + ) + ) .where(and(eq(permissions.entityType, 'workspace'), eq(permissions.entityId, workspaceId))) .orderBy(asc(user.email)), getPendingInvitationSummaries(workspaceId), @@ -409,8 +669,54 @@ export async function getWorkspaceMovePreflight( ? `This move is blocked: ${currentMembers} current member${currentMembers === 1 ? '' : 's'} plus ${projectedPendingInternalSeats} pending internal invitation reservation${projectedPendingInternalSeats === 1 ? '' : 's'} exceed the ${seatCapacity}-seat Enterprise capacity.` : null + const [sourceOrganization, entitlements, credentials, forkEdges, sourceImpact] = + await Promise.all([ + sourceOrganizationId ? getSourceOrganization(sourceOrganizationId) : null, + resolveMoveEntitlements(sourceOrganizationId, destinationOrganizationId), + collectWorkspaceCredentialSummary(workspaceId, sourceOrganizationId), + findCrossOrgForkEdges(workspaceId, destinationOrganizationId), + collectSourceOrganizationImpact(workspaceId, sourceOrganizationId, destinationOrganizationId), + ]) + + const boundedForkEdges = boundList(forkEdges, PREFLIGHT_LIST_LIMITS.forkEdges) + /** + * One truncation record covering every bounded list, so a partial review is + * never presented as a complete one. + */ + const droppedTotal = + (sourceImpact.truncated?.customBlocks ?? 0) + + (sourceImpact.truncated?.permissionGroups ?? 0) + + (sourceImpact.truncated?.collaboratorCaps ?? 0) + + boundedForkEdges.dropped + + credentials.truncatedCredentials + + credentials.truncatedEnvironmentVariableKeys + const mergedTruncation = + droppedTotal > 0 + ? { + customBlocks: sourceImpact.truncated?.customBlocks ?? 0, + permissionGroups: sourceImpact.truncated?.permissionGroups ?? 0, + collaboratorCaps: sourceImpact.truncated?.collaboratorCaps ?? 0, + forkEdges: boundedForkEdges.dropped, + credentials: credentials.truncatedCredentials, + environmentVariableKeys: credentials.truncatedEnvironmentVariableKeys, + } + : null + + const blockers = buildMoveBlockers({ + entitlements, + forkEdges, + pendingInvitationCount: sourceOrganizationId ? invitationRows.length : 0, + /** + * Seat capacity throws `seat-capacity-exceeded` in the transaction, so it + * belongs in `blockers` too. Leaving it only in `warning` let a client + * keying on the new field offer a confirmation guaranteed to fail. + */ + seatCapacityWarning: warning, + }) + return { workspace: workspaceRow, + sourceOrganization, destinationOrganization: destination, collaborators: collaboratorRows.map((row) => ({ userId: row.userId, @@ -418,12 +724,208 @@ export async function getWorkspaceMovePreflight( email: row.email, permission: row.permission, organizationMember: row.memberId !== null, + sourceOrganizationMember: row.sourceMemberId !== null, })), invitations: invitationRows.map(({ organizationId: _organizationId, ...row }) => row), + sourceOrganizationImpact: { + ...sourceImpact, + blockingForkEdges: boundedForkEdges.items, + truncated: mergedTruncation, + }, + credentials, + entitlements, + blockers, + notices: buildMoveNotices({ + sourceOrganization, + destinationOrganization: destination, + sourceImpact: { ...sourceImpact, truncated: mergedTruncation }, + credentials, + }), warning, } } +/** + * The conditions that make a move refuse outright, in the order an admin should + * resolve them. Each is re-checked inside the move transaction — this list is + * for presentation, never for authorization. + */ +function buildMoveBlockers(params: { + entitlements: WorkspaceMoveEntitlements + forkEdges: CrossOrgForkEdge[] + pendingInvitationCount: number + seatCapacityWarning: string | null +}): string[] { + const blockers: string[] = [] + if (params.seatCapacityWarning) { + blockers.push(params.seatCapacityWarning) + } + if (params.entitlements.capabilitiesLost.length > 0) { + blockers.push( + `The destination organization is not on Enterprise, so this workspace would lose ${formatList(params.entitlements.capabilitiesLost)}. Upgrade the destination or choose another organization.` + ) + } + if (params.forkEdges.length > 0) { + blockers.push( + `${params.forkEdges.length} fork ${params.forkEdges.length === 1 ? 'edge' : 'edges'} would span two organizations. Disconnect ${params.forkEdges.length === 1 ? 'it' : 'them'} from workspace settings before moving.` + ) + } + if (params.pendingInvitationCount > 0) { + blockers.push( + `${params.pendingInvitationCount} pending invitation${params.pendingInvitationCount === 1 ? '' : 's'} would be re-targeted at another organization. Let ${params.pendingInvitationCount === 1 ? 'it' : 'them'} be accepted or cancel ${params.pendingInvitationCount === 1 ? 'it' : 'them'} first.` + ) + } + return blockers +} + +/** Advisory consequences worth reading before confirming, but never blocking. */ +function buildMoveNotices(params: { + sourceOrganization: WorkspaceMoveSourceOrganization | null + destinationOrganization: WorkspaceMoveDestination + sourceImpact: Omit + credentials: WorkspaceMoveCredentialSummary +}): string[] { + const notices: string[] = [] + /** + * Truncation is reported before the source-organization early return: fork + * edges and credentials can be truncated on a personal source too, and a + * partial review must never present as a complete one. + */ + if (params.sourceImpact.truncated) { + const t = params.sourceImpact.truncated + notices.push( + `This review is incomplete — some lists were truncated to stay within response limits: ${t.customBlocks} custom block(s), ${t.permissionGroups} permission group(s), ${t.collaboratorCaps} collaborator cap(s), ${t.forkEdges} fork edge(s), ${t.credentials} credential(s) and ${t.environmentVariableKeys} environment variable(s) not shown.` + ) + } + if (!params.sourceOrganization) return notices + + notices.push( + `${params.destinationOrganization.name} gains this workspace's entire audit history, and ${params.sourceOrganization.name} loses visibility of it. Organization-scoped data drains follow the same boundary.` + ) + if (params.sourceImpact.unpublishedCustomBlocks.length > 0) { + const strandedDeployments = params.sourceImpact.unpublishedCustomBlocks.reduce( + (total, block) => total + block.sourceOrgElsewhereUsage.deployed, + 0 + ) + notices.push( + `${params.sourceImpact.unpublishedCustomBlocks.length} custom block${params.sourceImpact.unpublishedCustomBlocks.length === 1 ? '' : 's'} will be unpublished from ${params.sourceOrganization.name}${strandedDeployments > 0 ? `, breaking ${strandedDeployments} deployed workflow${strandedDeployments === 1 ? '' : 's'} that stay behind` : ''}.` + ) + } + const sourceBackedCredentials = params.credentials.items.filter( + (item) => item.backedBySourceOrgMember + ).length + if (sourceBackedCredentials > 0) { + notices.push( + `${sourceBackedCredentials} credential${sourceBackedCredentials === 1 ? '' : 's'} are backed by a ${params.sourceOrganization.name} member's identity, so ${params.destinationOrganization.name} inherits the ability to act as them.` + ) + } + const cappedCollaborators = params.sourceImpact.retainedCollaboratorCaps.filter( + (collaborator) => collaborator.sourceOrgLimitDollars !== null + ).length + if (cappedCollaborators > 0) { + notices.push( + `${cappedCollaborators} retained collaborator${cappedCollaborators === 1 ? '' : 's'} had a per-member usage cap in ${params.sourceOrganization.name} that will no longer apply. Re-apply it in ${params.destinationOrganization.name} if it should continue.` + ) + } + if (params.sourceImpact.brandingChanges) { + notices.push( + `The workspace will re-skin to ${params.destinationOrganization.name}'s branding immediately.` + ) + } + return notices +} + +/** + * Ceilings that keep a preflight response inside its contract's array bounds. + * A workspace with more rows than these is pathological, but silently emitting + * an oversized list makes `requestJson` reject the whole response on the + * client — the review surface would go blank rather than degrade. Truncate and + * say so instead; never drop rows without a notice. + */ +const PREFLIGHT_LIST_LIMITS = { + forkEdges: 500, + customBlocks: 500, + permissionGroups: 500, + collaboratorCaps: 1_000, + credentials: 1_000, + environmentVariableKeys: 1_000, +} as const + +/** Truncates to `limit`, returning what was dropped so callers can disclose it. */ +function boundList(items: T[], limit: number): { items: T[]; dropped: number } { + return items.length <= limit + ? { items, dropped: 0 } + : { items: items.slice(0, limit), dropped: items.length - limit } +} + +/** Renders a list as `a, b and c` for human-facing blocker copy. */ +function formatList(items: string[]): string { + if (items.length <= 1) return items[0] ?? '' + return `${items.slice(0, -1).join(', ')} and ${items[items.length - 1]}` +} + +/** + * Gathers everything the source organization loses, excluding fork edges, which + * the caller resolves separately because they also drive a blocker. + */ +async function collectSourceOrganizationImpact( + workspaceId: string, + sourceOrganizationId: string | null, + destinationOrganizationId: string +): Promise> { + if (!sourceOrganizationId) { + return { + unpublishedCustomBlocks: [], + detachedPermissionGroups: [], + strippedRetentionRules: { piiRedactionRules: 0, retentionOverrides: 0 }, + retainedCollaboratorCaps: [], + brandingChanges: false, + truncated: null, + } + } + + const [customBlocks, permissionGroups, retentionSettings, collaboratorCaps, brandingChanges] = + await Promise.all([ + findUnpublishableCustomBlocks(workspaceId, sourceOrganizationId), + findAttachedPermissionGroups(workspaceId), + db + .select({ dataRetentionSettings: organization.dataRetentionSettings }) + .from(organization) + .where(eq(organization.id, sourceOrganizationId)) + .limit(1), + findRetainedCollaboratorCaps(workspaceId, sourceOrganizationId), + willBrandingChange(sourceOrganizationId, destinationOrganizationId), + ]) + + const boundedBlocks = boundList(customBlocks.items, PREFLIGHT_LIST_LIMITS.customBlocks) + /** Enrichment already capped the slice, so the gap comes from the true total. */ + const droppedBlocks = Math.max(customBlocks.total - boundedBlocks.items.length, 0) + const boundedGroups = boundList(permissionGroups, PREFLIGHT_LIST_LIMITS.permissionGroups) + const boundedCaps = boundList(collaboratorCaps, PREFLIGHT_LIST_LIMITS.collaboratorCaps) + + return { + unpublishedCustomBlocks: boundedBlocks.items, + detachedPermissionGroups: boundedGroups.items, + strippedRetentionRules: countRetentionRulesForWorkspace( + retentionSettings[0]?.dataRetentionSettings, + workspaceId + ), + retainedCollaboratorCaps: boundedCaps.items, + brandingChanges, + truncated: + droppedBlocks + boundedGroups.dropped + boundedCaps.dropped > 0 + ? { + customBlocks: droppedBlocks, + permissionGroups: boundedGroups.dropped, + collaboratorCaps: boundedCaps.dropped, + forkEdges: 0, + credentials: 0, + environmentVariableKeys: 0, + } + : null, + } +} + /** * Moves one workspace and migrates every pending grant. Workspace ownership, * historical usage, credentials, and collaborator permissions are preserved; @@ -446,6 +948,22 @@ export async function moveWorkspaceToOrganization(params: { params.workspaceId, params.destinationOrganizationId ) + /** + * The source organization must be locked alongside the destination, but its + * id is only knowable by reading the workspace — which happens *after* the + * locks. Read it optimistically here, then re-verify under the locks and + * retry through the existing loop when it moved underneath us. + */ + let candidateSourceOrganizationId = await readWorkspaceOrganizationId(params.workspaceId) + /** + * Resolved outside the transaction on purpose — see the entitlement check + * inside it for why. Recomputed per attempt so a retry after a source-org + * change re-evaluates against the organization actually being left. + */ + let entitlements = await resolveMoveEntitlements( + candidateSourceOrganizationId, + params.destinationOrganizationId + ) let result: MoveTransactionResult | undefined for (let attempt = 0; attempt < 5; attempt += 1) { @@ -459,7 +977,20 @@ export async function moveWorkspaceToOrganization(params: { invitationIds: candidateInvitationIds, workspaceIds: [params.workspaceId], }) - await acquireOrganizationMutationLock(tx, params.destinationOrganizationId) + /** + * Both organizations are mutated, so both are locked — ascending by id, + * mirroring `acquireOrganizationUserMutationLocks`, so two concurrent + * moves swapping a workspace between the same pair cannot deadlock. + */ + for (const organizationId of [ + ...new Set( + [candidateSourceOrganizationId, params.destinationOrganizationId].filter( + (id): id is string => id !== null + ) + ), + ].sort()) { + await acquireOrganizationMutationLock(tx, organizationId) + } const durableOperationRequest: AdminWorkspaceMoveOperationRequest = { workspaceId: params.workspaceId, @@ -531,6 +1062,12 @@ export async function moveWorkspaceToOrganization(params: { 'workspace-owner-changed' ) } + if (workspaceRow.organizationId !== candidateSourceOrganizationId) { + throw new SourceOrganizationChangedError(workspaceRow.organizationId) + } + const sourceOrganizationId = workspaceRow.organizationId + /** Set by the in-transaction fence; the summary reports this, not the optimistic read. */ + let fencedEntitlements: WorkspaceMoveEntitlements | undefined const moveState = classifyWorkspaceMoveState(workspaceRow, params.destinationOrganizationId) const destination = await getDestinationOrganization(params.destinationOrganizationId, tx) @@ -548,17 +1085,64 @@ export async function moveWorkspaceToOrganization(params: { 'already-organization-workspace' ) } + const recordedAudit = existingDurableOperation + ? (parseAdminWorkspaceMoveOperationPayload(existingDurableOperation.payload)?.audit ?? + null) + : null + /** + * A retry of a confirmed operation must return what the original move + * did, not a blank. The durable payload persists the source + * organization and its losses precisely so this branch can rebuild + * them — discarding it here made the retry claim the source was + * unrecoverable while the payload was sitting right there. + */ + const recordedSource = await resolveRecordedSourceOrganization(recordedAudit, tx) + /** + * The workspace's own secrets are untouched by a move and by this + * no-op retry, so they are read rather than blanked: a retry that + * reported zero credentials told the admin the workspace had none. + */ + const replayedCredentials = await collectWorkspaceCredentialSummary( + params.workspaceId, + recordedSource.id, + tx + ) return { performedMove: false, + sourceOrganizationOutcome: recordedSource.id + ? { + sourceOrganizationId: recordedSource.id, + unpublishedCustomBlocks: recordedAudit?.unpublishedCustomBlocks ?? [], + detachedPermissionGroupIds: recordedAudit?.detachedPermissionGroupIds ?? [], + } + : null, previousBillingOwnerId: workspaceRow.billedAccountUserId, destinationOwnerId: destination.ownerId, organizationAssignedAt: null, - durableAudit: existingDurableOperation - ? (parseAdminWorkspaceMoveOperationPayload(existingDurableOperation.payload)?.audit ?? - null) - : null, + durableAudit: recordedAudit, invitationEvents: [], - summary: await getMovedWorkspaceSummary(tx, params.workspaceId, destination), + summary: await getMovedWorkspaceSummary(tx, params.workspaceId, destination, { + sourceOrganization: recordedSource.organization, + sourceOrganizationImpact: { + ...EMPTY_SOURCE_IMPACT, + truncated: buildAppliedTruncation({ + unpublishedCustomBlocks: 0, + detachedPermissionGroups: 0, + credentials: replayedCredentials, + }), + }, + credentials: replayedCredentials, + entitlements: { + sourceIsEnterprise: false, + destinationIsEnterprise: false, + capabilitiesLost: [], + }, + notices: recordedSource.unknown + ? [ + 'This workspace was already in the destination organization, so the organization it originally came from is no longer recoverable.', + ] + : [], + }), } satisfies MoveTransactionResult } @@ -601,6 +1185,116 @@ export async function moveWorkspaceToOrganization(params: { } } + /** + * The three org-to-org blockers, re-checked under the locks. Preflight + * evaluated them too, but a subscription can lapse, a fork can be + * created, and an invitation can arrive in between — and each of these + * either violates a cross-org invariant or silently rewrites a promise + * the source organization made. + */ + /** + * The fork check is NOT gated on an organization source. A personal + * workspace whose parent has since moved into an organization still + * produces a cross-organization edge when it lands in a different one, + * and the invariant admits no exceptions. Preflight already reports it + * unconditionally; gating it here would let the transaction accept a + * move preflight had refused. + */ + const forkEdges = await findCrossOrgForkEdges( + params.workspaceId, + params.destinationOrganizationId, + tx + ) + if (forkEdges.length > 0) { + throw new WorkspaceMoveError( + `${forkEdges.length} fork ${forkEdges.length === 1 ? 'edge' : 'edges'} would span two organizations. Disconnect the fork before moving this workspace.`, + 'fork-lineage-conflict' + ) + } + + /** + * An organization source requires a durable operation, and that is a + * caller contract rather than an operator-facing outcome. + * + * The source organization's "workspace moved out" audit is written + * after commit, so a process that dies in between loses it. Every + * other post-commit record recovers on retry, but this one cannot: + * the workspace has already left, so its source organization is no + * longer readable from the row. The durable payload is the only place + * that id survives, and the `already-moved` branch above replays the + * audit from it. + * + * Refusing here instead of reconstructing later is what keeps that a + * closed question. The two non-durable callers, the member transfer + * operation and Enterprise provisioning, both select through + * `ownedAttachableWorkspacesWhere`, which requires a null + * `organizationId`, so neither can reach this. The admin route always + * supplies one, because `operationId` is required by + * `adminDashboardWorkspaceMoveBodySchema`. This throws a plain Error + * on purpose: it is unreachable by construction today, and a future + * caller that reaches it has a wiring bug, not a bad request. + */ + if (sourceOrganizationId && !params.durableOperationId) { + throw new Error( + `Refusing to move workspace ${params.workspaceId} out of organization ${sourceOrganizationId} without a durable operation id: the source organization audit would not survive a crash before it is written.` + ) + } + + if (sourceOrganizationId) { + /** + * Entitlements are resolved BEFORE the transaction, not here. + * `isOrganizationOnEnterprisePlan` reads through the global client + * with no executor seam, so calling it inside the transaction trips + * the transaction tripwire outside production and reserves a second + * pool connection in it. The check is a precondition, not an + * invariant: a plan lapsing in the seconds between the read and the + * commit lands the workspace in an organization that just lost its + * entitlements, which is recoverable by moving it back — unlike a + * cross-organization artifact, which is not. + */ + /** + * Evaluate BOTH organizations under the locks when entitlement is + * subscription-backed, rather than trusting the pre-transaction + * verdict. That verdict is still what preflight reports, but as a + * blocker it is stale in both directions: a destination that lapsed + * after it was read, and a source that GAINED entitlement after it + * was read, which would otherwise skip the fence entirely. + * + * `isSubscriptionBackedEntitlement` is exported from the same module + * as `resolveOrganizationEnterprisePlan`'s short-circuits, so the two + * modes where entitlement is granted by deployment configuration — + * and no `subscription` row need exist — cannot drift away from this. + */ + /** + * Re-run the SAME resolver under the locks, on `tx`. The + * pre-transaction verdict is stale in both directions — a destination + * that lapsed after it was read, and a source that gained Enterprise + * after it was read, which would otherwise skip the check entirely. + * Reusing `resolveMoveEntitlements` rather than re-deriving the + * predicate here is what keeps the fence and preflight from ever + * disagreeing about what counts as a downgrade. + */ + fencedEntitlements = await resolveMoveEntitlements( + sourceOrganizationId, + params.destinationOrganizationId, + tx + ) + if (fencedEntitlements.capabilitiesLost.length > 0) { + throw new WorkspaceMoveError( + `The destination organization is not on Enterprise, so this workspace would lose ${fencedEntitlements.capabilitiesLost.join(', ')}`, + 'destination-entitlement-downgrade' + ) + } + + const pendingInvitations = await getPendingInvitationSummaries(params.workspaceId, tx) + if (pendingInvitations.length > 0) { + throw new WorkspaceMoveError( + `This workspace has ${pendingInvitations.length} pending invitation${pendingInvitations.length === 1 ? '' : 's'} scoped to its current organization. Let them be accepted or cancel them before moving it.`, + 'pending-invitations-present' + ) + } + } + const now = new Date() await expireLockedPendingInvitations(tx, candidateInvitationIds, now) const lockedInvitationIds = await lockCurrentPendingInvitations(tx, params.workspaceId, now) @@ -634,6 +1328,33 @@ export async function moveWorkspaceToOrganization(params: { } } + /** + * Enforce the cross-org invariants before the payer moves, while the + * source organization is still the one on the row. Unpublishing a + * custom block is the product's own `deleteCustomBlock`; the usage + * counts are captured first so the source org's audit entry can say how + * much it cost. + */ + const sourceOrganization = sourceOrganizationId + ? await getSourceOrganization(sourceOrganizationId, tx) + : null + const unpublishedCustomBlocks = sourceOrganizationId + ? await findSourceOrgCustomBlocksForWorkspace( + params.workspaceId, + sourceOrganizationId, + tx + ) + : [] + for (const block of unpublishedCustomBlocks) { + await deleteCustomBlock(block.id, tx) + } + const cleanup = sourceOrganizationId + ? await cleanupSourceOrganizationArtifactsTx(tx, { + workspaceId: params.workspaceId, + sourceOrganizationId, + }) + : { detachedPermissionGroupIds: [] } + await changeWorkspaceStoragePayerInTx(tx, { workspaceId: params.workspaceId, organizationId: params.destinationOrganizationId, @@ -664,6 +1385,9 @@ export async function moveWorkspaceToOrganization(params: { previousBillingOwnerId: workspaceRow.billedAccountUserId, newBillingOwnerId: destination.ownerId, organizationAssignedAt: now.toISOString(), + sourceOrganizationId, + unpublishedCustomBlocks, + detachedPermissionGroupIds: cleanup.detachedPermissionGroupIds, } : null if (params.durableOperationId && durableAudit) { @@ -692,6 +1416,19 @@ export async function moveWorkspaceToOrganization(params: { set: { permissionType: 'admin', updatedAt: now }, }) + /** + * Read against the PRE-move source organization, which is what + * `backedBySourceOrgMember` means. Every row this counts is workspace- + * scoped and travels with the move untouched, so unlike the source + * impact it is still fully reportable here — blanking it told the admin + * who just confirmed the move that the workspace carried no secrets. + */ + const movedCredentials = await collectWorkspaceCredentialSummary( + params.workspaceId, + sourceOrganizationId, + tx + ) + return { performedMove: true, previousBillingOwnerId: workspaceRow.billedAccountUserId, @@ -699,7 +1436,61 @@ export async function moveWorkspaceToOrganization(params: { organizationAssignedAt: now, durableAudit, invitationEvents: migration.invitationEvents, - summary: await getMovedWorkspaceSummary(tx, params.workspaceId, destination), + sourceOrganizationOutcome: sourceOrganizationId + ? { + sourceOrganizationId, + unpublishedCustomBlocks: unpublishedCustomBlocks.map(({ id, type, name }) => ({ + id, + type, + name, + })), + detachedPermissionGroupIds: cleanup.detachedPermissionGroupIds, + } + : null, + summary: await getMovedWorkspaceSummary(tx, params.workspaceId, destination, { + sourceOrganization, + /** + * What the move actually did, not what preflight projected. The + * rest of the impact described the pre-move state and is not + * recoverable — or meaningful — once the workspace has landed. + */ + sourceOrganizationImpact: { + ...EMPTY_SOURCE_IMPACT, + /** + * Usage counts are zero here rather than measured: they describe + * how much breaks in the source organization, and the reads that + * produce them are not transaction-safe. Preflight carries the + * real numbers; this reports which blocks were unpublished. + */ + unpublishedCustomBlocks: boundList( + unpublishedCustomBlocks, + PREFLIGHT_LIST_LIMITS.customBlocks + ).items.map((block) => ({ + ...block, + movingWorkspaceUsage: { live: 0, deployed: 0 }, + sourceOrgElsewhereUsage: { live: 0, deployed: 0 }, + })), + detachedPermissionGroups: boundList( + cleanup.detachedPermissionGroupIds, + PREFLIGHT_LIST_LIMITS.permissionGroups + ).items.map((permissionGroupId) => ({ permissionGroupId, name: '' })), + /** The applied response is bounded by the same limits as preflight. */ + truncated: buildAppliedTruncation({ + unpublishedCustomBlocks: unpublishedCustomBlocks.length, + detachedPermissionGroups: cleanup.detachedPermissionGroupIds.length, + credentials: movedCredentials, + }), + }, + credentials: movedCredentials, + /** + * The fenced result, not the optimistic one: the response must + * describe the entitlement state the move was actually allowed + * against, or a destination that gained Enterprise between the two + * reads is reported as a downgrade it no longer is. + */ + entitlements: fencedEntitlements ?? entitlements, + notices: [], + }), } satisfies MoveTransactionResult }) break @@ -708,6 +1499,14 @@ export async function moveWorkspaceToOrganization(params: { candidateInvitationIds = error.invitationIds continue } + if (error instanceof SourceOrganizationChangedError) { + candidateSourceOrganizationId = error.organizationId + entitlements = await resolveMoveEntitlements( + candidateSourceOrganizationId, + params.destinationOrganizationId + ) + continue + } if (isConcurrentPendingInvitationInsert(error)) { candidateInvitationIds = await findInvitationMigrationLockIds( params.workspaceId, @@ -740,6 +1539,24 @@ export async function moveWorkspaceToOrganization(params: { recovered: true, }) } + /** + * Replay the source organization's loss audit on this path too. The write + * is fire-and-forget after commit, so the retry that reaches this branch is + * often the one recovering from a process that died before it landed. + * `recordAuditOnce` keys make it a no-op when it already did. + */ + if (result.sourceOrganizationOutcome) { + await recordSourceOrganizationMoveAudit({ + workspaceId: params.workspaceId, + sourceOrganizationId: result.sourceOrganizationOutcome.sourceOrganizationId, + destinationOrganizationId: params.destinationOrganizationId, + adminEmail: params.adminEmail, + auditActor: params.auditActor, + auditOperationId: params.auditOperationId, + unpublishedCustomBlocks: result.sourceOrganizationOutcome.unpublishedCustomBlocks, + detachedPermissionGroupIds: result.sourceOrganizationOutcome.detachedPermissionGroupIds, + }) + } logger.info('Workspace was already in destination organization', { workspaceId: params.workspaceId, destinationOrganizationId: params.destinationOrganizationId, @@ -749,6 +1566,19 @@ export async function moveWorkspaceToOrganization(params: { invalidateWorkspaceTableLimitsCache(params.workspaceId) + if (result.sourceOrganizationOutcome) { + await recordSourceOrganizationMoveAudit({ + workspaceId: params.workspaceId, + sourceOrganizationId: result.sourceOrganizationOutcome.sourceOrganizationId, + destinationOrganizationId: params.destinationOrganizationId, + adminEmail: params.adminEmail, + auditActor: params.auditActor, + auditOperationId: params.auditOperationId, + unpublishedCustomBlocks: result.sourceOrganizationOutcome.unpublishedCustomBlocks, + detachedPermissionGroupIds: result.sourceOrganizationOutcome.detachedPermissionGroupIds, + }) + } + if (params.auditOperationId && result.durableAudit) { await recordDurableWorkspaceMoveAudit( params.auditOperationId, @@ -836,6 +1666,86 @@ async function recordWorkspaceMoveAudit({ } } +/** + * Records what the source organization lost, in the source organization's own + * audit view. + * + * The workspace-scoped move entry above is visible only to the *destination* + * after the move — `buildOrgScopeCondition` scopes org audit reads by the + * organization's current workspaces — so without this the organization that + * lost the workspace has no record of it at all. `workspaceId: null` plus + * `metadata.organizationId` is that condition's org-level branch, which + * resolves to the source and nowhere else. + */ +async function recordSourceOrganizationMoveAudit(params: { + workspaceId: string + sourceOrganizationId: string + destinationOrganizationId: string + adminEmail: string + auditActor?: { id: string | null; name: string; email: string | null } + auditOperationId?: string + unpublishedCustomBlocks: Array<{ id: string; type: string; name: string }> + detachedPermissionGroupIds: string[] +}): Promise { + const actor = { + actorId: params.auditActor ? params.auditActor.id : null, + actorName: params.auditActor?.name ?? 'Admin Panel', + actorEmail: params.auditActor?.email ?? params.adminEmail, + } + + const moveOut = { + workspaceId: null, + ...actor, + action: AuditAction.ORGANIZATION_UPDATED, + resourceType: AuditResourceType.ORGANIZATION, + resourceId: params.sourceOrganizationId, + description: 'Workspace moved out of this organization', + metadata: { + organizationId: params.sourceOrganizationId, + workspaceId: params.workspaceId, + destinationOrganizationId: params.destinationOrganizationId, + unpublishedCustomBlockIds: params.unpublishedCustomBlocks.map((block) => block.id), + detachedPermissionGroupIds: params.detachedPermissionGroupIds, + }, + } as const + + if (params.auditOperationId) { + await recordAuditOnce( + `${params.auditOperationId}:workspace-move-source:${params.workspaceId}`, + moveOut + ) + } else { + recordAudit(moveOut) + } + + for (const block of params.unpublishedCustomBlocks) { + const unpublished = { + workspaceId: null, + ...actor, + action: AuditAction.CUSTOM_BLOCK_DELETED, + resourceType: AuditResourceType.CUSTOM_BLOCK, + resourceId: block.id, + resourceName: block.name, + description: `Unpublished custom block "${block.name}"`, + metadata: { + organizationId: params.sourceOrganizationId, + type: block.type, + reason: 'workspace-moved-to-another-organization', + workspaceId: params.workspaceId, + destinationOrganizationId: params.destinationOrganizationId, + }, + } as const + if (params.auditOperationId) { + await recordAuditOnce( + `${params.auditOperationId}:custom-block-unpublished:${block.id}`, + unpublished + ) + } else { + recordAudit(unpublished) + } + } +} + async function recordDurableWorkspaceMoveAudit( operationId: string, workspaceId: string, @@ -986,8 +1896,70 @@ export async function getWorkspaceMoveOperation( destinationOrganizationId, operationPayload.audit ) + /** + * Reconstruct the source organization from the durable payload. The payer + * transfer already overwrote `workspace.organizationId`, so the row cannot + * supply it — and this reload is the path the admin UI takes after a lost + * response, which is exactly when the operator most needs to see what the + * move did and where it came from. + */ + const recordedSource = await resolveRecordedSourceOrganization(operationPayload.audit, db) + + /** + * Replay the source organization's loss audit. `recordAuditOnce` keys make it + * idempotent, so this is a no-op when the original write landed and a repair + * when the process died between commit and that fire-and-forget write. + */ + if (recordedSource.id) { + await recordSourceOrganizationMoveAudit({ + workspaceId, + sourceOrganizationId: recordedSource.id, + destinationOrganizationId, + adminEmail: operationPayload.audit.actor.email ?? 'admin-api@sim.ai', + auditActor: operationPayload.audit.actor, + auditOperationId: operationId, + unpublishedCustomBlocks: operationPayload.audit.unpublishedCustomBlocks ?? [], + detachedPermissionGroupIds: operationPayload.audit.detachedPermissionGroupIds ?? [], + }) + } + + /** + * The workspace's secrets are workspace-scoped and travel with the move, so + * the reload reads them rather than reporting a blank. `backedBySourceOrgMember` + * resolves against the recorded source; a personal or unrecorded source has no + * members to match, which is exactly what a `null` id asks for. + */ + const credentials = await collectWorkspaceCredentialSummary(workspaceId, recordedSource.id) + return toWorkspaceMoveOperationView( - await getMovedWorkspaceSummary(db, workspaceId, destination), + await getMovedWorkspaceSummary(db, workspaceId, destination, { + sourceOrganization: recordedSource.organization, + sourceOrganizationImpact: { + ...EMPTY_SOURCE_IMPACT, + truncated: buildAppliedTruncation({ + unpublishedCustomBlocks: 0, + detachedPermissionGroups: 0, + credentials, + }), + }, + credentials, + entitlements: { + sourceIsEnterprise: false, + destinationIsEnterprise: false, + capabilitiesLost: [], + }, + /** + * A recorded `null` means the workspace came from a personal source and + * there is nothing to name — not that the record is defective. + */ + notices: recordedSource.unknown + ? [ + recordedSource.recorded + ? 'The organization this workspace came from has since been deleted, so it can no longer be named.' + : 'This move was recorded before the source organization was persisted, so it cannot be reported.', + ] + : [], + }), operationId ) } @@ -1074,11 +2046,13 @@ async function searchWorkspaceById(workspaceId: string): Promise, } as const +/** + * Source-org context captured BEFORE the payer transfer rewrites + * `workspace.organizationId`. Without it the post-move summary cannot name the + * organization the workspace came from, because the row no longer records it. + */ +interface AppliedMoveContext { + sourceOrganization: WorkspaceMoveSourceOrganization | null + sourceOrganizationImpact: WorkspaceMoveSourceImpact + credentials: WorkspaceMoveCredentialSummary + entitlements: WorkspaceMoveEntitlements + notices: string[] +} + async function getMovedWorkspaceSummary( executor: DbOrTx, workspaceId: string, - destination: WorkspaceMoveDestination + destination: WorkspaceMoveDestination, + appliedContext?: AppliedMoveContext ): Promise { const [movedRow] = await executor .select({ @@ -1736,11 +2739,13 @@ async function getMovedWorkspaceSummary( ownerEmail: user.email, workspaceMode: workspace.workspaceMode, organizationId: workspace.organizationId, + organizationName: organization.name, billedAccountUserId: workspace.billedAccountUserId, archivedAt: workspace.archivedAt, }) .from(workspace) .innerJoin(user, eq(user.id, workspace.ownerId)) + .leftJoin(organization, eq(organization.id, workspace.organizationId)) .where(eq(workspace.id, workspaceId)) .limit(1) if (!movedRow) { @@ -1759,6 +2764,7 @@ async function getMovedWorkspaceSummary( email: user.email, permission: permissions.permissionType, memberId: member.id, + sourceMemberId: sourceMember.id, }) .from(permissions) .innerJoin(user, eq(user.id, permissions.userId)) @@ -1766,10 +2772,20 @@ async function getMovedWorkspaceSummary( member, and(eq(member.userId, permissions.userId), eq(member.organizationId, destination.id)) ) + .leftJoin( + sourceMember, + and( + eq(sourceMember.userId, permissions.userId), + appliedContext?.sourceOrganization + ? eq(sourceMember.organizationId, appliedContext.sourceOrganization.id) + : sql`false` + ) + ) .where(and(eq(permissions.entityType, 'workspace'), eq(permissions.entityId, workspaceId))) return { workspace: workspaceRow, + sourceOrganization: appliedContext?.sourceOrganization ?? null, destinationOrganization: destination, collaborators: collaboratorRows.map((row) => ({ userId: row.userId, @@ -1777,10 +2793,54 @@ async function getMovedWorkspaceSummary( email: row.email, permission: row.permission, organizationMember: row.memberId !== null, + sourceOrganizationMember: row.sourceMemberId !== null, })), invitations: (await getPendingInvitationSummaries(workspaceId, executor)).map( ({ organizationId: _organizationId, ...row }) => row ), + sourceOrganizationImpact: appliedContext?.sourceOrganizationImpact ?? EMPTY_SOURCE_IMPACT, + credentials: appliedContext?.credentials ?? EMPTY_CREDENTIAL_SUMMARY, + entitlements: appliedContext?.entitlements ?? { + sourceIsEnterprise: false, + destinationIsEnterprise: false, + capabilitiesLost: [], + }, + blockers: [], + notices: appliedContext?.notices ?? [], warning: null, } } + +/** A move that is already applied has no source-org context to report. */ +const EMPTY_SOURCE_IMPACT: WorkspaceMoveSourceImpact = { + unpublishedCustomBlocks: [], + blockingForkEdges: [], + detachedPermissionGroups: [], + strippedRetentionRules: { piiRedactionRules: 0, retentionOverrides: 0 }, + retainedCollaboratorCaps: [], + brandingChanges: false, + truncated: null, +} + +const EMPTY_CREDENTIAL_SUMMARY: WorkspaceMoveCredentialSummary = { + items: [], + credentialGroupCount: 0, + environmentVariableKeys: [], + byokKeyCount: 0, + truncatedCredentials: 0, + truncatedEnvironmentVariableKeys: 0, +} + +/** + * The workspace's current organization, read outside the move transaction so + * both organizations can be locked in a deterministic order. Always re-verified + * under the locks — see {@link SourceOrganizationChangedError}. + */ +async function readWorkspaceOrganizationId(workspaceId: string): Promise { + const [row] = await db + .select({ organizationId: workspace.organizationId }) + .from(workspace) + .where(eq(workspace.id, workspaceId)) + .limit(1) + return row?.organizationId ?? null +} diff --git a/apps/sim/providers/cost-policy.test.ts b/apps/sim/providers/cost-policy.test.ts index 5d7ebd9869e..48ced0fffcd 100644 --- a/apps/sim/providers/cost-policy.test.ts +++ b/apps/sim/providers/cost-policy.test.ts @@ -226,6 +226,23 @@ describe('installStreamingCostPolicy', () => { expect(output.cost).toMatchObject({ input: 0, output: 0, total: 0.75, toolCost: 0.75 }) }) + it('adds late failed Function cost once without applying the model multiplier', () => { + const output = { + cost: { input: 1, output: 2, total: 3.25, toolCost: 0.25 }, + } as NormalizedBlockOutput + const failedFunctionToolCost = { total: 0 } + installStreamingCostPolicy( + output, + { billable: false, multiplier: 0 }, + () => failedFunctionToolCost.total + ) + + failedFunctionToolCost.total = 0.125 + + expect(output.cost).toMatchObject({ input: 0, output: 0, total: 0.375, toolCost: 0.375 }) + expect(output.cost).toMatchObject({ total: 0.375, toolCost: 0.375 }) + }) + it('zeroes model cost written by a provider for a model Sim does not host', () => { const output = { cost: { input: 0, output: 0, total: 0 } } as NormalizedBlockOutput installStreamingCostPolicy(output, resolveModelCostPolicy(SELF_KEYED_MODEL)) diff --git a/apps/sim/providers/cost-policy.ts b/apps/sim/providers/cost-policy.ts index 7c67d82dee8..b116a90c472 100644 --- a/apps/sim/providers/cost-policy.ts +++ b/apps/sim/providers/cost-policy.ts @@ -266,12 +266,23 @@ export function resolveProxiedModelCost(cost: unknown): ModelCost { */ export function installStreamingCostPolicy( output: NormalizedBlockOutput, - policy: ModelCostPolicy + policy: ModelCostPolicy, + additionalToolCost?: () => number ): void { let raw = output.cost as ModelCost | undefined Object.defineProperty(output, 'cost', { - get: () => applyModelCostPolicy(raw, policy), + get: () => { + const projected = applyModelCostPolicy(raw, policy) + const additional = additionalToolCost?.() ?? 0 + if (!Number.isFinite(additional) || additional <= 0) return projected + + return { + ...projected, + toolCost: roundCost((projected.toolCost ?? 0) + additional), + total: roundCost(projected.total + additional), + } + }, set: (value: ModelCost | undefined) => { raw = value }, diff --git a/apps/sim/providers/index.test.ts b/apps/sim/providers/index.test.ts index 22efa428d51..f6f962b71e5 100644 --- a/apps/sim/providers/index.test.ts +++ b/apps/sim/providers/index.test.ts @@ -278,6 +278,33 @@ describe('executeProviderRequest — BYOK regression', () => { expect(result.cost?.total).toBeCloseTo(0.00675, 8) }) + it('adds failed Function cost once alongside successful tool results', async () => { + mockGetApiKeyWithBYOK.mockResolvedValue({ apiKey: 'sk-byok', isBYOK: true }) + mockExecuteTool.mockResolvedValueOnce({ + success: false, + output: { cost: { total: 0.004 } }, + error: 'execution failed', + }) + mockExecuteRequest.mockImplementationOnce(async () => { + const execution = await executeProviderTool('function_execute', {}) + expect(execution.rawResponse.success).toBe(false) + return { + ...makeAnthropicResponse(), + toolResults: [{ cost: { total: 0.005 } }], + } as ProviderResponse + }) + + const result = (await executeProviderRequest('anthropic', { + model: 'claude-opus-4-6', + workspaceId: 'ws-1', + tools: [makeProviderTool('function_execute', 'credential')], + })) as ProviderResponse + + expect(result.cost).toMatchObject({ input: 0, output: 0 }) + expect(result.cost?.toolCost).toBeCloseTo(0.009, 8) + expect(result.cost?.total).toBeCloseTo(0.009, 8) + }) + /** * Gemini hands the same cost object to its response and its model segment. * Adding tool cost by mutation would charge it to the segment too. diff --git a/apps/sim/providers/index.ts b/apps/sim/providers/index.ts index d8ecbb04388..f4db8ee535a 100644 --- a/apps/sim/providers/index.ts +++ b/apps/sim/providers/index.ts @@ -152,14 +152,18 @@ function isReadableStream(response: any): response is ReadableStream { * stream drain — long after this function returns — so the policy is installed * on the live output object rather than applied to a value. */ -function applyStreamingCostPolicy(response: StreamingExecution, policy: ModelCostPolicy): void { +function applyStreamingCostPolicy( + response: StreamingExecution, + policy: ModelCostPolicy, + additionalToolCost?: () => number +): void { const output = response.execution?.output if (!output || typeof output !== 'object') { logger.warn('Streaming output unavailable at intercept time; cost policy not applied') return } - installStreamingCostPolicy(output, policy) + installStreamingCostPolicy(output, policy, additionalToolCost) const segments = output.providerTiming?.timeSegments if (Array.isArray(segments)) { @@ -224,16 +228,19 @@ export async function executeProviderRequest( const provenanceSafeRequest = await omitUnsafeProviderFileAttachments(sanitizedRequest) const modelSafeRequest = provenanceSafeRequest const toolIdentities = assignProviderToolIdentities(modelSafeRequest.tools) - const requestRuntimeContext = - toolIdentities.toolIdByWireId.size > 0 + const failedFunctionToolCost = { total: 0 } + const requestRuntimeContext: ProviderRuntimeContext = { + ...runtimeContext, + failedFunctionToolCost, + ...(toolIdentities.toolIdByWireId.size > 0 ? { - ...runtimeContext, toolIdByWireId: new Map([ ...(runtimeContext?.toolIdByWireId ?? []), ...toolIdentities.toolIdByWireId, ]), } - : runtimeContext + : {}), + } if (modelSafeRequest.responseFormat) { const structuredOutputInstructions = generateStructuredOutputInstructions( @@ -254,7 +261,11 @@ export async function executeProviderRequest( if (isStreamingExecution(response)) { logger.info('Provider returned StreamingExecution', { isBYOK }) - applyStreamingCostPolicy(response, resolveModelCostPolicy(sanitizedRequest.model, isBYOK)) + applyStreamingCostPolicy( + response, + resolveModelCostPolicy(sanitizedRequest.model, isBYOK), + () => failedFunctionToolCost.total + ) projectStreamingExecutionToolIdentities(response, toolIdentities) return response } @@ -300,7 +311,7 @@ export async function executeProviderRequest( applySegmentCostPolicy(response.timing.timeSegments, costPolicy) } - const toolCost = sumToolCosts(response.toolResults) + const toolCost = sumToolCosts(response.toolResults) + failedFunctionToolCost.total if (toolCost > 0 && response.cost) { // Replaced rather than mutated: a provider-supplied cost can be the same // object it also handed to a time segment, and tool cost belongs only to diff --git a/apps/sim/providers/runtime-context.test.ts b/apps/sim/providers/runtime-context.test.ts index c4761eea0b6..6a7c718b2da 100644 --- a/apps/sim/providers/runtime-context.test.ts +++ b/apps/sim/providers/runtime-context.test.ts @@ -147,6 +147,35 @@ describe('provider runtime context', () => { ) }) + it('accumulates cost only for failed canonical Function results', async () => { + const failedFunctionToolCost = { total: 0 } + const context = { + failedFunctionToolCost, + toolIdByWireId: new Map([['function_execute__sim_2', 'function_execute']]), + } + + mockExecuteTool + .mockResolvedValueOnce({ + success: false, + output: { cost: { total: 0.125 } }, + error: 'execution failed', + }) + .mockResolvedValueOnce({ success: true, output: { cost: { total: 4 } } }) + .mockResolvedValueOnce({ + success: false, + output: { cost: { total: 8 } }, + error: 'other tool failed', + }) + + await runWithProviderRuntimeContext(context, () => + executeProviderTool('function_execute__sim_2', {}) + ) + await runWithProviderRuntimeContext(context, () => executeProviderTool('function_execute', {})) + await runWithProviderRuntimeContext(context, () => executeProviderTool('exa_search', {})) + + expect(failedFunctionToolCost.total).toBe(0.125) + }) + it('rebinds a prompt-exposed environment placeholder for the exact tool call', async () => { const sourceRegistry = new ResolvedSecretTraceRegistry([ { diff --git a/apps/sim/providers/runtime-context.ts b/apps/sim/providers/runtime-context.ts index 2e602a83ba0..3cfea3e1eab 100644 --- a/apps/sim/providers/runtime-context.ts +++ b/apps/sim/providers/runtime-context.ts @@ -19,6 +19,8 @@ export interface ProviderRuntimeContext { executionContext?: ExecutionContext /** Request-scoped provider wire ids mapped back to canonical tool registry ids. */ toolIdByWireId?: ReadonlyMap + /** Failed canonical Function cost omitted from provider tool-result collections. */ + failedFunctionToolCost?: { total: number } } export type ExecuteProviderToolOptions = ExecuteToolOptions @@ -88,6 +90,20 @@ function withoutChildTraceHandle(response: ToolResponse): ToolResponse { } } +function accumulateFailedFunctionToolCost( + toolId: string, + result: ToolResponse, + accumulator: ProviderRuntimeContext['failedFunctionToolCost'] +): void { + if (toolId !== 'function_execute' || result.success || !accumulator) return + if (!isRecordLike(result.output) || !isRecordLike(result.output.cost)) return + + const total = result.output.cost.total + if (typeof total === 'number' && Number.isFinite(total) && total > 0) { + accumulator.total += total + } +} + export async function executeProviderTool( toolId: string, params: Parameters[1], @@ -120,6 +136,11 @@ export async function executeProviderTool( ...(executionContext ? { executionContext } : {}), resolvedSecretTraceRegistry: toolCallRegistry, }) + accumulateFailedFunctionToolCost( + executionToolId, + result, + runtimeContext?.failedFunctionToolCost + ) if (!registry || !toolCallRegistry) { return { rawResponse: result, modelResponse: withoutChildTraceHandle(result) } } diff --git a/apps/sim/providers/utils.ts b/apps/sim/providers/utils.ts index 017f7479a5d..d838e0ef217 100644 --- a/apps/sim/providers/utils.ts +++ b/apps/sim/providers/utils.ts @@ -816,16 +816,18 @@ export async function transformBlockTool( toolDescription = workflowMetadata.description } } - } else if (toolId === 'function_execute' && resolvedResourceParams.secretScope === 'selected') { - // Scoping alone would leave the model guessing: the secrets are injected - // server-side and nothing else advertises them. Names only — values never - // enter the provider request, matching the copilot's workspace-context rule. - // `StoredTool.params` holds strings, so a multi-select arrives JSON-encoded; - // the executor's paramsTransform parses it later, but this runs before that. - const mounted = readMountedSecretNames(resolvedResourceParams.mountedSecrets) - toolDescription = mounted.length - ? `${toolDescription}\n\nWorkspace secret names available to this code: ${mounted.join(', ')}. Reference one with the exact {{NAME}} syntax. Its value is bound only while the code executes and is not included in the model request. No other secrets are readable.` - : `${toolDescription}\n\nThis code has no access to workspace secrets.` + } else if (toolId === 'function_execute') { + if (resolvedResourceParams.secretScope === 'selected') { + // Scoping alone would leave the model guessing: the secrets are injected + // server-side and nothing else advertises them. Names only — values never + // enter the provider request, matching the copilot's workspace-context rule. + // `StoredTool.params` holds strings, so a multi-select arrives JSON-encoded; + // the executor's paramsTransform parses it later, but this runs before that. + const mounted = readMountedSecretNames(resolvedResourceParams.mountedSecrets) + toolDescription = mounted.length + ? `${toolDescription}\n\nWorkspace secret names available to this code: ${mounted.join(', ')}. Reference one with the exact {{NAME}} syntax. Its value is bound only while the code executes and is not included in the model request. No other secrets are readable.` + : `${toolDescription}\n\nThis code has no access to workspace secrets.` + } } const blockParamsFn = blockDef?.tools?.config?.params as diff --git a/apps/sim/public/library/ai-agents-for-marketing-automation/cover.jpg b/apps/sim/public/library/ai-agents-for-marketing-automation/cover.jpg new file mode 100644 index 00000000000..eece276fabb Binary files /dev/null and b/apps/sim/public/library/ai-agents-for-marketing-automation/cover.jpg differ diff --git a/apps/sim/public/library/best-ai-agents-for-lead-enrichment-2026/cover.jpg b/apps/sim/public/library/best-ai-agents-for-lead-enrichment-2026/cover.jpg new file mode 100644 index 00000000000..fbe5053d604 Binary files /dev/null and b/apps/sim/public/library/best-ai-agents-for-lead-enrichment-2026/cover.jpg differ diff --git a/apps/sim/scripts/backfill-table-workflow-deployments.test.ts b/apps/sim/scripts/backfill-table-workflow-deployments.test.ts new file mode 100644 index 00000000000..e4ad4a1f520 --- /dev/null +++ b/apps/sim/scripts/backfill-table-workflow-deployments.test.ts @@ -0,0 +1,414 @@ +/** + * @vitest-environment node + */ +import { dbChainMockFns, resetDbChainMock } from '@sim/testing' +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' + +const { mockLoadRuntimeSecrets, mockPerformFullDeploy } = vi.hoisted(() => ({ + mockLoadRuntimeSecrets: vi.fn(), + mockPerformFullDeploy: vi.fn(), +})) + +vi.mock('@sim/runtime-secrets', () => ({ + loadRuntimeSecrets: mockLoadRuntimeSecrets, +})) + +vi.mock('@/lib/workflows/orchestration/deploy', () => ({ + performFullDeploy: mockPerformFullDeploy, +})) + +import { + backfillTableWorkflowDeployments, + deployTableWorkflow, + parseTableWorkflowDeploymentBackfillArgs, + postgresTableWorkflowDeploymentStore, + prepareTableWorkflowDeploymentBackfillEnvironment, + TABLE_WORKFLOW_DEPLOYMENT_BATCH_SIZE, + type TableWorkflowDeploymentCandidate, + type TableWorkflowDeploymentStore, +} from '@/scripts/backfill-table-workflow-deployments' + +const ORIGINAL_ENV = { + DATABASE_URL: process.env.DATABASE_URL, + DATABASE_URL_WEB: process.env.DATABASE_URL_WEB, + REDIS_TLS_SERVERNAME: process.env.REDIS_TLS_SERVERNAME, + REDIS_URL: process.env.REDIS_URL, + SIM_ENV_SECRET_ID: process.env.SIM_ENV_SECRET_ID, +} + +interface MockSqlQuery { + toSQL(): { sql: string } +} + +function restoreEnvironmentVariable(key: keyof typeof ORIGINAL_ENV): void { + const value = ORIGINAL_ENV[key] + if (value === undefined) { + Reflect.deleteProperty(process.env, key) + } else { + process.env[key] = value + } +} + +function candidate(workflowId: string): TableWorkflowDeploymentCandidate { + return { + workflowId, + workspaceId: 'workspace-1', + userId: 'user-1', + } +} + +function store( + overrides: Partial = {} +): TableWorkflowDeploymentStore { + return { + assertIntegrity: vi.fn().mockResolvedValue(undefined), + listCandidates: vi.fn().mockResolvedValue([]), + isDeployed: vi.fn().mockResolvedValue(false), + ...overrides, + } +} + +describe('backfillTableWorkflowDeployments', () => { + beforeEach(() => { + vi.clearAllMocks() + resetDbChainMock() + }) + + afterEach(() => { + restoreEnvironmentVariable('DATABASE_URL') + restoreEnvironmentVariable('DATABASE_URL_WEB') + restoreEnvironmentVariable('REDIS_TLS_SERVERNAME') + restoreEnvironmentVariable('REDIS_URL') + restoreEnvironmentVariable('SIM_ENV_SECRET_ID') + }) + + it.each([ + ['production', '/production/sim/env-vars'], + ['staging', '/staging/sim/env-vars'], + ] as const)( + 'loads the %s runtime secret before database modules are needed', + async (environment, runtimeSecretId) => { + Reflect.deleteProperty(process.env, 'DATABASE_URL') + Reflect.deleteProperty(process.env, 'DATABASE_URL_WEB') + Reflect.deleteProperty(process.env, 'REDIS_TLS_SERVERNAME') + Reflect.deleteProperty(process.env, 'REDIS_URL') + Reflect.deleteProperty(process.env, 'SIM_ENV_SECRET_ID') + mockLoadRuntimeSecrets.mockImplementation(async () => { + process.env.DATABASE_URL = `postgres://${environment}/database` + process.env.REDIS_TLS_SERVERNAME = `cache.${environment}.internal` + process.env.REDIS_URL = `rediss://cache.${environment}.internal:6379` + }) + + await prepareTableWorkflowDeploymentBackfillEnvironment([`--environment=${environment}`]) + + expect(process.env.SIM_ENV_SECRET_ID).toBe(runtimeSecretId) + expect(process.env.REDIS_TLS_SERVERNAME).toBeUndefined() + expect(process.env.REDIS_URL).toBeUndefined() + expect(mockLoadRuntimeSecrets).toHaveBeenCalledTimes(1) + } + ) + + it('keeps the existing local DATABASE_URL mode when no environment is requested', async () => { + process.env.DATABASE_URL = 'postgres://local/database' + process.env.REDIS_URL = 'redis://localhost:6379' + + await prepareTableWorkflowDeploymentBackfillEnvironment([]) + + expect(mockLoadRuntimeSecrets).not.toHaveBeenCalled() + expect(process.env.DATABASE_URL).toBe('postgres://local/database') + expect(process.env.REDIS_URL).toBe('redis://localhost:6379') + }) + + it('rejects unsupported, unknown, duplicate, and locally configured staging arguments', async () => { + expect(() => parseTableWorkflowDeploymentBackfillArgs(['--environment=prod'])).toThrow( + 'Unsupported backfill environment: prod' + ) + expect(() => parseTableWorkflowDeploymentBackfillArgs(['--dry-run'])).toThrow( + 'Unknown argument: --dry-run' + ) + expect(() => + parseTableWorkflowDeploymentBackfillArgs(['--environment=staging', '--environment=staging']) + ).toThrow('can only be provided once') + + process.env.DATABASE_URL = 'postgres://local/database' + await expect( + prepareTableWorkflowDeploymentBackfillEnvironment(['--environment=staging']) + ).rejects.toThrow('local configuration cannot override staging') + expect(mockLoadRuntimeSecrets).not.toHaveBeenCalled() + }) + + it('silently excludes missing and archived workflow references from integrity checks', async () => { + await postgresTableWorkflowDeploymentStore.assertIntegrity() + + const referenceQuery = dbChainMockFns.execute.mock.calls[2]?.[0] as MockSqlQuery + const queryText = referenceQuery.toSQL().sql + expect(queryText).toContain('INNER JOIN workflow') + expect(queryText).toContain('workflow.archived_at IS NULL') + expect(queryText).not.toContain('LEFT JOIN workflow') + expect(queryText).not.toContain('workflow.id IS NULL') + }) + + it('deploys bounded keyset pages and verifies the final desired state', async () => { + const listCandidates = vi + .fn() + .mockResolvedValueOnce([candidate('workflow-a'), candidate('workflow-b')]) + .mockResolvedValueOnce([candidate('workflow-c')]) + .mockResolvedValueOnce([]) + .mockResolvedValueOnce([]) + const deploymentState = new Map() + const isDeployed = vi + .fn() + .mockImplementation(async (workflowId) => deploymentState.get(workflowId) ?? false) + const deploy = vi.fn(async (workflow: TableWorkflowDeploymentCandidate) => { + deploymentState.set(workflow.workflowId, true) + return { + success: true, + activeDeployment: { + deploymentVersionId: `version-${workflow.workflowId}`, + version: 1, + deployedAt: new Date().toISOString(), + }, + } + }) + const backfillStore = store({ listCandidates, isDeployed }) + + await expect( + backfillTableWorkflowDeployments(backfillStore, deploy, { batchSize: 2 }) + ).resolves.toEqual({ + scanned: 3, + deployed: 3, + alreadyDeployed: 0, + skippedLocked: 0, + skippedUndeployable: 0, + }) + expect(listCandidates.mock.calls).toEqual([ + ['', 2], + ['workflow-b', 2], + ['workflow-c', 2], + ['', 1], + ]) + expect(backfillStore.assertIntegrity).toHaveBeenCalledTimes(2) + expect(deploy.mock.calls.map(([workflow]) => workflow.workflowId)).toEqual([ + 'workflow-a', + 'workflow-b', + 'workflow-c', + ]) + }) + + it('does not redeploy an already deployed workflow', async () => { + const listCandidates = vi + .fn() + .mockResolvedValueOnce([candidate('workflow-a')]) + .mockResolvedValueOnce([]) + .mockResolvedValueOnce([]) + const deploy = vi.fn() + + await expect( + backfillTableWorkflowDeployments( + store({ + listCandidates, + isDeployed: vi.fn().mockResolvedValue(true), + }), + deploy + ) + ).resolves.toEqual({ + scanned: 1, + deployed: 0, + alreadyDeployed: 1, + skippedLocked: 0, + skippedUndeployable: 0, + }) + expect(deploy).not.toHaveBeenCalled() + }) + + it('reports and skips locked workflows while continuing the backfill', async () => { + const listCandidates = vi + .fn() + .mockResolvedValueOnce([candidate('workflow-a'), candidate('workflow-b')]) + .mockResolvedValueOnce([]) + .mockResolvedValueOnce([candidate('workflow-a')]) + .mockResolvedValueOnce([]) + const deploymentState = new Map() + const isDeployed = vi + .fn() + .mockImplementation(async (workflowId) => deploymentState.get(workflowId) ?? false) + const deploy = vi.fn(async (workflow: TableWorkflowDeploymentCandidate) => { + if (workflow.workflowId === 'workflow-a') { + return { + success: false, + error: 'Workflow is locked by its containing folder', + errorCode: 'locked' as const, + } + } + deploymentState.set(workflow.workflowId, true) + return { + success: true, + activeDeployment: { + deploymentVersionId: `version-${workflow.workflowId}`, + version: 1, + deployedAt: new Date().toISOString(), + }, + } + }) + + await expect( + backfillTableWorkflowDeployments(store({ listCandidates, isDeployed }), deploy) + ).resolves.toEqual({ + scanned: 2, + deployed: 1, + alreadyDeployed: 0, + skippedLocked: 1, + skippedUndeployable: 0, + }) + expect(deploy.mock.calls.map(([workflow]) => workflow.workflowId)).toEqual([ + 'workflow-a', + 'workflow-b', + ]) + expect(listCandidates.mock.calls.slice(-2)).toEqual([ + ['', 1], + ['workflow-a', 1], + ]) + }) + + it('reports and skips undeployable workflows while continuing the backfill', async () => { + const listCandidates = vi + .fn() + .mockResolvedValueOnce([candidate('workflow-a'), candidate('workflow-b')]) + .mockResolvedValueOnce([]) + .mockResolvedValueOnce([candidate('workflow-a')]) + .mockResolvedValueOnce([]) + const deploymentState = new Map() + const isDeployed = vi + .fn() + .mockImplementation(async (workflowId) => deploymentState.get(workflowId) ?? false) + const deploy = vi.fn(async (workflow: TableWorkflowDeploymentCandidate) => { + if (workflow.workflowId === 'workflow-a') { + return { + success: false, + error: 'Missing required fields for WhatsApp Webhook: Verification Token, App Secret', + errorCode: 'validation' as const, + } + } + deploymentState.set(workflow.workflowId, true) + return { + success: true, + activeDeployment: { + deploymentVersionId: `version-${workflow.workflowId}`, + version: 1, + deployedAt: new Date().toISOString(), + }, + } + }) + + await expect( + backfillTableWorkflowDeployments(store({ listCandidates, isDeployed }), deploy) + ).resolves.toEqual({ + scanned: 2, + deployed: 1, + alreadyDeployed: 0, + skippedLocked: 0, + skippedUndeployable: 1, + }) + expect(deploy.mock.calls.map(([workflow]) => workflow.workflowId)).toEqual([ + 'workflow-a', + 'workflow-b', + ]) + expect(listCandidates.mock.calls.slice(-2)).toEqual([ + ['', 1], + ['workflow-a', 1], + ]) + }) + + it('fails fast when canonical deployment fails', async () => { + const listCandidates = vi + .fn() + .mockResolvedValueOnce([candidate('workflow-a'), candidate('workflow-b')]) + const deploy = vi.fn().mockResolvedValue({ + success: false, + error: 'invalid trigger configuration', + }) + + await expect( + backfillTableWorkflowDeployments(store({ listCandidates }), deploy) + ).rejects.toThrow('Failed to deploy table workflow workflow-a: invalid trigger configuration') + expect(deploy).toHaveBeenCalledTimes(1) + }) + + it('fails when deployment does not activate or persist a valid active version', async () => { + const firstList = vi + .fn() + .mockResolvedValueOnce([candidate('workflow-a')]) + await expect( + backfillTableWorkflowDeployments(store({ listCandidates: firstList }), async () => ({ + success: true, + activeDeployment: null, + })) + ).rejects.toThrow('did not reach an active deployment state') + + const secondList = vi + .fn() + .mockResolvedValueOnce([candidate('workflow-b')]) + await expect( + backfillTableWorkflowDeployments(store({ listCandidates: secondList }), async () => ({ + success: true, + activeDeployment: { + deploymentVersionId: 'version-b', + version: 1, + deployedAt: new Date().toISOString(), + }, + })) + ).rejects.toThrow('completed without a valid active version') + }) + + it('rejects invalid batch and page behavior before it can loop or skip data', async () => { + const invalidBatchStore = store() + await expect( + backfillTableWorkflowDeployments(invalidBatchStore, vi.fn(), { batchSize: 0 }) + ).rejects.toThrow('positive integer') + expect(invalidBatchStore.assertIntegrity).not.toHaveBeenCalled() + + const oversizedStore = store({ + listCandidates: vi.fn().mockResolvedValue([candidate('workflow-a'), candidate('workflow-b')]), + }) + await expect( + backfillTableWorkflowDeployments(oversizedStore, vi.fn(), { batchSize: 1 }) + ).rejects.toThrow('oversized page') + + const duplicateStore = store({ + listCandidates: vi.fn().mockResolvedValue([candidate('workflow-a'), candidate('workflow-a')]), + }) + await expect( + backfillTableWorkflowDeployments(duplicateStore, vi.fn(), { batchSize: 2 }) + ).rejects.toThrow('duplicate workflow ids') + }) + + it('uses the canonical deployer with backfill attribution and a stable idempotency key', async () => { + mockPerformFullDeploy.mockResolvedValue({ + success: true, + activeDeployment: { + deploymentVersionId: 'version-1', + version: 1, + deployedAt: new Date().toISOString(), + }, + }) + + await deployTableWorkflow(candidate('workflow-1')) + + expect(mockPerformFullDeploy).toHaveBeenCalledWith({ + workflowId: 'workflow-1', + userId: 'user-1', + actorId: 'table-workflow-deployment-backfill', + captureAnalytics: false, + requestId: 'table-workflow-deployment-backfill:v2:workflow-1', + idempotencyKey: 'table-workflow-deployment-backfill:v2:workflow-1', + }) + }) + + it('uses the repository batch-size default', async () => { + const listCandidates = vi.fn().mockResolvedValue([]) + + await backfillTableWorkflowDeployments(store({ listCandidates }), vi.fn()) + + expect(listCandidates).toHaveBeenNthCalledWith(1, '', TABLE_WORKFLOW_DEPLOYMENT_BATCH_SIZE) + }) +}) diff --git a/apps/sim/scripts/backfill-table-workflow-deployments.ts b/apps/sim/scripts/backfill-table-workflow-deployments.ts new file mode 100644 index 00000000000..5c8ddd77556 --- /dev/null +++ b/apps/sim/scripts/backfill-table-workflow-deployments.ts @@ -0,0 +1,515 @@ +#!/usr/bin/env bun + +/** + * Deploys every mutable workflow referenced by a table workflow group. + * + * The script is idempotent and resumable: it pages over only workflows that do + * not have the full desired state, and each deployment uses a stable idempotency + * key. It runs the canonical deployment orchestration so webhook, schedule, + * MCP, audit, and notification side effects stay consistent with a + * user-initiated deployment. + * + * Usage: + * DATABASE_URL=... bun run apps/sim/scripts/backfill-table-workflow-deployments.ts + * AWS_PROFILE=sim-admin bun --no-env-file apps/sim/scripts/backfill-table-workflow-deployments.ts --environment=staging + * AWS_PROFILE=sim-admin bun --no-env-file apps/sim/scripts/backfill-table-workflow-deployments.ts --environment=production + */ + +import { createLogger } from '@sim/logger' +import { loadRuntimeSecrets } from '@sim/runtime-secrets' +import { getErrorMessage } from '@sim/utils/errors' +import { sql } from 'drizzle-orm' +import type { PerformFullDeployResult } from '@/lib/workflows/orchestration/deploy' + +const logger = createLogger('BackfillTableWorkflowDeployments') + +export const TABLE_WORKFLOW_DEPLOYMENT_BATCH_SIZE = 25 +const BACKFILL_ACTOR_ID = 'table-workflow-deployment-backfill' +const BACKFILL_OPERATION_VERSION = 'v2' +const RUNTIME_SECRET_IDS = { + production: '/production/sim/env-vars', + staging: '/staging/sim/env-vars', +} as const +/** Container-private services that a locally executed hosted backfill must not initialize. */ +const LOCAL_HOSTED_OMITTED_VARIABLES = ['REDIS_URL', 'REDIS_TLS_SERVERNAME'] as const + +type TableWorkflowDeploymentBackfillEnvironment = keyof typeof RUNTIME_SECRET_IDS + +interface TableWorkflowDeploymentBackfillCliOptions { + environment?: TableWorkflowDeploymentBackfillEnvironment +} + +export interface TableWorkflowDeploymentCandidate { + workflowId: string + workspaceId: string + userId: string +} + +export interface TableWorkflowDeploymentStore { + assertIntegrity(): Promise + listCandidates( + afterWorkflowId: string, + limit: number + ): Promise + isDeployed(workflowId: string): Promise +} + +export interface TableWorkflowDeploymentSummary { + scanned: number + deployed: number + alreadyDeployed: number + skippedLocked: number + skippedUndeployable: number +} + +interface TableWorkflowDeploymentBackfillOptions { + batchSize?: number +} + +export type DeployTableWorkflow = ( + candidate: TableWorkflowDeploymentCandidate +) => Promise + +interface InvalidTableSchemaRow extends Record { + table_id: string +} + +interface InvalidWorkflowGroupRow extends Record { + group_index: string + table_id: string +} + +interface InvalidWorkflowReferenceRow extends Record { + table_id: string + table_workspace_id: string + workflow_id: string + workflow_workspace_id: string +} + +interface MultipleActiveVersionsRow extends Record { + active_version_count: number + workflow_id: string +} + +interface CandidateRow extends Record { + user_id: string + workflow_id: string + workspace_id: string +} + +interface DeploymentStateRow extends Record { + active_version_count: number + is_deployed: boolean +} + +function isTableWorkflowDeploymentBackfillEnvironment( + value: string +): value is TableWorkflowDeploymentBackfillEnvironment { + return Object.hasOwn(RUNTIME_SECRET_IDS, value) +} + +/** Parses the deliberately small CLI surface for the backfill. */ +export function parseTableWorkflowDeploymentBackfillArgs( + args: readonly string[] +): TableWorkflowDeploymentBackfillCliOptions { + let environment: TableWorkflowDeploymentBackfillCliOptions['environment'] + + for (const arg of args) { + if (!arg.startsWith('--environment=')) { + throw new Error(`Unknown argument: ${arg}`) + } + if (environment) { + throw new Error('The --environment argument can only be provided once') + } + + const requestedEnvironment = arg.slice('--environment='.length) + if (!isTableWorkflowDeploymentBackfillEnvironment(requestedEnvironment)) { + throw new Error(`Unsupported backfill environment: ${requestedEnvironment || '(empty)'}`) + } + environment = requestedEnvironment + } + + return { environment } +} + +/** Loads staging configuration before modules that read database settings are imported. */ +export async function prepareTableWorkflowDeploymentBackfillEnvironment( + args: readonly string[] +): Promise { + const { environment } = parseTableWorkflowDeploymentBackfillArgs(args) + if (!environment) return + + const runtimeSecretId = RUNTIME_SECRET_IDS[environment] + const configuredSecretId = process.env.SIM_ENV_SECRET_ID + if (configuredSecretId && configuredSecretId !== runtimeSecretId) { + throw new Error( + `SIM_ENV_SECRET_ID is already set to ${configuredSecretId}; expected ${runtimeSecretId}` + ) + } + + const configuredDatabaseVariables = ['DATABASE_URL', 'DATABASE_URL_WEB'].filter( + (key) => key in process.env + ) + if (configuredDatabaseVariables.length > 0) { + throw new Error( + `Unset ${configuredDatabaseVariables.join(', ')} before using --environment=${environment} so local configuration cannot override ${environment}` + ) + } + + process.env.SIM_ENV_SECRET_ID = runtimeSecretId + await loadRuntimeSecrets() + + if (!process.env.DATABASE_URL && !process.env.DATABASE_URL_WEB) { + throw new Error(`${runtimeSecretId} did not provide a database URL`) + } + + for (const key of LOCAL_HOSTED_OMITTED_VARIABLES) { + Reflect.deleteProperty(process.env, key) + } +} + +async function getDatabase() { + const { db } = await import('@sim/db') + return db +} + +function validateCandidatePage( + candidates: TableWorkflowDeploymentCandidate[], + afterWorkflowId: string, + limit: number +): string | null { + if (candidates.length === 0) return null + if (candidates.length > limit) { + throw new Error('Table workflow deployment store returned an oversized page') + } + + const pageIds = new Set(candidates.map((candidate) => candidate.workflowId)) + if (pageIds.size !== candidates.length) { + throw new Error('Table workflow deployment store returned duplicate workflow ids') + } + + const lastWorkflowId = candidates.at(-1)?.workflowId + if (!lastWorkflowId || lastWorkflowId === afterWorkflowId) { + throw new Error('Table workflow deployment store returned a non-advancing page') + } + return lastWorkflowId +} + +async function assertOnlySkippedCandidatesRemain( + store: TableWorkflowDeploymentStore, + skippedWorkflowIds: ReadonlySet +): Promise { + let afterWorkflowId = '' + for (;;) { + const candidates = await store.listCandidates(afterWorkflowId, 1) + const lastWorkflowId = validateCandidatePage(candidates, afterWorkflowId, 1) + if (!lastWorkflowId) return + + const unexpectedCandidate = candidates.find( + (candidate) => !skippedWorkflowIds.has(candidate.workflowId) + ) + if (unexpectedCandidate) { + throw new Error( + `Table workflow deployment backfill left workflow ${unexpectedCandidate.workflowId} undeployed` + ) + } + afterWorkflowId = lastWorkflowId + } +} + +/** Ensures the table group references can be traversed without silently dropping corrupt data. */ +async function assertTableWorkflowIntegrity(): Promise { + const db = await getDatabase() + const [invalidSchema] = await db.execute(sql` + SELECT id AS table_id + FROM user_table_definitions + WHERE schema ? 'workflowGroups' + AND jsonb_typeof(schema->'workflowGroups') IS DISTINCT FROM 'array' + LIMIT 1 + `) + if (invalidSchema) { + throw new Error( + `Table ${invalidSchema.table_id} has a workflowGroups value that is not an array` + ) + } + + const [invalidGroup] = await db.execute(sql` + SELECT + table_definition.id AS table_id, + workflow_group.ordinality::text AS group_index + FROM user_table_definitions AS table_definition + CROSS JOIN LATERAL jsonb_array_elements( + COALESCE(table_definition.schema->'workflowGroups', '[]'::jsonb) + ) WITH ORDINALITY AS workflow_group(value, ordinality) + WHERE jsonb_typeof(workflow_group.value) IS DISTINCT FROM 'object' + OR NOT (workflow_group.value ? 'workflowId') + OR jsonb_typeof(workflow_group.value->'workflowId') IS DISTINCT FROM 'string' + OR ( + workflow_group.value->>'workflowId' = '' + AND ( + NOT (workflow_group.value ? 'enrichmentId') + OR jsonb_typeof(workflow_group.value->'enrichmentId') IS DISTINCT FROM 'string' + OR workflow_group.value->>'enrichmentId' = '' + ) + ) + LIMIT 1 + `) + if (invalidGroup) { + throw new Error( + `Table ${invalidGroup.table_id} workflow group ${invalidGroup.group_index} has an invalid workflowId` + ) + } + + const [invalidReference] = await db.execute(sql` + WITH table_workflow_groups AS ( + SELECT + table_definition.id AS table_id, + table_definition.workspace_id AS table_workspace_id, + workflow_group.value->>'workflowId' AS workflow_id + FROM user_table_definitions AS table_definition + CROSS JOIN LATERAL jsonb_array_elements( + COALESCE(table_definition.schema->'workflowGroups', '[]'::jsonb) + ) AS workflow_group(value) + ) + SELECT + table_workflow_groups.table_id, + table_workflow_groups.table_workspace_id, + table_workflow_groups.workflow_id, + workflow.workspace_id AS workflow_workspace_id + FROM table_workflow_groups + INNER JOIN workflow ON workflow.id = table_workflow_groups.workflow_id + AND workflow.archived_at IS NULL + WHERE table_workflow_groups.workflow_id <> '' + AND workflow.workspace_id IS DISTINCT FROM table_workflow_groups.table_workspace_id + LIMIT 1 + `) + if (invalidReference) { + throw new Error( + `Table ${invalidReference.table_id} in workspace ${invalidReference.table_workspace_id} ` + + `references workflow ${invalidReference.workflow_id} in workspace ${invalidReference.workflow_workspace_id}` + ) + } + + const [multipleActiveVersions] = await db.execute(sql` + WITH referenced_workflow_ids AS ( + SELECT DISTINCT workflow_group.value->>'workflowId' AS workflow_id + FROM user_table_definitions AS table_definition + CROSS JOIN LATERAL jsonb_array_elements( + COALESCE(table_definition.schema->'workflowGroups', '[]'::jsonb) + ) AS workflow_group(value) + INNER JOIN workflow ON workflow.id = workflow_group.value->>'workflowId' + AND workflow.archived_at IS NULL + WHERE workflow_group.value->>'workflowId' <> '' + ) + SELECT + deployment_version.workflow_id, + COUNT(*)::int AS active_version_count + FROM workflow_deployment_version AS deployment_version + INNER JOIN referenced_workflow_ids + ON referenced_workflow_ids.workflow_id = deployment_version.workflow_id + WHERE deployment_version.is_active = true + GROUP BY deployment_version.workflow_id + HAVING COUNT(*) > 1 + LIMIT 1 + `) + if (multipleActiveVersions) { + throw new Error( + `Workflow ${multipleActiveVersions.workflow_id} has ${multipleActiveVersions.active_version_count} active deployment versions` + ) + } +} + +export const postgresTableWorkflowDeploymentStore: TableWorkflowDeploymentStore = { + assertIntegrity: assertTableWorkflowIntegrity, + + async listCandidates(afterWorkflowId, limit) { + const db = await getDatabase() + const rows = await db.execute(sql` + WITH referenced_workflow_ids AS ( + SELECT DISTINCT workflow_group.value->>'workflowId' AS workflow_id + FROM user_table_definitions AS table_definition + CROSS JOIN LATERAL jsonb_array_elements( + COALESCE(table_definition.schema->'workflowGroups', '[]'::jsonb) + ) AS workflow_group(value) + WHERE workflow_group.value->>'workflowId' <> '' + ) + SELECT + workflow.id AS workflow_id, + workflow.workspace_id AS workspace_id, + workflow.user_id + FROM referenced_workflow_ids + INNER JOIN workflow ON workflow.id = referenced_workflow_ids.workflow_id + AND workflow.archived_at IS NULL + WHERE workflow.id COLLATE "C" > ${afterWorkflowId}::text COLLATE "C" + AND ( + workflow.is_deployed = false + OR NOT EXISTS ( + SELECT 1 + FROM workflow_deployment_version AS active_version + WHERE active_version.workflow_id = workflow.id + AND active_version.is_active = true + ) + ) + ORDER BY workflow.id COLLATE "C" + LIMIT ${limit} + `) + + return rows.map((row) => ({ + workflowId: row.workflow_id, + workspaceId: row.workspace_id, + userId: row.user_id, + })) + }, + + async isDeployed(workflowId) { + const db = await getDatabase() + const [state] = await db.execute(sql` + SELECT + workflow.is_deployed, + (COUNT(deployment_version.id) FILTER (WHERE deployment_version.is_active))::int + AS active_version_count + FROM workflow + LEFT JOIN workflow_deployment_version AS deployment_version + ON deployment_version.workflow_id = workflow.id + WHERE workflow.id = ${workflowId} + GROUP BY workflow.id, workflow.is_deployed + `) + if (!state) { + throw new Error(`Workflow ${workflowId} disappeared during the deployment backfill`) + } + if (state.active_version_count > 1) { + throw new Error( + `Workflow ${workflowId} has ${state.active_version_count} active deployment versions` + ) + } + return state.is_deployed && state.active_version_count === 1 + }, +} + +/** Deploys one table workflow through the same orchestration used by application surfaces. */ +export async function deployTableWorkflow( + candidate: TableWorkflowDeploymentCandidate +): Promise { + const { performFullDeploy } = await import('@/lib/workflows/orchestration/deploy') + return performFullDeploy({ + workflowId: candidate.workflowId, + userId: candidate.userId, + actorId: BACKFILL_ACTOR_ID, + captureAnalytics: false, + requestId: `${BACKFILL_ACTOR_ID}:${BACKFILL_OPERATION_VERSION}:${candidate.workflowId}`, + idempotencyKey: `${BACKFILL_ACTOR_ID}:${BACKFILL_OPERATION_VERSION}:${candidate.workflowId}`, + }) +} + +/** + * Reaches and verifies that every mutable table workflow has an active deployment. + */ +export async function backfillTableWorkflowDeployments( + store: TableWorkflowDeploymentStore, + deploy: DeployTableWorkflow, + options: TableWorkflowDeploymentBackfillOptions = {} +): Promise { + const batchSize = options.batchSize ?? TABLE_WORKFLOW_DEPLOYMENT_BATCH_SIZE + if (!Number.isInteger(batchSize) || batchSize <= 0) { + throw new Error('Table workflow deployment backfill batch size must be a positive integer') + } + + await store.assertIntegrity() + + const summary: TableWorkflowDeploymentSummary = { + scanned: 0, + deployed: 0, + alreadyDeployed: 0, + skippedLocked: 0, + skippedUndeployable: 0, + } + const skippedWorkflowIds = new Set() + let afterWorkflowId = '' + + for (;;) { + const candidates = await store.listCandidates(afterWorkflowId, batchSize) + const lastWorkflowId = validateCandidatePage(candidates, afterWorkflowId, batchSize) + if (!lastWorkflowId) break + + for (const candidate of candidates) { + summary.scanned += 1 + if (await store.isDeployed(candidate.workflowId)) { + summary.alreadyDeployed += 1 + } else { + logger.info('Deploying workflow referenced by a table workflow group', { + workflowId: candidate.workflowId, + workspaceId: candidate.workspaceId, + }) + const result = await deploy(candidate) + if (!result.success) { + if (result.errorCode === 'locked') { + skippedWorkflowIds.add(candidate.workflowId) + summary.skippedLocked += 1 + logger.warn('Skipping locked workflow referenced by a table workflow group', { + workflowId: candidate.workflowId, + workspaceId: candidate.workspaceId, + reason: result.error ?? 'Workflow is locked', + }) + continue + } + if (result.errorCode === 'validation') { + skippedWorkflowIds.add(candidate.workflowId) + summary.skippedUndeployable += 1 + logger.warn('Skipping undeployable workflow referenced by a table workflow group', { + workflowId: candidate.workflowId, + workspaceId: candidate.workspaceId, + reason: result.error ?? 'Workflow deployment validation failed', + }) + continue + } + throw new Error( + `Failed to deploy table workflow ${candidate.workflowId}: ${result.error ?? 'deployment returned no error'}` + ) + } + if (!result.activeDeployment) { + throw new Error( + `Table workflow ${candidate.workflowId} did not reach an active deployment state` + ) + } + if (!(await store.isDeployed(candidate.workflowId))) { + throw new Error( + `Table workflow ${candidate.workflowId} deployment completed without a valid active version` + ) + } + summary.deployed += 1 + } + } + + afterWorkflowId = lastWorkflowId + } + + await store.assertIntegrity() + await assertOnlySkippedCandidatesRemain(store, skippedWorkflowIds) + + return summary +} + +export async function runTableWorkflowDeploymentBackfill(): Promise { + logger.info('Starting table workflow deployment backfill') + const summary = await backfillTableWorkflowDeployments( + postgresTableWorkflowDeploymentStore, + deployTableWorkflow + ) + logger.info('Table workflow deployment backfill completed', summary) +} + +async function main(): Promise { + await prepareTableWorkflowDeploymentBackfillEnvironment(process.argv.slice(2)) + await runTableWorkflowDeploymentBackfill() +} + +if ((import.meta as { main?: boolean }).main) { + main() + .then(() => process.exit(0)) + .catch((error: unknown) => { + logger.error('Table workflow deployment backfill failed', { + error: getErrorMessage(error), + }) + process.exit(1) + }) +} diff --git a/apps/sim/scripts/build-function-daytona-snapshot.ts b/apps/sim/scripts/build-function-daytona-snapshot.ts index 5dfae90c3cf..2c6b85b0be6 100644 --- a/apps/sim/scripts/build-function-daytona-snapshot.ts +++ b/apps/sim/scripts/build-function-daytona-snapshot.ts @@ -20,6 +20,7 @@ import { isImmutableDaytonaSnapshotRef, } from '@sim/utils/sandbox-references' import { + FUNCTION_DAYTONA_DISK_GB, FUNCTION_SANDBOX_CPU_COUNT, FUNCTION_SANDBOX_MEMORY_GB, } from '@/lib/execution/remote-sandbox/function-resources' @@ -53,7 +54,7 @@ const APT_INSTALL = 'DEBIAN_FRONTEND=noninteractive apt-get install -y --no-inst const RESOURCES = { cpu: FUNCTION_SANDBOX_CPU_COUNT, memory: FUNCTION_SANDBOX_MEMORY_GB, - disk: 10, + disk: FUNCTION_DAYTONA_DISK_GB, } as const export function createFunctionImage(manifest: FunctionSandboxParityManifest) { diff --git a/apps/sim/tools/elasticsearch/bulk.ts b/apps/sim/tools/elasticsearch/bulk.ts index 0d76f00d406..6aea23fb085 100644 --- a/apps/sim/tools/elasticsearch/bulk.ts +++ b/apps/sim/tools/elasticsearch/bulk.ts @@ -2,49 +2,9 @@ import type { ElasticsearchBulkParams, ElasticsearchBulkResponse, } from '@/tools/elasticsearch/types' +import { buildAuthHeaders, buildBaseUrl } from '@/tools/elasticsearch/utils' import type { ToolConfig } from '@/tools/types' -function buildBaseUrl(params: ElasticsearchBulkParams): string { - if (params.deploymentType === 'cloud' && params.cloudId) { - const parts = params.cloudId.split(':') - if (parts.length >= 2) { - try { - const decoded = Buffer.from(parts[1], 'base64').toString('utf-8') - const [esHost] = decoded.split('$') - if (esHost) { - return `https://${parts[0]}.${esHost}` - } - } catch { - // Fallback - } - } - throw new Error('Invalid Cloud ID format') - } - - if (!params.host) { - throw new Error('Host is required for self-hosted deployments') - } - - return params.host.replace(/\/$/, '') -} - -function buildAuthHeaders(params: ElasticsearchBulkParams): Record { - const headers: Record = { - 'Content-Type': 'application/x-ndjson', - } - - if (params.authMethod === 'api_key' && params.apiKey) { - headers.Authorization = `ApiKey ${params.apiKey}` - } else if (params.authMethod === 'basic_auth' && params.username && params.password) { - const credentials = Buffer.from(`${params.username}:${params.password}`).toString('base64') - headers.Authorization = `Basic ${credentials}` - } else { - throw new Error('Invalid authentication configuration') - } - - return headers -} - export const bulkTool: ToolConfig = { id: 'elasticsearch_bulk', name: 'Elasticsearch Bulk Operations', @@ -127,7 +87,8 @@ export const bulkTool: ToolConfig buildAuthHeaders(params), + headers: (params) => buildAuthHeaders(params, 'application/x-ndjson'), + redirectPolicy: () => ({ mode: 'legacy', sendCredentialsOnCrossOriginRedirect: false }), body: (params) => { // The body should be NDJSON format - we pass it as raw string // Ensure it ends with a newline diff --git a/apps/sim/tools/elasticsearch/cluster_health.test.ts b/apps/sim/tools/elasticsearch/cluster_health.test.ts new file mode 100644 index 00000000000..2690de93f69 --- /dev/null +++ b/apps/sim/tools/elasticsearch/cluster_health.test.ts @@ -0,0 +1,83 @@ +/** + * @vitest-environment node + */ +import { describe, expect, it } from 'vitest' +import { ElasticsearchBlock } from '@/blocks/blocks/elasticsearch' +import * as elasticsearchTools from '@/tools/elasticsearch' +import { prepareToolRequest } from '@/tools/request-transport' +import type { ToolConfig } from '@/tools/types' + +const CONNECTION = { + deploymentType: 'self_hosted', + host: 'https://es.example.com', + authMethod: 'api_key', + apiKey: 'test-key', +} as const + +function mapBlockParams(params: Record): Record { + const config = ElasticsearchBlock.tools.config + if (!config?.params) throw new Error('block has no params mapper') + return config.params(params) as Record +} + +/** + * `tools/request-transport.ts` reads `params.timeout` as the outbound HTTP + * deadline in milliseconds. A tool param of that name therefore arms a client + * abort as a side effect of asking Elasticsearch to wait. + */ +describe('cluster health timeout does not arm a client abort', () => { + it('sends the wait on the wire and leaves no client deadline', () => { + const prepared = prepareToolRequest( + elasticsearchTools.elasticsearchClusterHealthTool as ToolConfig, + { ...CONNECTION, clusterTimeout: '30s', waitForStatus: 'yellow' } + ) + expect(prepared.url).toContain('timeout=30s') + expect(prepared.url).toContain('wait_for_status=yellow') + expect(prepared.timeout).toBeUndefined() + }) + + it('ignores a stray transport timeout left in saved state', () => { + const mapped = mapBlockParams({ operation: 'elasticsearch_cluster_health', timeout: '30' }) + expect(mapped.timeout).toBeUndefined() + expect(mapped.clusterTimeout).toBe('30s') + }) + + it('declares no param named timeout', () => { + expect( + Object.keys(elasticsearchTools.elasticsearchClusterHealthTool.params ?? {}) + ).not.toContain('timeout') + }) +}) + +describe('cluster health timeout units', () => { + it.each([ + ['30', '30s'], + ['30s', '30s'], + ['1m', '1m'], + ['500ms', '500ms'], + ['2h', '2h'], + ])('maps %o to %o', (input, expected) => { + const mapped = mapBlockParams({ operation: 'elasticsearch_cluster_health', timeout: input }) + expect(mapped.clusterTimeout).toBe(expected) + }) + + it('emits nothing for a blank timeout', () => { + expect( + mapBlockParams({ operation: 'elasticsearch_cluster_health', timeout: ' ' }) + ).not.toHaveProperty('clusterTimeout') + }) +}) + +describe('list indices system-index opt-in', () => { + it('coerces the dropdown string to a real boolean', () => { + expect( + mapBlockParams({ operation: 'elasticsearch_list_indices', includeSystemIndices: 'true' }) + ).toMatchObject({ includeSystemIndices: true }) + }) + + it('omits the flag when the dropdown is left at its default', () => { + expect( + mapBlockParams({ operation: 'elasticsearch_list_indices', includeSystemIndices: '' }) + ).not.toHaveProperty('includeSystemIndices') + }) +}) diff --git a/apps/sim/tools/elasticsearch/cluster_health.ts b/apps/sim/tools/elasticsearch/cluster_health.ts index 4d33ced6c66..399574f9d07 100644 --- a/apps/sim/tools/elasticsearch/cluster_health.ts +++ b/apps/sim/tools/elasticsearch/cluster_health.ts @@ -2,49 +2,9 @@ import type { ElasticsearchClusterHealthParams, ElasticsearchClusterHealthResponse, } from '@/tools/elasticsearch/types' +import { buildAuthHeaders, buildBaseUrl } from '@/tools/elasticsearch/utils' import type { ToolConfig } from '@/tools/types' -function buildBaseUrl(params: ElasticsearchClusterHealthParams): string { - if (params.deploymentType === 'cloud' && params.cloudId) { - const parts = params.cloudId.split(':') - if (parts.length >= 2) { - try { - const decoded = Buffer.from(parts[1], 'base64').toString('utf-8') - const [esHost] = decoded.split('$') - if (esHost) { - return `https://${parts[0]}.${esHost}` - } - } catch { - // Fallback - } - } - throw new Error('Invalid Cloud ID format') - } - - if (!params.host) { - throw new Error('Host is required for self-hosted deployments') - } - - return params.host.replace(/\/$/, '') -} - -function buildAuthHeaders(params: ElasticsearchClusterHealthParams): Record { - const headers: Record = { - 'Content-Type': 'application/json', - } - - if (params.authMethod === 'api_key' && params.apiKey) { - headers.Authorization = `ApiKey ${params.apiKey}` - } else if (params.authMethod === 'basic_auth' && params.username && params.password) { - const credentials = Buffer.from(`${params.username}:${params.password}`).toString('base64') - headers.Authorization = `Basic ${credentials}` - } else { - throw new Error('Invalid authentication configuration') - } - - return headers -} - export const clusterHealthTool: ToolConfig< ElasticsearchClusterHealthParams, ElasticsearchClusterHealthResponse @@ -100,10 +60,11 @@ export const clusterHealthTool: ToolConfig< required: false, description: 'Wait until cluster reaches this status: green, yellow, or red', }, - timeout: { + clusterTimeout: { type: 'string', required: false, - description: 'Timeout for the wait operation (e.g., 30s, 1m)', + description: + 'How long Elasticsearch waits for the cluster to reach the requested status, as an Elasticsearch time value (e.g., 30s, 1m). Not named "timeout": that name is reserved by the tool transport as a client-side abort deadline in milliseconds.', }, }, @@ -116,8 +77,8 @@ export const clusterHealthTool: ToolConfig< if (params.waitForStatus) { queryParams.push(`wait_for_status=${params.waitForStatus}`) } - if (params.timeout) { - queryParams.push(`timeout=${encodeURIComponent(params.timeout)}`) + if (params.clusterTimeout) { + queryParams.push(`timeout=${encodeURIComponent(params.clusterTimeout)}`) } if (queryParams.length > 0) { url += `?${queryParams.join('&')}` @@ -127,6 +88,7 @@ export const clusterHealthTool: ToolConfig< }, method: 'GET', headers: (params) => buildAuthHeaders(params), + redirectPolicy: () => ({ mode: 'legacy', sendCredentialsOnCrossOriginRedirect: false }), }, transformResponse: async (response: Response) => { diff --git a/apps/sim/tools/elasticsearch/cluster_stats.ts b/apps/sim/tools/elasticsearch/cluster_stats.ts index bdb71153dec..b635c74cadb 100644 --- a/apps/sim/tools/elasticsearch/cluster_stats.ts +++ b/apps/sim/tools/elasticsearch/cluster_stats.ts @@ -2,49 +2,9 @@ import type { ElasticsearchClusterStatsParams, ElasticsearchClusterStatsResponse, } from '@/tools/elasticsearch/types' +import { buildAuthHeaders, buildBaseUrl } from '@/tools/elasticsearch/utils' import type { ToolConfig } from '@/tools/types' -function buildBaseUrl(params: ElasticsearchClusterStatsParams): string { - if (params.deploymentType === 'cloud' && params.cloudId) { - const parts = params.cloudId.split(':') - if (parts.length >= 2) { - try { - const decoded = Buffer.from(parts[1], 'base64').toString('utf-8') - const [esHost] = decoded.split('$') - if (esHost) { - return `https://${parts[0]}.${esHost}` - } - } catch { - // Fallback - } - } - throw new Error('Invalid Cloud ID format') - } - - if (!params.host) { - throw new Error('Host is required for self-hosted deployments') - } - - return params.host.replace(/\/$/, '') -} - -function buildAuthHeaders(params: ElasticsearchClusterStatsParams): Record { - const headers: Record = { - 'Content-Type': 'application/json', - } - - if (params.authMethod === 'api_key' && params.apiKey) { - headers.Authorization = `ApiKey ${params.apiKey}` - } else if (params.authMethod === 'basic_auth' && params.username && params.password) { - const credentials = Buffer.from(`${params.username}:${params.password}`).toString('base64') - headers.Authorization = `Basic ${credentials}` - } else { - throw new Error('Invalid authentication configuration') - } - - return headers -} - export const clusterStatsTool: ToolConfig< ElasticsearchClusterStatsParams, ElasticsearchClusterStatsResponse @@ -104,6 +64,7 @@ export const clusterStatsTool: ToolConfig< }, method: 'GET', headers: (params) => buildAuthHeaders(params), + redirectPolicy: () => ({ mode: 'legacy', sendCredentialsOnCrossOriginRedirect: false }), }, transformResponse: async (response: Response) => { @@ -167,6 +128,7 @@ export const clusterStatsTool: ToolConfig< status: { type: 'string', description: 'Cluster health status', + optional: true, }, nodes: { type: 'object', diff --git a/apps/sim/tools/elasticsearch/count.ts b/apps/sim/tools/elasticsearch/count.ts index 64ea6636021..19a555f12bd 100644 --- a/apps/sim/tools/elasticsearch/count.ts +++ b/apps/sim/tools/elasticsearch/count.ts @@ -2,49 +2,9 @@ import type { ElasticsearchCountParams, ElasticsearchCountResponse, } from '@/tools/elasticsearch/types' +import { buildAuthHeaders, buildBaseUrl } from '@/tools/elasticsearch/utils' import type { ToolConfig } from '@/tools/types' -function buildBaseUrl(params: ElasticsearchCountParams): string { - if (params.deploymentType === 'cloud' && params.cloudId) { - const parts = params.cloudId.split(':') - if (parts.length >= 2) { - try { - const decoded = Buffer.from(parts[1], 'base64').toString('utf-8') - const [esHost] = decoded.split('$') - if (esHost) { - return `https://${parts[0]}.${esHost}` - } - } catch { - // Fallback - } - } - throw new Error('Invalid Cloud ID format') - } - - if (!params.host) { - throw new Error('Host is required for self-hosted deployments') - } - - return params.host.replace(/\/$/, '') -} - -function buildAuthHeaders(params: ElasticsearchCountParams): Record { - const headers: Record = { - 'Content-Type': 'application/json', - } - - if (params.authMethod === 'api_key' && params.apiKey) { - headers.Authorization = `ApiKey ${params.apiKey}` - } else if (params.authMethod === 'basic_auth' && params.username && params.password) { - const credentials = Buffer.from(`${params.username}:${params.password}`).toString('base64') - headers.Authorization = `Basic ${credentials}` - } else { - throw new Error('Invalid authentication configuration') - } - - return headers -} - export const countTool: ToolConfig = { id: 'elasticsearch_count', name: 'Elasticsearch Count', @@ -114,6 +74,7 @@ export const countTool: ToolConfig buildAuthHeaders(params), + redirectPolicy: () => ({ mode: 'legacy', sendCredentialsOnCrossOriginRedirect: false }), body: (params) => { if (params.query) { try { diff --git a/apps/sim/tools/elasticsearch/create_index.ts b/apps/sim/tools/elasticsearch/create_index.ts index e795540cc6d..44a3c97f6a0 100644 --- a/apps/sim/tools/elasticsearch/create_index.ts +++ b/apps/sim/tools/elasticsearch/create_index.ts @@ -2,49 +2,9 @@ import type { ElasticsearchCreateIndexParams, ElasticsearchIndexResponse, } from '@/tools/elasticsearch/types' +import { buildAuthHeaders, buildBaseUrl } from '@/tools/elasticsearch/utils' import type { ToolConfig } from '@/tools/types' -function buildBaseUrl(params: ElasticsearchCreateIndexParams): string { - if (params.deploymentType === 'cloud' && params.cloudId) { - const parts = params.cloudId.split(':') - if (parts.length >= 2) { - try { - const decoded = Buffer.from(parts[1], 'base64').toString('utf-8') - const [esHost] = decoded.split('$') - if (esHost) { - return `https://${parts[0]}.${esHost}` - } - } catch { - // Fallback - } - } - throw new Error('Invalid Cloud ID format') - } - - if (!params.host) { - throw new Error('Host is required for self-hosted deployments') - } - - return params.host.replace(/\/$/, '') -} - -function buildAuthHeaders(params: ElasticsearchCreateIndexParams): Record { - const headers: Record = { - 'Content-Type': 'application/json', - } - - if (params.authMethod === 'api_key' && params.apiKey) { - headers.Authorization = `ApiKey ${params.apiKey}` - } else if (params.authMethod === 'basic_auth' && params.username && params.password) { - const credentials = Buffer.from(`${params.username}:${params.password}`).toString('base64') - headers.Authorization = `Basic ${credentials}` - } else { - throw new Error('Invalid authentication configuration') - } - - return headers -} - export const createIndexTool: ToolConfig< ElasticsearchCreateIndexParams, ElasticsearchIndexResponse @@ -120,6 +80,7 @@ export const createIndexTool: ToolConfig< }, method: 'PUT', headers: (params) => buildAuthHeaders(params), + redirectPolicy: () => ({ mode: 'legacy', sendCredentialsOnCrossOriginRedirect: false }), body: (params) => { const body: Record = {} @@ -180,10 +141,12 @@ export const createIndexTool: ToolConfig< shards_acknowledged: { type: 'boolean', description: 'Whether the shards were acknowledged', + optional: true, }, index: { type: 'string', description: 'Created index name', + optional: true, }, }, } diff --git a/apps/sim/tools/elasticsearch/delete_document.ts b/apps/sim/tools/elasticsearch/delete_document.ts index da78582d032..0572fc714fa 100644 --- a/apps/sim/tools/elasticsearch/delete_document.ts +++ b/apps/sim/tools/elasticsearch/delete_document.ts @@ -2,49 +2,9 @@ import type { ElasticsearchDeleteDocumentParams, ElasticsearchDocumentResponse, } from '@/tools/elasticsearch/types' +import { buildAuthHeaders, buildBaseUrl } from '@/tools/elasticsearch/utils' import type { ToolConfig } from '@/tools/types' -function buildBaseUrl(params: ElasticsearchDeleteDocumentParams): string { - if (params.deploymentType === 'cloud' && params.cloudId) { - const parts = params.cloudId.split(':') - if (parts.length >= 2) { - try { - const decoded = Buffer.from(parts[1], 'base64').toString('utf-8') - const [esHost] = decoded.split('$') - if (esHost) { - return `https://${parts[0]}.${esHost}` - } - } catch { - // Fallback - } - } - throw new Error('Invalid Cloud ID format') - } - - if (!params.host) { - throw new Error('Host is required for self-hosted deployments') - } - - return params.host.replace(/\/$/, '') -} - -function buildAuthHeaders(params: ElasticsearchDeleteDocumentParams): Record { - const headers: Record = { - 'Content-Type': 'application/json', - } - - if (params.authMethod === 'api_key' && params.apiKey) { - headers.Authorization = `ApiKey ${params.apiKey}` - } else if (params.authMethod === 'basic_auth' && params.username && params.password) { - const credentials = Buffer.from(`${params.username}:${params.password}`).toString('base64') - headers.Authorization = `Basic ${credentials}` - } else { - throw new Error('Invalid authentication configuration') - } - - return headers -} - export const deleteDocumentTool: ToolConfig< ElasticsearchDeleteDocumentParams, ElasticsearchDocumentResponse @@ -127,6 +87,7 @@ export const deleteDocumentTool: ToolConfig< }, method: 'DELETE', headers: (params) => buildAuthHeaders(params), + redirectPolicy: () => ({ mode: 'legacy', sendCredentialsOnCrossOriginRedirect: false }), }, transformResponse: async (response: Response) => { @@ -182,6 +143,7 @@ export const deleteDocumentTool: ToolConfig< _version: { type: 'number', description: 'Document version', + optional: true, }, result: { type: 'string', diff --git a/apps/sim/tools/elasticsearch/delete_index.ts b/apps/sim/tools/elasticsearch/delete_index.ts index 73b2f82f5e4..62154f1df60 100644 --- a/apps/sim/tools/elasticsearch/delete_index.ts +++ b/apps/sim/tools/elasticsearch/delete_index.ts @@ -2,49 +2,9 @@ import type { ElasticsearchDeleteIndexParams, ElasticsearchIndexResponse, } from '@/tools/elasticsearch/types' +import { buildAuthHeaders, buildBaseUrl } from '@/tools/elasticsearch/utils' import type { ToolConfig } from '@/tools/types' -function buildBaseUrl(params: ElasticsearchDeleteIndexParams): string { - if (params.deploymentType === 'cloud' && params.cloudId) { - const parts = params.cloudId.split(':') - if (parts.length >= 2) { - try { - const decoded = Buffer.from(parts[1], 'base64').toString('utf-8') - const [esHost] = decoded.split('$') - if (esHost) { - return `https://${parts[0]}.${esHost}` - } - } catch { - // Fallback - } - } - throw new Error('Invalid Cloud ID format') - } - - if (!params.host) { - throw new Error('Host is required for self-hosted deployments') - } - - return params.host.replace(/\/$/, '') -} - -function buildAuthHeaders(params: ElasticsearchDeleteIndexParams): Record { - const headers: Record = { - 'Content-Type': 'application/json', - } - - if (params.authMethod === 'api_key' && params.apiKey) { - headers.Authorization = `ApiKey ${params.apiKey}` - } else if (params.authMethod === 'basic_auth' && params.username && params.password) { - const credentials = Buffer.from(`${params.username}:${params.password}`).toString('base64') - headers.Authorization = `Basic ${credentials}` - } else { - throw new Error('Invalid authentication configuration') - } - - return headers -} - export const deleteIndexTool: ToolConfig< ElasticsearchDeleteIndexParams, ElasticsearchIndexResponse @@ -110,6 +70,7 @@ export const deleteIndexTool: ToolConfig< }, method: 'DELETE', headers: (params) => buildAuthHeaders(params), + redirectPolicy: () => ({ mode: 'legacy', sendCredentialsOnCrossOriginRedirect: false }), }, transformResponse: async (response: Response) => { diff --git a/apps/sim/tools/elasticsearch/get_document.ts b/apps/sim/tools/elasticsearch/get_document.ts index e25b26b0541..6b62b51fd8b 100644 --- a/apps/sim/tools/elasticsearch/get_document.ts +++ b/apps/sim/tools/elasticsearch/get_document.ts @@ -2,49 +2,9 @@ import type { ElasticsearchDocumentResponse, ElasticsearchGetDocumentParams, } from '@/tools/elasticsearch/types' +import { buildAuthHeaders, buildBaseUrl } from '@/tools/elasticsearch/utils' import type { ToolConfig } from '@/tools/types' -function buildBaseUrl(params: ElasticsearchGetDocumentParams): string { - if (params.deploymentType === 'cloud' && params.cloudId) { - const parts = params.cloudId.split(':') - if (parts.length >= 2) { - try { - const decoded = Buffer.from(parts[1], 'base64').toString('utf-8') - const [esHost] = decoded.split('$') - if (esHost) { - return `https://${parts[0]}.${esHost}` - } - } catch { - // Fallback - } - } - throw new Error('Invalid Cloud ID format') - } - - if (!params.host) { - throw new Error('Host is required for self-hosted deployments') - } - - return params.host.replace(/\/$/, '') -} - -function buildAuthHeaders(params: ElasticsearchGetDocumentParams): Record { - const headers: Record = { - 'Content-Type': 'application/json', - } - - if (params.authMethod === 'api_key' && params.apiKey) { - headers.Authorization = `ApiKey ${params.apiKey}` - } else if (params.authMethod === 'basic_auth' && params.username && params.password) { - const credentials = Buffer.from(`${params.username}:${params.password}`).toString('base64') - headers.Authorization = `Basic ${credentials}` - } else { - throw new Error('Invalid authentication configuration') - } - - return headers -} - export const getDocumentTool: ToolConfig< ElasticsearchGetDocumentParams, ElasticsearchDocumentResponse @@ -139,6 +99,7 @@ export const getDocumentTool: ToolConfig< }, method: 'GET', headers: (params) => buildAuthHeaders(params), + redirectPolicy: () => ({ mode: 'legacy', sendCredentialsOnCrossOriginRedirect: false }), }, transformResponse: async (response: Response) => { @@ -195,6 +156,7 @@ export const getDocumentTool: ToolConfig< _version: { type: 'number', description: 'Document version', + optional: true, }, found: { type: 'boolean', @@ -203,6 +165,7 @@ export const getDocumentTool: ToolConfig< _source: { type: 'json', description: 'Document content', + optional: true, }, }, } diff --git a/apps/sim/tools/elasticsearch/get_index.ts b/apps/sim/tools/elasticsearch/get_index.ts index c268d99e78c..4b6ea6a0fca 100644 --- a/apps/sim/tools/elasticsearch/get_index.ts +++ b/apps/sim/tools/elasticsearch/get_index.ts @@ -2,49 +2,9 @@ import type { ElasticsearchGetIndexParams, ElasticsearchIndexInfoResponse, } from '@/tools/elasticsearch/types' +import { buildAuthHeaders, buildBaseUrl } from '@/tools/elasticsearch/utils' import type { ToolConfig } from '@/tools/types' -function buildBaseUrl(params: ElasticsearchGetIndexParams): string { - if (params.deploymentType === 'cloud' && params.cloudId) { - const parts = params.cloudId.split(':') - if (parts.length >= 2) { - try { - const decoded = Buffer.from(parts[1], 'base64').toString('utf-8') - const [esHost] = decoded.split('$') - if (esHost) { - return `https://${parts[0]}.${esHost}` - } - } catch { - // Fallback - } - } - throw new Error('Invalid Cloud ID format') - } - - if (!params.host) { - throw new Error('Host is required for self-hosted deployments') - } - - return params.host.replace(/\/$/, '') -} - -function buildAuthHeaders(params: ElasticsearchGetIndexParams): Record { - const headers: Record = { - 'Content-Type': 'application/json', - } - - if (params.authMethod === 'api_key' && params.apiKey) { - headers.Authorization = `ApiKey ${params.apiKey}` - } else if (params.authMethod === 'basic_auth' && params.username && params.password) { - const credentials = Buffer.from(`${params.username}:${params.password}`).toString('base64') - headers.Authorization = `Basic ${credentials}` - } else { - throw new Error('Invalid authentication configuration') - } - - return headers -} - export const getIndexTool: ToolConfig = { id: 'elasticsearch_get_index', @@ -108,6 +68,7 @@ export const getIndexTool: ToolConfig buildAuthHeaders(params), + redirectPolicy: () => ({ mode: 'legacy', sendCredentialsOnCrossOriginRedirect: false }), }, transformResponse: async (response: Response) => { @@ -122,7 +83,7 @@ export const getIndexTool: ToolConfig= 2) { - try { - const decoded = Buffer.from(parts[1], 'base64').toString('utf-8') - const [esHost] = decoded.split('$') - if (esHost) { - return `https://${parts[0]}.${esHost}` - } - } catch { - // Fallback - } - } - throw new Error('Invalid Cloud ID format') - } - - if (!params.host) { - throw new Error('Host is required for self-hosted deployments') - } - - return params.host.replace(/\/$/, '') -} - -function buildAuthHeaders(params: ElasticsearchIndexDocumentParams): Record { - const headers: Record = { - 'Content-Type': 'application/json', - } - - if (params.authMethod === 'api_key' && params.apiKey) { - headers.Authorization = `ApiKey ${params.apiKey}` - } else if (params.authMethod === 'basic_auth' && params.username && params.password) { - const credentials = Buffer.from(`${params.username}:${params.password}`).toString('base64') - headers.Authorization = `Basic ${credentials}` - } else { - throw new Error('Invalid authentication configuration') - } - - return headers -} - export const indexDocumentTool: ToolConfig< ElasticsearchIndexDocumentParams, ElasticsearchDocumentResponse @@ -133,6 +93,7 @@ export const indexDocumentTool: ToolConfig< }, method: (params) => (params.documentId ? 'PUT' : 'POST'), headers: (params) => buildAuthHeaders(params), + redirectPolicy: () => ({ mode: 'legacy', sendCredentialsOnCrossOriginRedirect: false }), body: (params) => { try { return JSON.parse(params.document) diff --git a/apps/sim/tools/elasticsearch/list_indices.ts b/apps/sim/tools/elasticsearch/list_indices.ts index aa1bb8569cd..72792adc694 100644 --- a/apps/sim/tools/elasticsearch/list_indices.ts +++ b/apps/sim/tools/elasticsearch/list_indices.ts @@ -2,57 +2,9 @@ import type { ElasticsearchListIndicesParams, ElasticsearchListIndicesResponse, } from '@/tools/elasticsearch/types' +import { buildAuthHeaders, buildBaseUrl } from '@/tools/elasticsearch/utils' import type { ToolConfig } from '@/tools/types' -/** - * Builds the base URL for Elasticsearch connections. - * Supports both self-hosted and Elastic Cloud deployments. - */ -function buildBaseUrl(params: ElasticsearchListIndicesParams): string { - if (params.deploymentType === 'cloud' && params.cloudId) { - const parts = params.cloudId.split(':') - if (parts.length >= 2) { - try { - const decoded = Buffer.from(parts[1], 'base64').toString('utf-8') - const [esHost] = decoded.split('$') - if (esHost) { - return `https://${parts[0]}.${esHost}` - } - } catch { - // Fallback - } - } - throw new Error('Invalid Cloud ID format') - } - - if (!params.host) { - throw new Error('Host is required for self-hosted deployments') - } - - return params.host.replace(/\/$/, '') -} - -/** - * Builds authentication headers for Elasticsearch requests. - * Supports API key and basic authentication methods. - */ -function buildAuthHeaders(params: ElasticsearchListIndicesParams): Record { - const headers: Record = { - 'Content-Type': 'application/json', - } - - if (params.authMethod === 'api_key' && params.apiKey) { - headers.Authorization = `ApiKey ${params.apiKey}` - } else if (params.authMethod === 'basic_auth' && params.username && params.password) { - const credentials = Buffer.from(`${params.username}:${params.password}`).toString('base64') - headers.Authorization = `Basic ${credentials}` - } else { - throw new Error('Invalid authentication configuration') - } - - return headers -} - export const listIndicesTool: ToolConfig< ElasticsearchListIndicesParams, ElasticsearchListIndicesResponse @@ -104,6 +56,12 @@ export const listIndicesTool: ToolConfig< visibility: 'user-only', description: 'Password for basic auth', }, + includeSystemIndices: { + type: 'boolean', + required: false, + description: + 'Include Elasticsearch system indices (names starting with "."). Omitted by default.', + }, }, request: { @@ -113,9 +71,10 @@ export const listIndicesTool: ToolConfig< }, method: 'GET', headers: (params) => buildAuthHeaders(params), + redirectPolicy: () => ({ mode: 'legacy', sendCredentialsOnCrossOriginRedirect: false }), }, - transformResponse: async (response: Response) => { + transformResponse: async (response: Response, params?: ElasticsearchListIndicesParams) => { if (!response.ok) { const errorText = await response.text() let errorMessage = `Elasticsearch error: ${response.status}` @@ -137,12 +96,14 @@ export const listIndicesTool: ToolConfig< const data = await response.json() - const indices = data - .filter((item: Record) => { - const indexName = item.index as string - return !indexName.startsWith('.') + const rows: Array> = Array.isArray(data) ? data : [] + + const indices = rows + .filter((item) => { + if (params?.includeSystemIndices) return true + return typeof item.index === 'string' ? !item.index.startsWith('.') : true }) - .map((item: Record) => ({ + .map((item) => ({ index: item.index as string, health: item.health as string, status: item.status as string, @@ -168,7 +129,8 @@ export const listIndicesTool: ToolConfig< }, indices: { type: 'json', - description: 'Array of index information objects', + description: + 'Array of index information objects (index, health, status, docsCount, storeSize, primaryShards, replicaShards). System indices are omitted unless includeSystemIndices is set.', }, }, } diff --git a/apps/sim/tools/elasticsearch/responses.test.ts b/apps/sim/tools/elasticsearch/responses.test.ts new file mode 100644 index 00000000000..967214c7cf3 --- /dev/null +++ b/apps/sim/tools/elasticsearch/responses.test.ts @@ -0,0 +1,142 @@ +/** + * @vitest-environment node + */ +import { describe, expect, it } from 'vitest' +import * as elasticsearchTools from '@/tools/elasticsearch' +import type { ToolConfig } from '@/tools/types' + +function jsonResponse(body: unknown): Response { + return new Response(JSON.stringify(body), { + status: 200, + headers: { 'Content-Type': 'application/json' }, + }) +} + +const GET_INDEX_BODY = { + 'logs-2024': { aliases: {}, mappings: { properties: {} }, settings: { index: {} } }, + 'logs-2025': { aliases: {}, mappings: { properties: {} }, settings: { index: {} } }, +} + +/** + * `GET /{index}` answers with a map keyed by index name. The tool previously + * declared an output called `index`, which appears nowhere in that body, so the + * whole payload was unreachable from the reference picker. + */ +describe('elasticsearch_get_index output shape', () => { + const tool = elasticsearchTools.elasticsearchGetIndexTool + + it('declares only outputs the transform actually produces', async () => { + const result = await tool.transformResponse!(jsonResponse(GET_INDEX_BODY)) + for (const declared of Object.keys(tool.outputs ?? {})) { + expect(Object.keys(result.output)).toContain(declared) + } + }) + + it('exposes every matched index under the declared aggregate', async () => { + const result = await tool.transformResponse!(jsonResponse(GET_INDEX_BODY)) + expect(Object.keys(result.output.indices as object)).toEqual(['logs-2024', 'logs-2025']) + }) + + it('keeps the raw per-index keys so references saved earlier still resolve', async () => { + const result = await tool.transformResponse!(jsonResponse(GET_INDEX_BODY)) + expect(result.output['logs-2024']).toEqual(GET_INDEX_BODY['logs-2024']) + }) + + it('lets an index literally named indices win its own key', async () => { + const body = { indices: { aliases: {}, mappings: {}, settings: {} } } + const result = await tool.transformResponse!(jsonResponse(body)) + expect(result.output.indices).toEqual(body.indices) + }) +}) + +describe('elasticsearch_list_indices filtering', () => { + const tool = elasticsearchTools.elasticsearchListIndicesTool + const rows = [ + { index: 'products', health: 'green', status: 'open', 'docs.count': '5', pri: '1', rep: '1' }, + { index: '.kibana', health: 'green', status: 'open', 'docs.count': '2', pri: '1', rep: '0' }, + ] + + it('omits system indices by default', async () => { + const result = await tool.transformResponse!(jsonResponse(rows), {} as never) + expect((result.output.indices as Array<{ index: string }>).map((i) => i.index)).toEqual([ + 'products', + ]) + }) + + it('includes them when the caller opts in', async () => { + const result = await tool.transformResponse!(jsonResponse(rows), { + includeSystemIndices: true, + } as never) + expect((result.output.indices as Array<{ index: string }>).map((i) => i.index)).toEqual([ + 'products', + '.kibana', + ]) + }) + + /** `item.index.startsWith` threw outright when a `_cat` row had no index column. */ + it('does not throw on a row with no index column', async () => { + const result = await tool.transformResponse!(jsonResponse([{ health: 'green' }]), {} as never) + expect(result.success).toBe(true) + expect(result.output.indices).toHaveLength(1) + }) + + it('does not throw when the body is not an array', async () => { + const result = await tool.transformResponse!(jsonResponse({ error: 'x' }), {} as never) + expect(result.output.indices).toEqual([]) + }) +}) + +/** + * `host` and a Cloud ID are user-supplied origins and every tool sends an + * `Authorization` header. `prepareToolRequest` only populates `redirectPolicy` + * from the tool, and the credential-stripping branch in + * `lib/core/security/input-validation.server.ts` is gated on that policy + * existing — so without one, a redirect carries the credential off-origin. + * + * `stripAuthOnRedirect` is deliberately not used: it drops `Authorization` on + * every hop including same-origin, which would break a reverse proxy in front + * of Elasticsearch issuing a legitimate same-origin redirect. + */ +describe('every Elasticsearch tool declines cross-origin credentials', () => { + const tools = Object.values(elasticsearchTools) as ToolConfig[] + + it('covers all thirteen tools', () => { + expect(tools).toHaveLength(13) + }) + + it.each(tools.map((tool) => [tool.id, tool] as const))('%s', (_id, tool) => { + expect(tool.request?.redirectPolicy?.({})).toEqual({ + mode: 'legacy', + sendCredentialsOnCrossOriginRedirect: false, + }) + expect(tool.request?.stripAuthOnRedirect).toBeUndefined() + }) +}) + +/** Outputs that are absent on a documented branch must be declared optional. */ +describe('conditionally-present outputs are declared optional', () => { + it.each([ + ['elasticsearchGetDocumentTool', ['_version', '_source']], + ['elasticsearchDeleteDocumentTool', ['_version']], + ['elasticsearchCreateIndexTool', ['shards_acknowledged', 'index']], + ['elasticsearchClusterStatsTool', ['status']], + ] as const)('%s', (name, fields) => { + const tool = (elasticsearchTools as Record)[name] + for (const field of fields) { + expect(tool.outputs?.[field]).toMatchObject({ optional: true }) + } + }) +}) + +/** The error branch must satisfy the same declared shape as the success branch. */ +describe('elasticsearch_get_index error branch', () => { + it('still exposes the declared aggregate on failure', async () => { + const tool = elasticsearchTools.elasticsearchGetIndexTool + const result = await tool.transformResponse!( + new Response(JSON.stringify({ error: { reason: 'no such index' } }), { status: 404 }) + ) + expect(result.success).toBe(false) + expect(result.error).toBe('no such index') + expect(result.output.indices).toEqual({}) + }) +}) diff --git a/apps/sim/tools/elasticsearch/search.ts b/apps/sim/tools/elasticsearch/search.ts index 7a0f7c595ef..9419c6a6152 100644 --- a/apps/sim/tools/elasticsearch/search.ts +++ b/apps/sim/tools/elasticsearch/search.ts @@ -2,54 +2,9 @@ import type { ElasticsearchSearchParams, ElasticsearchSearchResponse, } from '@/tools/elasticsearch/types' +import { buildAuthHeaders, buildBaseUrl } from '@/tools/elasticsearch/utils' import type { ToolConfig } from '@/tools/types' -// Helper to build base URL from connection params -function buildBaseUrl(params: ElasticsearchSearchParams): string { - if (params.deploymentType === 'cloud' && params.cloudId) { - // Parse Cloud ID: format is "name:base64data" - // The base64 data contains: es_host$kibana_host ($ separated) - const parts = params.cloudId.split(':') - if (parts.length >= 2) { - try { - const decoded = Buffer.from(parts[1], 'base64').toString('utf-8') - const [esHost] = decoded.split('$') - if (esHost) { - // Cloud endpoints are always HTTPS with port 443 - return `https://${parts[0]}.${esHost}` - } - } catch { - // If decoding fails, try using cloudId directly as host - } - } - throw new Error('Invalid Cloud ID format') - } - - if (!params.host) { - throw new Error('Host is required for self-hosted deployments') - } - - return params.host.replace(/\/$/, '') // Remove trailing slash -} - -// Helper to build auth headers -function buildAuthHeaders(params: ElasticsearchSearchParams): Record { - const headers: Record = { - 'Content-Type': 'application/json', - } - - if (params.authMethod === 'api_key' && params.apiKey) { - headers.Authorization = `ApiKey ${params.apiKey}` - } else if (params.authMethod === 'basic_auth' && params.username && params.password) { - const credentials = Buffer.from(`${params.username}:${params.password}`).toString('base64') - headers.Authorization = `Basic ${credentials}` - } else { - throw new Error('Invalid authentication configuration') - } - - return headers -} - export const searchTool: ToolConfig = { id: 'elasticsearch_search', name: 'Elasticsearch Search', @@ -154,6 +109,7 @@ export const searchTool: ToolConfig buildAuthHeaders(params), + redirectPolicy: () => ({ mode: 'legacy', sendCredentialsOnCrossOriginRedirect: false }), body: (params) => { const body: Record = {} diff --git a/apps/sim/tools/elasticsearch/types.ts b/apps/sim/tools/elasticsearch/types.ts index 280e5562efe..ac351b635a1 100644 --- a/apps/sim/tools/elasticsearch/types.ts +++ b/apps/sim/tools/elasticsearch/types.ts @@ -2,7 +2,7 @@ import type { ToolResponse } from '@/tools/types' // Base params for all Elasticsearch tools -interface ElasticsearchBaseParams { +export interface ElasticsearchBaseParams { // Connection configuration deploymentType: 'self_hosted' | 'cloud' host?: string // For self-hosted @@ -105,12 +105,20 @@ interface ElasticsearchGetMappingParams extends ElasticsearchBaseParams { // Cluster Operations export interface ElasticsearchClusterHealthParams extends ElasticsearchBaseParams { waitForStatus?: 'green' | 'yellow' | 'red' - timeout?: string + /** + * Server-side wait, sent as the `timeout` query parameter. Deliberately not + * named `timeout`: `tools/request-transport.ts` reads `params.timeout` as the + * outbound client abort deadline in milliseconds. + */ + clusterTimeout?: string } export interface ElasticsearchClusterStatsParams extends ElasticsearchBaseParams {} -export interface ElasticsearchListIndicesParams extends ElasticsearchBaseParams {} +export interface ElasticsearchListIndicesParams extends ElasticsearchBaseParams { + /** Include indices whose name starts with `.` (Elasticsearch system indices). */ + includeSystemIndices?: boolean +} interface ElasticsearchIndexInfo { index: string @@ -185,15 +193,23 @@ export interface ElasticsearchIndexResponse extends ToolResponse { } } +/** One entry of a `GET /{index}` response, keyed by index name. */ +interface ElasticsearchIndexState { + aliases?: Record + mappings?: Record + settings?: Record +} + +/** + * `indices` is the declared aggregate map of every matched index. The raw + * per-index keys are spread alongside it so references saved before `indices` + * existed keep resolving; an index legitimately named `indices` is spread last + * and therefore wins, which is what widens the declared type here. + */ export interface ElasticsearchIndexInfoResponse extends ToolResponse { - output: Record< - string, - { - aliases: Record - mappings: Record - settings: Record - } - > + output: Record> & { + indices: ElasticsearchIndexState | Record + } } interface ElasticsearchIndexExistsResponse extends ToolResponse { diff --git a/apps/sim/tools/elasticsearch/update_document.ts b/apps/sim/tools/elasticsearch/update_document.ts index e337b175af5..5d90adeb996 100644 --- a/apps/sim/tools/elasticsearch/update_document.ts +++ b/apps/sim/tools/elasticsearch/update_document.ts @@ -2,49 +2,9 @@ import type { ElasticsearchDocumentResponse, ElasticsearchUpdateDocumentParams, } from '@/tools/elasticsearch/types' +import { buildAuthHeaders, buildBaseUrl } from '@/tools/elasticsearch/utils' import type { ToolConfig } from '@/tools/types' -function buildBaseUrl(params: ElasticsearchUpdateDocumentParams): string { - if (params.deploymentType === 'cloud' && params.cloudId) { - const parts = params.cloudId.split(':') - if (parts.length >= 2) { - try { - const decoded = Buffer.from(parts[1], 'base64').toString('utf-8') - const [esHost] = decoded.split('$') - if (esHost) { - return `https://${parts[0]}.${esHost}` - } - } catch { - // Fallback - } - } - throw new Error('Invalid Cloud ID format') - } - - if (!params.host) { - throw new Error('Host is required for self-hosted deployments') - } - - return params.host.replace(/\/$/, '') -} - -function buildAuthHeaders(params: ElasticsearchUpdateDocumentParams): Record { - const headers: Record = { - 'Content-Type': 'application/json', - } - - if (params.authMethod === 'api_key' && params.apiKey) { - headers.Authorization = `ApiKey ${params.apiKey}` - } else if (params.authMethod === 'basic_auth' && params.username && params.password) { - const credentials = Buffer.from(`${params.username}:${params.password}`).toString('base64') - headers.Authorization = `Basic ${credentials}` - } else { - throw new Error('Invalid authentication configuration') - } - - return headers -} - export const updateDocumentTool: ToolConfig< ElasticsearchUpdateDocumentParams, ElasticsearchDocumentResponse @@ -132,6 +92,7 @@ export const updateDocumentTool: ToolConfig< }, method: 'POST', headers: (params) => buildAuthHeaders(params), + redirectPolicy: () => ({ mode: 'legacy', sendCredentialsOnCrossOriginRedirect: false }), body: (params) => { try { return { doc: JSON.parse(params.document) } diff --git a/apps/sim/tools/elasticsearch/utils.test.ts b/apps/sim/tools/elasticsearch/utils.test.ts new file mode 100644 index 00000000000..f90dae92d3d --- /dev/null +++ b/apps/sim/tools/elasticsearch/utils.test.ts @@ -0,0 +1,216 @@ +/** + * @vitest-environment node + */ +import { describe, expect, it } from 'vitest' +import * as elasticsearchTools from '@/tools/elasticsearch' +import { buildBaseUrl, parseCloudId } from '@/tools/elasticsearch/utils' +import { prepareToolRequest } from '@/tools/request-transport' +import type { ToolConfig } from '@/tools/types' + +function cloudId(payload: string, label = 'my-deployment'): string { + return `${label}:${Buffer.from(payload).toString('base64')}` +} + +const PARENT_DOMAIN = 'us-east-1.aws.found.io' +const ES_UUID = 'a1b2c3d4e5f60718293a4b5c6d7e8f90' +const KIBANA_UUID = '0f9e8d7c6b5a4938271605f4e3d2c1b0' + +describe('parseCloudId', () => { + it('resolves the Elasticsearch UUID as the host, not the deployment label', () => { + const url = parseCloudId(cloudId(`${PARENT_DOMAIN}$${ES_UUID}$${KIBANA_UUID}`)) + expect(url).toBe(`https://${ES_UUID}.${PARENT_DOMAIN}`) + expect(url).not.toContain('my-deployment') + }) + + it('ignores a colon inside the deployment label by splitting at the last colon', () => { + const url = parseCloudId(cloudId(`${PARENT_DOMAIN}$${ES_UUID}$${KIBANA_UUID}`, 'eu:prod')) + expect(url).toBe(`https://${ES_UUID}.${PARENT_DOMAIN}`) + }) + + it('applies a per-service port taken from the last colon of the component', () => { + const url = parseCloudId(cloudId(`${PARENT_DOMAIN}$${ES_UUID}:9243$${KIBANA_UUID}`)) + expect(url).toBe(`https://${ES_UUID}.${PARENT_DOMAIN}:9243`) + }) + + it('inherits the parent domain port when the service component has none', () => { + const url = parseCloudId(cloudId(`${PARENT_DOMAIN}:9243$${ES_UUID}$${KIBANA_UUID}`)) + expect(url).toBe(`https://${ES_UUID}.${PARENT_DOMAIN}:9243`) + }) + + it('omits an explicit :443, which is the Elastic Cloud default', () => { + const url = parseCloudId(cloudId(`${PARENT_DOMAIN}:443$${ES_UUID}$${KIBANA_UUID}`)) + expect(url).toBe(`https://${ES_UUID}.${PARENT_DOMAIN}`) + }) + + it('rejects an @ in the Elasticsearch component that would redirect credentials', () => { + const hostile = cloudId(`${PARENT_DOMAIN}$${ES_UUID}@evil.example.com$${KIBANA_UUID}`) + expect(() => parseCloudId(hostile)).toThrow(/Invalid Cloud ID/) + }) + + it.each(['#', '?', '/', '\\'])('rejects %s in the parent domain component', (character) => { + const hostile = cloudId(`${PARENT_DOMAIN}${character}x$${ES_UUID}$${KIBANA_UUID}`) + expect(() => parseCloudId(hostile)).toThrow(/Invalid Cloud ID/) + }) + + it('rejects a non-numeric port that would smuggle a host into the authority', () => { + const hostile = cloudId(`${PARENT_DOMAIN}$${ES_UUID}:80@evil.example.com$${KIBANA_UUID}`) + expect(() => parseCloudId(hostile)).toThrow(/Invalid Cloud ID/) + }) + + it('rejects a backslash, which the URL parser treats as an authority terminator', () => { + const hostile = cloudId(`${PARENT_DOMAIN}$${ES_UUID}\\evil.example.com$${KIBANA_UUID}`) + expect(() => parseCloudId(hostile)).toThrow(/Invalid Cloud ID/) + }) + + it('rejects a malformed parent-domain port even when the service port is valid', () => { + const hostile = cloudId(`${PARENT_DOMAIN}:abc$${ES_UUID}:9243$${KIBANA_UUID}`) + expect(() => parseCloudId(hostile)).toThrow(/Invalid Cloud ID/) + }) + + /** + * `extractPortFromName` splits at the last colon, so a second colon survives + * in the *name* half. Both halves are numeric here on purpose: the all-digits + * port check passes, so only the reject set can catch it. Left unchecked this + * assembles `https://.:9243:5`, which surfaces as a bare + * `TypeError: Invalid URL` inside the transport instead of naming the real + * problem. + */ + it('rejects a second colon in the parent domain when both halves are numeric', () => { + const hostile = cloudId(`${PARENT_DOMAIN}:9243:5$${ES_UUID}$${KIBANA_UUID}`) + expect(() => parseCloudId(hostile)).toThrow(/Invalid Cloud ID/) + }) + + it('rejects a second colon in the parent domain when the service port is explicit', () => { + const hostile = cloudId(`${PARENT_DOMAIN}:9243:5$${ES_UUID}:9200$${KIBANA_UUID}`) + expect(() => parseCloudId(hostile)).toThrow(/Invalid Cloud ID/) + }) + + it('rejects a second colon in the Elasticsearch component', () => { + const hostile = cloudId(`${PARENT_DOMAIN}$${ES_UUID}:9243:5$${KIBANA_UUID}`) + expect(() => parseCloudId(hostile)).toThrow(/Invalid Cloud ID/) + }) + + /** + * Distinct from the cases above: here the trailing half is non-numeric, so + * the all-digits port check rejects it before the reject set is reached. + * Kept separate so neither check can silently stop carrying its own case. + */ + it('rejects a second colon whose trailing half is not a port', () => { + const hostile = cloudId(`${PARENT_DOMAIN}:9243:evil@x$${ES_UUID}:9243$${KIBANA_UUID}`) + expect(() => parseCloudId(hostile)).toThrow(/Invalid Cloud ID/) + }) + + it('rejects a payload with fewer than three $-separated components', () => { + expect(() => parseCloudId(cloudId(`${PARENT_DOMAIN}$${ES_UUID}`))).toThrow(/Invalid Cloud ID/) + }) + + it('rejects an empty Elasticsearch component', () => { + expect(() => parseCloudId(cloudId(`${PARENT_DOMAIN}$$${KIBANA_UUID}`))).toThrow( + /Invalid Cloud ID/ + ) + }) +}) + +describe('buildBaseUrl', () => { + it('strips a trailing slash from a self-hosted host', () => { + expect( + buildBaseUrl({ + deploymentType: 'self_hosted', + host: 'https://es.example.com/', + authMethod: 'api_key', + }) + ).toBe('https://es.example.com') + }) + + /** + * The block marks `cloudId` required and hides `host` when cloud is selected, + * but a user who switches the dropdown keeps their old `host` in saved state. + * Falling back to it would send the cloud credential to the previous cluster. + */ + it('never falls back to host for a cloud deployment missing its Cloud ID', () => { + expect(() => + buildBaseUrl({ + deploymentType: 'cloud', + host: 'https://stale-self-hosted.example.com', + authMethod: 'api_key', + }) + ).toThrow(/Cloud ID is required/) + }) + + /** + * `deploymentType` resolves to `user-or-llm`, so a model on the agent + * tool-calling path can emit a near miss for `cloud`. Treating every + * non-`cloud` value as self-hosted routes the credential to the stale host, + * which is the same disclosure the branch above exists to prevent. + */ + it.each(['Cloud', 'CLOUD', 'elastic_cloud', 'cloud ', 'elasticCloud', ''])( + 'rejects %o rather than routing it to the stale self-hosted host', + (deploymentType) => { + expect(() => + buildBaseUrl({ + deploymentType: deploymentType as unknown as 'cloud', + host: 'https://stale-self-hosted.example.com', + cloudId: cloudId(`${PARENT_DOMAIN}$${ES_UUID}$${KIBANA_UUID}`), + authMethod: 'api_key', + }) + ).toThrow(/Unsupported deployment type/) + } + ) + + /** Legacy saved state can carry no value at all; that still means self-hosted. */ + it('treats a nullish deployment type as self-hosted', () => { + expect( + buildBaseUrl({ + deploymentType: undefined as unknown as 'self_hosted', + host: 'https://es.example.com/', + authMethod: 'api_key', + }) + ).toBe('https://es.example.com') + }) + + it('requires a host when self-hosted', () => { + expect(() => buildBaseUrl({ deploymentType: 'self_hosted', authMethod: 'api_key' })).toThrow( + /Host is required/ + ) + }) +}) + +describe('every Elasticsearch tool resolves the same cloud host', () => { + const tools = Object.values(elasticsearchTools) as ToolConfig[] + const params = { + deploymentType: 'cloud', + cloudId: cloudId(`${PARENT_DOMAIN}$${ES_UUID}$${KIBANA_UUID}`), + authMethod: 'api_key', + apiKey: 'test-key', + index: 'products', + documentId: 'doc-1', + document: '{}', + operations: '{}', + } + + it('covers all thirteen tools', () => { + expect(tools).toHaveLength(13) + }) + + it.each(tools.map((tool) => [tool.id, tool] as const))('%s', (_id, tool) => { + const url = typeof tool.request.url === 'function' ? tool.request.url(params) : tool.request.url + expect(new URL(url).host).toBe(`${ES_UUID}.${PARENT_DOMAIN}`) + }) +}) + +/** + * `_bulk` answers `application/json` with HTTP 406, so it is the one tool that + * must override the shared header builder's default media type. + */ +describe('elasticsearch_bulk wire format', () => { + it('sends the bulk content type Elasticsearch requires', () => { + const prepared = prepareToolRequest(elasticsearchTools.elasticsearchBulkTool as ToolConfig, { + deploymentType: 'self_hosted', + host: 'https://es.example.com', + authMethod: 'api_key', + apiKey: 'test-key', + operations: '{"index":{"_index":"products","_id":"1"}}\n{"name":"Widget"}', + }) + expect(prepared.headers.get('content-type')).toBe('application/x-ndjson') + }) +}) diff --git a/apps/sim/tools/elasticsearch/utils.ts b/apps/sim/tools/elasticsearch/utils.ts new file mode 100644 index 00000000000..77ecad69366 --- /dev/null +++ b/apps/sim/tools/elasticsearch/utils.ts @@ -0,0 +1,148 @@ +import type { ElasticsearchBaseParams } from '@/tools/elasticsearch/types' + +/** + * Default port for Elastic Cloud endpoints, matching `defaultCloudPort` in + * Beats' `libbeat/cloudid/cloudid.go`. + */ +const DEFAULT_CLOUD_PORT = '443' + +/** + * Characters that must not appear in a decoded Cloud ID component name. An `@` + * would turn the rest of the authority into a host and send the credential + * headers to an attacker-controlled origin; `#`, `?` and `/` truncate the + * authority. Mirrors the `strings.IndexAny(component, "#@?/")` reject set in + * Beats, plus two additions: + * + * - `\`, which the WHATWG URL parser treats as a path separator for special + * schemes and which therefore truncates the authority exactly as `/` does. + * - `:`, because `extractPortFromName` has already split the component at its + * last colon, so a colon surviving in the *name* half means the component + * carried two. The all-digits port check below does not catch that when the + * trailing half is numeric: `found.io:9243:5` yields name `found.io:9243` + * and port `5`, which assembles `https://.found.io:9243:5` and fails + * as a bare `TypeError: Invalid URL` inside the transport. A hostname cannot + * contain a colon, so rejecting it here cannot refuse a legitimate ID. + */ +const CLOUD_ID_REJECTED_CHARACTERS = /[#@?/\\:]/ + +/** + * Splits a Cloud ID component of the form `name:port` at its last colon. + * Mirrors `extractPortFromName` in Beats' `libbeat/cloudid/cloudid.go`. + */ +function extractPortFromName(word: string, defaultPort: string): { name: string; port: string } { + const index = word.lastIndexOf(':') + if (index < 0) return { name: word, port: defaultPort } + return { name: word.slice(0, index), port: word.slice(index + 1) } +} + +/** + * Decodes an Elastic Cloud ID into the Elasticsearch endpoint it addresses. + * + * A Cloud ID is `:`. + * The reachable host is `.` — the deployment label is a + * human-readable name that resolves to nothing. + * + * @throws when the Cloud ID is malformed or contains an unsafe component. + */ +export function parseCloudId(cloudId: string): string { + const separatorIndex = cloudId.lastIndexOf(':') + const encoded = separatorIndex >= 0 ? cloudId.slice(separatorIndex + 1) : cloudId + + const decoded = Buffer.from(encoded, 'base64').toString('utf-8') + + const words = decoded.split('$') + if (words.length < 3) { + throw new Error('Invalid Cloud ID format') + } + + const parentDomain = extractPortFromName(words[0], DEFAULT_CLOUD_PORT) + const elasticsearch = extractPortFromName(words[1], parentDomain.port) + + if (!parentDomain.name || !elasticsearch.name) { + throw new Error('Invalid Cloud ID format') + } + + for (const component of [parentDomain.name, elasticsearch.name]) { + if (CLOUD_ID_REJECTED_CHARACTERS.test(component)) { + throw new Error('Invalid Cloud ID format') + } + } + + for (const port of [parentDomain.port, elasticsearch.port]) { + if (!/^\d+$/.test(port)) { + throw new Error('Invalid Cloud ID format') + } + } + + const host = `${elasticsearch.name}.${parentDomain.name}` + return elasticsearch.port === DEFAULT_CLOUD_PORT + ? `https://${host}` + : `https://${host}:${elasticsearch.port}` +} + +/** + * Resolves the Elasticsearch base URL for a tool invocation, from either an + * Elastic Cloud ID or a self-hosted host URL. + * + * The deployment type alone selects the branch. A cloud invocation must never + * fall back to `host`: the block hides `host` when cloud is selected but keeps + * its previous value in saved state, so a fallback sends the cloud credential + * to whatever cluster the user was pointed at before they switched. + * + * An unrecognized deployment type is rejected rather than treated as + * self-hosted, because that fallthrough is the same disclosure by another + * route. `deploymentType` is `required: true` with no explicit `visibility`, + * which `tools/params.ts` resolves to `user-or-llm`, so a model supplies it on + * the agent tool-calling path; a near miss such as `Cloud` is not `=== 'cloud'` + * and would otherwise select the self-hosted branch. Only a nullish value still + * means self-hosted — that is the dropdown's own default (`value: () => + * 'self_hosted'`) and the shape of state saved before the field was touched. + */ +export function buildBaseUrl(params: ElasticsearchBaseParams): string { + if (params.deploymentType === 'cloud') { + if (!params.cloudId) { + throw new Error('Cloud ID is required for cloud deployments') + } + return parseCloudId(params.cloudId) + } + + if (params.deploymentType != null && params.deploymentType !== 'self_hosted') { + throw new Error( + `Unsupported deployment type "${params.deploymentType}". Expected "self_hosted" or "cloud".` + ) + } + + if (!params.host) { + throw new Error('Host is required for self-hosted deployments') + } + + return params.host.replace(/\/$/, '') +} + +/** + * Builds the content-type and authorization headers shared by every + * Elasticsearch tool. + * + * @param contentType overrides the default JSON media type. The `_bulk` + * endpoint requires `application/x-ndjson` and answers `application/json` with + * HTTP 406, so that tool must pass its own. + */ +export function buildAuthHeaders( + params: ElasticsearchBaseParams, + contentType = 'application/json' +): Record { + const headers: Record = { + 'Content-Type': contentType, + } + + if (params.authMethod === 'api_key' && params.apiKey) { + headers.Authorization = `ApiKey ${params.apiKey}` + } else if (params.authMethod === 'basic_auth' && params.username && params.password) { + const credentials = Buffer.from(`${params.username}:${params.password}`).toString('base64') + headers.Authorization = `Basic ${credentials}` + } else { + throw new Error('Invalid authentication configuration') + } + + return headers +} diff --git a/apps/sim/tools/file/index.ts b/apps/sim/tools/file/index.ts index feda5c045db..0e53a887898 100644 --- a/apps/sim/tools/file/index.ts +++ b/apps/sim/tools/file/index.ts @@ -9,6 +9,7 @@ export { fileAppendTool } from '@/tools/file/append' export { fileCompressTool, fileDecompressTool } from '@/tools/file/compress' export { fileGetContentTool, fileGetTool, fileReadTool } from '@/tools/file/get' export { fileManageSharingTool } from '@/tools/file/manage-sharing' +export { fileSearchTool } from '@/tools/file/search' export { fileWriteTool } from '@/tools/file/write' export const fileParseTool = fileParserTool diff --git a/apps/sim/tools/file/search.test.ts b/apps/sim/tools/file/search.test.ts new file mode 100644 index 00000000000..5e8718a3e84 --- /dev/null +++ b/apps/sim/tools/file/search.test.ts @@ -0,0 +1,84 @@ +import { describe, expect, it } from 'vitest' +import { fileOperations } from '@/lib/workspace-files/application/operations' +import { fileSearchTool } from '@/tools/file/search' + +describe('fileSearchTool', () => { + it('uses the shared protected read operation and admits executor delegation', () => { + expect(fileOperations.searchContent).toMatchObject({ + id: 'files.search_content', + minimumRole: 'read', + workspaceApiKey: 'allow', + delegatedServices: ['copilot', 'executor'], + }) + }) + + it('keeps the query model-visible and the configured limit user-only', () => { + expect(fileSearchTool.params.query).toMatchObject({ + required: true, + visibility: 'user-or-llm', + }) + expect(fileSearchTool.params.maxResults).toMatchObject({ + required: false, + visibility: 'user-only', + }) + }) + + it('requests fail-closed secret provenance for returned excerpts', () => { + expect(fileSearchTool.operation.secretProvenance?.response).toEqual({ + incomplete: 'reject', + }) + }) + + it('defaults the hard cap to 50 without coercing model parameters during serialization', () => { + expect(fileSearchTool.operation.input({ query: 'needle' })).toEqual({ + query: 'needle', + maxResults: 50, + }) + }) + + it('describes structured results and index coverage counters', () => { + expect(fileSearchTool.outputs.results).toMatchObject({ + type: 'array', + items: { + type: 'object', + properties: { + fileId: { type: 'string' }, + lineNumber: { type: 'number' }, + text: { type: 'string' }, + }, + }, + }) + expect(fileSearchTool.outputs.indexStatus).toMatchObject({ + properties: { + readyFiles: { type: 'number' }, + pendingFiles: { type: 'number' }, + failedFiles: { type: 'number' }, + skippedFiles: { type: 'number' }, + partialFiles: { type: 'number' }, + }, + }) + }) + + it('returns structured result objects to the workflow', async () => { + const result = await fileSearchTool.transformResponse( + Response.json({ + success: true, + data: { + results: [{ fileId: 'file-1', lineNumber: 2, text: 'needle' }], + count: 1, + truncated: false, + complete: true, + indexStatus: { + readyFiles: 1, + pendingFiles: 0, + failedFiles: 0, + skippedFiles: 0, + partialFiles: 0, + }, + }, + }) + ) + + expect(result.output.results).toEqual([{ fileId: 'file-1', lineNumber: 2, text: 'needle' }]) + }) +}) diff --git a/apps/sim/tools/file/search.ts b/apps/sim/tools/file/search.ts new file mode 100644 index 00000000000..ff111593e88 --- /dev/null +++ b/apps/sim/tools/file/search.ts @@ -0,0 +1,109 @@ +import { FILE_SEARCH_DEFAULT_MAX_RESULTS } from '@/lib/workspace-files/search/constants' +import type { FileSearchOutput } from '@/tools/file/types' +import type { InternalToolConfig, ToolResponse } from '@/tools/types' + +interface FileSearchParams { + query: string + maxResults?: number +} + +interface FileSearchResponse extends ToolResponse { + output: FileSearchOutput +} + +export const fileSearchTool: InternalToolConfig = { + id: 'file_search', + name: 'File Search', + description: + 'Search indexed text across active workspace files using literal smart-case substring matching.', + version: '1.0.0', + params: { + query: { + type: 'string', + required: true, + visibility: 'user-or-llm', + description: + 'Literal text to find (3-512 characters). Uppercase Unicode letters make matching case-sensitive.', + }, + maxResults: { + type: 'number', + required: false, + visibility: 'user-only', + description: 'Hard result cap configured by the workflow builder (1-200, default 50).', + }, + }, + operation: { + secretProvenance: { response: { incomplete: 'reject' } }, + input: (params) => ({ + query: params.query, + maxResults: params.maxResults ?? FILE_SEARCH_DEFAULT_MAX_RESULTS, + }), + }, + transformResponse: async (response): Promise => { + const body = await response.json() + if (!response.ok || !body.success) { + return { + success: false, + output: { + results: [], + count: 0, + truncated: false, + complete: false, + indexStatus: { + readyFiles: 0, + pendingFiles: 0, + failedFiles: 0, + skippedFiles: 0, + partialFiles: 0, + }, + }, + error: body.error || 'Failed to search workspace files', + } + } + return { success: true, output: body.data } + }, + outputs: { + results: { + type: 'array', + description: 'Matching logical lines with their workspace file ID and 1-based line number.', + items: { + type: 'object', + properties: { + fileId: { type: 'string', description: 'Canonical workspace file ID.' }, + lineNumber: { type: 'number', description: '1-based logical line number.' }, + text: { type: 'string', description: 'Matching line or bounded match-centered preview.' }, + }, + }, + }, + count: { type: 'number', description: 'Number of returned matching lines.' }, + truncated: { + type: 'boolean', + description: 'Whether more matching lines exist beyond the configured hard cap.', + }, + complete: { + type: 'boolean', + description: + 'Whether indexing has no pending or failed current revisions; skipped and partial coverage is reported separately.', + }, + indexStatus: { + type: 'object', + description: 'Current workspace search-index coverage by file status.', + properties: { + readyFiles: { type: 'number', description: 'Files whose current revision is searchable.' }, + pendingFiles: { type: 'number', description: 'Files still waiting to be indexed.' }, + failedFiles: { + type: 'number', + description: 'Files whose current indexing attempt failed.', + }, + skippedFiles: { + type: 'number', + description: 'Files intentionally excluded because they are unsupported or oversized.', + }, + partialFiles: { + type: 'number', + description: 'Searchable files whose extracted text was truncated by the parser or cap.', + }, + }, + }, + }, +} diff --git a/apps/sim/tools/file/types.ts b/apps/sim/tools/file/types.ts index 1d5e773a061..f9d3a466cc5 100644 --- a/apps/sim/tools/file/types.ts +++ b/apps/sim/tools/file/types.ts @@ -1,6 +1,24 @@ import type { UserFile } from '@/executor/types' import type { TableRow, ToolResponse } from '@/tools/types' +export interface FileSearchOutput { + results: Array<{ + fileId: string + lineNumber: number + text: string + }> + count: number + truncated: boolean + complete: boolean + indexStatus: { + readyFiles: number + pendingFiles: number + failedFiles: number + skippedFiles: number + partialFiles: number + } +} + export interface FileParserInput { filePath?: string | string[] file?: UserFile | UserFile[] | FileUploadInput | FileUploadInput[] diff --git a/apps/sim/tools/file/write.ts b/apps/sim/tools/file/write.ts index 25c779b07b8..8f1f6227443 100644 --- a/apps/sim/tools/file/write.ts +++ b/apps/sim/tools/file/write.ts @@ -1,9 +1,11 @@ import type { InternalToolConfig, ToolResponse } from '@/tools/types' interface FileWriteParams { - fileName: string - content: string + fileName?: string + content?: string + fileInput?: unknown contentType?: string + overwrite?: boolean workspaceId?: string } @@ -11,29 +13,44 @@ export const fileWriteTool: InternalToolConfig = id: 'file_write', name: 'File Write', description: - 'Create a new workspace file. If a file with the same name already exists, a numeric suffix is added (e.g., "data (1).csv").', + 'Create a new workspace file, either from text content or from an existing file. If a file with the same name already exists, a numeric suffix is added (e.g., "data (1).csv") unless overwrite is enabled.', version: '1.0.0', params: { fileName: { type: 'string', - required: true, + required: false, visibility: 'user-or-llm', description: - 'File name (e.g., "data.csv"). If a file with this name exists, a numeric suffix is added automatically.', + 'File name (e.g., "data.csv"). Required when writing text; optional when storing a file, which keeps its own name unless this overrides it. If the name already exists, a numeric suffix is added automatically unless overwrite is enabled.', }, content: { type: 'string', - required: true, + required: false, visibility: 'user-or-llm', - description: 'The text content to write to the file.', + description: + 'The text content to write to the file. Provide exactly one of content or fileInput.', + }, + fileInput: { + type: 'file', + required: false, + visibility: 'user-or-llm', + description: + 'An existing file to store in the workspace, such as one produced by an earlier tool. Use this for anything that is not text — PDFs, images, audio, archives. Provide exactly one of content or fileInput.', }, contentType: { type: 'string', required: false, visibility: 'user-only', description: - 'MIME type for new files (e.g., "text/plain"). Auto-detected from file extension if omitted.', + 'MIME type for new files (e.g., "text/plain"). Auto-detected from the file extension, or taken from the stored file, if omitted.', + }, + overwrite: { + type: 'boolean', + required: false, + visibility: 'user-only', + description: + 'Replace the contents of an existing file at the exact target path (folder and name) instead of creating a suffixed copy. Creates the file when that path does not exist yet.', }, }, @@ -42,10 +59,15 @@ export const fileWriteTool: InternalToolConfig = operation: 'write', fileName: params.fileName, content: params.content, + fileInput: params.fileInput, contentType: params.contentType, + overwrite: params.overwrite, workspaceId: params.workspaceId, }), secretProvenance: { + // Only the text branch carries caller-authored content. A stored file's + // bytes come from an already-tracked object, whose own provenance follows + // it rather than being re-derived from this request. request: () => [{ key: 'content', inputPaths: [['content']] }], }, }, diff --git a/apps/sim/tools/function/execute.test.ts b/apps/sim/tools/function/execute.test.ts index 053ce3c3372..f4dc0573265 100644 --- a/apps/sim/tools/function/execute.test.ts +++ b/apps/sim/tools/function/execute.test.ts @@ -120,4 +120,41 @@ describe('Function Execute Tool', () => { expect(body[PRIVATE_SECRET_PROVENANCE_FIELD]).toEqual(bundle) expect(JSON.stringify(body)).not.toContain('plaintext') }) + + it('preserves sandbox cost in a successful Function result', async () => { + const cost = { input: 0, output: 0, total: 0.00012345 } + const result = await functionExecuteTool.transformResponse?.( + Response.json({ + success: true, + output: { result: 42, stdout: 'done', cost }, + }), + { code: 'return 42' } + ) + + expect(result).toMatchObject({ + success: true, + output: { result: 42, stdout: 'done', cost }, + }) + }) + + it('preserves sandbox cost in a failed Function result', async () => { + const cost = { input: 0, output: 0, total: 0.00012345 } + const result = await functionExecuteTool.transformResponse?.( + Response.json( + { + success: false, + error: 'boom', + output: { result: null, stdout: 'trace', cost }, + }, + { status: 422 } + ), + { code: 'throw new Error("boom")' } + ) + + expect(result).toMatchObject({ + success: false, + output: { result: null, stdout: 'trace', cost }, + error: 'boom', + }) + }) }) diff --git a/apps/sim/tools/function/execute.ts b/apps/sim/tools/function/execute.ts index de9b27a3121..ab2cde32a1d 100644 --- a/apps/sim/tools/function/execute.ts +++ b/apps/sim/tools/function/execute.ts @@ -5,12 +5,44 @@ import { normalizeStringRecord, normalizeWorkflowVariables, } from '@/lib/core/utils/records' +import { isUserFileWithMetadata } from '@/lib/core/utils/user-file' import { DEFAULT_EXECUTION_TIMEOUT_MS } from '@/lib/execution/constants' import { DEFAULT_CODE_LANGUAGE } from '@/lib/execution/languages' import { PRIVATE_SECRET_PROVENANCE_FIELD } from '@/lib/execution/private-tool-metadata' +import { SANDBOX_INPUT_DIR, SANDBOX_OUTPUT_DIR } from '@/lib/execution/remote-sandbox/sandbox-paths' +import type { UserFile } from '@/executor/types' import type { CodeExecutionInput, CodeExecutionOutput } from '@/tools/function/types' import type { InternalToolConfig } from '@/tools/types' +/** + * Normalizes the mounted-file param, which advanced-mode template resolution + * delivers as a JSON string rather than an array. + * + * Deliberately not `normalizeFileInput` from `@/blocks/utils`: that module + * reaches the providers store and Sim's icon set, so importing it here would + * drag React and zustand into the tool registry's module graph. + */ +function normalizeSandboxInputFiles(value: unknown): FunctionExecuteBody['files'] { + if (!value) return undefined + + let parsed = value + if (typeof parsed === 'string') { + try { + parsed = JSON.parse(parsed) + } catch { + return undefined + } + } + + const files = Array.isArray(parsed) ? parsed : [parsed] + const userFiles = files.filter((file): file is UserFile => isUserFileWithMetadata(file)) + if (userFiles.length === 0) return undefined + // Copied onto fresh objects because the boundary schema is `.passthrough()`: + // its inferred type carries an index signature, which a declared interface + // like UserFile cannot satisfy directly. + return userFiles.map((file) => ({ ...file })) +} + /** Builds the canonical Function protocol body for both HTTP compatibility and in-process calls. */ export function buildFunctionExecuteBody(params: CodeExecutionInput): FunctionExecuteBody { const codeContent = Array.isArray(params.code) @@ -35,6 +67,7 @@ export function buildFunctionExecuteBody(params: CodeExecutionInput): FunctionEx overwriteFileId: params.overwriteFileId, inputs: params.inputs, outputs: params.outputs, + files: normalizeSandboxInputFiles(params.files), envVars: normalizeStringRecord(params.envVars), workflowVariables: normalizeWorkflowVariables(params.workflowVariables), blockData: normalizeRecord(params.blockData), @@ -60,8 +93,9 @@ export function buildFunctionExecuteBody(params: CodeExecutionInput): FunctionEx export const functionExecuteTool: InternalToolConfig = { id: 'function_execute', name: 'Function Execute', - description: - 'Execute JavaScript, Python, or shell scripts in a secure sandbox. For JS: fetch() is available, code runs in an async IIFE wrapper. Shell includes general utilities such as jq, curl, git, and rg. Use outputPath/outputTable to persist returned data, or outputSandboxPath + outputPath to export a file created inside the sandbox into the workspace.', + description: `Execute JavaScript, Python, or shell scripts in a secure sandbox. For JS: fetch() is available, code runs in an async IIFE wrapper. Shell includes general utilities such as jq, curl, git, and rg. Use outputPath/outputTable to persist returned data, or outputSandboxPath + outputPath to export a file created inside the sandbox into the workspace. Naming outputSandboxPath exports only those paths — the /tmp/sim/outputs directory is not harvested in the same call, so use one or the other. +To read a file, pass its id in \`files\`: each one is mounted read-only under ${SANDBOX_INPUT_DIR}. List that directory to find them rather than guessing a path — names are sanitized and de-duplicated, so they do not always match the original. +To return a file, write it to ${SANDBOX_OUTPUT_DIR}. Everything there comes back in this tool's \`files\` output as a platform file object, which another tool that takes a file accepts directly — no upload step in between.`, version: '1.0.0', params: { @@ -133,6 +167,12 @@ export const functionExecuteTool: InternalToolConfig } + /** + * Platform file objects mounted into the sandbox before the code runs. Unlike + * {@link CodeExecutionInput.inputs}, which names workspace VFS paths, these are + * the objects tools exchange — so an upstream block's output reaches the + * sandbox without a trip through the workspace. + */ + files?: UserFile[] /** Workspace sandbox whose dependency set this execution runs against. */ sandboxId?: string /** @@ -76,5 +84,12 @@ export interface CodeExecutionOutput extends ToolResponse { output: { result: any stdout: string + /** Files harvested from the sandbox output directory, already persisted. */ + files: UserFile[] + cost?: { + input: number + output: number + total: number + } } } diff --git a/apps/sim/tools/generated/tool-ids.ts b/apps/sim/tools/generated/tool-ids.ts index 6f157b3e485..fed8cbfc0aa 100644 --- a/apps/sim/tools/generated/tool-ids.ts +++ b/apps/sim/tools/generated/tool-ids.ts @@ -3,7 +3,7 @@ /** Every registered tool id, including versioned variants. */ const toolIds: string[] = JSON.parse( - '["a2a_cancel_task","a2a_get_agent_card","a2a_get_task","a2a_send_message","affinity_batch_update_entity_fields","affinity_batch_update_list_entry_fields","affinity_create_list","affinity_create_list_field_dropdown_option","affinity_create_merge","affinity_create_note","affinity_create_reminder","affinity_delete_list_field_dropdown_option","affinity_delete_note","affinity_get_company","affinity_get_current_user","affinity_get_entity_field_value","affinity_get_list","affinity_get_list_entry","affinity_get_list_entry_field","affinity_get_list_field_dropdown_option","affinity_get_merge","affinity_get_merge_task","affinity_get_note","affinity_get_opportunity","affinity_get_person","affinity_get_saved_view","affinity_get_transcript","affinity_get_user","affinity_list_calls","affinity_list_chat_messages","affinity_list_companies","affinity_list_coworker_connections","affinity_list_emails","affinity_list_entity_field_values","affinity_list_entity_list_entries","affinity_list_entity_lists","affinity_list_entity_notes","affinity_list_entity_relationships","affinity_list_field_dropdown_options","affinity_list_field_metadata","affinity_list_field_value_changes","affinity_list_investor_executive_connections","affinity_list_list_entries","affinity_list_list_entry_field_value_changes","affinity_list_list_entry_fields","affinity_list_list_field_dropdown_options","affinity_list_list_fields","affinity_list_lists","affinity_list_meetings","affinity_list_merge_tasks","affinity_list_merges","affinity_list_note_attached_companies","affinity_list_note_attached_opportunities","affinity_list_note_attached_persons","affinity_list_note_replies","affinity_list_notes","affinity_list_opportunities","affinity_list_persons","affinity_list_reminders","affinity_list_saved_view_entries","affinity_list_saved_views","affinity_list_transcript_fragments","affinity_list_transcripts","affinity_list_users","affinity_search_companies","affinity_search_files","affinity_search_list_entries","affinity_search_notes","affinity_search_persons","affinity_semantic_search","affinity_update_entity_field_value","affinity_update_list_entry_field","affinity_update_list_field_dropdown_option","affinity_update_note","agentmail_create_draft","agentmail_create_inbox","agentmail_delete_draft","agentmail_delete_inbox","agentmail_delete_thread","agentmail_forward_message","agentmail_get_draft","agentmail_get_inbox","agentmail_get_message","agentmail_get_thread","agentmail_list_drafts","agentmail_list_inboxes","agentmail_list_messages","agentmail_list_threads","agentmail_reply_message","agentmail_send_draft","agentmail_send_message","agentmail_update_draft","agentmail_update_inbox","agentmail_update_message","agentmail_update_thread","agentphone_create_call","agentphone_create_contact","agentphone_create_number","agentphone_delete_contact","agentphone_get_call","agentphone_get_call_transcript","agentphone_get_contact","agentphone_get_conversation","agentphone_get_conversation_messages","agentphone_get_number_messages","agentphone_get_usage","agentphone_get_usage_daily","agentphone_get_usage_monthly","agentphone_list_calls","agentphone_list_contacts","agentphone_list_conversations","agentphone_list_numbers","agentphone_react_to_message","agentphone_release_number","agentphone_send_message","agentphone_update_contact","agentphone_update_conversation","agiloft_async_status","agiloft_attach_file","agiloft_attachment_info","agiloft_create_record","agiloft_delete_record","agiloft_get_choice_line_id","agiloft_list_tables","agiloft_lock_record","agiloft_nlp_search","agiloft_read_record","agiloft_remove_attachment","agiloft_retrieve_attachment","agiloft_run_action_button","agiloft_saved_search","agiloft_search_records","agiloft_select_records","agiloft_update_record","agiloft_upsert_record","ahrefs_anchors","ahrefs_backlinks","ahrefs_backlinks_stats","ahrefs_batch_analysis","ahrefs_broken_backlinks","ahrefs_domain_rating","ahrefs_domain_rating_history","ahrefs_keyword_overview","ahrefs_keywords_history","ahrefs_metrics","ahrefs_metrics_history","ahrefs_organic_competitors","ahrefs_organic_keywords","ahrefs_paid_pages","ahrefs_rank_tracker_competitors_overview","ahrefs_rank_tracker_competitors_stats","ahrefs_rank_tracker_overview","ahrefs_rank_tracker_serp_overview","ahrefs_refdomains_history","ahrefs_referring_domains","ahrefs_related_terms","ahrefs_site_audit_page_explorer","ahrefs_top_pages","airtable_create_records","airtable_delete_records","airtable_get_base_schema","airtable_get_record","airtable_list_bases","airtable_list_records","airtable_list_tables","airtable_update_multiple_records","airtable_update_record","airtable_upsert_records","airweave_search","algolia_add_record","algolia_batch_operations","algolia_browse_records","algolia_clear_records","algolia_copy_move_index","algolia_delete_by_filter","algolia_delete_index","algolia_delete_record","algolia_get_record","algolia_get_records","algolia_get_settings","algolia_get_task_status","algolia_list_indices","algolia_partial_update_record","algolia_search","algolia_update_settings","amplitude_event_segmentation","amplitude_funnels","amplitude_get_active_users","amplitude_get_revenue","amplitude_group_identify","amplitude_identify_user","amplitude_list_events","amplitude_realtime_active_users","amplitude_retention","amplitude_send_event","amplitude_user_activity","amplitude_user_profile","amplitude_user_search","apify_get_dataset_items","apify_get_run","apify_run_actor_async","apify_run_actor_sync","apify_run_task","apollo_account_bulk_create","apollo_account_bulk_update","apollo_account_create","apollo_account_search","apollo_account_update","apollo_contact_bulk_create","apollo_contact_bulk_update","apollo_contact_create","apollo_contact_search","apollo_contact_update","apollo_email_accounts","apollo_opportunity_create","apollo_opportunity_get","apollo_opportunity_search","apollo_opportunity_update","apollo_organization_bulk_enrich","apollo_organization_enrich","apollo_organization_search","apollo_people_bulk_enrich","apollo_people_enrich","apollo_people_search","apollo_sequence_add_contacts","apollo_sequence_search","apollo_task_create","apollo_task_search","appconfig_create_application","appconfig_create_configuration_profile","appconfig_create_environment","appconfig_create_hosted_configuration_version","appconfig_delete_application","appconfig_delete_configuration_profile","appconfig_delete_environment","appconfig_delete_hosted_configuration_version","appconfig_get_application","appconfig_get_configuration","appconfig_get_configuration_profile","appconfig_get_deployment","appconfig_get_environment","appconfig_get_hosted_configuration_version","appconfig_list_applications","appconfig_list_configuration_profiles","appconfig_list_deployment_strategies","appconfig_list_deployments","appconfig_list_environments","appconfig_list_hosted_configuration_versions","appconfig_start_deployment","appconfig_stop_deployment","appconfig_update_application","appconfig_update_configuration_profile","appconfig_update_environment","arxiv_get_author_papers","arxiv_get_paper","arxiv_search","asana_add_comment","asana_add_followers","asana_create_project","asana_create_section","asana_create_subtask","asana_create_task","asana_delete_task","asana_get_project","asana_get_projects","asana_get_task","asana_list_sections","asana_list_workspaces","asana_search_tasks","asana_update_task","ashby_add_candidate_tag","ashby_anonymize_candidate","ashby_change_application_source","ashby_change_application_stage","ashby_create_application","ashby_create_candidate","ashby_create_note","ashby_delete_application","ashby_get_application","ashby_get_candidate","ashby_get_job","ashby_get_job_posting","ashby_get_offer","ashby_list_applications","ashby_list_archive_reasons","ashby_list_candidate_tags","ashby_list_candidates","ashby_list_custom_fields","ashby_list_departments","ashby_list_interviews","ashby_list_job_postings","ashby_list_jobs","ashby_list_locations","ashby_list_notes","ashby_list_offers","ashby_list_openings","ashby_list_sources","ashby_list_users","ashby_remove_candidate_tag","ashby_search_candidates","ashby_set_custom_field_value","ashby_set_custom_field_values","ashby_update_candidate","athena_batch_get_query_execution","athena_create_named_query","athena_delete_named_query","athena_get_named_query","athena_get_query_execution","athena_get_query_results","athena_list_databases","athena_list_named_queries","athena_list_query_executions","athena_list_table_metadata","athena_start_query","athena_stop_query","attio_assert_record","attio_create_attribute","attio_create_comment","attio_create_list","attio_create_list_entry","attio_create_note","attio_create_object","attio_create_record","attio_create_task","attio_create_webhook","attio_delete_comment","attio_delete_list_entry","attio_delete_note","attio_delete_record","attio_delete_task","attio_delete_webhook","attio_get_attribute","attio_get_comment","attio_get_list","attio_get_list_entry","attio_get_member","attio_get_note","attio_get_object","attio_get_record","attio_get_task","attio_get_thread","attio_get_webhook","attio_list_attributes","attio_list_lists","attio_list_members","attio_list_notes","attio_list_objects","attio_list_records","attio_list_tasks","attio_list_threads","attio_list_webhooks","attio_query_list_entries","attio_search_records","attio_update_attribute","attio_update_list","attio_update_list_entry","attio_update_object","attio_update_record","attio_update_task","attio_update_webhook","azure_data_explorer_create_table","azure_data_explorer_drop_table","azure_data_explorer_ingest_from_query","azure_data_explorer_ingest_inline","azure_data_explorer_list_databases","azure_data_explorer_list_functions","azure_data_explorer_list_tables","azure_data_explorer_management","azure_data_explorer_query","azure_data_explorer_show_database_schema","azure_data_explorer_show_ingestion_failures","azure_data_explorer_show_operations","azure_data_explorer_show_table_details","azure_data_explorer_show_table_schema","azure_devops_add_comment","azure_devops_create_work_item","azure_devops_get_build_log","azure_devops_get_build_timeline","azure_devops_get_comments","azure_devops_get_pipeline","azure_devops_get_pipeline_run","azure_devops_get_work_item","azure_devops_get_work_items_batch","azure_devops_get_work_items_between_builds","azure_devops_list_build_logs","azure_devops_list_builds","azure_devops_list_pipeline_runs","azure_devops_list_pipelines","azure_devops_query_work_items","azure_devops_update_work_item","bitbucket_approve_pull_request","bitbucket_create_branch","bitbucket_create_pull_request","bitbucket_create_pull_request_comment","bitbucket_decline_pull_request","bitbucket_delete_branch","bitbucket_get_commit","bitbucket_get_file","bitbucket_get_file_metadata","bitbucket_get_pipeline","bitbucket_get_pipeline_step_log","bitbucket_get_pull_request","bitbucket_get_pull_request_diff","bitbucket_get_pull_request_diffstat","bitbucket_get_pull_request_merge_task_status","bitbucket_get_repository","bitbucket_list_branches","bitbucket_list_commits","bitbucket_list_directory","bitbucket_list_pipeline_steps","bitbucket_list_pipelines","bitbucket_list_pull_request_comments","bitbucket_list_pull_request_commit_statuses","bitbucket_list_pull_requests","bitbucket_list_repositories","bitbucket_list_workspaces","bitbucket_merge_pull_request","bitbucket_request_pull_request_changes","bitbucket_stop_pipeline","bitbucket_trigger_pipeline","box_copy_file","box_create_folder","box_delete_file","box_delete_folder","box_download_file","box_get_file_info","box_list_folder_items","box_search","box_sign_cancel_request","box_sign_create_request","box_sign_get_request","box_sign_list_requests","box_sign_resend_request","box_update_file","box_upload_file","brandfetch_get_brand","brandfetch_search","brex_archive_budget","brex_create_budget","brex_create_spend_limit","brex_create_transfer","brex_create_vendor","brex_get_budget","brex_get_cash_account","brex_get_company","brex_get_current_user","brex_get_expense","brex_get_spend_limit","brex_get_transfer","brex_get_user","brex_get_vendor","brex_list_budgets","brex_list_card_accounts","brex_list_card_statements","brex_list_card_transactions","brex_list_cards","brex_list_cash_accounts","brex_list_cash_statements","brex_list_cash_transactions","brex_list_departments","brex_list_expenses","brex_list_locations","brex_list_spend_limits","brex_list_titles","brex_list_transfers","brex_list_users","brex_list_vendors","brex_match_receipt","brex_update_expense","brex_update_vendor","brex_upload_receipt","brightdata_cancel_snapshot","brightdata_discover","brightdata_download_snapshot","brightdata_scrape_dataset","brightdata_scrape_url","brightdata_serp_search","brightdata_snapshot_status","brightdata_sync_scrape","browser_use_run_task","buffer_create_idea","buffer_create_post","buffer_delete_post","buffer_edit_post","buffer_get_account","buffer_get_channels","buffer_get_idea_groups","buffer_get_ideas","buffer_get_post","buffer_get_posts","calcom_cancel_booking","calcom_confirm_booking","calcom_create_booking","calcom_create_event_type","calcom_create_schedule","calcom_decline_booking","calcom_delete_event_type","calcom_delete_schedule","calcom_get_booking","calcom_get_default_schedule","calcom_get_event_type","calcom_get_schedule","calcom_get_slots","calcom_list_bookings","calcom_list_event_types","calcom_list_schedules","calcom_reschedule_booking","calcom_update_event_type","calcom_update_schedule","calendly_cancel_event","calendly_create_event_invitee","calendly_create_invitee_no_show","calendly_create_scheduling_link","calendly_create_webhook","calendly_delete_invitee_no_show","calendly_delete_webhook","calendly_get_current_user","calendly_get_event_invitee","calendly_get_event_type","calendly_get_scheduled_event","calendly_get_user","calendly_list_event_invitees","calendly_list_event_type_available_times","calendly_list_event_types","calendly_list_organization_memberships","calendly_list_routing_form_submissions","calendly_list_routing_forms","calendly_list_scheduled_events","calendly_list_user_availability_schedules","calendly_list_user_busy_times","calendly_list_webhooks","cbinsights_chat","cbinsights_get_commercial_maturity_history","cbinsights_get_exit_probability_history","cbinsights_get_mosaic_history","cbinsights_get_org_business_relationships","cbinsights_get_org_funding_window","cbinsights_get_org_fundings","cbinsights_get_org_investments","cbinsights_get_org_management_and_board","cbinsights_get_org_outlook","cbinsights_get_org_portfolio_exits","cbinsights_get_org_revenue","cbinsights_get_scouting_report","cbinsights_get_strategy_map","cbinsights_list_business_relationships","cbinsights_list_funding_window","cbinsights_list_fundings","cbinsights_list_investments","cbinsights_list_management_and_board","cbinsights_list_outlook","cbinsights_list_portfolio_exits","cbinsights_list_revenue","cbinsights_lookup_organizations","cbinsights_rag","cbinsights_search_firmographics","clay_populate","clerk_add_organization_member","clerk_ban_user","clerk_create_actor_token","clerk_create_allowlist_identifier","clerk_create_blocklist_identifier","clerk_create_organization","clerk_create_organization_invitation","clerk_create_user","clerk_delete_allowlist_identifier","clerk_delete_blocklist_identifier","clerk_delete_organization","clerk_delete_user","clerk_get_jwt_template","clerk_get_organization","clerk_get_session","clerk_get_user","clerk_get_user_oauth_token","clerk_list_allowlist_identifiers","clerk_list_blocklist_identifiers","clerk_list_jwt_templates","clerk_list_organization_invitations","clerk_list_organization_memberships","clerk_list_organizations","clerk_list_sessions","clerk_list_users","clerk_lock_user","clerk_remove_organization_member","clerk_revoke_actor_token","clerk_revoke_session","clerk_unban_user","clerk_unlock_user","clerk_update_organization","clerk_update_organization_membership","clerk_update_user","clickhouse_count_rows","clickhouse_create_database","clickhouse_create_table","clickhouse_delete","clickhouse_describe_table","clickhouse_drop_database","clickhouse_drop_partition","clickhouse_drop_table","clickhouse_execute","clickhouse_insert","clickhouse_insert_rows","clickhouse_introspect","clickhouse_kill_query","clickhouse_list_clusters","clickhouse_list_databases","clickhouse_list_mutations","clickhouse_list_partitions","clickhouse_list_running_queries","clickhouse_list_tables","clickhouse_optimize_table","clickhouse_query","clickhouse_rename_table","clickhouse_show_create_table","clickhouse_table_stats","clickhouse_truncate_table","clickhouse_update","clickup_add_tag_to_task","clickup_create_checklist","clickup_create_checklist_item","clickup_create_comment","clickup_create_folder","clickup_create_list","clickup_create_task","clickup_create_time_entry","clickup_delete_checklist","clickup_delete_checklist_item","clickup_delete_comment","clickup_delete_task","clickup_delete_time_entry","clickup_get_comments","clickup_get_custom_fields","clickup_get_folders","clickup_get_list_members","clickup_get_lists","clickup_get_running_timer","clickup_get_space_tags","clickup_get_spaces","clickup_get_task","clickup_get_task_members","clickup_get_tasks","clickup_get_time_entries","clickup_get_workspaces","clickup_remove_custom_field_value","clickup_remove_tag_from_task","clickup_search_tasks","clickup_set_custom_field_value","clickup_start_timer","clickup_stop_timer","clickup_update_checklist","clickup_update_checklist_item","clickup_update_comment","clickup_update_task","clickup_update_time_entry","clickup_upload_attachment","cloudflare_create_access_application","cloudflare_create_access_policy","cloudflare_create_access_service_token","cloudflare_create_dns_record","cloudflare_create_r2_bucket","cloudflare_create_rate_limit_rule","cloudflare_create_ruleset","cloudflare_create_ruleset_rule","cloudflare_create_zone","cloudflare_delete_access_application","cloudflare_delete_access_policy","cloudflare_delete_dns_record","cloudflare_delete_r2_bucket","cloudflare_delete_ruleset_rule","cloudflare_delete_zone","cloudflare_dns_analytics","cloudflare_get_access_application","cloudflare_get_r2_bucket","cloudflare_get_ruleset","cloudflare_get_ruleset_entrypoint","cloudflare_get_tunnel","cloudflare_get_tunnel_configuration","cloudflare_get_worker_script_settings","cloudflare_get_zone","cloudflare_get_zone_settings","cloudflare_list_access_applications","cloudflare_list_access_groups","cloudflare_list_access_identity_providers","cloudflare_list_access_policies","cloudflare_list_access_service_tokens","cloudflare_list_certificates","cloudflare_list_dns_records","cloudflare_list_managed_ruleset_overrides","cloudflare_list_r2_buckets","cloudflare_list_rate_limit_rules","cloudflare_list_rulesets","cloudflare_list_tunnels","cloudflare_list_worker_routes","cloudflare_list_worker_scripts","cloudflare_list_zones","cloudflare_purge_cache","cloudflare_revoke_access_service_token","cloudflare_update_access_application","cloudflare_update_access_policy","cloudflare_update_dns_record","cloudflare_update_rate_limit_rule","cloudflare_update_ruleset_rule","cloudflare_update_zone_setting","cloudformation_cancel_update_stack","cloudformation_create_change_set","cloudformation_create_stack","cloudformation_delete_stack","cloudformation_describe_change_set","cloudformation_describe_stack_drift_detection_status","cloudformation_describe_stack_events","cloudformation_describe_stacks","cloudformation_detect_stack_drift","cloudformation_execute_change_set","cloudformation_get_template","cloudformation_get_template_summary","cloudformation_list_stack_resources","cloudformation_update_stack","cloudformation_validate_template","cloudwatch_describe_alarm_history","cloudwatch_describe_alarms","cloudwatch_describe_log_groups","cloudwatch_describe_log_streams","cloudwatch_filter_log_events","cloudwatch_get_log_events","cloudwatch_get_metric_statistics","cloudwatch_list_metrics","cloudwatch_mute_alarm","cloudwatch_put_log_group_retention","cloudwatch_put_metric_data","cloudwatch_query_logs","cloudwatch_unmute_alarm","codepipeline_disable_stage_transition","codepipeline_enable_stage_transition","codepipeline_get_pipeline","codepipeline_get_pipeline_execution","codepipeline_get_pipeline_state","codepipeline_list_action_executions","codepipeline_list_pipeline_executions","codepipeline_list_pipelines","codepipeline_put_approval_result","codepipeline_retry_stage_execution","codepipeline_start_execution","codepipeline_stop_execution","confluence_add_label","confluence_create_blogpost","confluence_create_comment","confluence_create_page","confluence_create_page_property","confluence_create_space","confluence_create_space_property","confluence_delete_attachment","confluence_delete_blogpost","confluence_delete_comment","confluence_delete_label","confluence_delete_page","confluence_delete_page_property","confluence_delete_space","confluence_delete_space_property","confluence_get_blogpost","confluence_get_page_ancestors","confluence_get_page_children","confluence_get_page_descendants","confluence_get_page_version","confluence_get_pages_by_label","confluence_get_space","confluence_get_task","confluence_get_user","confluence_list_attachments","confluence_list_blogposts","confluence_list_blogposts_in_space","confluence_list_comments","confluence_list_labels","confluence_list_page_properties","confluence_list_page_versions","confluence_list_pages_in_space","confluence_list_space_labels","confluence_list_space_permissions","confluence_list_space_properties","confluence_list_spaces","confluence_list_tasks","confluence_retrieve","confluence_search","confluence_search_in_space","confluence_update","confluence_update_blogpost","confluence_update_comment","confluence_update_space","confluence_update_task","confluence_upload_attachment","context_dev_classify_naics","context_dev_classify_sic","context_dev_crawl","context_dev_extract","context_dev_extract_product","context_dev_extract_products","context_dev_get_brand","context_dev_get_brand_by_email","context_dev_get_brand_by_name","context_dev_get_brand_by_ticker","context_dev_identify_transaction","context_dev_map","context_dev_scrape_fonts","context_dev_scrape_html","context_dev_scrape_images","context_dev_scrape_markdown","context_dev_scrape_styleguide","context_dev_screenshot","context_dev_search","convex_action","convex_document_deltas","convex_list_documents","convex_list_tables","convex_mutation","convex_query","convex_run_function","crowdstrike_create_indicators","crowdstrike_delete_indicators","crowdstrike_delete_rtr_session","crowdstrike_execute_rtr_command","crowdstrike_get_alert_details","crowdstrike_get_case_details","crowdstrike_get_host_group_details","crowdstrike_get_indicator_details","crowdstrike_get_rtr_command_status","crowdstrike_get_sensor_aggregates","crowdstrike_get_sensor_details","crowdstrike_get_vulnerability_details","crowdstrike_init_rtr_session","crowdstrike_perform_host_action","crowdstrike_perform_host_group_action","crowdstrike_query_alerts","crowdstrike_query_cases","crowdstrike_query_host_groups","crowdstrike_query_indicators","crowdstrike_query_sensors","crowdstrike_query_vulnerabilities","crowdstrike_update_alerts","crowdstrike_update_indicators","crunchbase_autocomplete","crunchbase_get_acquisition","crunchbase_get_entity","crunchbase_get_entity_card","crunchbase_get_fields_metadata","crunchbase_get_funding_round","crunchbase_get_organization","crunchbase_get_person","crunchbase_list_deleted_entities","crunchbase_search_acquisitions","crunchbase_search_entities","crunchbase_search_funding_rounds","crunchbase_search_organizations","crunchbase_search_people","cursor_add_followup","cursor_add_followup_v2","cursor_delete_agent","cursor_delete_agent_v2","cursor_download_artifact","cursor_download_artifact_v2","cursor_get_agent","cursor_get_agent_v2","cursor_get_api_key_info","cursor_get_api_key_info_v2","cursor_get_conversation","cursor_get_conversation_v2","cursor_launch_agent","cursor_launch_agent_v2","cursor_list_agents","cursor_list_agents_v2","cursor_list_artifacts","cursor_list_artifacts_v2","cursor_list_models","cursor_list_models_v2","cursor_list_repositories","cursor_list_repositories_v2","cursor_stop_agent","cursor_stop_agent_v2","dagster_delete_run","dagster_get_asset","dagster_get_run","dagster_get_run_logs","dagster_launch_run","dagster_list_assets","dagster_list_jobs","dagster_list_runs","dagster_list_schedules","dagster_list_sensors","dagster_materialize_assets","dagster_reexecute_run","dagster_report_asset_materialization","dagster_start_schedule","dagster_start_sensor","dagster_stop_schedule","dagster_stop_sensor","dagster_terminate_run","dagster_wipe_asset","databricks_cancel_run","databricks_execute_sql","databricks_get_cluster","databricks_get_job","databricks_get_run","databricks_get_run_output","databricks_get_statement","databricks_list_clusters","databricks_list_jobs","databricks_list_runs","databricks_list_warehouses","databricks_run_job","datadog_add_incident_todo","datadog_cancel_downtime","datadog_create_dashboard","datadog_create_downtime","datadog_create_event","datadog_create_incident","datadog_create_monitor","datadog_create_slo","datadog_delete_dashboard","datadog_delete_slo","datadog_get_browser_synthetics_results","datadog_get_dashboard","datadog_get_incident","datadog_get_monitor","datadog_get_security_signal","datadog_get_slo","datadog_get_slo_history","datadog_get_synthetics_results","datadog_get_synthetics_test","datadog_list_dashboards","datadog_list_downtimes","datadog_list_incidents","datadog_list_monitors","datadog_list_security_rules","datadog_list_security_signals","datadog_list_services","datadog_list_slos","datadog_list_synthetics_tests","datadog_mute_monitor","datadog_query_logs","datadog_query_timeseries","datadog_search_spans","datadog_send_logs","datadog_submit_metrics","datadog_trigger_synthetics_tests","datadog_unmute_monitor","datadog_update_incident","datadog_update_security_signal_assignee","datadog_update_security_signal_state","datadog_update_slo","datadog_update_synthetics_status","datagma_enrich_company","datagma_enrich_person","datagma_find_email","datagma_find_phone","datagma_get_credits","daytona_create_sandbox","daytona_delete_sandbox","daytona_download_file","daytona_execute_command","daytona_get_sandbox","daytona_git_clone","daytona_list_files","daytona_list_sandboxes","daytona_run_code","daytona_start_sandbox","daytona_stop_sandbox","daytona_upload_file","deployed_block_executor","deployments_deploy","deployments_get_version","deployments_list_versions","deployments_promote","deployments_undeploy","devin_append_session_tags","devin_archive_session","devin_create_session","devin_get_session","devin_get_session_tags","devin_list_session_attachments","devin_list_session_messages","devin_list_sessions","devin_replace_session_tags","devin_send_message","devin_terminate_session","discord_add_reaction","discord_archive_thread","discord_assign_role","discord_ban_member","discord_bulk_delete_messages","discord_create_channel","discord_create_invite","discord_create_role","discord_create_thread","discord_create_webhook","discord_delete_channel","discord_delete_invite","discord_delete_message","discord_delete_role","discord_delete_webhook","discord_edit_message","discord_execute_webhook","discord_get_channel","discord_get_invite","discord_get_member","discord_get_messages","discord_get_pinned_messages","discord_get_server","discord_get_user","discord_get_webhook","discord_join_thread","discord_kick_member","discord_leave_thread","discord_list_channels","discord_list_roles","discord_pin_message","discord_remove_reaction","discord_remove_role","discord_send_message","discord_unban_member","discord_unpin_message","discord_update_channel","discord_update_member","discord_update_role","docusign_create_from_template","docusign_download_document","docusign_get_envelope","docusign_list_envelopes","docusign_list_recipients","docusign_list_templates","docusign_send_envelope","docusign_void_envelope","downdetector_get_company","downdetector_get_company_attribution","downdetector_get_company_baseline","downdetector_get_company_events","downdetector_get_company_incidents","downdetector_get_company_indicators","downdetector_get_company_last_15","downdetector_get_company_status","downdetector_get_provider","downdetector_get_reports","downdetector_get_site_companies","downdetector_list_categories","downdetector_list_incidents","downdetector_list_sites","downdetector_search_companies","dropbox_copy","dropbox_create_folder","dropbox_create_shared_link","dropbox_delete","dropbox_download","dropbox_get_metadata","dropbox_list_folder","dropbox_list_revisions","dropbox_list_shared_links","dropbox_move","dropbox_restore","dropbox_search","dropbox_upload","dropcontact_enrich_contact","dspy_chain_of_thought","dspy_predict","dspy_react","dub_bulk_create_links","dub_bulk_delete_links","dub_bulk_update_links","dub_create_link","dub_create_tag","dub_delete_link","dub_get_analytics","dub_get_events","dub_get_link","dub_get_links_count","dub_get_qr_code","dub_list_domains","dub_list_folders","dub_list_links","dub_list_tags","dub_update_link","dub_upsert_link","duckduckgo_search","dynamodb_delete","dynamodb_get","dynamodb_introspect","dynamodb_put","dynamodb_query","dynamodb_scan","dynamodb_update","dynatrace_add_problem_comment","dynatrace_add_tags","dynatrace_close_problem","dynatrace_create_settings_object","dynatrace_create_slo","dynatrace_delete_problem_comment","dynatrace_delete_settings_object","dynatrace_delete_slo","dynatrace_delete_tag","dynatrace_execute_synthetic_monitors","dynatrace_get_attack","dynatrace_get_audit_logs","dynatrace_get_entity","dynatrace_get_event","dynatrace_get_metric","dynatrace_get_problem","dynatrace_get_problem_comment","dynatrace_get_security_problem","dynatrace_get_settings_object","dynatrace_get_slo","dynatrace_get_synthetic_batch","dynatrace_ingest_event","dynatrace_ingest_logs","dynatrace_ingest_metrics","dynatrace_list_attacks","dynatrace_list_entities","dynatrace_list_entity_types","dynatrace_list_events","dynatrace_list_metrics","dynatrace_list_problem_comments","dynatrace_list_problems","dynatrace_list_remediation_items","dynatrace_list_security_problems","dynatrace_list_settings_objects","dynatrace_list_settings_schemas","dynatrace_list_slos","dynatrace_list_synthetic_monitors","dynatrace_list_tags","dynatrace_mute_security_problem","dynatrace_mute_security_problems","dynatrace_query_metrics","dynatrace_search_logs","dynatrace_unmute_security_problem","dynatrace_unmute_security_problems","dynatrace_update_problem_comment","dynatrace_update_settings_object","dynatrace_update_slo","elasticsearch_bulk","elasticsearch_cluster_health","elasticsearch_cluster_stats","elasticsearch_count","elasticsearch_create_index","elasticsearch_delete_document","elasticsearch_delete_index","elasticsearch_get_document","elasticsearch_get_index","elasticsearch_index_document","elasticsearch_list_indices","elasticsearch_search","elasticsearch_update_document","elevenlabs_audio_isolation","elevenlabs_edit_voice_settings","elevenlabs_get_user","elevenlabs_get_voice","elevenlabs_get_voice_settings","elevenlabs_list_models","elevenlabs_list_voices","elevenlabs_sound_effects","elevenlabs_speech_to_speech","elevenlabs_tts","emailbison_attach_leads_to_campaign","emailbison_attach_tags_to_leads","emailbison_create_campaign","emailbison_create_lead","emailbison_create_tag","emailbison_get_lead","emailbison_list_campaigns","emailbison_list_leads","emailbison_list_replies","emailbison_list_tags","emailbison_update_campaign","emailbison_update_campaign_status","emailbison_update_lead","embeddings_cohere","embeddings_gemini","embeddings_mistral","embeddings_openai","embeddings_openrouter","enrich_check_credits","enrich_company_funding","enrich_company_lookup","enrich_company_revenue","enrich_disposable_email_check","enrich_email_to_ip","enrich_email_to_person_lite","enrich_email_to_phone","enrich_email_to_profile","enrich_find_email","enrich_get_post_details","enrich_ip_to_company","enrich_linkedin_profile","enrich_linkedin_to_personal_email","enrich_linkedin_to_work_email","enrich_phone_finder","enrich_reverse_hash_lookup","enrich_sales_pointer_people","enrich_search_company","enrich_search_company_activities","enrich_search_company_employees","enrich_search_jobs","enrich_search_logo","enrich_search_people","enrich_search_people_activities","enrich_search_post_comments","enrich_search_post_comments_by_url","enrich_search_post_reactions","enrich_search_post_reactions_by_url","enrich_search_posts","enrich_search_similar_companies","enrich_verify_email","enrichment_run","enrow_find_email","enrow_verify_email","exa_agent","exa_answer","exa_find_similar_links","exa_get_contents","exa_search","extend_parser","extend_parser_v2","fathom_get_summary","fathom_get_transcript","fathom_list_meeting_types","fathom_list_meetings","fathom_list_team_members","fathom_list_teams","file_append","file_compress","file_decompress","file_fetch","file_get","file_get_content","file_manage_sharing","file_parser","file_parser_v2","file_parser_v3","file_read","file_write","findymail_find_email_from_linkedin","findymail_find_email_from_name","findymail_find_emails_by_domain","findymail_find_employees","findymail_find_phone","findymail_get_company","findymail_get_credits","findymail_lookup_technologies","findymail_reverse_email_lookup","findymail_search_technologies","findymail_verify_email","firecrawl_agent","firecrawl_batch_scrape","firecrawl_batch_scrape_status","firecrawl_cancel_crawl","firecrawl_crawl","firecrawl_crawl_status","firecrawl_credit_usage","firecrawl_extract","firecrawl_extract_status","firecrawl_map","firecrawl_parse","firecrawl_scrape","firecrawl_search","fireflies_add_to_live_meeting","fireflies_create_bite","fireflies_delete_transcript","fireflies_get_transcript","fireflies_get_user","fireflies_list_bites","fireflies_list_contacts","fireflies_list_transcripts","fireflies_list_users","fireflies_upload_audio","flint_create_task","flint_generate_pages","flint_get_task","function_execute","gamma_check_status","gamma_generate","gamma_generate_from_template","gamma_list_folders","gamma_list_themes","github_add_assignees","github_add_assignees_v2","github_add_labels","github_add_labels_v2","github_cancel_workflow_run","github_cancel_workflow_run_v2","github_check_star","github_check_star_v2","github_close_issue","github_close_issue_v2","github_close_pr","github_close_pr_v2","github_comment","github_comment_v2","github_compare_commits","github_compare_commits_v2","github_create_branch","github_create_branch_v2","github_create_comment_reaction","github_create_comment_reaction_v2","github_create_file","github_create_file_v2","github_create_gist","github_create_gist_v2","github_create_issue","github_create_issue_reaction","github_create_issue_reaction_v2","github_create_issue_v2","github_create_milestone","github_create_milestone_v2","github_create_pr","github_create_pr_review","github_create_pr_review_v2","github_create_pr_v2","github_create_project","github_create_project_v2","github_create_release","github_create_release_v2","github_delete_branch","github_delete_branch_v2","github_delete_comment","github_delete_comment_reaction","github_delete_comment_reaction_v2","github_delete_comment_v2","github_delete_file","github_delete_file_v2","github_delete_gist","github_delete_gist_v2","github_delete_issue_reaction","github_delete_issue_reaction_v2","github_delete_milestone","github_delete_milestone_v2","github_delete_project","github_delete_project_v2","github_delete_release","github_delete_release_v2","github_fork_gist","github_fork_gist_v2","github_fork_repo","github_fork_repo_v2","github_get_branch","github_get_branch_protection","github_get_branch_protection_v2","github_get_branch_v2","github_get_commit","github_get_commit_v2","github_get_file_content","github_get_file_content_v2","github_get_gist","github_get_gist_v2","github_get_issue","github_get_issue_v2","github_get_latest_release","github_get_latest_release_v2","github_get_milestone","github_get_milestone_v2","github_get_pr_files","github_get_pr_files_v2","github_get_project","github_get_project_v2","github_get_readme","github_get_readme_v2","github_get_release","github_get_release_v2","github_get_tree","github_get_tree_v2","github_get_workflow","github_get_workflow_run","github_get_workflow_run_v2","github_get_workflow_v2","github_issue_comment","github_issue_comment_v2","github_job_logs","github_latest_commit","github_latest_commit_v2","github_list_branches","github_list_branches_v2","github_list_commits","github_list_commits_v2","github_list_forks","github_list_forks_v2","github_list_gists","github_list_gists_v2","github_list_issue_comments","github_list_issue_comments_v2","github_list_issues","github_list_issues_v2","github_list_milestones","github_list_milestones_v2","github_list_pr_comments","github_list_pr_comments_v2","github_list_projects","github_list_projects_v2","github_list_prs","github_list_prs_v2","github_list_releases","github_list_releases_v2","github_list_review_threads","github_list_stargazers","github_list_stargazers_v2","github_list_tags","github_list_tags_v2","github_list_workflow_runs","github_list_workflow_runs_v2","github_list_workflows","github_list_workflows_v2","github_merge_pr","github_merge_pr_v2","github_pr","github_pr_v2","github_remove_label","github_remove_label_v2","github_reply_review_thread","github_repo_info","github_repo_info_v2","github_request_reviewers","github_request_reviewers_v2","github_rerun_workflow","github_rerun_workflow_v2","github_resolve_review_thread","github_search_code","github_search_code_v2","github_search_commits","github_search_commits_v2","github_search_issues","github_search_issues_v2","github_search_repos","github_search_repos_v2","github_search_users","github_search_users_v2","github_star_gist","github_star_gist_v2","github_star_repo","github_star_repo_v2","github_status_check_rollup","github_trigger_workflow","github_trigger_workflow_v2","github_unstar_gist","github_unstar_gist_v2","github_unstar_repo","github_unstar_repo_v2","github_update_branch_protection","github_update_branch_protection_v2","github_update_comment","github_update_comment_v2","github_update_file","github_update_file_v2","github_update_gist","github_update_gist_v2","github_update_issue","github_update_issue_v2","github_update_milestone","github_update_milestone_v2","github_update_pr","github_update_pr_v2","github_update_project","github_update_project_v2","github_update_release","github_update_release_v2","gitlab_activate_user","gitlab_add_member","gitlab_add_saml_group_link","gitlab_approve_access_request","gitlab_approve_merge_request","gitlab_approve_user","gitlab_ban_user","gitlab_block_user","gitlab_cancel_pipeline","gitlab_compare_branches","gitlab_create_branch","gitlab_create_file","gitlab_create_issue","gitlab_create_issue_note","gitlab_create_merge_request","gitlab_create_merge_request_note","gitlab_create_pipeline","gitlab_create_release","gitlab_create_user","gitlab_deactivate_user","gitlab_delete_branch","gitlab_delete_issue","gitlab_delete_saml_group_link","gitlab_delete_user","gitlab_delete_user_identity","gitlab_deny_access_request","gitlab_get_file","gitlab_get_group","gitlab_get_issue","gitlab_get_job_log","gitlab_get_merge_request","gitlab_get_merge_request_changes","gitlab_get_pipeline","gitlab_get_project","gitlab_invite_member","gitlab_list_access_requests","gitlab_list_branches","gitlab_list_commits","gitlab_list_groups","gitlab_list_invitations","gitlab_list_issues","gitlab_list_members","gitlab_list_merge_requests","gitlab_list_pipeline_jobs","gitlab_list_pipelines","gitlab_list_projects","gitlab_list_releases","gitlab_list_repository_tree","gitlab_list_saml_group_links","gitlab_list_user_memberships","gitlab_merge_merge_request","gitlab_play_job","gitlab_reject_user","gitlab_remove_member","gitlab_retry_pipeline","gitlab_revoke_invitation","gitlab_search_users","gitlab_unban_user","gitlab_unblock_user","gitlab_update_file","gitlab_update_invitation","gitlab_update_issue","gitlab_update_member","gitlab_update_merge_request","gitlab_update_user","gmail_add_label","gmail_add_label_v2","gmail_archive","gmail_archive_v2","gmail_create_label_v2","gmail_delete","gmail_delete_draft_v2","gmail_delete_label_v2","gmail_delete_v2","gmail_draft","gmail_draft_v2","gmail_edit_draft_v2","gmail_get_draft_v2","gmail_get_thread_v2","gmail_list_drafts_v2","gmail_list_labels_v2","gmail_list_threads_v2","gmail_mark_read","gmail_mark_read_v2","gmail_mark_unread","gmail_mark_unread_v2","gmail_move","gmail_move_v2","gmail_read","gmail_read_v2","gmail_remove_label","gmail_remove_label_v2","gmail_search","gmail_search_v2","gmail_send","gmail_send_v2","gmail_trash_thread_v2","gmail_unarchive","gmail_unarchive_v2","gmail_untrash_thread_v2","gmail_update_label_v2","gong_aggregate_activity","gong_aggregate_by_period","gong_answered_scorecards","gong_ask_anything","gong_assign_flow_prospects","gong_create_call","gong_day_by_day_activity","gong_get_brief","gong_get_call","gong_get_call_transcript","gong_get_coaching","gong_get_extensive_calls","gong_get_folder_content","gong_get_logs","gong_get_prospect_flows","gong_get_user","gong_interaction_stats","gong_list_calls","gong_list_flows","gong_list_library_folders","gong_list_scorecards","gong_list_trackers","gong_list_users","gong_list_workspaces","gong_lookup_email","gong_lookup_phone","gong_purge_email_address","gong_purge_phone_number","gong_unassign_flow_prospects","google_ads_ad_performance","google_ads_campaign_performance","google_ads_list_ad_groups","google_ads_list_campaigns","google_ads_list_customers","google_ads_search","google_appsheet_add_rows","google_appsheet_delete_rows","google_appsheet_edit_rows","google_appsheet_find_rows","google_bigquery_create_dataset","google_bigquery_create_table","google_bigquery_delete_dataset","google_bigquery_delete_table","google_bigquery_get_query_results","google_bigquery_get_table","google_bigquery_insert_rows","google_bigquery_list_datasets","google_bigquery_list_table_data","google_bigquery_list_tables","google_bigquery_query","google_books_volume_details","google_books_volume_search","google_calendar_create","google_calendar_create_calendar","google_calendar_create_calendar_v2","google_calendar_create_v2","google_calendar_delete","google_calendar_delete_calendar","google_calendar_delete_calendar_v2","google_calendar_delete_v2","google_calendar_freebusy","google_calendar_freebusy_v2","google_calendar_get","google_calendar_get_v2","google_calendar_instances","google_calendar_instances_v2","google_calendar_invite","google_calendar_invite_v2","google_calendar_list","google_calendar_list_acl","google_calendar_list_acl_v2","google_calendar_list_calendars","google_calendar_list_calendars_v2","google_calendar_list_v2","google_calendar_move","google_calendar_move_v2","google_calendar_quick_add","google_calendar_quick_add_v2","google_calendar_share_calendar","google_calendar_share_calendar_v2","google_calendar_unshare_calendar","google_calendar_unshare_calendar_v2","google_calendar_update","google_calendar_update_acl","google_calendar_update_acl_v2","google_calendar_update_calendar","google_calendar_update_calendar_v2","google_calendar_update_v2","google_contacts_create","google_contacts_delete","google_contacts_get","google_contacts_list","google_contacts_search","google_contacts_update","google_docs_create","google_docs_create_named_range","google_docs_create_paragraph_bullets","google_docs_delete_content_range","google_docs_delete_named_range","google_docs_delete_paragraph_bullets","google_docs_insert_image","google_docs_insert_page_break","google_docs_insert_table","google_docs_insert_text","google_docs_read","google_docs_replace_text","google_docs_update_paragraph_style","google_docs_update_text_style","google_docs_write","google_drive_copy","google_drive_create_comment","google_drive_create_folder","google_drive_delete","google_drive_delete_comment","google_drive_download","google_drive_export","google_drive_get_about","google_drive_get_content","google_drive_get_file","google_drive_get_revision","google_drive_list","google_drive_list_comments","google_drive_list_permissions","google_drive_list_revisions","google_drive_move","google_drive_search","google_drive_share","google_drive_trash","google_drive_unshare","google_drive_untrash","google_drive_update","google_drive_upload","google_forms_batch_update","google_forms_create_form","google_forms_create_watch","google_forms_delete_watch","google_forms_get_form","google_forms_get_responses","google_forms_list_watches","google_forms_renew_watch","google_forms_set_publish_settings","google_groups_add_alias","google_groups_add_member","google_groups_create_group","google_groups_delete_group","google_groups_get_group","google_groups_get_member","google_groups_get_settings","google_groups_has_member","google_groups_list_aliases","google_groups_list_groups","google_groups_list_members","google_groups_remove_alias","google_groups_remove_member","google_groups_update_group","google_groups_update_member","google_groups_update_settings","google_maps_air_quality","google_maps_directions","google_maps_distance_matrix","google_maps_elevation","google_maps_geocode","google_maps_geolocate","google_maps_place_details","google_maps_places_nearby","google_maps_places_search","google_maps_pollen","google_maps_reverse_geocode","google_maps_snap_to_roads","google_maps_solar","google_maps_speed_limits","google_maps_timezone","google_maps_validate_address","google_meet_create_space","google_meet_end_conference","google_meet_get_conference_record","google_meet_get_space","google_meet_list_conference_records","google_meet_list_participants","google_pagespeed_analyze","google_search","google_sheets_append","google_sheets_append_v2","google_sheets_batch_clear_v2","google_sheets_batch_get_v2","google_sheets_batch_update_v2","google_sheets_clear_v2","google_sheets_copy_sheet_v2","google_sheets_create_spreadsheet_v2","google_sheets_delete_rows_v2","google_sheets_delete_sheet_v2","google_sheets_delete_spreadsheet_v2","google_sheets_get_spreadsheet_v2","google_sheets_read","google_sheets_read_v2","google_sheets_update","google_sheets_update_v2","google_sheets_write","google_sheets_write_v2","google_slides_add_image","google_slides_add_slide","google_slides_batch_update","google_slides_copy_presentation","google_slides_create","google_slides_create_line","google_slides_create_paragraph_bullets","google_slides_create_shape","google_slides_create_sheets_chart","google_slides_create_table","google_slides_create_video","google_slides_delete_object","google_slides_delete_paragraph_bullets","google_slides_delete_table_column","google_slides_delete_table_row","google_slides_delete_text","google_slides_duplicate_object","google_slides_export_presentation","google_slides_get_page","google_slides_get_thumbnail","google_slides_group_objects","google_slides_insert_table_columns","google_slides_insert_table_rows","google_slides_insert_text","google_slides_merge_table_cells","google_slides_read","google_slides_refresh_sheets_chart","google_slides_replace_all_shapes_with_image","google_slides_replace_all_shapes_with_sheets_chart","google_slides_replace_all_text","google_slides_replace_image","google_slides_reroute_line","google_slides_ungroup_objects","google_slides_unmerge_table_cells","google_slides_update_image_properties","google_slides_update_line_category","google_slides_update_line_properties","google_slides_update_page_element_alt_text","google_slides_update_page_element_transform","google_slides_update_page_elements_z_order","google_slides_update_page_properties","google_slides_update_paragraph_style","google_slides_update_shape_properties","google_slides_update_slide_properties","google_slides_update_slides_position","google_slides_update_table_border_properties","google_slides_update_table_cell_properties","google_slides_update_table_column_properties","google_slides_update_table_row_properties","google_slides_update_text_style","google_slides_update_video_properties","google_slides_write","google_tasks_create","google_tasks_delete","google_tasks_get","google_tasks_list","google_tasks_list_task_lists","google_tasks_update","google_translate_detect","google_translate_text","google_vault_add_held_accounts","google_vault_add_matters_permissions","google_vault_close_matters","google_vault_create_matters","google_vault_create_matters_export","google_vault_create_matters_holds","google_vault_create_saved_query","google_vault_delete_matters","google_vault_delete_matters_export","google_vault_delete_matters_holds","google_vault_delete_saved_query","google_vault_download_export_file","google_vault_list_matters","google_vault_list_matters_export","google_vault_list_matters_holds","google_vault_list_saved_queries","google_vault_remove_held_accounts","google_vault_remove_matters_permissions","google_vault_reopen_matters","google_vault_undelete_matters","google_vault_update_matters","google_vault_update_matters_holds","grafana_check_data_source_health","grafana_create_alert_rule","grafana_create_annotation","grafana_create_contact_point","grafana_create_dashboard","grafana_create_folder","grafana_delete_alert_rule","grafana_delete_annotation","grafana_delete_contact_point","grafana_delete_dashboard","grafana_delete_folder","grafana_get_alert_rule","grafana_get_alert_rule_group","grafana_get_dashboard","grafana_get_data_source","grafana_get_folder","grafana_get_health","grafana_list_alert_rules","grafana_list_annotations","grafana_list_contact_points","grafana_list_dashboards","grafana_list_data_sources","grafana_list_folders","grafana_move_folder","grafana_query_data_source","grafana_update_alert_rule","grafana_update_annotation","grafana_update_contact_point","grafana_update_dashboard","grafana_update_folder","grain_create_hook","grain_create_hook_v2","grain_delete_hook","grain_delete_hook_v2","grain_get_recording","grain_get_transcript","grain_list_hooks","grain_list_hooks_v2","grain_list_meeting_types","grain_list_recordings","grain_list_teams","grain_list_views","granola_create_webhook_endpoint","granola_delete_webhook_endpoint","granola_get_note","granola_get_transcript","granola_list_audit_events","granola_list_folders","granola_list_notes","granola_list_webhook_endpoints","granola_update_webhook_endpoint","greenhouse_get_application","greenhouse_get_candidate","greenhouse_get_job","greenhouse_get_user","greenhouse_list_applications","greenhouse_list_candidates","greenhouse_list_departments","greenhouse_list_job_stages","greenhouse_list_jobs","greenhouse_list_offices","greenhouse_list_users","greptile_index_repo","greptile_query","greptile_search","greptile_status","guardrails_validate","harmonic_batch_get_people","harmonic_clear_people_saved_search_net_new_results","harmonic_enrich_person","harmonic_get_company_employees","harmonic_get_email_enrichment_job","harmonic_get_email_enrichment_usage","harmonic_get_enrichment_status","harmonic_get_people_saved_search_net_new_results","harmonic_get_people_saved_search_results","harmonic_get_person","harmonic_list_people_saved_searches","harmonic_search_people_scout","harmonic_submit_email_enrichment_job","hex_cancel_run","hex_create_collection","hex_create_group","hex_deactivate_user","hex_delete_group","hex_get_collection","hex_get_data_connection","hex_get_group","hex_get_project","hex_get_project_runs","hex_get_queried_tables","hex_get_run_status","hex_list_collections","hex_list_data_connections","hex_list_groups","hex_list_projects","hex_list_users","hex_run_project","hex_update_collection","hex_update_group","hex_update_project","http_request","hubspot_add_list_memberships","hubspot_create_appointment","hubspot_create_association","hubspot_create_company","hubspot_create_contact","hubspot_create_deal","hubspot_create_email","hubspot_create_line_item","hubspot_create_list","hubspot_create_note","hubspot_create_ticket","hubspot_delete_association","hubspot_delete_company","hubspot_delete_contact","hubspot_delete_deal","hubspot_delete_line_item","hubspot_delete_ticket","hubspot_get_appointment","hubspot_get_association_labels","hubspot_get_cart","hubspot_get_company","hubspot_get_contact","hubspot_get_deal","hubspot_get_email","hubspot_get_line_item","hubspot_get_list","hubspot_get_list_memberships","hubspot_get_marketing_event","hubspot_get_note","hubspot_get_properties","hubspot_get_quote","hubspot_get_ticket","hubspot_get_users","hubspot_list_appointments","hubspot_list_associations","hubspot_list_carts","hubspot_list_companies","hubspot_list_contacts","hubspot_list_deals","hubspot_list_emails","hubspot_list_line_items","hubspot_list_lists","hubspot_list_marketing_events","hubspot_list_notes","hubspot_list_owners","hubspot_list_quotes","hubspot_list_tickets","hubspot_remove_list_memberships","hubspot_search_companies","hubspot_search_contacts","hubspot_search_deals","hubspot_search_emails","hubspot_search_line_items","hubspot_search_notes","hubspot_search_quotes","hubspot_search_tickets","hubspot_update_appointment","hubspot_update_company","hubspot_update_contact","hubspot_update_deal","hubspot_update_line_item","hubspot_update_ticket","huggingface_chat","hunter_companies_find","hunter_discover","hunter_domain_search","hunter_email_count","hunter_email_finder","hunter_email_verifier","iam_add_user_to_group","iam_attach_role_policy","iam_attach_user_policy","iam_create_access_key","iam_create_role","iam_create_user","iam_delete_access_key","iam_delete_role","iam_delete_user","iam_detach_role_policy","iam_detach_user_policy","iam_get_role","iam_get_user","iam_list_attached_role_policies","iam_list_attached_user_policies","iam_list_groups","iam_list_policies","iam_list_roles","iam_list_users","iam_remove_user_from_group","iam_simulate_principal_policy","icypeas_find_email","icypeas_verify_email","identity_center_check_assignment_deletion_status","identity_center_check_assignment_status","identity_center_create_account_assignment","identity_center_delete_account_assignment","identity_center_describe_account","identity_center_get_group","identity_center_get_user","identity_center_list_account_assignments","identity_center_list_accounts","identity_center_list_groups","identity_center_list_instances","identity_center_list_permission_sets","image_generate","incidentio_actions_create","incidentio_actions_list","incidentio_actions_show","incidentio_actions_update","incidentio_alert_events_create","incidentio_alerts_list","incidentio_alerts_resolve","incidentio_alerts_show","incidentio_catalog_entries_list","incidentio_catalog_types_list","incidentio_custom_fields_create","incidentio_custom_fields_delete","incidentio_custom_fields_list","incidentio_custom_fields_show","incidentio_custom_fields_update","incidentio_escalation_paths_create","incidentio_escalation_paths_delete","incidentio_escalation_paths_list","incidentio_escalation_paths_show","incidentio_escalation_paths_update","incidentio_escalations_cancel","incidentio_escalations_create","incidentio_escalations_list","incidentio_escalations_show","incidentio_follow_ups_create","incidentio_follow_ups_list","incidentio_follow_ups_show","incidentio_follow_ups_update","incidentio_incident_alerts_list","incidentio_incident_memberships_create","incidentio_incident_memberships_revoke","incidentio_incident_participants_list","incidentio_incident_roles_create","incidentio_incident_roles_delete","incidentio_incident_roles_list","incidentio_incident_roles_show","incidentio_incident_roles_update","incidentio_incident_statuses_list","incidentio_incident_timestamps_list","incidentio_incident_timestamps_show","incidentio_incident_types_list","incidentio_incident_updates_list","incidentio_incidents_create","incidentio_incidents_list","incidentio_incidents_show","incidentio_incidents_update","incidentio_on_call_now","incidentio_schedule_entries_list","incidentio_schedule_overrides_create","incidentio_schedule_overrides_list","incidentio_schedules_create","incidentio_schedules_delete","incidentio_schedules_list","incidentio_schedules_show","incidentio_schedules_update","incidentio_severities_list","incidentio_teams_list","incidentio_teams_show","incidentio_users_list","incidentio_users_show","incidentio_workflows_create","incidentio_workflows_delete","incidentio_workflows_list","incidentio_workflows_show","incidentio_workflows_update","infisical_create_secret","infisical_delete_secret","infisical_get_secret","infisical_list_secrets","infisical_update_secret","instagram_delete_comment","instagram_download_media","instagram_get_account_insights","instagram_get_container_status","instagram_get_conversation_messages","instagram_get_media","instagram_get_media_insights","instagram_get_message","instagram_get_profile","instagram_get_publishing_limit","instagram_hide_comment","instagram_list_comments","instagram_list_conversations","instagram_list_media","instagram_list_stories","instagram_private_reply","instagram_publish_carousel","instagram_publish_image","instagram_publish_reel","instagram_publish_story","instagram_publish_video","instagram_reply_to_comment","instagram_send_text_message","instagram_set_comments_enabled","instantly_activate_campaign","instantly_create_campaign","instantly_create_lead","instantly_create_lead_list","instantly_delete_campaign","instantly_delete_leads","instantly_get_lead","instantly_list_campaigns","instantly_list_emails","instantly_list_lead_lists","instantly_list_leads","instantly_patch_campaign","instantly_patch_lead","instantly_pause_campaign","instantly_reply_to_email","instantly_update_lead_interest_status","intercom_assign_conversation_v2","intercom_attach_contact_to_company_v2","intercom_close_conversation_v2","intercom_create_company","intercom_create_company_v2","intercom_create_contact","intercom_create_contact_v2","intercom_create_event_v2","intercom_create_message","intercom_create_message_v2","intercom_create_note_v2","intercom_create_tag_v2","intercom_create_ticket","intercom_create_ticket_v2","intercom_delete_contact","intercom_delete_contact_v2","intercom_detach_contact_from_company_v2","intercom_get_company","intercom_get_company_v2","intercom_get_contact","intercom_get_contact_v2","intercom_get_conversation","intercom_get_conversation_v2","intercom_get_ticket","intercom_get_ticket_v2","intercom_list_admins_v2","intercom_list_companies","intercom_list_companies_v2","intercom_list_contacts","intercom_list_contacts_v2","intercom_list_conversations","intercom_list_conversations_v2","intercom_list_tags_v2","intercom_open_conversation_v2","intercom_reply_conversation","intercom_reply_conversation_v2","intercom_search_contacts","intercom_search_contacts_v2","intercom_search_conversations","intercom_search_conversations_v2","intercom_snooze_conversation_v2","intercom_tag_contact_v2","intercom_tag_conversation_v2","intercom_untag_contact_v2","intercom_update_contact","intercom_update_contact_v2","intercom_update_ticket_v2","jina_read_url","jina_search","jira_add_attachment","jira_add_comment","jira_add_watcher","jira_add_worklog","jira_assign_issue","jira_bulk_read","jira_create_issue_link","jira_delete_attachment","jira_delete_comment","jira_delete_issue","jira_delete_issue_link","jira_delete_worklog","jira_get_attachments","jira_get_comments","jira_get_fields","jira_get_project","jira_get_transitions","jira_get_users","jira_get_worklogs","jira_list_issue_types","jira_list_projects","jira_remove_watcher","jira_retrieve","jira_search_issues","jira_search_users","jira_transition_issue","jira_update","jira_update_comment","jira_update_worklog","jira_write","jotform_add_label_resources","jotform_clone_form","jotform_create_form","jotform_create_label","jotform_create_question","jotform_create_questions","jotform_create_report","jotform_create_submission","jotform_create_submissions","jotform_create_webhook","jotform_delete_form","jotform_delete_label","jotform_delete_question","jotform_delete_report","jotform_delete_submission","jotform_delete_webhook","jotform_get_form","jotform_get_form_properties","jotform_get_history","jotform_get_label","jotform_get_question","jotform_get_report","jotform_get_settings","jotform_get_submission","jotform_get_usage","jotform_get_user","jotform_list_form_files","jotform_list_form_reports","jotform_list_form_submissions","jotform_list_forms","jotform_list_label_resources","jotform_list_labels","jotform_list_questions","jotform_list_reports","jotform_list_submissions","jotform_list_subusers","jotform_list_webhooks","jotform_remove_label_resources","jotform_update_form_properties","jotform_update_label","jotform_update_question","jotform_update_settings","jotform_update_submission","jsm_add_comment","jsm_add_customer","jsm_add_organization","jsm_add_participants","jsm_answer_approval","jsm_attach_form","jsm_copy_forms","jsm_create_object","jsm_create_organization","jsm_create_request","jsm_delete_form","jsm_delete_object","jsm_externalise_form","jsm_get_approvals","jsm_get_comments","jsm_get_customers","jsm_get_form","jsm_get_form_answers","jsm_get_form_structure","jsm_get_form_templates","jsm_get_issue_forms","jsm_get_object","jsm_get_object_schema","jsm_get_object_type_attributes","jsm_get_organizations","jsm_get_participants","jsm_get_queues","jsm_get_request","jsm_get_request_type_fields","jsm_get_request_types","jsm_get_requests","jsm_get_service_desks","jsm_get_sla","jsm_get_transitions","jsm_internalise_form","jsm_list_object_schemas","jsm_list_object_types","jsm_reopen_form","jsm_save_form_answers","jsm_search_objects_aql","jsm_submit_form","jsm_transition_request","jsm_update_object","jupyter_copy_content","jupyter_create_file","jupyter_create_session","jupyter_delete_content","jupyter_delete_session","jupyter_get_content","jupyter_interrupt_kernel","jupyter_list_contents","jupyter_list_kernels","jupyter_list_kernelspecs","jupyter_list_sessions","jupyter_rename_content","jupyter_restart_kernel","jupyter_start_kernel","jupyter_stop_kernel","jupyter_upload_file","kalshi_amend_order","kalshi_amend_order_v2","kalshi_cancel_order","kalshi_cancel_order_v2","kalshi_create_order","kalshi_create_order_v2","kalshi_get_balance","kalshi_get_balance_v2","kalshi_get_candlesticks","kalshi_get_candlesticks_v2","kalshi_get_event","kalshi_get_event_candlesticks","kalshi_get_event_candlesticks_v2","kalshi_get_event_v2","kalshi_get_events","kalshi_get_events_v2","kalshi_get_exchange_announcements","kalshi_get_exchange_announcements_v2","kalshi_get_exchange_schedule","kalshi_get_exchange_schedule_v2","kalshi_get_exchange_status","kalshi_get_exchange_status_v2","kalshi_get_fills","kalshi_get_fills_v2","kalshi_get_market","kalshi_get_market_v2","kalshi_get_markets","kalshi_get_markets_v2","kalshi_get_order","kalshi_get_order_v2","kalshi_get_orderbook","kalshi_get_orderbook_v2","kalshi_get_orders","kalshi_get_orders_v2","kalshi_get_positions","kalshi_get_positions_v2","kalshi_get_series_by_ticker","kalshi_get_series_by_ticker_v2","kalshi_get_series_list","kalshi_get_series_list_v2","kalshi_get_settlements","kalshi_get_settlements_v2","kalshi_get_trades","kalshi_get_trades_v2","ketch_get_consent","ketch_get_subscriptions","ketch_invoke_right","ketch_set_consent","ketch_set_subscriptions","knowledge_create_document","knowledge_delete_chunk","knowledge_delete_document","knowledge_get_connector","knowledge_get_document","knowledge_list_chunks","knowledge_list_connectors","knowledge_list_documents","knowledge_list_tags","knowledge_search","knowledge_trigger_sync","knowledge_update_chunk","knowledge_upload_chunk","knowledge_upsert_document","lambda_add_permission","lambda_create_alias","lambda_create_event_source_mapping","lambda_create_function","lambda_create_function_url_config","lambda_delete_alias","lambda_delete_event_source_mapping","lambda_delete_function","lambda_delete_function_concurrency","lambda_delete_function_event_invoke_config","lambda_delete_function_url_config","lambda_delete_provisioned_concurrency_config","lambda_get_account_settings","lambda_get_alias","lambda_get_event_source_mapping","lambda_get_function","lambda_get_function_concurrency","lambda_get_function_configuration","lambda_get_function_event_invoke_config","lambda_get_function_recursion_config","lambda_get_function_url_config","lambda_get_layer_version","lambda_get_policy","lambda_get_provisioned_concurrency_config","lambda_get_runtime_management_config","lambda_invoke","lambda_list_aliases","lambda_list_event_source_mappings","lambda_list_function_event_invoke_configs","lambda_list_function_url_configs","lambda_list_functions","lambda_list_layer_versions","lambda_list_layers","lambda_list_provisioned_concurrency_configs","lambda_list_tags","lambda_list_versions_by_function","lambda_publish_version","lambda_put_function_concurrency","lambda_put_function_event_invoke_config","lambda_put_function_recursion_config","lambda_put_provisioned_concurrency_config","lambda_put_runtime_management_config","lambda_remove_permission","lambda_tag_resource","lambda_untag_resource","lambda_update_alias","lambda_update_event_source_mapping","lambda_update_function_code","lambda_update_function_configuration","lambda_update_function_url_config","langsmith_create_feedback","langsmith_create_run","langsmith_create_runs_batch","langsmith_get_run","langsmith_update_run","latex_compile","latex_get_package","latex_list_fonts","latex_search_packages","launchdarkly_create_flag","launchdarkly_delete_flag","launchdarkly_get_audit_log","launchdarkly_get_flag","launchdarkly_get_flag_status","launchdarkly_list_environments","launchdarkly_list_flags","launchdarkly_list_members","launchdarkly_list_projects","launchdarkly_list_segments","launchdarkly_toggle_flag","launchdarkly_update_flag","leadmagic_company_search","leadmagic_email_to_profile","leadmagic_find_email","leadmagic_find_mobile","leadmagic_get_credits","leadmagic_profile_search","leadmagic_profile_to_email","leadmagic_role_finder","leadmagic_validate_email","lemlist_get_activities","lemlist_get_lead","lemlist_send_email","linear_add_label_to_issue","linear_add_label_to_project","linear_archive_issue","linear_archive_label","linear_archive_project","linear_create_attachment","linear_create_comment","linear_create_customer","linear_create_customer_request","linear_create_customer_status","linear_create_customer_tier","linear_create_cycle","linear_create_favorite","linear_create_issue","linear_create_issue_relation","linear_create_label","linear_create_project","linear_create_project_label","linear_create_project_milestone","linear_create_project_status","linear_create_project_update","linear_create_workflow_state","linear_delete_attachment","linear_delete_comment","linear_delete_customer","linear_delete_customer_status","linear_delete_customer_tier","linear_delete_issue","linear_delete_issue_relation","linear_delete_project","linear_delete_project_label","linear_delete_project_milestone","linear_delete_project_status","linear_get_active_cycle","linear_get_customer","linear_get_cycle","linear_get_issue","linear_get_project","linear_get_viewer","linear_list_attachments","linear_list_comments","linear_list_customer_requests","linear_list_customer_statuses","linear_list_customer_tiers","linear_list_customers","linear_list_cycles","linear_list_favorites","linear_list_issue_relations","linear_list_labels","linear_list_notifications","linear_list_project_labels","linear_list_project_milestones","linear_list_project_statuses","linear_list_project_updates","linear_list_projects","linear_list_teams","linear_list_users","linear_list_workflow_states","linear_merge_customers","linear_read_issues","linear_remove_label_from_issue","linear_remove_label_from_project","linear_search_issues","linear_unarchive_issue","linear_update_attachment","linear_update_comment","linear_update_customer","linear_update_customer_request","linear_update_customer_status","linear_update_customer_tier","linear_update_issue","linear_update_label","linear_update_notification","linear_update_project","linear_update_project_label","linear_update_project_milestone","linear_update_project_status","linear_update_workflow_state","linkedin_get_profile","linkedin_share_post","linkup_search","linq_add_participant","linq_check_imessage","linq_check_rcs","linq_create_attachment","linq_create_chat","linq_create_contact_card","linq_create_webhook_subscription","linq_delete_attachment","linq_delete_message","linq_delete_webhook_subscription","linq_edit_message","linq_get_attachment","linq_get_chat","linq_get_contact_card","linq_get_message","linq_get_webhook_subscription","linq_leave_chat","linq_list_chats","linq_list_messages","linq_list_phone_numbers","linq_list_thread","linq_list_webhook_events","linq_list_webhook_subscriptions","linq_mark_chat_read","linq_react_to_message","linq_remove_participant","linq_send_message","linq_send_voice_memo","linq_share_contact_card","linq_start_typing","linq_stop_typing","linq_update_chat","linq_update_contact_card","linq_update_webhook_subscription","llm_chat","logfire_get_token_info","logfire_get_trace","logfire_query","logfire_search_records","logrocket_create_release","logrocket_get_audit_logs","logrocket_get_highlights","logrocket_identify_user","logrocket_list_exported_sessions","logrocket_request_highlights","logs_get","logs_get_execution","logs_get_run_details","logs_query","logs_query_runs","loops_check_contact_suppression","loops_create_contact","loops_create_contact_property","loops_delete_contact","loops_find_contact","loops_get_transactional_email","loops_list_contact_properties","loops_list_mailing_lists","loops_list_transactional_emails","loops_remove_contact_suppression","loops_send_event","loops_send_transactional_email","loops_update_contact","luma_add_guests","luma_cancel_event","luma_create_event","luma_get_event","luma_get_guest","luma_get_guests","luma_list_events","luma_lookup_event","luma_send_invites","luma_update_event","luma_update_guest_status","mailchimp_add_member","mailchimp_add_member_tags","mailchimp_add_or_update_member","mailchimp_add_segment_member","mailchimp_add_subscriber_to_automation","mailchimp_archive_member","mailchimp_create_audience","mailchimp_create_batch_operation","mailchimp_create_campaign","mailchimp_create_interest","mailchimp_create_interest_category","mailchimp_create_landing_page","mailchimp_create_merge_field","mailchimp_create_segment","mailchimp_create_template","mailchimp_delete_audience","mailchimp_delete_batch_operation","mailchimp_delete_campaign","mailchimp_delete_interest","mailchimp_delete_interest_category","mailchimp_delete_landing_page","mailchimp_delete_member","mailchimp_delete_merge_field","mailchimp_delete_segment","mailchimp_delete_template","mailchimp_get_audience","mailchimp_get_audiences","mailchimp_get_automation","mailchimp_get_automations","mailchimp_get_batch_operation","mailchimp_get_batch_operations","mailchimp_get_campaign","mailchimp_get_campaign_content","mailchimp_get_campaign_report","mailchimp_get_campaign_reports","mailchimp_get_campaigns","mailchimp_get_interest","mailchimp_get_interest_categories","mailchimp_get_interest_category","mailchimp_get_interests","mailchimp_get_landing_page","mailchimp_get_landing_pages","mailchimp_get_member","mailchimp_get_member_tags","mailchimp_get_members","mailchimp_get_merge_field","mailchimp_get_merge_fields","mailchimp_get_segment","mailchimp_get_segment_members","mailchimp_get_segments","mailchimp_get_template","mailchimp_get_templates","mailchimp_pause_automation","mailchimp_publish_landing_page","mailchimp_remove_member_tags","mailchimp_remove_segment_member","mailchimp_replicate_campaign","mailchimp_schedule_campaign","mailchimp_send_campaign","mailchimp_set_campaign_content","mailchimp_start_automation","mailchimp_unarchive_member","mailchimp_unpublish_landing_page","mailchimp_unschedule_campaign","mailchimp_update_audience","mailchimp_update_campaign","mailchimp_update_interest","mailchimp_update_interest_category","mailchimp_update_landing_page","mailchimp_update_member","mailchimp_update_merge_field","mailchimp_update_segment","mailchimp_update_template","mailgun_add_list_member","mailgun_create_mailing_list","mailgun_get_domain","mailgun_get_mailing_list","mailgun_get_message","mailgun_list_domains","mailgun_list_messages","mailgun_send_message","managed_agent_archive_session","managed_agent_create_session","managed_agent_delete_session","managed_agent_get_session","managed_agent_interrupt_session","managed_agent_list_events","managed_agent_respond_custom_tool","managed_agent_respond_tool_confirmation","managed_agent_run_session","managed_agent_send_message","managed_agent_update_session","mem0_add_memories","mem0_get_memories","mem0_search_memories","memory_add","memory_delete","memory_get","memory_get_all","microsoft_ad_add_directory_role_member","microsoft_ad_add_group_member","microsoft_ad_add_user_app_role_assignment","microsoft_ad_assign_license","microsoft_ad_create_group","microsoft_ad_create_user","microsoft_ad_delete_group","microsoft_ad_delete_user","microsoft_ad_get_conditional_access_policy","microsoft_ad_get_device","microsoft_ad_get_group","microsoft_ad_get_user","microsoft_ad_list_authentication_methods","microsoft_ad_list_conditional_access_policies","microsoft_ad_list_devices","microsoft_ad_list_directory_audits","microsoft_ad_list_directory_role_members","microsoft_ad_list_directory_roles","microsoft_ad_list_group_members","microsoft_ad_list_groups","microsoft_ad_list_service_principal_app_role_assignments","microsoft_ad_list_service_principals","microsoft_ad_list_sign_ins","microsoft_ad_list_subscribed_skus","microsoft_ad_list_user_app_role_assignments","microsoft_ad_list_user_devices","microsoft_ad_list_user_licenses","microsoft_ad_list_users","microsoft_ad_remove_directory_role_member","microsoft_ad_remove_group_member","microsoft_ad_remove_user_app_role_assignment","microsoft_ad_reset_password","microsoft_ad_revoke_sign_in_sessions","microsoft_ad_set_password","microsoft_ad_update_group","microsoft_ad_update_user","microsoft_dataverse_associate","microsoft_dataverse_create_multiple","microsoft_dataverse_create_record","microsoft_dataverse_delete_record","microsoft_dataverse_disassociate","microsoft_dataverse_download_file","microsoft_dataverse_execute_action","microsoft_dataverse_execute_function","microsoft_dataverse_fetchxml_query","microsoft_dataverse_get_entity_metadata","microsoft_dataverse_get_record","microsoft_dataverse_list_records","microsoft_dataverse_search","microsoft_dataverse_update_multiple","microsoft_dataverse_update_record","microsoft_dataverse_upload_file","microsoft_dataverse_upsert_record","microsoft_dataverse_whoami","microsoft_dynamics_365_close_case","microsoft_dynamics_365_close_opportunity","microsoft_dynamics_365_create_record","microsoft_dynamics_365_get_record","microsoft_dynamics_365_list_records","microsoft_dynamics_365_qualify_lead","microsoft_dynamics_365_search_records","microsoft_dynamics_365_update_record","microsoft_excel_clear_range","microsoft_excel_create_table","microsoft_excel_delete_worksheet","microsoft_excel_format_range","microsoft_excel_read","microsoft_excel_read_v2","microsoft_excel_sort_range","microsoft_excel_table_add","microsoft_excel_worksheet_add","microsoft_excel_write","microsoft_excel_write_v2","microsoft_planner_create_bucket","microsoft_planner_create_plan","microsoft_planner_create_task","microsoft_planner_delete_bucket","microsoft_planner_delete_plan","microsoft_planner_delete_task","microsoft_planner_get_plan_details","microsoft_planner_get_task_details","microsoft_planner_list_buckets","microsoft_planner_list_plans","microsoft_planner_read_bucket","microsoft_planner_read_plan","microsoft_planner_read_task","microsoft_planner_update_bucket","microsoft_planner_update_plan","microsoft_planner_update_plan_details","microsoft_planner_update_task","microsoft_planner_update_task_details","microsoft_teams_delete_channel_message","microsoft_teams_delete_chat_message","microsoft_teams_get_message","microsoft_teams_list_channel_members","microsoft_teams_list_channels","microsoft_teams_list_chat_members","microsoft_teams_list_chats","microsoft_teams_list_team_members","microsoft_teams_list_teams","microsoft_teams_read_channel","microsoft_teams_read_chat","microsoft_teams_reply_to_message","microsoft_teams_set_reaction","microsoft_teams_unset_reaction","microsoft_teams_update_channel_message","microsoft_teams_update_chat_message","microsoft_teams_write_channel","microsoft_teams_write_chat","microsoft_word_append","microsoft_word_create","microsoft_word_create_from_template","microsoft_word_export_pdf","microsoft_word_list","microsoft_word_read","microsoft_word_replace_text","microsoft_word_update","millionverifier_get_credits","millionverifier_verify_email","mintlify_create_agent_job","mintlify_create_assistant_message","mintlify_detect_ai_prose","mintlify_get_agent_job","mintlify_get_assistant_caller_stats","mintlify_get_assistant_conversations","mintlify_get_feedback","mintlify_get_feedback_by_page","mintlify_get_page_content","mintlify_get_searches","mintlify_get_update_status","mintlify_get_views","mintlify_get_visitors","mintlify_search","mintlify_send_agent_message","mintlify_trigger_automation","mintlify_trigger_preview","mintlify_trigger_update","mistral_parser","mistral_parser_v2","mistral_parser_v3","modal_call_function","modal_chat_completion","modal_list_models","monday_archive_item","monday_change_column_value","monday_create_board","monday_create_column","monday_create_group","monday_create_item","monday_create_subitem","monday_create_update","monday_delete_item","monday_duplicate_item","monday_get_board","monday_get_groups","monday_get_item","monday_get_items","monday_list_boards","monday_move_item_to_group","monday_search_items","monday_update_item","mongodb_delete","mongodb_execute","mongodb_insert","mongodb_introspect","mongodb_query","mongodb_update","mssql_delete","mssql_execute","mssql_insert","mssql_introspect","mssql_query","mssql_update","mysql_delete","mysql_execute","mysql_insert","mysql_introspect","mysql_query","mysql_update","neo4j_create","neo4j_delete","neo4j_execute","neo4j_introspect","neo4j_merge","neo4j_query","neo4j_update","netsuite_attach_record","netsuite_batch_create_records","netsuite_batch_delete_records","netsuite_batch_get_records","netsuite_batch_update_records","netsuite_batch_upsert_records","netsuite_create_record","netsuite_delete_record","netsuite_detach_record","netsuite_execute_action","netsuite_execute_dataset","netsuite_execute_suiteql","netsuite_get_async_result","netsuite_get_async_status","netsuite_get_governance_limits","netsuite_get_record","netsuite_get_record_form","netsuite_get_record_metadata","netsuite_get_select_options","netsuite_get_server_time","netsuite_get_subresource","netsuite_list_datasets","netsuite_list_record_types","netsuite_list_records","netsuite_transform_record","netsuite_update_record","netsuite_upsert_record","neverbounce_get_credits","neverbounce_verify_email","new_relic_create_deployment_event","new_relic_get_entity","new_relic_nrql_query","new_relic_search_entities","notion_add_database_row","notion_add_database_row_v2","notion_append_blocks","notion_append_blocks_v2","notion_create_comment","notion_create_comment_v2","notion_create_database","notion_create_database_v2","notion_create_page","notion_create_page_v2","notion_delete_block","notion_delete_block_v2","notion_list_comments","notion_list_comments_v2","notion_list_users","notion_list_users_v2","notion_query_database","notion_query_database_v2","notion_read","notion_read_database","notion_read_database_v2","notion_read_v2","notion_retrieve_block","notion_retrieve_block_children","notion_retrieve_block_children_v2","notion_retrieve_block_v2","notion_retrieve_user","notion_retrieve_user_v2","notion_search","notion_search_v2","notion_update_block","notion_update_block_v2","notion_update_page","notion_update_page_v2","notion_write","notion_write_v2","obsidian_append_active","obsidian_append_note","obsidian_append_periodic_note","obsidian_create_note","obsidian_delete_note","obsidian_execute_command","obsidian_get_active","obsidian_get_note","obsidian_get_periodic_note","obsidian_list_commands","obsidian_list_files","obsidian_open_file","obsidian_patch_active","obsidian_patch_note","obsidian_search","okta_activate_group_rule","okta_activate_user","okta_add_user_to_group","okta_assign_group_to_app","okta_assign_user_role","okta_assign_user_to_app","okta_clear_user_sessions","okta_create_group","okta_create_group_rule","okta_create_user","okta_deactivate_group_rule","okta_deactivate_user","okta_delete_group","okta_delete_group_rule","okta_delete_user","okta_enroll_factor","okta_get_app","okta_get_factor","okta_get_group","okta_get_group_rule","okta_get_logs","okta_get_session","okta_get_user","okta_list_app_groups","okta_list_app_users","okta_list_apps","okta_list_factors","okta_list_group_members","okta_list_group_rules","okta_list_groups","okta_list_user_roles","okta_list_users","okta_remove_group_from_app","okta_remove_user_from_app","okta_remove_user_from_group","okta_remove_user_role","okta_reset_all_factors","okta_reset_factor","okta_reset_password","okta_revoke_session","okta_suspend_user","okta_unsuspend_user","okta_update_group","okta_update_user","onedrive_copy","onedrive_create_folder","onedrive_create_share_link","onedrive_delete","onedrive_download","onedrive_get_drive_info","onedrive_get_item","onedrive_list","onedrive_move","onedrive_search","onedrive_upload","onepassword_create_item","onepassword_delete_item","onepassword_get_item","onepassword_get_item_file","onepassword_get_vault","onepassword_list_items","onepassword_list_vaults","onepassword_replace_item","onepassword_resolve_secret","onepassword_update_item","openai_embeddings","openai_image","outlook_calendar_create_event","outlook_calendar_delete_event","outlook_calendar_get_event","outlook_calendar_list_events","outlook_calendar_respond","outlook_calendar_update_event","outlook_copy","outlook_create_folder","outlook_delete","outlook_draft","outlook_forward","outlook_get_attachment","outlook_list_attachments","outlook_list_folders","outlook_mark_read","outlook_mark_unread","outlook_move","outlook_read","outlook_reply","outlook_reply_all","outlook_search","outlook_send","outlook_update_message","pagerduty_add_note","pagerduty_create_incident","pagerduty_get_incident","pagerduty_get_service","pagerduty_list_escalation_policies","pagerduty_list_incident_alerts","pagerduty_list_incidents","pagerduty_list_oncalls","pagerduty_list_schedules","pagerduty_list_services","pagerduty_list_users","pagerduty_merge_incidents","pagerduty_send_event","pagerduty_snooze_incident","pagerduty_update_incident","parallel_deep_research","parallel_extract","parallel_search","pdl_autocomplete","pdl_bulk_company_enrich","pdl_bulk_person_enrich","pdl_clean_company","pdl_clean_location","pdl_clean_school","pdl_company_enrich","pdl_company_search","pdl_person_enrich","pdl_person_identify","pdl_person_search","perplexity_chat","perplexity_search","persona_approve_inquiry","persona_create_account","persona_create_inquiry","persona_create_report","persona_decline_inquiry","persona_expire_inquiry","persona_generate_inquiry_link","persona_get_account","persona_get_case","persona_get_document","persona_get_inquiry","persona_get_report","persona_get_verification","persona_import_accounts","persona_list_accounts","persona_list_cases","persona_list_inquiries","persona_list_inquiry_templates","persona_list_reports","persona_mark_inquiry_for_review","persona_print_inquiry_pdf","persona_redact_account","persona_redact_inquiry","persona_resume_inquiry","persona_update_account","persona_update_inquiry","pinecone_delete_vectors","pinecone_describe_index","pinecone_describe_index_stats","pinecone_fetch","pinecone_generate_embeddings","pinecone_list_indexes","pinecone_list_vector_ids","pinecone_search_text","pinecone_search_vector","pinecone_update_vector","pinecone_upsert_text","pipedrive_create_activity","pipedrive_create_deal","pipedrive_create_lead","pipedrive_create_project","pipedrive_delete_lead","pipedrive_get_activities","pipedrive_get_all_deals","pipedrive_get_deal","pipedrive_get_files","pipedrive_get_leads","pipedrive_get_mail_messages","pipedrive_get_mail_thread","pipedrive_get_pipeline_deals","pipedrive_get_pipelines","pipedrive_get_projects","pipedrive_update_activity","pipedrive_update_deal","pipedrive_update_lead","pitchbook_company_active_investors","pitchbook_company_bio","pitchbook_company_deal_service_providers","pitchbook_company_deals","pitchbook_company_financials","pitchbook_company_general_service_providers","pitchbook_company_industries","pitchbook_company_investors","pitchbook_company_most_recent_debt_financing","pitchbook_company_most_recent_financials","pitchbook_company_most_recent_financing","pitchbook_company_search","pitchbook_company_similar_companies","pitchbook_company_social_analytics","pitchbook_company_updates","pitchbook_company_vc_exit_predictions","pitchbook_contracts_history","pitchbook_cost_of_calls","pitchbook_credit_history","pitchbook_credit_news","pitchbook_credit_news_bulk","pitchbook_credit_news_most_recent","pitchbook_credit_news_search","pitchbook_deal_bio","pitchbook_deal_cap_table_history","pitchbook_deal_debt_lenders","pitchbook_deal_detailed","pitchbook_deal_investors","pitchbook_deal_multiples","pitchbook_deal_search","pitchbook_deal_service_providers","pitchbook_deal_stock_info","pitchbook_deal_tranche_info","pitchbook_deal_updates","pitchbook_deal_valuation","pitchbook_entity_affiliates","pitchbook_entity_locations","pitchbook_entity_news","pitchbook_entity_people","pitchbook_entity_updates","pitchbook_fund_active_investments","pitchbook_fund_benchmark","pitchbook_fund_bio","pitchbook_fund_cash_flows","pitchbook_fund_commitments","pitchbook_fund_investment_preferences","pitchbook_fund_investments","pitchbook_fund_performance","pitchbook_fund_search","pitchbook_fund_team","pitchbook_fund_updates","pitchbook_investor_active_investments","pitchbook_investor_bio","pitchbook_investor_board_seats","pitchbook_investor_deal_service_providers","pitchbook_investor_funds","pitchbook_investor_general_service_providers","pitchbook_investor_investments","pitchbook_investor_last_closed_fund","pitchbook_investor_preferences","pitchbook_investor_search","pitchbook_investor_updates","pitchbook_limited_partner_actual_allocations","pitchbook_limited_partner_bio","pitchbook_limited_partner_commitment_aggregates","pitchbook_limited_partner_commitment_preferences","pitchbook_limited_partner_commitments_detailed","pitchbook_limited_partner_search","pitchbook_limited_partner_service_providers","pitchbook_limited_partner_target_allocations","pitchbook_limited_partner_updates","pitchbook_lookup_table_structure","pitchbook_lookup_tables","pitchbook_patent_detailed","pitchbook_patent_search","pitchbook_people_search","pitchbook_person_bio","pitchbook_person_contact","pitchbook_person_education_work","pitchbook_sandbox_entities","pitchbook_search","pitchbook_service_provider_bio","pitchbook_service_provider_search","pitchbook_service_provider_updates","pitchbook_serviced_companies","pitchbook_serviced_deals","pitchbook_serviced_funds","pitchbook_serviced_investors","pitchbook_serviced_limited_partners","pitchbook_shared_search","pitchbook_usage_report","polymarket_get_activity","polymarket_get_event","polymarket_get_events","polymarket_get_holders","polymarket_get_last_trade_price","polymarket_get_leaderboard","polymarket_get_market","polymarket_get_markets","polymarket_get_midpoint","polymarket_get_orderbook","polymarket_get_positions","polymarket_get_price","polymarket_get_price_history","polymarket_get_series","polymarket_get_series_by_id","polymarket_get_spread","polymarket_get_tags","polymarket_get_tick_size","polymarket_get_trades","polymarket_search","postgresql_delete","postgresql_execute","postgresql_insert","postgresql_introspect","postgresql_query","postgresql_update","posthog_batch_events","posthog_capture_event","posthog_create_annotation","posthog_create_cohort","posthog_create_dashboard","posthog_create_experiment","posthog_create_feature_flag","posthog_create_insight","posthog_create_survey","posthog_delete_feature_flag","posthog_delete_person","posthog_delete_survey","posthog_evaluate_flags","posthog_get_cohort","posthog_get_dashboard","posthog_get_event_definition","posthog_get_experiment","posthog_get_feature_flag","posthog_get_insight","posthog_get_organization","posthog_get_person","posthog_get_project","posthog_get_property_definition","posthog_get_session_recording","posthog_get_survey","posthog_list_actions","posthog_list_annotations","posthog_list_cohorts","posthog_list_dashboards","posthog_list_event_definitions","posthog_list_experiments","posthog_list_feature_flags","posthog_list_insights","posthog_list_organizations","posthog_list_persons","posthog_list_projects","posthog_list_property_definitions","posthog_list_recording_playlists","posthog_list_session_recordings","posthog_list_surveys","posthog_query","posthog_update_cohort","posthog_update_event_definition","posthog_update_experiment","posthog_update_feature_flag","posthog_update_insight","posthog_update_property_definition","posthog_update_survey","profound_bot_logs","profound_bots_report","profound_category_assets","profound_category_personas","profound_category_prompts","profound_category_tags","profound_category_topics","profound_citation_prompts","profound_citations_report","profound_list_assets","profound_list_categories","profound_list_domains","profound_list_models","profound_list_optimizations","profound_list_personas","profound_list_regions","profound_optimization_analysis","profound_prompt_answers","profound_prompt_volume","profound_query_fanouts","profound_raw_logs","profound_referrals_report","profound_sentiment_report","profound_visibility_report","prospeo_account_information","prospeo_bulk_enrich_company","prospeo_bulk_enrich_person","prospeo_enrich_company","prospeo_enrich_person","prospeo_search_company","prospeo_search_person","prospeo_search_suggestions","pulse_parser","pulse_parser_v2","qdrant_fetch_points","qdrant_search_vector","qdrant_upsert_points","quartr_get_audio","quartr_get_company","quartr_get_event","quartr_get_event_summary","quartr_get_report","quartr_get_slide_deck","quartr_get_transcript","quartr_list_audio","quartr_list_companies","quartr_list_document_types","quartr_list_documents","quartr_list_event_types","quartr_list_events","quartr_list_live_events","quartr_list_reports","quartr_list_slide_decks","quartr_list_transcripts","quiver_image_to_svg","quiver_list_models","quiver_text_to_svg","rabbitmq_create_binding","rabbitmq_create_exchange","rabbitmq_create_policy","rabbitmq_create_queue","rabbitmq_delete_binding","rabbitmq_delete_exchange","rabbitmq_delete_policy","rabbitmq_delete_queue","rabbitmq_get_exchange","rabbitmq_get_messages","rabbitmq_get_overview","rabbitmq_get_queue","rabbitmq_health_check","rabbitmq_list_bindings","rabbitmq_list_channels","rabbitmq_list_connections","rabbitmq_list_consumers","rabbitmq_list_exchange_bindings","rabbitmq_list_exchanges","rabbitmq_list_nodes","rabbitmq_list_policies","rabbitmq_list_queues","rabbitmq_list_vhosts","rabbitmq_publish_message","rabbitmq_purge_queue","railway_create_environment","railway_create_project","railway_create_service","railway_delete_environment","railway_delete_project","railway_delete_service","railway_delete_variable","railway_deploy_service","railway_get_deployment","railway_get_deployment_logs","railway_get_project","railway_list_deployments","railway_list_project_members","railway_list_projects","railway_list_variables","railway_restart_deployment","railway_rollback_deployment","railway_transfer_project","railway_update_project","railway_upsert_variable","rb2b_credit_check","rb2b_email_to_activity","rb2b_hem_to_best_linkedin","rb2b_hem_to_business_profile","rb2b_hem_to_linkedin","rb2b_hem_to_maid","rb2b_ip_to_company","rb2b_ip_to_hem","rb2b_ip_to_maid","rb2b_linkedin_slug_search","rb2b_linkedin_to_best_personal_email","rb2b_linkedin_to_business_profile","rb2b_linkedin_to_hashed_emails","rb2b_linkedin_to_mobile_phone","rb2b_linkedin_to_personal_email","rds_delete","rds_execute","rds_insert","rds_introspect","rds_query","rds_update","reddit_delete","reddit_edit","reddit_get_comments","reddit_get_controversial","reddit_get_info","reddit_get_me","reddit_get_messages","reddit_get_posts","reddit_get_saved","reddit_get_subreddit_info","reddit_get_subreddit_rules","reddit_get_user","reddit_get_user_comments","reddit_get_user_posts","reddit_hide","reddit_hot_posts","reddit_list_my_subreddits","reddit_lock","reddit_mark_all_read","reddit_mark_read","reddit_marknsfw","reddit_mod_approve","reddit_mod_distinguish","reddit_mod_remove","reddit_mod_sticky","reddit_reply","reddit_report","reddit_save","reddit_search","reddit_search_subreddits","reddit_send_message","reddit_submit_post","reddit_subscribe","reddit_unhide","reddit_unlock","reddit_unmarknsfw","reddit_unsave","reddit_vote","redis_command","redis_delete","redis_exists","redis_expire","redis_get","redis_hdel","redis_hget","redis_hgetall","redis_hset","redis_incr","redis_incrby","redis_keys","redis_llen","redis_lpop","redis_lpush","redis_lrange","redis_persist","redis_rpop","redis_rpush","redis_set","redis_setnx","redis_ttl","reducto_parser","reducto_parser_v2","resend_cancel_email","resend_create_audience","resend_create_broadcast","resend_create_contact","resend_delete_audience","resend_delete_contact","resend_get_audience","resend_get_broadcast","resend_get_contact","resend_get_email","resend_list_audiences","resend_list_contacts","resend_list_domains","resend_send","resend_send_broadcast","resend_update_contact","revenuecat_create_purchase","revenuecat_defer_google_subscription","revenuecat_delete_customer","revenuecat_get_customer","revenuecat_grant_entitlement","revenuecat_list_offerings","revenuecat_refund_google_subscription","revenuecat_revoke_entitlement","revenuecat_revoke_google_subscription","revenuecat_update_subscriber_attributes","rippling_bulk_create_custom_object_records","rippling_bulk_delete_custom_object_records","rippling_bulk_update_custom_object_records","rippling_create_business_partner","rippling_create_business_partner_group","rippling_create_custom_app","rippling_create_custom_object","rippling_create_custom_object_field","rippling_create_custom_object_record","rippling_create_custom_page","rippling_create_custom_setting","rippling_create_department","rippling_create_draft_hires","rippling_create_object_category","rippling_create_title","rippling_create_work_location","rippling_delete_business_partner","rippling_delete_business_partner_group","rippling_delete_custom_app","rippling_delete_custom_object","rippling_delete_custom_object_field","rippling_delete_custom_object_record","rippling_delete_custom_page","rippling_delete_custom_setting","rippling_delete_object_category","rippling_delete_title","rippling_delete_work_location","rippling_get_business_partner","rippling_get_business_partner_group","rippling_get_current_user","rippling_get_custom_app","rippling_get_custom_object","rippling_get_custom_object_field","rippling_get_custom_object_record","rippling_get_custom_object_record_by_external_id","rippling_get_custom_page","rippling_get_custom_setting","rippling_get_department","rippling_get_employment_type","rippling_get_job_function","rippling_get_object_category","rippling_get_report_run","rippling_get_supergroup","rippling_get_team","rippling_get_title","rippling_get_user","rippling_get_work_location","rippling_get_worker","rippling_list_business_partner_groups","rippling_list_business_partners","rippling_list_companies","rippling_list_custom_apps","rippling_list_custom_fields","rippling_list_custom_object_fields","rippling_list_custom_object_records","rippling_list_custom_objects","rippling_list_custom_pages","rippling_list_custom_settings","rippling_list_departments","rippling_list_employment_types","rippling_list_entitlements","rippling_list_job_functions","rippling_list_object_categories","rippling_list_supergroup_exclusion_members","rippling_list_supergroup_inclusion_members","rippling_list_supergroup_members","rippling_list_supergroups","rippling_list_teams","rippling_list_titles","rippling_list_users","rippling_list_work_locations","rippling_list_workers","rippling_query_custom_object_records","rippling_trigger_report_run","rippling_update_custom_app","rippling_update_custom_object","rippling_update_custom_object_field","rippling_update_custom_object_record","rippling_update_custom_page","rippling_update_custom_setting","rippling_update_department","rippling_update_object_category","rippling_update_supergroup_exclusion_members","rippling_update_supergroup_inclusion_members","rippling_update_title","rippling_update_work_location","rocketlane_add_field_option","rocketlane_add_project_members","rocketlane_add_task_assignees","rocketlane_add_task_dependencies","rocketlane_add_task_followers","rocketlane_archive_project","rocketlane_assign_placeholders","rocketlane_create_field","rocketlane_create_phase","rocketlane_create_project","rocketlane_create_space","rocketlane_create_space_document","rocketlane_create_task","rocketlane_create_time_entry","rocketlane_create_time_off","rocketlane_delete_field","rocketlane_delete_phase","rocketlane_delete_project","rocketlane_delete_space","rocketlane_delete_space_document","rocketlane_delete_task","rocketlane_delete_time_entry","rocketlane_delete_time_off","rocketlane_get_field","rocketlane_get_invoice","rocketlane_get_invoice_line_items","rocketlane_get_invoice_payments","rocketlane_get_phase","rocketlane_get_project","rocketlane_get_space","rocketlane_get_space_document","rocketlane_get_task","rocketlane_get_time_entry","rocketlane_get_time_off","rocketlane_get_user","rocketlane_import_template","rocketlane_list_fields","rocketlane_list_invoices","rocketlane_list_phases","rocketlane_list_placeholders","rocketlane_list_projects","rocketlane_list_resource_allocations","rocketlane_list_space_documents","rocketlane_list_spaces","rocketlane_list_tasks","rocketlane_list_time_entries","rocketlane_list_time_entry_categories","rocketlane_list_time_offs","rocketlane_list_users","rocketlane_move_task_to_phase","rocketlane_remove_project_members","rocketlane_remove_task_assignees","rocketlane_remove_task_dependencies","rocketlane_remove_task_followers","rocketlane_search_time_entries","rocketlane_unassign_placeholders","rocketlane_update_field","rocketlane_update_field_option","rocketlane_update_phase","rocketlane_update_project","rocketlane_update_space","rocketlane_update_space_document","rocketlane_update_task","rocketlane_update_time_entry","rootly_acknowledge_alert","rootly_add_incident_event","rootly_add_subscribers","rootly_assign_incident_role","rootly_create_action_item","rootly_create_alert","rootly_create_incident","rootly_create_status_page_event","rootly_delete_action_item","rootly_delete_incident","rootly_escalate_alert","rootly_get_alert","rootly_get_incident","rootly_list_action_items","rootly_list_alerts","rootly_list_causes","rootly_list_environments","rootly_list_escalation_policies","rootly_list_functionalities","rootly_list_incident_events","rootly_list_incident_roles","rootly_list_incident_types","rootly_list_incidents","rootly_list_on_calls","rootly_list_playbooks","rootly_list_retrospectives","rootly_list_schedules","rootly_list_services","rootly_list_severities","rootly_list_teams","rootly_list_users","rootly_mitigate_incident","rootly_remove_subscribers","rootly_resolve_alert","rootly_resolve_incident","rootly_run_workflow","rootly_snooze_alert","rootly_unassign_incident_role","rootly_update_action_item","rootly_update_alert","rootly_update_incident","s3_copy_object","s3_create_bucket","s3_delete_bucket","s3_delete_object","s3_delete_objects","s3_get_object","s3_head_object","s3_list_buckets","s3_list_objects","s3_presigned_url","s3_put_object","salesforce_create_account","salesforce_create_case","salesforce_create_contact","salesforce_create_custom_field","salesforce_create_custom_object","salesforce_create_lead","salesforce_create_opportunity","salesforce_create_task","salesforce_delete_account","salesforce_delete_case","salesforce_delete_contact","salesforce_delete_custom_field","salesforce_delete_lead","salesforce_delete_opportunity","salesforce_delete_task","salesforce_describe_object","salesforce_get_accounts","salesforce_get_cases","salesforce_get_contacts","salesforce_get_dashboard","salesforce_get_leads","salesforce_get_opportunities","salesforce_get_report","salesforce_get_tasks","salesforce_list_dashboards","salesforce_list_objects","salesforce_list_report_types","salesforce_list_reports","salesforce_query","salesforce_query_more","salesforce_refresh_dashboard","salesforce_run_report","salesforce_tooling_query","salesforce_update_account","salesforce_update_case","salesforce_update_contact","salesforce_update_custom_field","salesforce_update_lead","salesforce_update_opportunity","salesforce_update_task","sap_concur_approve_expense_report","sap_concur_associate_attendees","sap_concur_create_cash_advance","sap_concur_create_expected_expense","sap_concur_create_expense_report","sap_concur_create_list_item","sap_concur_create_purchase_request","sap_concur_create_quick_expense","sap_concur_create_quick_expense_with_image","sap_concur_create_report_comment","sap_concur_create_travel_request","sap_concur_create_user","sap_concur_delete_expected_expense","sap_concur_delete_expense","sap_concur_delete_expense_report","sap_concur_delete_list_item","sap_concur_delete_travel_request","sap_concur_delete_user","sap_concur_get_allocation","sap_concur_get_budget","sap_concur_get_cash_advance","sap_concur_get_expected_expense","sap_concur_get_expense","sap_concur_get_expense_report","sap_concur_get_itemizations","sap_concur_get_itinerary","sap_concur_get_list","sap_concur_get_list_item","sap_concur_get_purchase_request","sap_concur_get_receipt","sap_concur_get_receipt_status","sap_concur_get_request_cash_advance","sap_concur_get_travel_profile","sap_concur_get_travel_request","sap_concur_get_user","sap_concur_issue_cash_advance","sap_concur_list_allocations","sap_concur_list_attendee_associations","sap_concur_list_budget_categories","sap_concur_list_budgets","sap_concur_list_exceptions","sap_concur_list_expected_expenses","sap_concur_list_expense_reports","sap_concur_list_expenses","sap_concur_list_itineraries","sap_concur_list_list_items","sap_concur_list_lists","sap_concur_list_receipts","sap_concur_list_report_comments","sap_concur_list_reports_to_approve","sap_concur_list_travel_profiles_summary","sap_concur_list_travel_request_comments","sap_concur_list_travel_requests","sap_concur_list_users","sap_concur_move_travel_request","sap_concur_recall_expense_report","sap_concur_remove_all_attendees","sap_concur_search_locations","sap_concur_search_users","sap_concur_send_back_expense_report","sap_concur_submit_expense_report","sap_concur_update_allocation","sap_concur_update_expected_expense","sap_concur_update_expense","sap_concur_update_expense_report","sap_concur_update_list_item","sap_concur_update_travel_request","sap_concur_update_user","sap_concur_upload_exchange_rates","sap_concur_upload_receipt_image","sap_s4hana_create_business_partner","sap_s4hana_create_purchase_order","sap_s4hana_create_purchase_requisition","sap_s4hana_create_sales_order","sap_s4hana_delete_sales_order","sap_s4hana_get_billing_document","sap_s4hana_get_business_partner","sap_s4hana_get_customer","sap_s4hana_get_inbound_delivery","sap_s4hana_get_material_document","sap_s4hana_get_outbound_delivery","sap_s4hana_get_product","sap_s4hana_get_purchase_order","sap_s4hana_get_purchase_requisition","sap_s4hana_get_sales_order","sap_s4hana_get_supplier","sap_s4hana_get_supplier_invoice","sap_s4hana_list_billing_documents","sap_s4hana_list_business_partners","sap_s4hana_list_customers","sap_s4hana_list_inbound_deliveries","sap_s4hana_list_material_documents","sap_s4hana_list_material_stock","sap_s4hana_list_outbound_deliveries","sap_s4hana_list_products","sap_s4hana_list_purchase_orders","sap_s4hana_list_purchase_requisitions","sap_s4hana_list_sales_orders","sap_s4hana_list_supplier_invoices","sap_s4hana_list_suppliers","sap_s4hana_odata_query","sap_s4hana_update_business_partner","sap_s4hana_update_customer","sap_s4hana_update_product","sap_s4hana_update_purchase_order","sap_s4hana_update_purchase_requisition","sap_s4hana_update_sales_order","sap_s4hana_update_supplier","search_tool","secrets_manager_create_secret","secrets_manager_delete_secret","secrets_manager_describe_secret","secrets_manager_get_secret","secrets_manager_list_secrets","secrets_manager_restore_secret","secrets_manager_rotate_secret","secrets_manager_tag_resource","secrets_manager_untag_resource","secrets_manager_update_secret","semrush_backlinks","semrush_backlinks_anchors","semrush_backlinks_competitors","semrush_backlinks_geo_distribution","semrush_backlinks_indexed_pages","semrush_backlinks_overview","semrush_backlinks_tld_distribution","semrush_batch_keyword_overview","semrush_broad_match_keywords","semrush_domain_ad_copies","semrush_domain_ad_history","semrush_domain_organic_competitors","semrush_domain_organic_keywords","semrush_domain_overview","semrush_domain_overview_all","semrush_domain_overview_history","semrush_domain_paid_competitors","semrush_domain_paid_keywords","semrush_domain_pla_copies","semrush_domain_pla_keywords","semrush_domain_vs_domain","semrush_keyword_ad_history","semrush_keyword_difficulty","semrush_keyword_overview","semrush_keyword_overview_all","semrush_keyword_questions","semrush_organic_results","semrush_paid_results","semrush_referring_domains","semrush_referring_ips","semrush_related_keywords","semrush_subdomain_ad_copies","semrush_subdomain_organic_keywords","semrush_subdomain_overview","semrush_subdomain_overview_all","semrush_subdomain_overview_history","semrush_subdomain_paid_keywords","semrush_top_domains","semrush_url_organic_keywords","semrush_url_overview","semrush_url_overview_all","semrush_url_overview_history","semrush_url_paid_keywords","semrush_winners_and_losers","sendblue_evaluate_service","sendblue_get_message","sendblue_send_group_message","sendblue_send_message","sendblue_send_typing_indicator","sendgrid_add_contact","sendgrid_add_contacts_to_list","sendgrid_create_list","sendgrid_create_template","sendgrid_create_template_version","sendgrid_delete_contacts","sendgrid_delete_list","sendgrid_delete_template","sendgrid_get_contact","sendgrid_get_list","sendgrid_get_template","sendgrid_list_all_lists","sendgrid_list_templates","sendgrid_remove_contacts_from_list","sendgrid_search_contacts","sendgrid_send_mail","sentry_events_get","sentry_events_list","sentry_issues_get","sentry_issues_list","sentry_issues_update","sentry_projects_create","sentry_projects_get","sentry_projects_list","sentry_projects_update","sentry_releases_create","sentry_releases_deploy","sentry_releases_list","sentry_teams_list","serper_search","servicenow_add_incident_comment","servicenow_aggregate","servicenow_close_incident","servicenow_create_change_request","servicenow_create_incident","servicenow_create_record","servicenow_delete_record","servicenow_download_attachment","servicenow_find_user","servicenow_get_change_next_states","servicenow_get_change_request","servicenow_get_ci","servicenow_get_incident","servicenow_get_knowledge_article","servicenow_get_requested_item","servicenow_list_approvals","servicenow_list_attachments","servicenow_list_catalog_items","servicenow_list_change_requests","servicenow_list_change_tasks","servicenow_list_ci_relationships","servicenow_list_group_members","servicenow_list_incidents","servicenow_list_requested_items","servicenow_order_catalog_item","servicenow_read_record","servicenow_resolve_incident","servicenow_search_cis","servicenow_search_knowledge","servicenow_update_approval","servicenow_update_change_request","servicenow_update_change_state","servicenow_update_incident","servicenow_update_record","servicenow_upload_attachment","ses_create_configuration_set","ses_create_email_identity","ses_create_template","ses_delete_email_identity","ses_delete_suppressed_destination","ses_delete_template","ses_get_account","ses_get_email_identity","ses_get_suppressed_destination","ses_get_template","ses_list_identities","ses_list_suppressed_destinations","ses_list_templates","ses_put_suppressed_destination","ses_send_bulk_email","ses_send_custom_verification_email","ses_send_email","ses_send_templated_email","ses_update_template","sftp_delete","sftp_download","sftp_list","sftp_mkdir","sftp_upload","sharepoint_add_list_items","sharepoint_create_list","sharepoint_create_page","sharepoint_delete_file","sharepoint_delete_list_item","sharepoint_delete_page","sharepoint_download_file","sharepoint_get_drive_item","sharepoint_get_list","sharepoint_get_list_item","sharepoint_list_sites","sharepoint_publish_page","sharepoint_read_page","sharepoint_update_list","sharepoint_update_page","sharepoint_upload_file","shopify_adjust_inventory","shopify_cancel_order","shopify_create_customer","shopify_create_fulfillment","shopify_create_product","shopify_delete_customer","shopify_delete_product","shopify_get_collection","shopify_get_customer","shopify_get_inventory_level","shopify_get_order","shopify_get_product","shopify_list_collections","shopify_list_customers","shopify_list_inventory_items","shopify_list_locations","shopify_list_orders","shopify_list_products","shopify_update_customer","shopify_update_order","shopify_update_product","similarweb_bounce_rate","similarweb_page_views","similarweb_pages_per_visit","similarweb_traffic_visits","similarweb_visit_duration","similarweb_website_overview","sixtyfour_enrich_company","sixtyfour_enrich_lead","sixtyfour_find_email","sixtyfour_find_phone","slack_add_reaction","slack_archive_conversation","slack_canvas","slack_create_channel_canvas","slack_create_conversation","slack_delete_canvas","slack_delete_message","slack_delete_scheduled_message","slack_download","slack_edit_canvas","slack_ephemeral_message","slack_get_canvas","slack_get_channel_history","slack_get_channel_info","slack_get_message","slack_get_permalink","slack_get_thread","slack_get_thread_replies","slack_get_user","slack_get_user_presence","slack_invite_to_conversation","slack_list_canvases","slack_list_channels","slack_list_members","slack_list_scheduled_messages","slack_list_users","slack_lookup_canvas_sections","slack_message","slack_message_reader","slack_open_view","slack_publish_view","slack_push_view","slack_remove_reaction","slack_rename_conversation","slack_schedule_message","slack_set_conversation_purpose","slack_set_conversation_topic","slack_set_status","slack_set_suggested_prompts","slack_set_title","slack_update_message","slack_update_view","smartlead_add_email_accounts_to_campaign","smartlead_add_leads_to_campaign","smartlead_create_campaign","smartlead_create_lead_list","smartlead_delete_campaign","smartlead_delete_campaign_webhook","smartlead_delete_lead_from_campaign","smartlead_delete_lead_list","smartlead_duplicate_campaign","smartlead_export_campaign_leads","smartlead_get_campaign","smartlead_get_campaign_analytics","smartlead_get_campaign_analytics_by_date","smartlead_get_campaign_lead_statistics","smartlead_get_campaign_mailbox_statistics","smartlead_get_campaign_sequences","smartlead_get_campaign_statistics","smartlead_get_campaign_top_level_analytics_by_date","smartlead_get_campaign_webhook_summary","smartlead_get_lead_by_email","smartlead_get_lead_by_id","smartlead_get_lead_list","smartlead_get_lead_message_history","smartlead_list_campaign_email_accounts","smartlead_list_campaign_leads","smartlead_list_campaign_webhooks","smartlead_list_campaigns","smartlead_list_clients","smartlead_list_email_accounts","smartlead_list_inbox_replies","smartlead_list_lead_activities","smartlead_list_lead_categories","smartlead_list_lead_lists","smartlead_mark_lead_complete","smartlead_pause_lead","smartlead_remove_email_accounts_from_campaign","smartlead_resume_lead","smartlead_save_campaign_sequences","smartlead_unsubscribe_lead_from_campaign","smartlead_unsubscribe_lead_globally","smartlead_update_campaign_schedule","smartlead_update_campaign_settings","smartlead_update_campaign_status","smartlead_update_lead","smartlead_update_lead_category","smartlead_update_lead_list","smartlead_upsert_campaign_webhook","sms_send","smtp_send_mail","snowflake_alter_warehouse","snowflake_call_procedure","snowflake_cancel_statement","snowflake_cancel_task_run","snowflake_delete_rows","snowflake_execute_sql","snowflake_get_statement","snowflake_get_task","snowflake_get_task_run","snowflake_get_task_run_output","snowflake_get_warehouse","snowflake_insert_rows","snowflake_introspect_schema","snowflake_list_copy_history","snowflake_list_databases","snowflake_list_query_history","snowflake_list_schemas","snowflake_list_tables","snowflake_list_task_runs","snowflake_list_tasks","snowflake_list_warehouses","snowflake_load_data","snowflake_resume_task","snowflake_resume_warehouse","snowflake_run_task","snowflake_suspend_task","snowflake_suspend_warehouse","snowflake_unload_data","snowflake_update_rows","snowflake_upsert_rows","splunk_cancel_search_job","splunk_create_search_job","splunk_dispatch_saved_search","splunk_get_fired_alerts","splunk_get_saved_search","splunk_get_search_job","splunk_get_search_results","splunk_list_apps","splunk_list_fired_alerts","splunk_list_indexes","splunk_list_saved_searches","splunk_run_search","sportmonks_core_get_cities","sportmonks_core_get_city","sportmonks_core_get_continent","sportmonks_core_get_continents","sportmonks_core_get_countries","sportmonks_core_get_country","sportmonks_core_get_entity_filters","sportmonks_core_get_my_usage","sportmonks_core_get_region","sportmonks_core_get_regions","sportmonks_core_get_timezones","sportmonks_core_get_type","sportmonks_core_get_type_by_entity","sportmonks_core_get_types","sportmonks_core_search_cities","sportmonks_core_search_countries","sportmonks_core_search_regions","sportmonks_football_expected_by_player","sportmonks_football_expected_by_team","sportmonks_football_get_all_commentaries","sportmonks_football_get_all_fixtures","sportmonks_football_get_all_players","sportmonks_football_get_all_rivals","sportmonks_football_get_all_teams","sportmonks_football_get_all_transfer_rumours","sportmonks_football_get_all_transfers","sportmonks_football_get_brackets_by_season","sportmonks_football_get_coach","sportmonks_football_get_coaches","sportmonks_football_get_coaches_by_country","sportmonks_football_get_commentaries_by_fixture","sportmonks_football_get_current_leagues_by_team","sportmonks_football_get_expected_lineups_by_player","sportmonks_football_get_expected_lineups_by_team","sportmonks_football_get_extended_team_squad","sportmonks_football_get_fixture","sportmonks_football_get_fixtures_by_date","sportmonks_football_get_fixtures_by_date_range","sportmonks_football_get_fixtures_by_date_range_for_team","sportmonks_football_get_fixtures_by_ids","sportmonks_football_get_grouped_standings_by_round","sportmonks_football_get_head_to_head","sportmonks_football_get_inplay_livescores","sportmonks_football_get_latest_coaches","sportmonks_football_get_latest_fixtures","sportmonks_football_get_latest_livescores","sportmonks_football_get_latest_players","sportmonks_football_get_latest_totw","sportmonks_football_get_latest_transfers","sportmonks_football_get_league","sportmonks_football_get_leagues","sportmonks_football_get_leagues_by_country","sportmonks_football_get_leagues_by_date","sportmonks_football_get_leagues_by_team","sportmonks_football_get_live_leagues","sportmonks_football_get_live_probabilities","sportmonks_football_get_live_probabilities_by_fixture","sportmonks_football_get_live_standings_by_league","sportmonks_football_get_livescores","sportmonks_football_get_match_facts","sportmonks_football_get_match_facts_by_date_range","sportmonks_football_get_match_facts_by_fixture","sportmonks_football_get_match_facts_by_league","sportmonks_football_get_past_fixtures_by_tv_station","sportmonks_football_get_player","sportmonks_football_get_players_by_country","sportmonks_football_get_postmatch_news","sportmonks_football_get_postmatch_news_by_season","sportmonks_football_get_predictability_by_league","sportmonks_football_get_prematch_news","sportmonks_football_get_prematch_news_by_season","sportmonks_football_get_prematch_news_upcoming","sportmonks_football_get_probabilities","sportmonks_football_get_probabilities_by_fixture","sportmonks_football_get_referee","sportmonks_football_get_referees","sportmonks_football_get_referees_by_country","sportmonks_football_get_referees_by_season","sportmonks_football_get_rivals_by_team","sportmonks_football_get_round","sportmonks_football_get_round_statistics","sportmonks_football_get_rounds","sportmonks_football_get_rounds_by_season","sportmonks_football_get_schedules_by_season","sportmonks_football_get_schedules_by_season_and_team","sportmonks_football_get_schedules_by_team","sportmonks_football_get_season","sportmonks_football_get_seasons","sportmonks_football_get_seasons_by_team","sportmonks_football_get_stage","sportmonks_football_get_stage_statistics","sportmonks_football_get_stages","sportmonks_football_get_stages_by_season","sportmonks_football_get_standing_corrections_by_season","sportmonks_football_get_standings","sportmonks_football_get_standings_by_round","sportmonks_football_get_standings_by_season","sportmonks_football_get_state","sportmonks_football_get_states","sportmonks_football_get_team","sportmonks_football_get_team_rankings","sportmonks_football_get_team_rankings_by_date","sportmonks_football_get_team_rankings_by_team","sportmonks_football_get_team_squad","sportmonks_football_get_team_squad_by_season","sportmonks_football_get_teams_by_country","sportmonks_football_get_teams_by_season","sportmonks_football_get_topscorers_by_season","sportmonks_football_get_topscorers_by_stage","sportmonks_football_get_totw","sportmonks_football_get_totw_by_round","sportmonks_football_get_transfer","sportmonks_football_get_transfer_rumour","sportmonks_football_get_transfer_rumours_between_dates","sportmonks_football_get_transfer_rumours_by_player","sportmonks_football_get_transfer_rumours_by_team","sportmonks_football_get_transfers_between_dates","sportmonks_football_get_transfers_by_player","sportmonks_football_get_transfers_by_team","sportmonks_football_get_tv_station","sportmonks_football_get_tv_stations","sportmonks_football_get_tv_stations_by_fixture","sportmonks_football_get_upcoming_fixtures_by_market","sportmonks_football_get_upcoming_fixtures_by_tv_station","sportmonks_football_get_value_bets","sportmonks_football_get_value_bets_by_fixture","sportmonks_football_get_venue","sportmonks_football_get_venues","sportmonks_football_get_venues_by_season","sportmonks_football_search_coaches","sportmonks_football_search_fixtures","sportmonks_football_search_leagues","sportmonks_football_search_players","sportmonks_football_search_referees","sportmonks_football_search_rounds","sportmonks_football_search_seasons","sportmonks_football_search_stages","sportmonks_football_search_teams","sportmonks_football_search_venues","sportmonks_motorsport_get_all_fixtures","sportmonks_motorsport_get_current_leagues_by_team","sportmonks_motorsport_get_driver","sportmonks_motorsport_get_driver_standings","sportmonks_motorsport_get_driver_standings_by_season","sportmonks_motorsport_get_drivers","sportmonks_motorsport_get_drivers_by_country","sportmonks_motorsport_get_drivers_by_season","sportmonks_motorsport_get_fixture","sportmonks_motorsport_get_fixtures_by_date","sportmonks_motorsport_get_fixtures_by_date_range","sportmonks_motorsport_get_fixtures_by_ids","sportmonks_motorsport_get_laps_by_fixture","sportmonks_motorsport_get_laps_by_fixture_and_driver","sportmonks_motorsport_get_laps_by_fixture_and_lap","sportmonks_motorsport_get_latest_laps_by_fixture","sportmonks_motorsport_get_latest_pitstops_by_fixture","sportmonks_motorsport_get_latest_stints_by_fixture","sportmonks_motorsport_get_latest_updated_drivers","sportmonks_motorsport_get_latest_updated_fixtures","sportmonks_motorsport_get_league","sportmonks_motorsport_get_leagues","sportmonks_motorsport_get_leagues_by_country","sportmonks_motorsport_get_leagues_by_date","sportmonks_motorsport_get_leagues_by_live","sportmonks_motorsport_get_leagues_by_team","sportmonks_motorsport_get_livescores","sportmonks_motorsport_get_pitstops_by_fixture","sportmonks_motorsport_get_pitstops_by_fixture_and_driver","sportmonks_motorsport_get_pitstops_by_fixture_and_lap","sportmonks_motorsport_get_race_results_by_season_and_driver","sportmonks_motorsport_get_race_results_by_season_and_team","sportmonks_motorsport_get_schedules_by_season","sportmonks_motorsport_get_season","sportmonks_motorsport_get_seasons","sportmonks_motorsport_get_stage","sportmonks_motorsport_get_stages","sportmonks_motorsport_get_stages_by_season","sportmonks_motorsport_get_state","sportmonks_motorsport_get_states","sportmonks_motorsport_get_stints_by_fixture","sportmonks_motorsport_get_stints_by_fixture_and_driver","sportmonks_motorsport_get_stints_by_fixture_and_stint","sportmonks_motorsport_get_team","sportmonks_motorsport_get_team_standings","sportmonks_motorsport_get_team_standings_by_season","sportmonks_motorsport_get_teams","sportmonks_motorsport_get_teams_by_country","sportmonks_motorsport_get_teams_by_season","sportmonks_motorsport_get_venue","sportmonks_motorsport_get_venues","sportmonks_motorsport_get_venues_by_season","sportmonks_motorsport_search_drivers","sportmonks_motorsport_search_leagues","sportmonks_motorsport_search_stages","sportmonks_motorsport_search_teams","sportmonks_motorsport_search_venues","sportmonks_odds_get_all_historical_odds","sportmonks_odds_get_all_inplay_odds","sportmonks_odds_get_all_pre_match_odds","sportmonks_odds_get_all_premium_odds","sportmonks_odds_get_bookmaker","sportmonks_odds_get_bookmaker_event_ids_by_fixture","sportmonks_odds_get_bookmakers","sportmonks_odds_get_bookmakers_by_fixture","sportmonks_odds_get_inplay_odds_by_fixture","sportmonks_odds_get_inplay_odds_by_fixture_and_bookmaker","sportmonks_odds_get_inplay_odds_by_fixture_and_market","sportmonks_odds_get_last_updated_inplay_odds","sportmonks_odds_get_last_updated_pre_match_odds","sportmonks_odds_get_market","sportmonks_odds_get_markets","sportmonks_odds_get_pre_match_odds_by_fixture","sportmonks_odds_get_pre_match_odds_by_fixture_and_bookmaker","sportmonks_odds_get_pre_match_odds_by_fixture_and_market","sportmonks_odds_get_premium_odds_by_fixture","sportmonks_odds_get_premium_odds_by_fixture_and_bookmaker","sportmonks_odds_get_premium_odds_by_fixture_and_market","sportmonks_odds_get_updated_historical_odds_between","sportmonks_odds_get_updated_premium_odds_between","sportmonks_odds_search_bookmakers","sportmonks_odds_search_markets","spotify_add_playlist_cover","spotify_add_to_queue","spotify_add_tracks_to_playlist","spotify_check_following","spotify_check_playlist_followers","spotify_check_saved_albums","spotify_check_saved_audiobooks","spotify_check_saved_episodes","spotify_check_saved_shows","spotify_check_saved_tracks","spotify_create_playlist","spotify_follow_artists","spotify_follow_playlist","spotify_get_album","spotify_get_album_tracks","spotify_get_albums","spotify_get_artist","spotify_get_artist_albums","spotify_get_artist_top_tracks","spotify_get_artists","spotify_get_audiobook","spotify_get_audiobook_chapters","spotify_get_audiobooks","spotify_get_categories","spotify_get_current_user","spotify_get_currently_playing","spotify_get_devices","spotify_get_episode","spotify_get_episodes","spotify_get_followed_artists","spotify_get_markets","spotify_get_new_releases","spotify_get_playback_state","spotify_get_playlist","spotify_get_playlist_cover","spotify_get_playlist_tracks","spotify_get_queue","spotify_get_recently_played","spotify_get_saved_albums","spotify_get_saved_audiobooks","spotify_get_saved_episodes","spotify_get_saved_shows","spotify_get_saved_tracks","spotify_get_show","spotify_get_show_episodes","spotify_get_shows","spotify_get_top_artists","spotify_get_top_tracks","spotify_get_track","spotify_get_tracks","spotify_get_user_playlists","spotify_get_user_profile","spotify_pause","spotify_play","spotify_remove_saved_albums","spotify_remove_saved_audiobooks","spotify_remove_saved_episodes","spotify_remove_saved_shows","spotify_remove_saved_tracks","spotify_remove_tracks_from_playlist","spotify_reorder_playlist_items","spotify_replace_playlist_items","spotify_save_albums","spotify_save_audiobooks","spotify_save_episodes","spotify_save_shows","spotify_save_tracks","spotify_search","spotify_seek","spotify_set_repeat","spotify_set_shuffle","spotify_set_volume","spotify_skip_next","spotify_skip_previous","spotify_transfer_playback","spotify_unfollow_artists","spotify_unfollow_playlist","spotify_update_playlist","sqs_send","square_batch_retrieve_inventory_counts","square_cancel_invoice","square_cancel_payment","square_complete_payment","square_create_catalog_image","square_create_customer","square_create_invoice","square_create_order","square_create_payment","square_delete_catalog_object","square_delete_customer","square_delete_invoice","square_get_catalog_object","square_get_customer","square_get_invoice","square_get_location","square_get_order","square_get_payment","square_get_refund","square_list_catalog","square_list_customers","square_list_invoices","square_list_locations","square_list_payments","square_list_refunds","square_pay_order","square_publish_invoice","square_refund_payment","square_search_catalog_objects","square_search_customers","square_search_invoices","square_search_orders","square_update_customer","square_upsert_catalog_object","ssh_check_command_exists","ssh_check_file_exists","ssh_create_directory","ssh_delete_file","ssh_download_file","ssh_execute_command","ssh_execute_script","ssh_get_system_info","ssh_list_directory","ssh_move_rename","ssh_read_file_content","ssh_upload_file","ssh_write_file_content","stagehand_agent","stagehand_extract","stripe_cancel_payment_intent","stripe_cancel_subscription","stripe_capture_charge","stripe_capture_payment_intent","stripe_confirm_payment_intent","stripe_create_charge","stripe_create_customer","stripe_create_invoice","stripe_create_payment_intent","stripe_create_price","stripe_create_product","stripe_create_subscription","stripe_delete_customer","stripe_delete_invoice","stripe_delete_product","stripe_finalize_invoice","stripe_list_charges","stripe_list_customers","stripe_list_events","stripe_list_invoices","stripe_list_payment_intents","stripe_list_prices","stripe_list_products","stripe_list_subscriptions","stripe_pay_invoice","stripe_resume_subscription","stripe_retrieve_charge","stripe_retrieve_customer","stripe_retrieve_event","stripe_retrieve_invoice","stripe_retrieve_payment_intent","stripe_retrieve_price","stripe_retrieve_product","stripe_retrieve_subscription","stripe_search_charges","stripe_search_customers","stripe_search_invoices","stripe_search_payment_intents","stripe_search_prices","stripe_search_products","stripe_search_subscriptions","stripe_send_invoice","stripe_update_charge","stripe_update_customer","stripe_update_invoice","stripe_update_payment_intent","stripe_update_price","stripe_update_product","stripe_update_subscription","stripe_void_invoice","sts_assume_role","sts_assume_role_with_saml","sts_assume_role_with_web_identity","sts_get_access_key_info","sts_get_caller_identity","sts_get_session_token","stt_assemblyai","stt_assemblyai_v2","stt_deepgram","stt_deepgram_v2","stt_elevenlabs","stt_elevenlabs_v2","stt_gemini","stt_gemini_v2","stt_whisper","stt_whisper_v2","supabase_count","supabase_delete","supabase_get_row","supabase_insert","supabase_introspect","supabase_invoke_function","supabase_query","supabase_rpc","supabase_storage_copy","supabase_storage_create_bucket","supabase_storage_create_signed_upload_url","supabase_storage_create_signed_url","supabase_storage_delete","supabase_storage_delete_bucket","supabase_storage_download","supabase_storage_empty_bucket","supabase_storage_get_public_url","supabase_storage_list","supabase_storage_list_buckets","supabase_storage_move","supabase_storage_update_bucket","supabase_storage_upload","supabase_text_search","supabase_update","supabase_upsert","supabase_vector_search","table_batch_insert_rows","table_create","table_delete_row","table_delete_rows_by_filter","table_get_row","table_get_schema","table_insert_row","table_list","table_query_rows","table_query_rows_v2","table_update_row","table_update_rows_by_filter","table_upsert_row","tailscale_authorize_device","tailscale_create_auth_key","tailscale_delete_auth_key","tailscale_delete_device","tailscale_delete_user","tailscale_expire_device_key","tailscale_get_acl","tailscale_get_auth_key","tailscale_get_device","tailscale_get_device_routes","tailscale_get_dns_preferences","tailscale_get_dns_searchpaths","tailscale_list_auth_keys","tailscale_list_devices","tailscale_list_dns_nameservers","tailscale_list_users","tailscale_set_acl","tailscale_set_device_routes","tailscale_set_device_tags","tailscale_set_dns_nameservers","tailscale_set_dns_preferences","tailscale_set_dns_searchpaths","tailscale_suspend_user","tailscale_update_device_key","tavily_crawl","tavily_extract","tavily_map","tavily_search","telegram_copy_message","telegram_delete_message","telegram_edit_message_text","telegram_forward_message","telegram_get_chat","telegram_get_chat_member","telegram_message","telegram_pin_message","telegram_send_animation","telegram_send_audio","telegram_send_chat_action","telegram_send_contact","telegram_send_document","telegram_send_location","telegram_send_photo","telegram_send_poll","telegram_send_video","telegram_set_message_reaction","telegram_unpin_message","temporal_cancel_workflow","temporal_count_workflows","temporal_create_schedule","temporal_delete_schedule","temporal_describe_schedule","temporal_describe_task_queue","temporal_describe_workflow","temporal_get_workflow_history","temporal_list_schedules","temporal_list_workflows","temporal_pause_schedule","temporal_query_workflow","temporal_reset_workflow","temporal_signal_with_start","temporal_signal_workflow","temporal_start_workflow","temporal_terminate_workflow","temporal_trigger_schedule","temporal_unpause_schedule","temporal_update_workflow","textract_analyze_expense","textract_analyze_id","textract_parser","textract_parser_v2","thinking_tool","thrive_add_audience_managers","thrive_add_audience_members","thrive_add_user_tags","thrive_create_assignment","thrive_create_audience","thrive_create_completion","thrive_create_user","thrive_delete_assignment","thrive_delete_audience","thrive_delete_user","thrive_get_activity","thrive_get_assignment","thrive_get_audience","thrive_get_completion","thrive_get_content","thrive_get_cpd_category","thrive_get_cpd_entry","thrive_get_cpd_requirement","thrive_get_enrolment","thrive_get_skill_levels","thrive_get_tag","thrive_get_user_by_id","thrive_get_user_by_ref","thrive_list_assignments","thrive_list_audience_managers","thrive_list_audience_members","thrive_list_audiences","thrive_list_completions","thrive_list_enrolments","thrive_list_tags","thrive_query_activities","thrive_query_content","thrive_query_cpd_categories","thrive_query_cpd_entries","thrive_query_cpd_requirements","thrive_query_cpd_user_summaries","thrive_remove_audience_manager","thrive_remove_audience_member","thrive_remove_user_tags","thrive_replace_audience_managers","thrive_replace_audience_members","thrive_search_users","thrive_suspend_user","thrive_update_assignment","thrive_update_audience","thrive_update_user","thrive_update_user_skills","tiktok_get_post_status","tiktok_get_user","tiktok_list_videos","tiktok_query_videos","tiktok_upload_video_draft","tinybird_append_datasource","tinybird_delete_datasource_rows","tinybird_events","tinybird_get_job","tinybird_query","tinybird_query_pipe","tinybird_truncate_datasource","tinyfish_cancel_run","tinyfish_fetch","tinyfish_get_run","tinyfish_list_runs","tinyfish_list_vault_items","tinyfish_run","tinyfish_run_async","tinyfish_search","trello_add_checklist","trello_add_checklist_item","trello_add_comment","trello_add_label","trello_add_member","trello_create_board","trello_create_card","trello_create_list","trello_delete_card","trello_get_actions","trello_get_board","trello_get_card","trello_list_cards","trello_list_lists","trello_list_members","trello_remove_label","trello_remove_member","trello_search","trello_update_card","trello_update_checklist_item","trello_update_list","trigger_dev_activate_schedule","trigger_dev_add_run_tags","trigger_dev_batch_trigger_task","trigger_dev_cancel_run","trigger_dev_complete_waitpoint_token","trigger_dev_create_env_var","trigger_dev_create_schedule","trigger_dev_create_waitpoint_token","trigger_dev_deactivate_schedule","trigger_dev_delete_env_var","trigger_dev_delete_schedule","trigger_dev_execute_query","trigger_dev_get_batch","trigger_dev_get_batch_results","trigger_dev_get_deployment","trigger_dev_get_env_var","trigger_dev_get_latest_deployment","trigger_dev_get_query_schema","trigger_dev_get_queue","trigger_dev_get_run","trigger_dev_get_run_events","trigger_dev_get_run_result","trigger_dev_get_run_trace","trigger_dev_get_schedule","trigger_dev_get_waitpoint_token","trigger_dev_import_env_vars","trigger_dev_list_deployments","trigger_dev_list_env_vars","trigger_dev_list_queues","trigger_dev_list_runs","trigger_dev_list_schedules","trigger_dev_list_timezones","trigger_dev_list_waitpoint_tokens","trigger_dev_override_queue_concurrency","trigger_dev_pause_queue","trigger_dev_promote_deployment","trigger_dev_replay_run","trigger_dev_reschedule_run","trigger_dev_reset_queue_concurrency","trigger_dev_resume_queue","trigger_dev_trigger_task","trigger_dev_update_env_var","trigger_dev_update_run_metadata","trigger_dev_update_schedule","tts_azure","tts_cartesia","tts_deepgram","tts_elevenlabs","tts_google","tts_openai","tts_playht","twilio_send_sms","twilio_voice_get_recording","twilio_voice_list_calls","twilio_voice_make_call","typeform_create_form","typeform_delete_form","typeform_files","typeform_get_form","typeform_insights","typeform_list_forms","typeform_responses","typeform_update_form","upstash_redis_command","upstash_redis_delete","upstash_redis_exists","upstash_redis_expire","upstash_redis_get","upstash_redis_hget","upstash_redis_hgetall","upstash_redis_hset","upstash_redis_incr","upstash_redis_incrby","upstash_redis_keys","upstash_redis_lpush","upstash_redis_lrange","upstash_redis_set","upstash_redis_setnx","upstash_redis_ttl","uptimerobot_create_alert_contact","uptimerobot_create_maintenance_window","uptimerobot_create_monitor","uptimerobot_create_psp","uptimerobot_delete_alert_contact","uptimerobot_delete_maintenance_window","uptimerobot_delete_monitor","uptimerobot_delete_psp","uptimerobot_get_account","uptimerobot_get_alert_contact","uptimerobot_get_incident","uptimerobot_get_maintenance_window","uptimerobot_get_monitor","uptimerobot_get_psp","uptimerobot_list_alert_contacts","uptimerobot_list_incidents","uptimerobot_list_maintenance_windows","uptimerobot_list_monitors","uptimerobot_list_psps","uptimerobot_pause_monitor","uptimerobot_start_monitor","uptimerobot_update_maintenance_window","uptimerobot_update_monitor","uptimerobot_update_psp","vanta_download_document_file","vanta_get_control","vanta_get_document","vanta_get_framework","vanta_get_person","vanta_get_policy","vanta_get_risk_scenario","vanta_get_test","vanta_get_vendor","vanta_get_vulnerable_asset","vanta_list_control_documents","vanta_list_control_tests","vanta_list_controls","vanta_list_document_uploads","vanta_list_documents","vanta_list_framework_controls","vanta_list_frameworks","vanta_list_monitored_computers","vanta_list_people","vanta_list_policies","vanta_list_risk_scenarios","vanta_list_test_entities","vanta_list_tests","vanta_list_vendors","vanta_list_vulnerabilities","vanta_list_vulnerability_remediations","vanta_list_vulnerable_assets","vanta_submit_document","vanta_upload_document_file","vercel_add_domain","vercel_add_project_domain","vercel_cancel_deployment","vercel_create_alias","vercel_create_check","vercel_create_deployment","vercel_create_dns_record","vercel_create_edge_config","vercel_create_env_var","vercel_create_project","vercel_create_webhook","vercel_delete_alias","vercel_delete_deployment","vercel_delete_dns_record","vercel_delete_domain","vercel_delete_edge_config","vercel_delete_env_var","vercel_delete_project","vercel_delete_webhook","vercel_get_alias","vercel_get_check","vercel_get_deployment","vercel_get_deployment_events","vercel_get_domain","vercel_get_domain_config","vercel_get_edge_config","vercel_get_edge_config_items","vercel_get_env_vars","vercel_get_project","vercel_get_team","vercel_get_user","vercel_get_webhook","vercel_list_aliases","vercel_list_checks","vercel_list_deployment_files","vercel_list_deployments","vercel_list_dns_records","vercel_list_domains","vercel_list_edge_configs","vercel_list_project_domains","vercel_list_projects","vercel_list_team_members","vercel_list_teams","vercel_list_webhooks","vercel_pause_project","vercel_promote_deployment","vercel_remove_project_domain","vercel_rerequest_check","vercel_unpause_project","vercel_update_check","vercel_update_dns_record","vercel_update_edge_config_items","vercel_update_env_var","vercel_update_project","vercel_update_project_domain","vercel_verify_project_domain","video_falai","video_luma","video_minimax","video_runway","video_veo","vision_tool","vision_tool_v2","wealthbox_read_contact","wealthbox_read_note","wealthbox_read_task","wealthbox_write_contact","wealthbox_write_note","wealthbox_write_task","webflow_create_item","webflow_delete_item","webflow_get_item","webflow_list_items","webflow_update_item","webhook_request","whatsapp_get_media","whatsapp_mark_read","whatsapp_send_interactive","whatsapp_send_media","whatsapp_send_message","whatsapp_send_reaction","whatsapp_send_template","whatsapp_upload_media","wikipedia_content","wikipedia_random","wikipedia_search","wikipedia_summary","windchill_check_in_document","windchill_check_in_documents","windchill_check_out_document","windchill_check_out_documents","windchill_create_document","windchill_create_documents","windchill_delete_document","windchill_delete_documents","windchill_download_attachment","windchill_download_primary_content","windchill_get_document","windchill_get_document_structure","windchill_get_primary_content","windchill_get_valid_state_transitions","windchill_list_attachments","windchill_list_documents","windchill_revise_document","windchill_revise_documents","windchill_set_lifecycle_state","windchill_undo_check_out_document","windchill_undo_check_out_documents","windchill_update_common_properties","windchill_update_document","windchill_update_document_security_labels","windchill_update_documents","windchill_upload_attachments","windchill_upload_primary_content","wiza_company_enrichment","wiza_get_credits","wiza_individual_reveal","wiza_prospect_search","wordpress_create_category","wordpress_create_comment","wordpress_create_page","wordpress_create_post","wordpress_create_tag","wordpress_delete_category","wordpress_delete_comment","wordpress_delete_media","wordpress_delete_page","wordpress_delete_post","wordpress_delete_tag","wordpress_get_category","wordpress_get_current_user","wordpress_get_media","wordpress_get_page","wordpress_get_post","wordpress_get_tag","wordpress_get_user","wordpress_list_categories","wordpress_list_comments","wordpress_list_media","wordpress_list_pages","wordpress_list_posts","wordpress_list_tags","wordpress_list_users","wordpress_search_content","wordpress_update_category","wordpress_update_comment","wordpress_update_page","wordpress_update_post","wordpress_update_tag","wordpress_upload_media","workday_assign_onboarding","workday_change_job","workday_create_prehire","workday_get_compensation","workday_get_organizations","workday_get_worker","workday_hire_employee","workday_list_workers","workday_terminate_worker","workday_update_worker","workflow_executor","x_create_bookmark","x_create_tweet","x_delete_bookmark","x_delete_tweet","x_get_blocking","x_get_bookmarks","x_get_followers","x_get_following","x_get_liked_tweets","x_get_liking_users","x_get_me","x_get_personalized_trends","x_get_quote_tweets","x_get_retweeted_by","x_get_trends_by_woeid","x_get_tweets_by_ids","x_get_usage","x_get_user_mentions","x_get_user_timeline","x_get_user_tweets","x_hide_reply","x_manage_block","x_manage_follow","x_manage_like","x_manage_mute","x_manage_retweet","x_read","x_search","x_search_tweets","x_search_users","x_user","x_write","youtube_channel_info","youtube_channel_playlists","youtube_channel_videos","youtube_comments","youtube_playlist_items","youtube_search","youtube_trending","youtube_video_categories","youtube_video_details","zendesk_autocomplete_organizations","zendesk_create_organization","zendesk_create_organizations_bulk","zendesk_create_ticket","zendesk_create_tickets_bulk","zendesk_create_user","zendesk_create_users_bulk","zendesk_delete_organization","zendesk_delete_ticket","zendesk_delete_user","zendesk_get_current_user","zendesk_get_organization","zendesk_get_organizations","zendesk_get_ticket","zendesk_get_tickets","zendesk_get_user","zendesk_get_users","zendesk_merge_tickets","zendesk_search","zendesk_search_count","zendesk_search_users","zendesk_update_organization","zendesk_update_ticket","zendesk_update_tickets_bulk","zendesk_update_user","zendesk_update_users_bulk","zep_add_messages","zep_add_user","zep_create_thread","zep_delete_thread","zep_get_context","zep_get_messages","zep_get_threads","zep_get_user","zep_get_user_threads","zerobounce_get_credits","zerobounce_verify_email","zoho_desk_add_comment","zoho_desk_get_attachment","zoho_desk_get_contact","zoho_desk_get_thread","zoho_desk_get_ticket","zoho_desk_list_comments","zoho_desk_list_organizations","zoho_desk_list_threads","zoho_desk_list_tickets","zoho_desk_update_ticket","zoom_create_meeting","zoom_delete_meeting","zoom_delete_recording","zoom_get_meeting","zoom_get_meeting_invitation","zoom_get_meeting_recordings","zoom_list_meetings","zoom_list_past_participants","zoom_list_recordings","zoom_update_meeting","zoominfo_enrich_companies","zoominfo_enrich_contacts","zoominfo_search_companies","zoominfo_search_contacts","zoominfo_search_intent","zoominfo_search_news"]' + '["a2a_cancel_task","a2a_get_agent_card","a2a_get_task","a2a_send_message","affinity_batch_update_entity_fields","affinity_batch_update_list_entry_fields","affinity_create_list","affinity_create_list_field_dropdown_option","affinity_create_merge","affinity_create_note","affinity_create_reminder","affinity_delete_list_field_dropdown_option","affinity_delete_note","affinity_get_company","affinity_get_current_user","affinity_get_entity_field_value","affinity_get_list","affinity_get_list_entry","affinity_get_list_entry_field","affinity_get_list_field_dropdown_option","affinity_get_merge","affinity_get_merge_task","affinity_get_note","affinity_get_opportunity","affinity_get_person","affinity_get_saved_view","affinity_get_transcript","affinity_get_user","affinity_list_calls","affinity_list_chat_messages","affinity_list_companies","affinity_list_coworker_connections","affinity_list_emails","affinity_list_entity_field_values","affinity_list_entity_list_entries","affinity_list_entity_lists","affinity_list_entity_notes","affinity_list_entity_relationships","affinity_list_field_dropdown_options","affinity_list_field_metadata","affinity_list_field_value_changes","affinity_list_investor_executive_connections","affinity_list_list_entries","affinity_list_list_entry_field_value_changes","affinity_list_list_entry_fields","affinity_list_list_field_dropdown_options","affinity_list_list_fields","affinity_list_lists","affinity_list_meetings","affinity_list_merge_tasks","affinity_list_merges","affinity_list_note_attached_companies","affinity_list_note_attached_opportunities","affinity_list_note_attached_persons","affinity_list_note_replies","affinity_list_notes","affinity_list_opportunities","affinity_list_persons","affinity_list_reminders","affinity_list_saved_view_entries","affinity_list_saved_views","affinity_list_transcript_fragments","affinity_list_transcripts","affinity_list_users","affinity_search_companies","affinity_search_files","affinity_search_list_entries","affinity_search_notes","affinity_search_persons","affinity_semantic_search","affinity_update_entity_field_value","affinity_update_list_entry_field","affinity_update_list_field_dropdown_option","affinity_update_note","agentmail_create_draft","agentmail_create_inbox","agentmail_delete_draft","agentmail_delete_inbox","agentmail_delete_thread","agentmail_forward_message","agentmail_get_draft","agentmail_get_inbox","agentmail_get_message","agentmail_get_thread","agentmail_list_drafts","agentmail_list_inboxes","agentmail_list_messages","agentmail_list_threads","agentmail_reply_message","agentmail_send_draft","agentmail_send_message","agentmail_update_draft","agentmail_update_inbox","agentmail_update_message","agentmail_update_thread","agentphone_create_call","agentphone_create_contact","agentphone_create_number","agentphone_delete_contact","agentphone_get_call","agentphone_get_call_transcript","agentphone_get_contact","agentphone_get_conversation","agentphone_get_conversation_messages","agentphone_get_number_messages","agentphone_get_usage","agentphone_get_usage_daily","agentphone_get_usage_monthly","agentphone_list_calls","agentphone_list_contacts","agentphone_list_conversations","agentphone_list_numbers","agentphone_react_to_message","agentphone_release_number","agentphone_send_message","agentphone_update_contact","agentphone_update_conversation","agiloft_async_status","agiloft_attach_file","agiloft_attachment_info","agiloft_create_record","agiloft_delete_record","agiloft_get_choice_line_id","agiloft_list_tables","agiloft_lock_record","agiloft_nlp_search","agiloft_read_record","agiloft_remove_attachment","agiloft_retrieve_attachment","agiloft_run_action_button","agiloft_saved_search","agiloft_search_records","agiloft_select_records","agiloft_update_record","agiloft_upsert_record","ahrefs_anchors","ahrefs_backlinks","ahrefs_backlinks_stats","ahrefs_batch_analysis","ahrefs_broken_backlinks","ahrefs_domain_rating","ahrefs_domain_rating_history","ahrefs_keyword_overview","ahrefs_keywords_history","ahrefs_metrics","ahrefs_metrics_history","ahrefs_organic_competitors","ahrefs_organic_keywords","ahrefs_paid_pages","ahrefs_rank_tracker_competitors_overview","ahrefs_rank_tracker_competitors_stats","ahrefs_rank_tracker_overview","ahrefs_rank_tracker_serp_overview","ahrefs_refdomains_history","ahrefs_referring_domains","ahrefs_related_terms","ahrefs_site_audit_page_explorer","ahrefs_top_pages","airtable_create_records","airtable_delete_records","airtable_get_base_schema","airtable_get_record","airtable_list_bases","airtable_list_records","airtable_list_tables","airtable_update_multiple_records","airtable_update_record","airtable_upsert_records","airweave_search","algolia_add_record","algolia_batch_operations","algolia_browse_records","algolia_clear_records","algolia_copy_move_index","algolia_delete_by_filter","algolia_delete_index","algolia_delete_record","algolia_get_record","algolia_get_records","algolia_get_settings","algolia_get_task_status","algolia_list_indices","algolia_partial_update_record","algolia_search","algolia_update_settings","amplitude_event_segmentation","amplitude_funnels","amplitude_get_active_users","amplitude_get_revenue","amplitude_group_identify","amplitude_identify_user","amplitude_list_events","amplitude_realtime_active_users","amplitude_retention","amplitude_send_event","amplitude_user_activity","amplitude_user_profile","amplitude_user_search","apify_get_dataset_items","apify_get_run","apify_run_actor_async","apify_run_actor_sync","apify_run_task","apollo_account_bulk_create","apollo_account_bulk_update","apollo_account_create","apollo_account_search","apollo_account_update","apollo_contact_bulk_create","apollo_contact_bulk_update","apollo_contact_create","apollo_contact_search","apollo_contact_update","apollo_email_accounts","apollo_opportunity_create","apollo_opportunity_get","apollo_opportunity_search","apollo_opportunity_update","apollo_organization_bulk_enrich","apollo_organization_enrich","apollo_organization_search","apollo_people_bulk_enrich","apollo_people_enrich","apollo_people_search","apollo_sequence_add_contacts","apollo_sequence_search","apollo_task_create","apollo_task_search","appconfig_create_application","appconfig_create_configuration_profile","appconfig_create_environment","appconfig_create_hosted_configuration_version","appconfig_delete_application","appconfig_delete_configuration_profile","appconfig_delete_environment","appconfig_delete_hosted_configuration_version","appconfig_get_application","appconfig_get_configuration","appconfig_get_configuration_profile","appconfig_get_deployment","appconfig_get_environment","appconfig_get_hosted_configuration_version","appconfig_list_applications","appconfig_list_configuration_profiles","appconfig_list_deployment_strategies","appconfig_list_deployments","appconfig_list_environments","appconfig_list_hosted_configuration_versions","appconfig_start_deployment","appconfig_stop_deployment","appconfig_update_application","appconfig_update_configuration_profile","appconfig_update_environment","arxiv_get_author_papers","arxiv_get_paper","arxiv_search","asana_add_comment","asana_add_followers","asana_create_project","asana_create_section","asana_create_subtask","asana_create_task","asana_delete_task","asana_get_project","asana_get_projects","asana_get_task","asana_list_sections","asana_list_workspaces","asana_search_tasks","asana_update_task","ashby_add_candidate_tag","ashby_anonymize_candidate","ashby_change_application_source","ashby_change_application_stage","ashby_create_application","ashby_create_candidate","ashby_create_note","ashby_delete_application","ashby_get_application","ashby_get_candidate","ashby_get_job","ashby_get_job_posting","ashby_get_offer","ashby_list_applications","ashby_list_archive_reasons","ashby_list_candidate_tags","ashby_list_candidates","ashby_list_custom_fields","ashby_list_departments","ashby_list_interviews","ashby_list_job_postings","ashby_list_jobs","ashby_list_locations","ashby_list_notes","ashby_list_offers","ashby_list_openings","ashby_list_sources","ashby_list_users","ashby_remove_candidate_tag","ashby_search_candidates","ashby_set_custom_field_value","ashby_set_custom_field_values","ashby_update_candidate","athena_batch_get_query_execution","athena_create_named_query","athena_delete_named_query","athena_get_named_query","athena_get_query_execution","athena_get_query_results","athena_list_databases","athena_list_named_queries","athena_list_query_executions","athena_list_table_metadata","athena_start_query","athena_stop_query","attio_assert_record","attio_create_attribute","attio_create_comment","attio_create_list","attio_create_list_entry","attio_create_note","attio_create_object","attio_create_record","attio_create_task","attio_create_webhook","attio_delete_comment","attio_delete_list_entry","attio_delete_note","attio_delete_record","attio_delete_task","attio_delete_webhook","attio_get_attribute","attio_get_comment","attio_get_list","attio_get_list_entry","attio_get_member","attio_get_note","attio_get_object","attio_get_record","attio_get_task","attio_get_thread","attio_get_webhook","attio_list_attributes","attio_list_lists","attio_list_members","attio_list_notes","attio_list_objects","attio_list_records","attio_list_tasks","attio_list_threads","attio_list_webhooks","attio_query_list_entries","attio_search_records","attio_update_attribute","attio_update_list","attio_update_list_entry","attio_update_object","attio_update_record","attio_update_task","attio_update_webhook","azure_data_explorer_create_table","azure_data_explorer_drop_table","azure_data_explorer_ingest_from_query","azure_data_explorer_ingest_inline","azure_data_explorer_list_databases","azure_data_explorer_list_functions","azure_data_explorer_list_tables","azure_data_explorer_management","azure_data_explorer_query","azure_data_explorer_show_database_schema","azure_data_explorer_show_ingestion_failures","azure_data_explorer_show_operations","azure_data_explorer_show_table_details","azure_data_explorer_show_table_schema","azure_devops_add_comment","azure_devops_create_work_item","azure_devops_get_build_log","azure_devops_get_build_timeline","azure_devops_get_comments","azure_devops_get_pipeline","azure_devops_get_pipeline_run","azure_devops_get_work_item","azure_devops_get_work_items_batch","azure_devops_get_work_items_between_builds","azure_devops_list_build_logs","azure_devops_list_builds","azure_devops_list_pipeline_runs","azure_devops_list_pipelines","azure_devops_query_work_items","azure_devops_update_work_item","bitbucket_approve_pull_request","bitbucket_create_branch","bitbucket_create_pull_request","bitbucket_create_pull_request_comment","bitbucket_decline_pull_request","bitbucket_delete_branch","bitbucket_get_commit","bitbucket_get_file","bitbucket_get_file_metadata","bitbucket_get_pipeline","bitbucket_get_pipeline_step_log","bitbucket_get_pull_request","bitbucket_get_pull_request_diff","bitbucket_get_pull_request_diffstat","bitbucket_get_pull_request_merge_task_status","bitbucket_get_repository","bitbucket_list_branches","bitbucket_list_commits","bitbucket_list_directory","bitbucket_list_pipeline_steps","bitbucket_list_pipelines","bitbucket_list_pull_request_comments","bitbucket_list_pull_request_commit_statuses","bitbucket_list_pull_requests","bitbucket_list_repositories","bitbucket_list_workspaces","bitbucket_merge_pull_request","bitbucket_request_pull_request_changes","bitbucket_stop_pipeline","bitbucket_trigger_pipeline","box_copy_file","box_create_folder","box_delete_file","box_delete_folder","box_download_file","box_get_file_info","box_list_folder_items","box_search","box_sign_cancel_request","box_sign_create_request","box_sign_get_request","box_sign_list_requests","box_sign_resend_request","box_update_file","box_upload_file","brandfetch_get_brand","brandfetch_search","brex_archive_budget","brex_create_budget","brex_create_spend_limit","brex_create_transfer","brex_create_vendor","brex_get_budget","brex_get_cash_account","brex_get_company","brex_get_current_user","brex_get_expense","brex_get_spend_limit","brex_get_transfer","brex_get_user","brex_get_vendor","brex_list_budgets","brex_list_card_accounts","brex_list_card_statements","brex_list_card_transactions","brex_list_cards","brex_list_cash_accounts","brex_list_cash_statements","brex_list_cash_transactions","brex_list_departments","brex_list_expenses","brex_list_locations","brex_list_spend_limits","brex_list_titles","brex_list_transfers","brex_list_users","brex_list_vendors","brex_match_receipt","brex_update_expense","brex_update_vendor","brex_upload_receipt","brightdata_cancel_snapshot","brightdata_discover","brightdata_download_snapshot","brightdata_scrape_dataset","brightdata_scrape_url","brightdata_serp_search","brightdata_snapshot_status","brightdata_sync_scrape","browser_use_run_task","buffer_create_idea","buffer_create_post","buffer_delete_post","buffer_edit_post","buffer_get_account","buffer_get_channels","buffer_get_idea_groups","buffer_get_ideas","buffer_get_post","buffer_get_posts","calcom_cancel_booking","calcom_confirm_booking","calcom_create_booking","calcom_create_event_type","calcom_create_schedule","calcom_decline_booking","calcom_delete_event_type","calcom_delete_schedule","calcom_get_booking","calcom_get_default_schedule","calcom_get_event_type","calcom_get_schedule","calcom_get_slots","calcom_list_bookings","calcom_list_event_types","calcom_list_schedules","calcom_reschedule_booking","calcom_update_event_type","calcom_update_schedule","calendly_cancel_event","calendly_create_event_invitee","calendly_create_invitee_no_show","calendly_create_scheduling_link","calendly_create_webhook","calendly_delete_invitee_no_show","calendly_delete_webhook","calendly_get_current_user","calendly_get_event_invitee","calendly_get_event_type","calendly_get_scheduled_event","calendly_get_user","calendly_list_event_invitees","calendly_list_event_type_available_times","calendly_list_event_types","calendly_list_organization_memberships","calendly_list_routing_form_submissions","calendly_list_routing_forms","calendly_list_scheduled_events","calendly_list_user_availability_schedules","calendly_list_user_busy_times","calendly_list_webhooks","cbinsights_chat","cbinsights_get_commercial_maturity_history","cbinsights_get_exit_probability_history","cbinsights_get_mosaic_history","cbinsights_get_org_business_relationships","cbinsights_get_org_funding_window","cbinsights_get_org_fundings","cbinsights_get_org_investments","cbinsights_get_org_management_and_board","cbinsights_get_org_outlook","cbinsights_get_org_portfolio_exits","cbinsights_get_org_revenue","cbinsights_get_scouting_report","cbinsights_get_strategy_map","cbinsights_list_business_relationships","cbinsights_list_funding_window","cbinsights_list_fundings","cbinsights_list_investments","cbinsights_list_management_and_board","cbinsights_list_outlook","cbinsights_list_portfolio_exits","cbinsights_list_revenue","cbinsights_lookup_organizations","cbinsights_rag","cbinsights_search_firmographics","clay_populate","clerk_add_organization_member","clerk_ban_user","clerk_create_actor_token","clerk_create_allowlist_identifier","clerk_create_blocklist_identifier","clerk_create_organization","clerk_create_organization_invitation","clerk_create_user","clerk_delete_allowlist_identifier","clerk_delete_blocklist_identifier","clerk_delete_organization","clerk_delete_user","clerk_get_jwt_template","clerk_get_organization","clerk_get_session","clerk_get_user","clerk_get_user_oauth_token","clerk_list_allowlist_identifiers","clerk_list_blocklist_identifiers","clerk_list_jwt_templates","clerk_list_organization_invitations","clerk_list_organization_memberships","clerk_list_organizations","clerk_list_sessions","clerk_list_users","clerk_lock_user","clerk_remove_organization_member","clerk_revoke_actor_token","clerk_revoke_session","clerk_unban_user","clerk_unlock_user","clerk_update_organization","clerk_update_organization_membership","clerk_update_user","clickhouse_count_rows","clickhouse_create_database","clickhouse_create_table","clickhouse_delete","clickhouse_describe_table","clickhouse_drop_database","clickhouse_drop_partition","clickhouse_drop_table","clickhouse_execute","clickhouse_insert","clickhouse_insert_rows","clickhouse_introspect","clickhouse_kill_query","clickhouse_list_clusters","clickhouse_list_databases","clickhouse_list_mutations","clickhouse_list_partitions","clickhouse_list_running_queries","clickhouse_list_tables","clickhouse_optimize_table","clickhouse_query","clickhouse_rename_table","clickhouse_show_create_table","clickhouse_table_stats","clickhouse_truncate_table","clickhouse_update","clickup_add_tag_to_task","clickup_create_checklist","clickup_create_checklist_item","clickup_create_comment","clickup_create_folder","clickup_create_list","clickup_create_task","clickup_create_time_entry","clickup_delete_checklist","clickup_delete_checklist_item","clickup_delete_comment","clickup_delete_task","clickup_delete_time_entry","clickup_get_comments","clickup_get_custom_fields","clickup_get_folders","clickup_get_list_members","clickup_get_lists","clickup_get_running_timer","clickup_get_space_tags","clickup_get_spaces","clickup_get_task","clickup_get_task_members","clickup_get_tasks","clickup_get_time_entries","clickup_get_workspaces","clickup_remove_custom_field_value","clickup_remove_tag_from_task","clickup_search_tasks","clickup_set_custom_field_value","clickup_start_timer","clickup_stop_timer","clickup_update_checklist","clickup_update_checklist_item","clickup_update_comment","clickup_update_task","clickup_update_time_entry","clickup_upload_attachment","cloudflare_create_access_application","cloudflare_create_access_policy","cloudflare_create_access_service_token","cloudflare_create_dns_record","cloudflare_create_r2_bucket","cloudflare_create_rate_limit_rule","cloudflare_create_ruleset","cloudflare_create_ruleset_rule","cloudflare_create_zone","cloudflare_delete_access_application","cloudflare_delete_access_policy","cloudflare_delete_dns_record","cloudflare_delete_r2_bucket","cloudflare_delete_ruleset_rule","cloudflare_delete_zone","cloudflare_dns_analytics","cloudflare_get_access_application","cloudflare_get_r2_bucket","cloudflare_get_ruleset","cloudflare_get_ruleset_entrypoint","cloudflare_get_tunnel","cloudflare_get_tunnel_configuration","cloudflare_get_worker_script_settings","cloudflare_get_zone","cloudflare_get_zone_settings","cloudflare_list_access_applications","cloudflare_list_access_groups","cloudflare_list_access_identity_providers","cloudflare_list_access_policies","cloudflare_list_access_service_tokens","cloudflare_list_certificates","cloudflare_list_dns_records","cloudflare_list_managed_ruleset_overrides","cloudflare_list_r2_buckets","cloudflare_list_rate_limit_rules","cloudflare_list_rulesets","cloudflare_list_tunnels","cloudflare_list_worker_routes","cloudflare_list_worker_scripts","cloudflare_list_zones","cloudflare_purge_cache","cloudflare_revoke_access_service_token","cloudflare_update_access_application","cloudflare_update_access_policy","cloudflare_update_dns_record","cloudflare_update_rate_limit_rule","cloudflare_update_ruleset_rule","cloudflare_update_zone_setting","cloudformation_cancel_update_stack","cloudformation_create_change_set","cloudformation_create_stack","cloudformation_delete_stack","cloudformation_describe_change_set","cloudformation_describe_stack_drift_detection_status","cloudformation_describe_stack_events","cloudformation_describe_stacks","cloudformation_detect_stack_drift","cloudformation_execute_change_set","cloudformation_get_template","cloudformation_get_template_summary","cloudformation_list_stack_resources","cloudformation_update_stack","cloudformation_validate_template","cloudwatch_describe_alarm_history","cloudwatch_describe_alarms","cloudwatch_describe_log_groups","cloudwatch_describe_log_streams","cloudwatch_filter_log_events","cloudwatch_get_log_events","cloudwatch_get_metric_statistics","cloudwatch_list_metrics","cloudwatch_mute_alarm","cloudwatch_put_log_group_retention","cloudwatch_put_metric_data","cloudwatch_query_logs","cloudwatch_unmute_alarm","codepipeline_disable_stage_transition","codepipeline_enable_stage_transition","codepipeline_get_pipeline","codepipeline_get_pipeline_execution","codepipeline_get_pipeline_state","codepipeline_list_action_executions","codepipeline_list_pipeline_executions","codepipeline_list_pipelines","codepipeline_put_approval_result","codepipeline_retry_stage_execution","codepipeline_start_execution","codepipeline_stop_execution","confluence_add_label","confluence_create_blogpost","confluence_create_comment","confluence_create_page","confluence_create_page_property","confluence_create_space","confluence_create_space_property","confluence_delete_attachment","confluence_delete_blogpost","confluence_delete_comment","confluence_delete_label","confluence_delete_page","confluence_delete_page_property","confluence_delete_space","confluence_delete_space_property","confluence_get_blogpost","confluence_get_page_ancestors","confluence_get_page_children","confluence_get_page_descendants","confluence_get_page_version","confluence_get_pages_by_label","confluence_get_space","confluence_get_task","confluence_get_user","confluence_list_attachments","confluence_list_blogposts","confluence_list_blogposts_in_space","confluence_list_comments","confluence_list_labels","confluence_list_page_properties","confluence_list_page_versions","confluence_list_pages_in_space","confluence_list_space_labels","confluence_list_space_permissions","confluence_list_space_properties","confluence_list_spaces","confluence_list_tasks","confluence_retrieve","confluence_search","confluence_search_in_space","confluence_update","confluence_update_blogpost","confluence_update_comment","confluence_update_space","confluence_update_task","confluence_upload_attachment","context_dev_classify_naics","context_dev_classify_sic","context_dev_crawl","context_dev_extract","context_dev_extract_product","context_dev_extract_products","context_dev_get_brand","context_dev_get_brand_by_email","context_dev_get_brand_by_name","context_dev_get_brand_by_ticker","context_dev_identify_transaction","context_dev_map","context_dev_scrape_fonts","context_dev_scrape_html","context_dev_scrape_images","context_dev_scrape_markdown","context_dev_scrape_styleguide","context_dev_screenshot","context_dev_search","convex_action","convex_document_deltas","convex_list_documents","convex_list_tables","convex_mutation","convex_query","convex_run_function","crowdstrike_create_indicators","crowdstrike_delete_indicators","crowdstrike_delete_rtr_session","crowdstrike_execute_rtr_command","crowdstrike_get_alert_details","crowdstrike_get_case_details","crowdstrike_get_host_group_details","crowdstrike_get_indicator_details","crowdstrike_get_rtr_command_status","crowdstrike_get_sensor_aggregates","crowdstrike_get_sensor_details","crowdstrike_get_vulnerability_details","crowdstrike_init_rtr_session","crowdstrike_perform_host_action","crowdstrike_perform_host_group_action","crowdstrike_query_alerts","crowdstrike_query_cases","crowdstrike_query_host_groups","crowdstrike_query_indicators","crowdstrike_query_sensors","crowdstrike_query_vulnerabilities","crowdstrike_update_alerts","crowdstrike_update_indicators","crunchbase_autocomplete","crunchbase_get_acquisition","crunchbase_get_entity","crunchbase_get_entity_card","crunchbase_get_fields_metadata","crunchbase_get_funding_round","crunchbase_get_organization","crunchbase_get_person","crunchbase_list_deleted_entities","crunchbase_search_acquisitions","crunchbase_search_entities","crunchbase_search_funding_rounds","crunchbase_search_organizations","crunchbase_search_people","cursor_add_followup","cursor_add_followup_v2","cursor_delete_agent","cursor_delete_agent_v2","cursor_download_artifact","cursor_download_artifact_v2","cursor_get_agent","cursor_get_agent_v2","cursor_get_api_key_info","cursor_get_api_key_info_v2","cursor_get_conversation","cursor_get_conversation_v2","cursor_launch_agent","cursor_launch_agent_v2","cursor_list_agents","cursor_list_agents_v2","cursor_list_artifacts","cursor_list_artifacts_v2","cursor_list_models","cursor_list_models_v2","cursor_list_repositories","cursor_list_repositories_v2","cursor_stop_agent","cursor_stop_agent_v2","dagster_delete_run","dagster_get_asset","dagster_get_run","dagster_get_run_logs","dagster_launch_run","dagster_list_assets","dagster_list_jobs","dagster_list_runs","dagster_list_schedules","dagster_list_sensors","dagster_materialize_assets","dagster_reexecute_run","dagster_report_asset_materialization","dagster_start_schedule","dagster_start_sensor","dagster_stop_schedule","dagster_stop_sensor","dagster_terminate_run","dagster_wipe_asset","databricks_cancel_run","databricks_execute_sql","databricks_get_cluster","databricks_get_job","databricks_get_run","databricks_get_run_output","databricks_get_statement","databricks_list_clusters","databricks_list_jobs","databricks_list_runs","databricks_list_warehouses","databricks_run_job","datadog_add_incident_todo","datadog_cancel_downtime","datadog_create_dashboard","datadog_create_downtime","datadog_create_event","datadog_create_incident","datadog_create_monitor","datadog_create_slo","datadog_delete_dashboard","datadog_delete_slo","datadog_get_browser_synthetics_results","datadog_get_dashboard","datadog_get_incident","datadog_get_monitor","datadog_get_security_signal","datadog_get_slo","datadog_get_slo_history","datadog_get_synthetics_results","datadog_get_synthetics_test","datadog_list_dashboards","datadog_list_downtimes","datadog_list_incidents","datadog_list_monitors","datadog_list_security_rules","datadog_list_security_signals","datadog_list_services","datadog_list_slos","datadog_list_synthetics_tests","datadog_mute_monitor","datadog_query_logs","datadog_query_timeseries","datadog_search_spans","datadog_send_logs","datadog_submit_metrics","datadog_trigger_synthetics_tests","datadog_unmute_monitor","datadog_update_incident","datadog_update_security_signal_assignee","datadog_update_security_signal_state","datadog_update_slo","datadog_update_synthetics_status","datagma_enrich_company","datagma_enrich_person","datagma_find_email","datagma_find_phone","datagma_get_credits","daytona_create_sandbox","daytona_delete_sandbox","daytona_download_file","daytona_execute_command","daytona_get_sandbox","daytona_git_clone","daytona_list_files","daytona_list_sandboxes","daytona_run_code","daytona_start_sandbox","daytona_stop_sandbox","daytona_upload_file","deployed_block_executor","deployments_deploy","deployments_get_version","deployments_list_versions","deployments_promote","deployments_undeploy","devin_append_session_tags","devin_archive_session","devin_create_session","devin_get_session","devin_get_session_tags","devin_list_session_attachments","devin_list_session_messages","devin_list_sessions","devin_replace_session_tags","devin_send_message","devin_terminate_session","discord_add_reaction","discord_archive_thread","discord_assign_role","discord_ban_member","discord_bulk_delete_messages","discord_create_channel","discord_create_invite","discord_create_role","discord_create_thread","discord_create_webhook","discord_delete_channel","discord_delete_invite","discord_delete_message","discord_delete_role","discord_delete_webhook","discord_edit_message","discord_execute_webhook","discord_get_channel","discord_get_invite","discord_get_member","discord_get_messages","discord_get_pinned_messages","discord_get_server","discord_get_user","discord_get_webhook","discord_join_thread","discord_kick_member","discord_leave_thread","discord_list_channels","discord_list_roles","discord_pin_message","discord_remove_reaction","discord_remove_role","discord_send_message","discord_unban_member","discord_unpin_message","discord_update_channel","discord_update_member","discord_update_role","docusign_create_from_template","docusign_download_document","docusign_get_envelope","docusign_list_envelopes","docusign_list_recipients","docusign_list_templates","docusign_send_envelope","docusign_void_envelope","downdetector_get_company","downdetector_get_company_attribution","downdetector_get_company_baseline","downdetector_get_company_events","downdetector_get_company_incidents","downdetector_get_company_indicators","downdetector_get_company_last_15","downdetector_get_company_status","downdetector_get_provider","downdetector_get_reports","downdetector_get_site_companies","downdetector_list_categories","downdetector_list_incidents","downdetector_list_sites","downdetector_search_companies","dropbox_copy","dropbox_create_folder","dropbox_create_shared_link","dropbox_delete","dropbox_download","dropbox_get_metadata","dropbox_list_folder","dropbox_list_revisions","dropbox_list_shared_links","dropbox_move","dropbox_restore","dropbox_search","dropbox_upload","dropcontact_enrich_contact","dspy_chain_of_thought","dspy_predict","dspy_react","dub_bulk_create_links","dub_bulk_delete_links","dub_bulk_update_links","dub_create_link","dub_create_tag","dub_delete_link","dub_get_analytics","dub_get_events","dub_get_link","dub_get_links_count","dub_get_qr_code","dub_list_domains","dub_list_folders","dub_list_links","dub_list_tags","dub_update_link","dub_upsert_link","duckduckgo_search","dynamodb_delete","dynamodb_get","dynamodb_introspect","dynamodb_put","dynamodb_query","dynamodb_scan","dynamodb_update","dynatrace_add_problem_comment","dynatrace_add_tags","dynatrace_close_problem","dynatrace_create_settings_object","dynatrace_create_slo","dynatrace_delete_problem_comment","dynatrace_delete_settings_object","dynatrace_delete_slo","dynatrace_delete_tag","dynatrace_execute_synthetic_monitors","dynatrace_get_attack","dynatrace_get_audit_logs","dynatrace_get_entity","dynatrace_get_event","dynatrace_get_metric","dynatrace_get_problem","dynatrace_get_problem_comment","dynatrace_get_security_problem","dynatrace_get_settings_object","dynatrace_get_slo","dynatrace_get_synthetic_batch","dynatrace_ingest_event","dynatrace_ingest_logs","dynatrace_ingest_metrics","dynatrace_list_attacks","dynatrace_list_entities","dynatrace_list_entity_types","dynatrace_list_events","dynatrace_list_metrics","dynatrace_list_problem_comments","dynatrace_list_problems","dynatrace_list_remediation_items","dynatrace_list_security_problems","dynatrace_list_settings_objects","dynatrace_list_settings_schemas","dynatrace_list_slos","dynatrace_list_synthetic_monitors","dynatrace_list_tags","dynatrace_mute_security_problem","dynatrace_mute_security_problems","dynatrace_query_metrics","dynatrace_search_logs","dynatrace_unmute_security_problem","dynatrace_unmute_security_problems","dynatrace_update_problem_comment","dynatrace_update_settings_object","dynatrace_update_slo","elasticsearch_bulk","elasticsearch_cluster_health","elasticsearch_cluster_stats","elasticsearch_count","elasticsearch_create_index","elasticsearch_delete_document","elasticsearch_delete_index","elasticsearch_get_document","elasticsearch_get_index","elasticsearch_index_document","elasticsearch_list_indices","elasticsearch_search","elasticsearch_update_document","elevenlabs_audio_isolation","elevenlabs_edit_voice_settings","elevenlabs_get_user","elevenlabs_get_voice","elevenlabs_get_voice_settings","elevenlabs_list_models","elevenlabs_list_voices","elevenlabs_sound_effects","elevenlabs_speech_to_speech","elevenlabs_tts","emailbison_attach_leads_to_campaign","emailbison_attach_tags_to_leads","emailbison_create_campaign","emailbison_create_lead","emailbison_create_tag","emailbison_get_lead","emailbison_list_campaigns","emailbison_list_leads","emailbison_list_replies","emailbison_list_tags","emailbison_update_campaign","emailbison_update_campaign_status","emailbison_update_lead","embeddings_cohere","embeddings_gemini","embeddings_mistral","embeddings_openai","embeddings_openrouter","enrich_check_credits","enrich_company_funding","enrich_company_lookup","enrich_company_revenue","enrich_disposable_email_check","enrich_email_to_ip","enrich_email_to_person_lite","enrich_email_to_phone","enrich_email_to_profile","enrich_find_email","enrich_get_post_details","enrich_ip_to_company","enrich_linkedin_profile","enrich_linkedin_to_personal_email","enrich_linkedin_to_work_email","enrich_phone_finder","enrich_reverse_hash_lookup","enrich_sales_pointer_people","enrich_search_company","enrich_search_company_activities","enrich_search_company_employees","enrich_search_jobs","enrich_search_logo","enrich_search_people","enrich_search_people_activities","enrich_search_post_comments","enrich_search_post_comments_by_url","enrich_search_post_reactions","enrich_search_post_reactions_by_url","enrich_search_posts","enrich_search_similar_companies","enrich_verify_email","enrichment_run","enrow_find_email","enrow_verify_email","exa_agent","exa_answer","exa_find_similar_links","exa_get_contents","exa_search","extend_parser","extend_parser_v2","fathom_get_summary","fathom_get_transcript","fathom_list_meeting_types","fathom_list_meetings","fathom_list_team_members","fathom_list_teams","file_append","file_compress","file_decompress","file_fetch","file_get","file_get_content","file_manage_sharing","file_parser","file_parser_v2","file_parser_v3","file_read","file_search","file_write","findymail_find_email_from_linkedin","findymail_find_email_from_name","findymail_find_emails_by_domain","findymail_find_employees","findymail_find_phone","findymail_get_company","findymail_get_credits","findymail_lookup_technologies","findymail_reverse_email_lookup","findymail_search_technologies","findymail_verify_email","firecrawl_agent","firecrawl_batch_scrape","firecrawl_batch_scrape_status","firecrawl_cancel_crawl","firecrawl_crawl","firecrawl_crawl_status","firecrawl_credit_usage","firecrawl_extract","firecrawl_extract_status","firecrawl_map","firecrawl_parse","firecrawl_scrape","firecrawl_search","fireflies_add_to_live_meeting","fireflies_create_bite","fireflies_delete_transcript","fireflies_get_transcript","fireflies_get_user","fireflies_list_bites","fireflies_list_contacts","fireflies_list_transcripts","fireflies_list_users","fireflies_upload_audio","flint_create_task","flint_generate_pages","flint_get_task","function_execute","gamma_check_status","gamma_generate","gamma_generate_from_template","gamma_list_folders","gamma_list_themes","github_add_assignees","github_add_assignees_v2","github_add_labels","github_add_labels_v2","github_cancel_workflow_run","github_cancel_workflow_run_v2","github_check_star","github_check_star_v2","github_close_issue","github_close_issue_v2","github_close_pr","github_close_pr_v2","github_comment","github_comment_v2","github_compare_commits","github_compare_commits_v2","github_create_branch","github_create_branch_v2","github_create_comment_reaction","github_create_comment_reaction_v2","github_create_file","github_create_file_v2","github_create_gist","github_create_gist_v2","github_create_issue","github_create_issue_reaction","github_create_issue_reaction_v2","github_create_issue_v2","github_create_milestone","github_create_milestone_v2","github_create_pr","github_create_pr_review","github_create_pr_review_v2","github_create_pr_v2","github_create_project","github_create_project_v2","github_create_release","github_create_release_v2","github_delete_branch","github_delete_branch_v2","github_delete_comment","github_delete_comment_reaction","github_delete_comment_reaction_v2","github_delete_comment_v2","github_delete_file","github_delete_file_v2","github_delete_gist","github_delete_gist_v2","github_delete_issue_reaction","github_delete_issue_reaction_v2","github_delete_milestone","github_delete_milestone_v2","github_delete_project","github_delete_project_v2","github_delete_release","github_delete_release_v2","github_fork_gist","github_fork_gist_v2","github_fork_repo","github_fork_repo_v2","github_get_branch","github_get_branch_protection","github_get_branch_protection_v2","github_get_branch_v2","github_get_commit","github_get_commit_v2","github_get_file_content","github_get_file_content_v2","github_get_gist","github_get_gist_v2","github_get_issue","github_get_issue_v2","github_get_latest_release","github_get_latest_release_v2","github_get_milestone","github_get_milestone_v2","github_get_pr_files","github_get_pr_files_v2","github_get_project","github_get_project_v2","github_get_readme","github_get_readme_v2","github_get_release","github_get_release_v2","github_get_tree","github_get_tree_v2","github_get_workflow","github_get_workflow_run","github_get_workflow_run_v2","github_get_workflow_v2","github_issue_comment","github_issue_comment_v2","github_job_logs","github_latest_commit","github_latest_commit_v2","github_list_branches","github_list_branches_v2","github_list_commits","github_list_commits_v2","github_list_forks","github_list_forks_v2","github_list_gists","github_list_gists_v2","github_list_issue_comments","github_list_issue_comments_v2","github_list_issues","github_list_issues_v2","github_list_milestones","github_list_milestones_v2","github_list_pr_comments","github_list_pr_comments_v2","github_list_projects","github_list_projects_v2","github_list_prs","github_list_prs_v2","github_list_releases","github_list_releases_v2","github_list_review_threads","github_list_stargazers","github_list_stargazers_v2","github_list_tags","github_list_tags_v2","github_list_workflow_runs","github_list_workflow_runs_v2","github_list_workflows","github_list_workflows_v2","github_merge_pr","github_merge_pr_v2","github_pr","github_pr_v2","github_remove_label","github_remove_label_v2","github_reply_review_thread","github_repo_info","github_repo_info_v2","github_request_reviewers","github_request_reviewers_v2","github_rerun_workflow","github_rerun_workflow_v2","github_resolve_review_thread","github_search_code","github_search_code_v2","github_search_commits","github_search_commits_v2","github_search_issues","github_search_issues_v2","github_search_repos","github_search_repos_v2","github_search_users","github_search_users_v2","github_star_gist","github_star_gist_v2","github_star_repo","github_star_repo_v2","github_status_check_rollup","github_trigger_workflow","github_trigger_workflow_v2","github_unstar_gist","github_unstar_gist_v2","github_unstar_repo","github_unstar_repo_v2","github_update_branch_protection","github_update_branch_protection_v2","github_update_comment","github_update_comment_v2","github_update_file","github_update_file_v2","github_update_gist","github_update_gist_v2","github_update_issue","github_update_issue_v2","github_update_milestone","github_update_milestone_v2","github_update_pr","github_update_pr_v2","github_update_project","github_update_project_v2","github_update_release","github_update_release_v2","gitlab_activate_user","gitlab_add_member","gitlab_add_saml_group_link","gitlab_approve_access_request","gitlab_approve_merge_request","gitlab_approve_user","gitlab_ban_user","gitlab_block_user","gitlab_cancel_pipeline","gitlab_compare_branches","gitlab_create_branch","gitlab_create_file","gitlab_create_issue","gitlab_create_issue_note","gitlab_create_merge_request","gitlab_create_merge_request_note","gitlab_create_pipeline","gitlab_create_release","gitlab_create_user","gitlab_deactivate_user","gitlab_delete_branch","gitlab_delete_issue","gitlab_delete_saml_group_link","gitlab_delete_user","gitlab_delete_user_identity","gitlab_deny_access_request","gitlab_get_file","gitlab_get_group","gitlab_get_issue","gitlab_get_job_log","gitlab_get_merge_request","gitlab_get_merge_request_changes","gitlab_get_pipeline","gitlab_get_project","gitlab_invite_member","gitlab_list_access_requests","gitlab_list_branches","gitlab_list_commits","gitlab_list_groups","gitlab_list_invitations","gitlab_list_issues","gitlab_list_members","gitlab_list_merge_requests","gitlab_list_pipeline_jobs","gitlab_list_pipelines","gitlab_list_projects","gitlab_list_releases","gitlab_list_repository_tree","gitlab_list_saml_group_links","gitlab_list_user_memberships","gitlab_merge_merge_request","gitlab_play_job","gitlab_reject_user","gitlab_remove_member","gitlab_retry_pipeline","gitlab_revoke_invitation","gitlab_search_users","gitlab_unban_user","gitlab_unblock_user","gitlab_update_file","gitlab_update_invitation","gitlab_update_issue","gitlab_update_member","gitlab_update_merge_request","gitlab_update_user","gmail_add_label","gmail_add_label_v2","gmail_archive","gmail_archive_v2","gmail_create_label_v2","gmail_delete","gmail_delete_draft_v2","gmail_delete_label_v2","gmail_delete_v2","gmail_draft","gmail_draft_v2","gmail_edit_draft_v2","gmail_get_draft_v2","gmail_get_thread_v2","gmail_list_drafts_v2","gmail_list_labels_v2","gmail_list_threads_v2","gmail_mark_read","gmail_mark_read_v2","gmail_mark_unread","gmail_mark_unread_v2","gmail_move","gmail_move_v2","gmail_read","gmail_read_v2","gmail_remove_label","gmail_remove_label_v2","gmail_search","gmail_search_v2","gmail_send","gmail_send_v2","gmail_trash_thread_v2","gmail_unarchive","gmail_unarchive_v2","gmail_untrash_thread_v2","gmail_update_label_v2","gong_aggregate_activity","gong_aggregate_by_period","gong_answered_scorecards","gong_ask_anything","gong_assign_flow_prospects","gong_create_call","gong_day_by_day_activity","gong_get_brief","gong_get_call","gong_get_call_transcript","gong_get_coaching","gong_get_extensive_calls","gong_get_folder_content","gong_get_logs","gong_get_prospect_flows","gong_get_user","gong_interaction_stats","gong_list_calls","gong_list_flows","gong_list_library_folders","gong_list_scorecards","gong_list_trackers","gong_list_users","gong_list_workspaces","gong_lookup_email","gong_lookup_phone","gong_purge_email_address","gong_purge_phone_number","gong_unassign_flow_prospects","google_ads_ad_performance","google_ads_campaign_performance","google_ads_list_ad_groups","google_ads_list_campaigns","google_ads_list_customers","google_ads_search","google_appsheet_add_rows","google_appsheet_delete_rows","google_appsheet_edit_rows","google_appsheet_find_rows","google_bigquery_create_dataset","google_bigquery_create_table","google_bigquery_delete_dataset","google_bigquery_delete_table","google_bigquery_get_query_results","google_bigquery_get_table","google_bigquery_insert_rows","google_bigquery_list_datasets","google_bigquery_list_table_data","google_bigquery_list_tables","google_bigquery_query","google_books_volume_details","google_books_volume_search","google_calendar_create","google_calendar_create_calendar","google_calendar_create_calendar_v2","google_calendar_create_v2","google_calendar_delete","google_calendar_delete_calendar","google_calendar_delete_calendar_v2","google_calendar_delete_v2","google_calendar_freebusy","google_calendar_freebusy_v2","google_calendar_get","google_calendar_get_v2","google_calendar_instances","google_calendar_instances_v2","google_calendar_invite","google_calendar_invite_v2","google_calendar_list","google_calendar_list_acl","google_calendar_list_acl_v2","google_calendar_list_calendars","google_calendar_list_calendars_v2","google_calendar_list_v2","google_calendar_move","google_calendar_move_v2","google_calendar_quick_add","google_calendar_quick_add_v2","google_calendar_share_calendar","google_calendar_share_calendar_v2","google_calendar_unshare_calendar","google_calendar_unshare_calendar_v2","google_calendar_update","google_calendar_update_acl","google_calendar_update_acl_v2","google_calendar_update_calendar","google_calendar_update_calendar_v2","google_calendar_update_v2","google_contacts_create","google_contacts_delete","google_contacts_get","google_contacts_list","google_contacts_search","google_contacts_update","google_docs_create","google_docs_create_named_range","google_docs_create_paragraph_bullets","google_docs_delete_content_range","google_docs_delete_named_range","google_docs_delete_paragraph_bullets","google_docs_insert_image","google_docs_insert_page_break","google_docs_insert_table","google_docs_insert_text","google_docs_read","google_docs_replace_text","google_docs_update_paragraph_style","google_docs_update_text_style","google_docs_write","google_drive_copy","google_drive_create_comment","google_drive_create_folder","google_drive_delete","google_drive_delete_comment","google_drive_download","google_drive_export","google_drive_get_about","google_drive_get_content","google_drive_get_file","google_drive_get_revision","google_drive_list","google_drive_list_comments","google_drive_list_permissions","google_drive_list_revisions","google_drive_move","google_drive_search","google_drive_share","google_drive_trash","google_drive_unshare","google_drive_untrash","google_drive_update","google_drive_upload","google_forms_batch_update","google_forms_create_form","google_forms_create_watch","google_forms_delete_watch","google_forms_get_form","google_forms_get_responses","google_forms_list_watches","google_forms_renew_watch","google_forms_set_publish_settings","google_groups_add_alias","google_groups_add_member","google_groups_create_group","google_groups_delete_group","google_groups_get_group","google_groups_get_member","google_groups_get_settings","google_groups_has_member","google_groups_list_aliases","google_groups_list_groups","google_groups_list_members","google_groups_remove_alias","google_groups_remove_member","google_groups_update_group","google_groups_update_member","google_groups_update_settings","google_maps_air_quality","google_maps_directions","google_maps_distance_matrix","google_maps_elevation","google_maps_geocode","google_maps_geolocate","google_maps_place_details","google_maps_places_nearby","google_maps_places_search","google_maps_pollen","google_maps_reverse_geocode","google_maps_snap_to_roads","google_maps_solar","google_maps_speed_limits","google_maps_timezone","google_maps_validate_address","google_meet_create_space","google_meet_end_conference","google_meet_get_conference_record","google_meet_get_space","google_meet_list_conference_records","google_meet_list_participants","google_pagespeed_analyze","google_search","google_sheets_append","google_sheets_append_v2","google_sheets_batch_clear_v2","google_sheets_batch_get_v2","google_sheets_batch_update_v2","google_sheets_clear_v2","google_sheets_copy_sheet_v2","google_sheets_create_spreadsheet_v2","google_sheets_delete_rows_v2","google_sheets_delete_sheet_v2","google_sheets_delete_spreadsheet_v2","google_sheets_get_spreadsheet_v2","google_sheets_read","google_sheets_read_v2","google_sheets_update","google_sheets_update_v2","google_sheets_write","google_sheets_write_v2","google_slides_add_image","google_slides_add_slide","google_slides_batch_update","google_slides_copy_presentation","google_slides_create","google_slides_create_line","google_slides_create_paragraph_bullets","google_slides_create_shape","google_slides_create_sheets_chart","google_slides_create_table","google_slides_create_video","google_slides_delete_object","google_slides_delete_paragraph_bullets","google_slides_delete_table_column","google_slides_delete_table_row","google_slides_delete_text","google_slides_duplicate_object","google_slides_export_presentation","google_slides_get_page","google_slides_get_thumbnail","google_slides_group_objects","google_slides_insert_table_columns","google_slides_insert_table_rows","google_slides_insert_text","google_slides_merge_table_cells","google_slides_read","google_slides_refresh_sheets_chart","google_slides_replace_all_shapes_with_image","google_slides_replace_all_shapes_with_sheets_chart","google_slides_replace_all_text","google_slides_replace_image","google_slides_reroute_line","google_slides_ungroup_objects","google_slides_unmerge_table_cells","google_slides_update_image_properties","google_slides_update_line_category","google_slides_update_line_properties","google_slides_update_page_element_alt_text","google_slides_update_page_element_transform","google_slides_update_page_elements_z_order","google_slides_update_page_properties","google_slides_update_paragraph_style","google_slides_update_shape_properties","google_slides_update_slide_properties","google_slides_update_slides_position","google_slides_update_table_border_properties","google_slides_update_table_cell_properties","google_slides_update_table_column_properties","google_slides_update_table_row_properties","google_slides_update_text_style","google_slides_update_video_properties","google_slides_write","google_tasks_create","google_tasks_delete","google_tasks_get","google_tasks_list","google_tasks_list_task_lists","google_tasks_update","google_translate_detect","google_translate_text","google_vault_add_held_accounts","google_vault_add_matters_permissions","google_vault_close_matters","google_vault_create_matters","google_vault_create_matters_export","google_vault_create_matters_holds","google_vault_create_saved_query","google_vault_delete_matters","google_vault_delete_matters_export","google_vault_delete_matters_holds","google_vault_delete_saved_query","google_vault_download_export_file","google_vault_list_matters","google_vault_list_matters_export","google_vault_list_matters_holds","google_vault_list_saved_queries","google_vault_remove_held_accounts","google_vault_remove_matters_permissions","google_vault_reopen_matters","google_vault_undelete_matters","google_vault_update_matters","google_vault_update_matters_holds","grafana_check_data_source_health","grafana_create_alert_rule","grafana_create_annotation","grafana_create_contact_point","grafana_create_dashboard","grafana_create_folder","grafana_delete_alert_rule","grafana_delete_annotation","grafana_delete_contact_point","grafana_delete_dashboard","grafana_delete_folder","grafana_get_alert_rule","grafana_get_alert_rule_group","grafana_get_dashboard","grafana_get_data_source","grafana_get_folder","grafana_get_health","grafana_list_alert_rules","grafana_list_annotations","grafana_list_contact_points","grafana_list_dashboards","grafana_list_data_sources","grafana_list_folders","grafana_move_folder","grafana_query_data_source","grafana_update_alert_rule","grafana_update_annotation","grafana_update_contact_point","grafana_update_dashboard","grafana_update_folder","grain_create_hook","grain_create_hook_v2","grain_delete_hook","grain_delete_hook_v2","grain_get_recording","grain_get_transcript","grain_list_hooks","grain_list_hooks_v2","grain_list_meeting_types","grain_list_recordings","grain_list_teams","grain_list_views","granola_create_webhook_endpoint","granola_delete_webhook_endpoint","granola_get_note","granola_get_transcript","granola_list_audit_events","granola_list_folders","granola_list_notes","granola_list_webhook_endpoints","granola_update_webhook_endpoint","greenhouse_get_application","greenhouse_get_candidate","greenhouse_get_job","greenhouse_get_user","greenhouse_list_applications","greenhouse_list_candidates","greenhouse_list_departments","greenhouse_list_job_stages","greenhouse_list_jobs","greenhouse_list_offices","greenhouse_list_users","greptile_index_repo","greptile_query","greptile_search","greptile_status","guardrails_validate","harmonic_batch_get_people","harmonic_clear_people_saved_search_net_new_results","harmonic_enrich_person","harmonic_get_company_employees","harmonic_get_email_enrichment_job","harmonic_get_email_enrichment_usage","harmonic_get_enrichment_status","harmonic_get_people_saved_search_net_new_results","harmonic_get_people_saved_search_results","harmonic_get_person","harmonic_list_people_saved_searches","harmonic_search_people_scout","harmonic_submit_email_enrichment_job","hex_cancel_run","hex_create_collection","hex_create_group","hex_deactivate_user","hex_delete_group","hex_get_collection","hex_get_data_connection","hex_get_group","hex_get_project","hex_get_project_runs","hex_get_queried_tables","hex_get_run_status","hex_list_collections","hex_list_data_connections","hex_list_groups","hex_list_projects","hex_list_users","hex_run_project","hex_update_collection","hex_update_group","hex_update_project","http_request","hubspot_add_list_memberships","hubspot_create_appointment","hubspot_create_association","hubspot_create_company","hubspot_create_contact","hubspot_create_deal","hubspot_create_email","hubspot_create_line_item","hubspot_create_list","hubspot_create_note","hubspot_create_ticket","hubspot_delete_association","hubspot_delete_company","hubspot_delete_contact","hubspot_delete_deal","hubspot_delete_line_item","hubspot_delete_ticket","hubspot_get_appointment","hubspot_get_association_labels","hubspot_get_cart","hubspot_get_company","hubspot_get_contact","hubspot_get_deal","hubspot_get_email","hubspot_get_line_item","hubspot_get_list","hubspot_get_list_memberships","hubspot_get_marketing_event","hubspot_get_note","hubspot_get_properties","hubspot_get_quote","hubspot_get_ticket","hubspot_get_users","hubspot_list_appointments","hubspot_list_associations","hubspot_list_carts","hubspot_list_companies","hubspot_list_contacts","hubspot_list_deals","hubspot_list_emails","hubspot_list_line_items","hubspot_list_lists","hubspot_list_marketing_events","hubspot_list_notes","hubspot_list_owners","hubspot_list_quotes","hubspot_list_tickets","hubspot_remove_list_memberships","hubspot_search_companies","hubspot_search_contacts","hubspot_search_deals","hubspot_search_emails","hubspot_search_line_items","hubspot_search_notes","hubspot_search_quotes","hubspot_search_tickets","hubspot_update_appointment","hubspot_update_company","hubspot_update_contact","hubspot_update_deal","hubspot_update_line_item","hubspot_update_ticket","huggingface_chat","hunter_companies_find","hunter_discover","hunter_domain_search","hunter_email_count","hunter_email_finder","hunter_email_verifier","iam_add_user_to_group","iam_attach_role_policy","iam_attach_user_policy","iam_create_access_key","iam_create_role","iam_create_user","iam_delete_access_key","iam_delete_role","iam_delete_user","iam_detach_role_policy","iam_detach_user_policy","iam_get_role","iam_get_user","iam_list_attached_role_policies","iam_list_attached_user_policies","iam_list_groups","iam_list_policies","iam_list_roles","iam_list_users","iam_remove_user_from_group","iam_simulate_principal_policy","icypeas_find_email","icypeas_verify_email","identity_center_check_assignment_deletion_status","identity_center_check_assignment_status","identity_center_create_account_assignment","identity_center_delete_account_assignment","identity_center_describe_account","identity_center_get_group","identity_center_get_user","identity_center_list_account_assignments","identity_center_list_accounts","identity_center_list_groups","identity_center_list_instances","identity_center_list_permission_sets","image_generate","incidentio_actions_create","incidentio_actions_list","incidentio_actions_show","incidentio_actions_update","incidentio_alert_events_create","incidentio_alerts_list","incidentio_alerts_resolve","incidentio_alerts_show","incidentio_catalog_entries_list","incidentio_catalog_types_list","incidentio_custom_fields_create","incidentio_custom_fields_delete","incidentio_custom_fields_list","incidentio_custom_fields_show","incidentio_custom_fields_update","incidentio_escalation_paths_create","incidentio_escalation_paths_delete","incidentio_escalation_paths_list","incidentio_escalation_paths_show","incidentio_escalation_paths_update","incidentio_escalations_cancel","incidentio_escalations_create","incidentio_escalations_list","incidentio_escalations_show","incidentio_follow_ups_create","incidentio_follow_ups_list","incidentio_follow_ups_show","incidentio_follow_ups_update","incidentio_incident_alerts_list","incidentio_incident_memberships_create","incidentio_incident_memberships_revoke","incidentio_incident_participants_list","incidentio_incident_roles_create","incidentio_incident_roles_delete","incidentio_incident_roles_list","incidentio_incident_roles_show","incidentio_incident_roles_update","incidentio_incident_statuses_list","incidentio_incident_timestamps_list","incidentio_incident_timestamps_show","incidentio_incident_types_list","incidentio_incident_updates_list","incidentio_incidents_create","incidentio_incidents_list","incidentio_incidents_show","incidentio_incidents_update","incidentio_on_call_now","incidentio_schedule_entries_list","incidentio_schedule_overrides_create","incidentio_schedule_overrides_list","incidentio_schedules_create","incidentio_schedules_delete","incidentio_schedules_list","incidentio_schedules_show","incidentio_schedules_update","incidentio_severities_list","incidentio_teams_list","incidentio_teams_show","incidentio_users_list","incidentio_users_show","incidentio_workflows_create","incidentio_workflows_delete","incidentio_workflows_list","incidentio_workflows_show","incidentio_workflows_update","infisical_create_secret","infisical_delete_secret","infisical_get_secret","infisical_list_secrets","infisical_update_secret","instagram_delete_comment","instagram_download_media","instagram_get_account_insights","instagram_get_container_status","instagram_get_conversation_messages","instagram_get_media","instagram_get_media_insights","instagram_get_message","instagram_get_profile","instagram_get_publishing_limit","instagram_hide_comment","instagram_list_comments","instagram_list_conversations","instagram_list_media","instagram_list_stories","instagram_private_reply","instagram_publish_carousel","instagram_publish_image","instagram_publish_reel","instagram_publish_story","instagram_publish_video","instagram_reply_to_comment","instagram_send_text_message","instagram_set_comments_enabled","instantly_activate_campaign","instantly_create_campaign","instantly_create_lead","instantly_create_lead_list","instantly_delete_campaign","instantly_delete_leads","instantly_get_lead","instantly_list_campaigns","instantly_list_emails","instantly_list_lead_lists","instantly_list_leads","instantly_patch_campaign","instantly_patch_lead","instantly_pause_campaign","instantly_reply_to_email","instantly_update_lead_interest_status","intercom_assign_conversation_v2","intercom_attach_contact_to_company_v2","intercom_close_conversation_v2","intercom_create_company","intercom_create_company_v2","intercom_create_contact","intercom_create_contact_v2","intercom_create_event_v2","intercom_create_message","intercom_create_message_v2","intercom_create_note_v2","intercom_create_tag_v2","intercom_create_ticket","intercom_create_ticket_v2","intercom_delete_contact","intercom_delete_contact_v2","intercom_detach_contact_from_company_v2","intercom_get_company","intercom_get_company_v2","intercom_get_contact","intercom_get_contact_v2","intercom_get_conversation","intercom_get_conversation_v2","intercom_get_ticket","intercom_get_ticket_v2","intercom_list_admins_v2","intercom_list_companies","intercom_list_companies_v2","intercom_list_contacts","intercom_list_contacts_v2","intercom_list_conversations","intercom_list_conversations_v2","intercom_list_tags_v2","intercom_open_conversation_v2","intercom_reply_conversation","intercom_reply_conversation_v2","intercom_search_contacts","intercom_search_contacts_v2","intercom_search_conversations","intercom_search_conversations_v2","intercom_snooze_conversation_v2","intercom_tag_contact_v2","intercom_tag_conversation_v2","intercom_untag_contact_v2","intercom_update_contact","intercom_update_contact_v2","intercom_update_ticket_v2","jina_read_url","jina_search","jira_add_attachment","jira_add_comment","jira_add_watcher","jira_add_worklog","jira_assign_issue","jira_bulk_read","jira_create_issue_link","jira_delete_attachment","jira_delete_comment","jira_delete_issue","jira_delete_issue_link","jira_delete_worklog","jira_get_attachments","jira_get_comments","jira_get_fields","jira_get_project","jira_get_transitions","jira_get_users","jira_get_worklogs","jira_list_issue_types","jira_list_projects","jira_remove_watcher","jira_retrieve","jira_search_issues","jira_search_users","jira_transition_issue","jira_update","jira_update_comment","jira_update_worklog","jira_write","jotform_add_label_resources","jotform_clone_form","jotform_create_form","jotform_create_label","jotform_create_question","jotform_create_questions","jotform_create_report","jotform_create_submission","jotform_create_submissions","jotform_create_webhook","jotform_delete_form","jotform_delete_label","jotform_delete_question","jotform_delete_report","jotform_delete_submission","jotform_delete_webhook","jotform_get_form","jotform_get_form_properties","jotform_get_history","jotform_get_label","jotform_get_question","jotform_get_report","jotform_get_settings","jotform_get_submission","jotform_get_usage","jotform_get_user","jotform_list_form_files","jotform_list_form_reports","jotform_list_form_submissions","jotform_list_forms","jotform_list_label_resources","jotform_list_labels","jotform_list_questions","jotform_list_reports","jotform_list_submissions","jotform_list_subusers","jotform_list_webhooks","jotform_remove_label_resources","jotform_update_form_properties","jotform_update_label","jotform_update_question","jotform_update_settings","jotform_update_submission","jsm_add_comment","jsm_add_customer","jsm_add_organization","jsm_add_participants","jsm_answer_approval","jsm_attach_form","jsm_copy_forms","jsm_create_object","jsm_create_organization","jsm_create_request","jsm_delete_form","jsm_delete_object","jsm_externalise_form","jsm_get_approvals","jsm_get_comments","jsm_get_customers","jsm_get_form","jsm_get_form_answers","jsm_get_form_structure","jsm_get_form_templates","jsm_get_issue_forms","jsm_get_object","jsm_get_object_schema","jsm_get_object_type_attributes","jsm_get_organizations","jsm_get_participants","jsm_get_queues","jsm_get_request","jsm_get_request_type_fields","jsm_get_request_types","jsm_get_requests","jsm_get_service_desks","jsm_get_sla","jsm_get_transitions","jsm_internalise_form","jsm_list_object_schemas","jsm_list_object_types","jsm_reopen_form","jsm_save_form_answers","jsm_search_objects_aql","jsm_submit_form","jsm_transition_request","jsm_update_object","jupyter_copy_content","jupyter_create_file","jupyter_create_session","jupyter_delete_content","jupyter_delete_session","jupyter_get_content","jupyter_interrupt_kernel","jupyter_list_contents","jupyter_list_kernels","jupyter_list_kernelspecs","jupyter_list_sessions","jupyter_rename_content","jupyter_restart_kernel","jupyter_start_kernel","jupyter_stop_kernel","jupyter_upload_file","kalshi_amend_order","kalshi_amend_order_v2","kalshi_cancel_order","kalshi_cancel_order_v2","kalshi_create_order","kalshi_create_order_v2","kalshi_get_balance","kalshi_get_balance_v2","kalshi_get_candlesticks","kalshi_get_candlesticks_v2","kalshi_get_event","kalshi_get_event_candlesticks","kalshi_get_event_candlesticks_v2","kalshi_get_event_v2","kalshi_get_events","kalshi_get_events_v2","kalshi_get_exchange_announcements","kalshi_get_exchange_announcements_v2","kalshi_get_exchange_schedule","kalshi_get_exchange_schedule_v2","kalshi_get_exchange_status","kalshi_get_exchange_status_v2","kalshi_get_fills","kalshi_get_fills_v2","kalshi_get_market","kalshi_get_market_v2","kalshi_get_markets","kalshi_get_markets_v2","kalshi_get_order","kalshi_get_order_v2","kalshi_get_orderbook","kalshi_get_orderbook_v2","kalshi_get_orders","kalshi_get_orders_v2","kalshi_get_positions","kalshi_get_positions_v2","kalshi_get_series_by_ticker","kalshi_get_series_by_ticker_v2","kalshi_get_series_list","kalshi_get_series_list_v2","kalshi_get_settlements","kalshi_get_settlements_v2","kalshi_get_trades","kalshi_get_trades_v2","ketch_get_consent","ketch_get_subscriptions","ketch_invoke_right","ketch_set_consent","ketch_set_subscriptions","knowledge_create_document","knowledge_delete_chunk","knowledge_delete_document","knowledge_get_connector","knowledge_get_document","knowledge_list_chunks","knowledge_list_connectors","knowledge_list_documents","knowledge_list_tags","knowledge_search","knowledge_trigger_sync","knowledge_update_chunk","knowledge_upload_chunk","knowledge_upsert_document","lambda_add_permission","lambda_create_alias","lambda_create_event_source_mapping","lambda_create_function","lambda_create_function_url_config","lambda_delete_alias","lambda_delete_event_source_mapping","lambda_delete_function","lambda_delete_function_concurrency","lambda_delete_function_event_invoke_config","lambda_delete_function_url_config","lambda_delete_provisioned_concurrency_config","lambda_get_account_settings","lambda_get_alias","lambda_get_event_source_mapping","lambda_get_function","lambda_get_function_concurrency","lambda_get_function_configuration","lambda_get_function_event_invoke_config","lambda_get_function_recursion_config","lambda_get_function_url_config","lambda_get_layer_version","lambda_get_policy","lambda_get_provisioned_concurrency_config","lambda_get_runtime_management_config","lambda_invoke","lambda_list_aliases","lambda_list_event_source_mappings","lambda_list_function_event_invoke_configs","lambda_list_function_url_configs","lambda_list_functions","lambda_list_layer_versions","lambda_list_layers","lambda_list_provisioned_concurrency_configs","lambda_list_tags","lambda_list_versions_by_function","lambda_publish_version","lambda_put_function_concurrency","lambda_put_function_event_invoke_config","lambda_put_function_recursion_config","lambda_put_provisioned_concurrency_config","lambda_put_runtime_management_config","lambda_remove_permission","lambda_tag_resource","lambda_untag_resource","lambda_update_alias","lambda_update_event_source_mapping","lambda_update_function_code","lambda_update_function_configuration","lambda_update_function_url_config","langsmith_create_feedback","langsmith_create_run","langsmith_create_runs_batch","langsmith_get_run","langsmith_update_run","latex_compile","latex_get_package","latex_list_fonts","latex_search_packages","launchdarkly_create_flag","launchdarkly_delete_flag","launchdarkly_get_audit_log","launchdarkly_get_flag","launchdarkly_get_flag_status","launchdarkly_list_environments","launchdarkly_list_flags","launchdarkly_list_members","launchdarkly_list_projects","launchdarkly_list_segments","launchdarkly_toggle_flag","launchdarkly_update_flag","leadmagic_company_search","leadmagic_email_to_profile","leadmagic_find_email","leadmagic_find_mobile","leadmagic_get_credits","leadmagic_profile_search","leadmagic_profile_to_email","leadmagic_role_finder","leadmagic_validate_email","lemlist_get_activities","lemlist_get_lead","lemlist_send_email","linear_add_label_to_issue","linear_add_label_to_project","linear_archive_issue","linear_archive_label","linear_archive_project","linear_create_attachment","linear_create_comment","linear_create_customer","linear_create_customer_request","linear_create_customer_status","linear_create_customer_tier","linear_create_cycle","linear_create_favorite","linear_create_issue","linear_create_issue_relation","linear_create_label","linear_create_project","linear_create_project_label","linear_create_project_milestone","linear_create_project_status","linear_create_project_update","linear_create_workflow_state","linear_delete_attachment","linear_delete_comment","linear_delete_customer","linear_delete_customer_status","linear_delete_customer_tier","linear_delete_issue","linear_delete_issue_relation","linear_delete_project","linear_delete_project_label","linear_delete_project_milestone","linear_delete_project_status","linear_get_active_cycle","linear_get_customer","linear_get_cycle","linear_get_issue","linear_get_project","linear_get_viewer","linear_list_attachments","linear_list_comments","linear_list_customer_requests","linear_list_customer_statuses","linear_list_customer_tiers","linear_list_customers","linear_list_cycles","linear_list_favorites","linear_list_issue_relations","linear_list_labels","linear_list_notifications","linear_list_project_labels","linear_list_project_milestones","linear_list_project_statuses","linear_list_project_updates","linear_list_projects","linear_list_teams","linear_list_users","linear_list_workflow_states","linear_merge_customers","linear_read_issues","linear_remove_label_from_issue","linear_remove_label_from_project","linear_search_issues","linear_unarchive_issue","linear_update_attachment","linear_update_comment","linear_update_customer","linear_update_customer_request","linear_update_customer_status","linear_update_customer_tier","linear_update_issue","linear_update_label","linear_update_notification","linear_update_project","linear_update_project_label","linear_update_project_milestone","linear_update_project_status","linear_update_workflow_state","linkedin_get_profile","linkedin_share_post","linkup_search","linq_add_participant","linq_check_imessage","linq_check_rcs","linq_create_attachment","linq_create_chat","linq_create_contact_card","linq_create_webhook_subscription","linq_delete_attachment","linq_delete_message","linq_delete_webhook_subscription","linq_edit_message","linq_get_attachment","linq_get_chat","linq_get_contact_card","linq_get_message","linq_get_webhook_subscription","linq_leave_chat","linq_list_chats","linq_list_messages","linq_list_phone_numbers","linq_list_thread","linq_list_webhook_events","linq_list_webhook_subscriptions","linq_mark_chat_read","linq_react_to_message","linq_remove_participant","linq_send_message","linq_send_voice_memo","linq_share_contact_card","linq_start_typing","linq_stop_typing","linq_update_chat","linq_update_contact_card","linq_update_webhook_subscription","llm_chat","logfire_get_token_info","logfire_get_trace","logfire_query","logfire_search_records","logrocket_create_release","logrocket_get_audit_logs","logrocket_get_highlights","logrocket_identify_user","logrocket_list_exported_sessions","logrocket_request_highlights","logs_get","logs_get_execution","logs_get_run_details","logs_query","logs_query_runs","loops_check_contact_suppression","loops_create_contact","loops_create_contact_property","loops_delete_contact","loops_find_contact","loops_get_transactional_email","loops_list_contact_properties","loops_list_mailing_lists","loops_list_transactional_emails","loops_remove_contact_suppression","loops_send_event","loops_send_transactional_email","loops_update_contact","luma_add_guests","luma_cancel_event","luma_create_event","luma_get_event","luma_get_guest","luma_get_guests","luma_list_events","luma_lookup_event","luma_send_invites","luma_update_event","luma_update_guest_status","mailchimp_add_member","mailchimp_add_member_tags","mailchimp_add_or_update_member","mailchimp_add_segment_member","mailchimp_add_subscriber_to_automation","mailchimp_archive_member","mailchimp_create_audience","mailchimp_create_batch_operation","mailchimp_create_campaign","mailchimp_create_interest","mailchimp_create_interest_category","mailchimp_create_landing_page","mailchimp_create_merge_field","mailchimp_create_segment","mailchimp_create_template","mailchimp_delete_audience","mailchimp_delete_batch_operation","mailchimp_delete_campaign","mailchimp_delete_interest","mailchimp_delete_interest_category","mailchimp_delete_landing_page","mailchimp_delete_member","mailchimp_delete_merge_field","mailchimp_delete_segment","mailchimp_delete_template","mailchimp_get_audience","mailchimp_get_audiences","mailchimp_get_automation","mailchimp_get_automations","mailchimp_get_batch_operation","mailchimp_get_batch_operations","mailchimp_get_campaign","mailchimp_get_campaign_content","mailchimp_get_campaign_report","mailchimp_get_campaign_reports","mailchimp_get_campaigns","mailchimp_get_interest","mailchimp_get_interest_categories","mailchimp_get_interest_category","mailchimp_get_interests","mailchimp_get_landing_page","mailchimp_get_landing_pages","mailchimp_get_member","mailchimp_get_member_tags","mailchimp_get_members","mailchimp_get_merge_field","mailchimp_get_merge_fields","mailchimp_get_segment","mailchimp_get_segment_members","mailchimp_get_segments","mailchimp_get_template","mailchimp_get_templates","mailchimp_pause_automation","mailchimp_publish_landing_page","mailchimp_remove_member_tags","mailchimp_remove_segment_member","mailchimp_replicate_campaign","mailchimp_schedule_campaign","mailchimp_send_campaign","mailchimp_set_campaign_content","mailchimp_start_automation","mailchimp_unarchive_member","mailchimp_unpublish_landing_page","mailchimp_unschedule_campaign","mailchimp_update_audience","mailchimp_update_campaign","mailchimp_update_interest","mailchimp_update_interest_category","mailchimp_update_landing_page","mailchimp_update_member","mailchimp_update_merge_field","mailchimp_update_segment","mailchimp_update_template","mailgun_add_list_member","mailgun_create_mailing_list","mailgun_get_domain","mailgun_get_mailing_list","mailgun_get_message","mailgun_list_domains","mailgun_list_messages","mailgun_send_message","managed_agent_archive_session","managed_agent_create_session","managed_agent_delete_session","managed_agent_get_session","managed_agent_interrupt_session","managed_agent_list_events","managed_agent_respond_custom_tool","managed_agent_respond_tool_confirmation","managed_agent_run_session","managed_agent_send_message","managed_agent_update_session","mem0_add_memories","mem0_get_memories","mem0_search_memories","memory_add","memory_delete","memory_get","memory_get_all","microsoft_ad_add_directory_role_member","microsoft_ad_add_group_member","microsoft_ad_add_user_app_role_assignment","microsoft_ad_assign_license","microsoft_ad_create_group","microsoft_ad_create_user","microsoft_ad_delete_group","microsoft_ad_delete_user","microsoft_ad_get_conditional_access_policy","microsoft_ad_get_device","microsoft_ad_get_group","microsoft_ad_get_user","microsoft_ad_list_authentication_methods","microsoft_ad_list_conditional_access_policies","microsoft_ad_list_devices","microsoft_ad_list_directory_audits","microsoft_ad_list_directory_role_members","microsoft_ad_list_directory_roles","microsoft_ad_list_group_members","microsoft_ad_list_groups","microsoft_ad_list_service_principal_app_role_assignments","microsoft_ad_list_service_principals","microsoft_ad_list_sign_ins","microsoft_ad_list_subscribed_skus","microsoft_ad_list_user_app_role_assignments","microsoft_ad_list_user_devices","microsoft_ad_list_user_licenses","microsoft_ad_list_users","microsoft_ad_remove_directory_role_member","microsoft_ad_remove_group_member","microsoft_ad_remove_user_app_role_assignment","microsoft_ad_reset_password","microsoft_ad_revoke_sign_in_sessions","microsoft_ad_set_password","microsoft_ad_update_group","microsoft_ad_update_user","microsoft_dataverse_associate","microsoft_dataverse_create_multiple","microsoft_dataverse_create_record","microsoft_dataverse_delete_record","microsoft_dataverse_disassociate","microsoft_dataverse_download_file","microsoft_dataverse_execute_action","microsoft_dataverse_execute_function","microsoft_dataverse_fetchxml_query","microsoft_dataverse_get_entity_metadata","microsoft_dataverse_get_record","microsoft_dataverse_list_records","microsoft_dataverse_search","microsoft_dataverse_update_multiple","microsoft_dataverse_update_record","microsoft_dataverse_upload_file","microsoft_dataverse_upsert_record","microsoft_dataverse_whoami","microsoft_dynamics_365_close_case","microsoft_dynamics_365_close_opportunity","microsoft_dynamics_365_create_record","microsoft_dynamics_365_get_record","microsoft_dynamics_365_list_records","microsoft_dynamics_365_qualify_lead","microsoft_dynamics_365_search_records","microsoft_dynamics_365_update_record","microsoft_excel_clear_range","microsoft_excel_create_table","microsoft_excel_delete_worksheet","microsoft_excel_format_range","microsoft_excel_read","microsoft_excel_read_v2","microsoft_excel_sort_range","microsoft_excel_table_add","microsoft_excel_worksheet_add","microsoft_excel_write","microsoft_excel_write_v2","microsoft_planner_create_bucket","microsoft_planner_create_plan","microsoft_planner_create_task","microsoft_planner_delete_bucket","microsoft_planner_delete_plan","microsoft_planner_delete_task","microsoft_planner_get_plan_details","microsoft_planner_get_task_details","microsoft_planner_list_buckets","microsoft_planner_list_plans","microsoft_planner_read_bucket","microsoft_planner_read_plan","microsoft_planner_read_task","microsoft_planner_update_bucket","microsoft_planner_update_plan","microsoft_planner_update_plan_details","microsoft_planner_update_task","microsoft_planner_update_task_details","microsoft_teams_delete_channel_message","microsoft_teams_delete_chat_message","microsoft_teams_get_message","microsoft_teams_list_channel_members","microsoft_teams_list_channels","microsoft_teams_list_chat_members","microsoft_teams_list_chats","microsoft_teams_list_team_members","microsoft_teams_list_teams","microsoft_teams_read_channel","microsoft_teams_read_chat","microsoft_teams_reply_to_message","microsoft_teams_set_reaction","microsoft_teams_unset_reaction","microsoft_teams_update_channel_message","microsoft_teams_update_chat_message","microsoft_teams_write_channel","microsoft_teams_write_chat","microsoft_word_append","microsoft_word_create","microsoft_word_create_from_template","microsoft_word_export_pdf","microsoft_word_list","microsoft_word_read","microsoft_word_replace_text","microsoft_word_update","millionverifier_get_credits","millionverifier_verify_email","mintlify_create_agent_job","mintlify_create_assistant_message","mintlify_detect_ai_prose","mintlify_get_agent_job","mintlify_get_assistant_caller_stats","mintlify_get_assistant_conversations","mintlify_get_feedback","mintlify_get_feedback_by_page","mintlify_get_page_content","mintlify_get_searches","mintlify_get_update_status","mintlify_get_views","mintlify_get_visitors","mintlify_search","mintlify_send_agent_message","mintlify_trigger_automation","mintlify_trigger_preview","mintlify_trigger_update","mistral_parser","mistral_parser_v2","mistral_parser_v3","modal_call_function","modal_chat_completion","modal_list_models","monday_archive_item","monday_change_column_value","monday_create_board","monday_create_column","monday_create_group","monday_create_item","monday_create_subitem","monday_create_update","monday_delete_item","monday_duplicate_item","monday_get_board","monday_get_groups","monday_get_item","monday_get_items","monday_list_boards","monday_move_item_to_group","monday_search_items","monday_update_item","mongodb_delete","mongodb_execute","mongodb_insert","mongodb_introspect","mongodb_query","mongodb_update","mssql_delete","mssql_execute","mssql_insert","mssql_introspect","mssql_query","mssql_update","mysql_delete","mysql_execute","mysql_insert","mysql_introspect","mysql_query","mysql_update","neo4j_create","neo4j_delete","neo4j_execute","neo4j_introspect","neo4j_merge","neo4j_query","neo4j_update","netsuite_attach_record","netsuite_batch_create_records","netsuite_batch_delete_records","netsuite_batch_get_records","netsuite_batch_update_records","netsuite_batch_upsert_records","netsuite_create_record","netsuite_delete_record","netsuite_detach_record","netsuite_execute_action","netsuite_execute_dataset","netsuite_execute_suiteql","netsuite_get_async_result","netsuite_get_async_status","netsuite_get_governance_limits","netsuite_get_record","netsuite_get_record_form","netsuite_get_record_metadata","netsuite_get_select_options","netsuite_get_server_time","netsuite_get_subresource","netsuite_list_datasets","netsuite_list_record_types","netsuite_list_records","netsuite_transform_record","netsuite_update_record","netsuite_upsert_record","neverbounce_get_credits","neverbounce_verify_email","new_relic_create_deployment_event","new_relic_get_entity","new_relic_nrql_query","new_relic_search_entities","notion_add_database_row","notion_add_database_row_v2","notion_append_blocks","notion_append_blocks_v2","notion_create_comment","notion_create_comment_v2","notion_create_database","notion_create_database_v2","notion_create_page","notion_create_page_v2","notion_delete_block","notion_delete_block_v2","notion_list_comments","notion_list_comments_v2","notion_list_users","notion_list_users_v2","notion_query_database","notion_query_database_v2","notion_read","notion_read_database","notion_read_database_v2","notion_read_v2","notion_retrieve_block","notion_retrieve_block_children","notion_retrieve_block_children_v2","notion_retrieve_block_v2","notion_retrieve_user","notion_retrieve_user_v2","notion_search","notion_search_v2","notion_update_block","notion_update_block_v2","notion_update_page","notion_update_page_v2","notion_write","notion_write_v2","obsidian_append_active","obsidian_append_note","obsidian_append_periodic_note","obsidian_create_note","obsidian_delete_note","obsidian_execute_command","obsidian_get_active","obsidian_get_note","obsidian_get_periodic_note","obsidian_list_commands","obsidian_list_files","obsidian_open_file","obsidian_patch_active","obsidian_patch_note","obsidian_search","okta_activate_group_rule","okta_activate_user","okta_add_user_to_group","okta_assign_group_to_app","okta_assign_user_role","okta_assign_user_to_app","okta_clear_user_sessions","okta_create_group","okta_create_group_rule","okta_create_user","okta_deactivate_group_rule","okta_deactivate_user","okta_delete_group","okta_delete_group_rule","okta_delete_user","okta_enroll_factor","okta_get_app","okta_get_factor","okta_get_group","okta_get_group_rule","okta_get_logs","okta_get_session","okta_get_user","okta_list_app_groups","okta_list_app_users","okta_list_apps","okta_list_factors","okta_list_group_members","okta_list_group_rules","okta_list_groups","okta_list_user_roles","okta_list_users","okta_remove_group_from_app","okta_remove_user_from_app","okta_remove_user_from_group","okta_remove_user_role","okta_reset_all_factors","okta_reset_factor","okta_reset_password","okta_revoke_session","okta_suspend_user","okta_unsuspend_user","okta_update_group","okta_update_user","onedrive_copy","onedrive_create_folder","onedrive_create_share_link","onedrive_delete","onedrive_download","onedrive_get_drive_info","onedrive_get_item","onedrive_list","onedrive_move","onedrive_search","onedrive_upload","onepassword_create_item","onepassword_delete_item","onepassword_get_item","onepassword_get_item_file","onepassword_get_vault","onepassword_list_items","onepassword_list_vaults","onepassword_replace_item","onepassword_resolve_secret","onepassword_update_item","openai_embeddings","openai_image","outlook_calendar_create_event","outlook_calendar_delete_event","outlook_calendar_get_event","outlook_calendar_list_events","outlook_calendar_respond","outlook_calendar_update_event","outlook_copy","outlook_create_folder","outlook_delete","outlook_draft","outlook_forward","outlook_get_attachment","outlook_list_attachments","outlook_list_folders","outlook_mark_read","outlook_mark_unread","outlook_move","outlook_read","outlook_reply","outlook_reply_all","outlook_search","outlook_send","outlook_update_message","pagerduty_add_note","pagerduty_create_incident","pagerduty_get_incident","pagerduty_get_service","pagerduty_list_escalation_policies","pagerduty_list_incident_alerts","pagerduty_list_incidents","pagerduty_list_oncalls","pagerduty_list_schedules","pagerduty_list_services","pagerduty_list_users","pagerduty_merge_incidents","pagerduty_send_event","pagerduty_snooze_incident","pagerduty_update_incident","parallel_deep_research","parallel_extract","parallel_search","pdl_autocomplete","pdl_bulk_company_enrich","pdl_bulk_person_enrich","pdl_clean_company","pdl_clean_location","pdl_clean_school","pdl_company_enrich","pdl_company_search","pdl_person_enrich","pdl_person_identify","pdl_person_search","perplexity_chat","perplexity_search","persona_approve_inquiry","persona_create_account","persona_create_inquiry","persona_create_report","persona_decline_inquiry","persona_expire_inquiry","persona_generate_inquiry_link","persona_get_account","persona_get_case","persona_get_document","persona_get_inquiry","persona_get_report","persona_get_verification","persona_import_accounts","persona_list_accounts","persona_list_cases","persona_list_inquiries","persona_list_inquiry_templates","persona_list_reports","persona_mark_inquiry_for_review","persona_print_inquiry_pdf","persona_redact_account","persona_redact_inquiry","persona_resume_inquiry","persona_update_account","persona_update_inquiry","pinecone_delete_vectors","pinecone_describe_index","pinecone_describe_index_stats","pinecone_fetch","pinecone_generate_embeddings","pinecone_list_indexes","pinecone_list_vector_ids","pinecone_search_text","pinecone_search_vector","pinecone_update_vector","pinecone_upsert_text","pipedrive_create_activity","pipedrive_create_deal","pipedrive_create_lead","pipedrive_create_project","pipedrive_delete_lead","pipedrive_get_activities","pipedrive_get_all_deals","pipedrive_get_deal","pipedrive_get_files","pipedrive_get_leads","pipedrive_get_mail_messages","pipedrive_get_mail_thread","pipedrive_get_pipeline_deals","pipedrive_get_pipelines","pipedrive_get_projects","pipedrive_update_activity","pipedrive_update_deal","pipedrive_update_lead","pitchbook_company_active_investors","pitchbook_company_bio","pitchbook_company_deal_service_providers","pitchbook_company_deals","pitchbook_company_financials","pitchbook_company_general_service_providers","pitchbook_company_industries","pitchbook_company_investors","pitchbook_company_most_recent_debt_financing","pitchbook_company_most_recent_financials","pitchbook_company_most_recent_financing","pitchbook_company_search","pitchbook_company_similar_companies","pitchbook_company_social_analytics","pitchbook_company_updates","pitchbook_company_vc_exit_predictions","pitchbook_contracts_history","pitchbook_cost_of_calls","pitchbook_credit_history","pitchbook_credit_news","pitchbook_credit_news_bulk","pitchbook_credit_news_most_recent","pitchbook_credit_news_search","pitchbook_deal_bio","pitchbook_deal_cap_table_history","pitchbook_deal_debt_lenders","pitchbook_deal_detailed","pitchbook_deal_investors","pitchbook_deal_multiples","pitchbook_deal_search","pitchbook_deal_service_providers","pitchbook_deal_stock_info","pitchbook_deal_tranche_info","pitchbook_deal_updates","pitchbook_deal_valuation","pitchbook_entity_affiliates","pitchbook_entity_locations","pitchbook_entity_news","pitchbook_entity_people","pitchbook_entity_updates","pitchbook_fund_active_investments","pitchbook_fund_benchmark","pitchbook_fund_bio","pitchbook_fund_cash_flows","pitchbook_fund_commitments","pitchbook_fund_investment_preferences","pitchbook_fund_investments","pitchbook_fund_performance","pitchbook_fund_search","pitchbook_fund_team","pitchbook_fund_updates","pitchbook_investor_active_investments","pitchbook_investor_bio","pitchbook_investor_board_seats","pitchbook_investor_deal_service_providers","pitchbook_investor_funds","pitchbook_investor_general_service_providers","pitchbook_investor_investments","pitchbook_investor_last_closed_fund","pitchbook_investor_preferences","pitchbook_investor_search","pitchbook_investor_updates","pitchbook_limited_partner_actual_allocations","pitchbook_limited_partner_bio","pitchbook_limited_partner_commitment_aggregates","pitchbook_limited_partner_commitment_preferences","pitchbook_limited_partner_commitments_detailed","pitchbook_limited_partner_search","pitchbook_limited_partner_service_providers","pitchbook_limited_partner_target_allocations","pitchbook_limited_partner_updates","pitchbook_lookup_table_structure","pitchbook_lookup_tables","pitchbook_patent_detailed","pitchbook_patent_search","pitchbook_people_search","pitchbook_person_bio","pitchbook_person_contact","pitchbook_person_education_work","pitchbook_sandbox_entities","pitchbook_search","pitchbook_service_provider_bio","pitchbook_service_provider_search","pitchbook_service_provider_updates","pitchbook_serviced_companies","pitchbook_serviced_deals","pitchbook_serviced_funds","pitchbook_serviced_investors","pitchbook_serviced_limited_partners","pitchbook_shared_search","pitchbook_usage_report","polymarket_get_activity","polymarket_get_event","polymarket_get_events","polymarket_get_holders","polymarket_get_last_trade_price","polymarket_get_leaderboard","polymarket_get_market","polymarket_get_markets","polymarket_get_midpoint","polymarket_get_orderbook","polymarket_get_positions","polymarket_get_price","polymarket_get_price_history","polymarket_get_series","polymarket_get_series_by_id","polymarket_get_spread","polymarket_get_tags","polymarket_get_tick_size","polymarket_get_trades","polymarket_search","postgresql_delete","postgresql_execute","postgresql_insert","postgresql_introspect","postgresql_query","postgresql_update","posthog_batch_events","posthog_capture_event","posthog_create_annotation","posthog_create_cohort","posthog_create_dashboard","posthog_create_experiment","posthog_create_feature_flag","posthog_create_insight","posthog_create_survey","posthog_delete_feature_flag","posthog_delete_person","posthog_delete_survey","posthog_evaluate_flags","posthog_get_cohort","posthog_get_dashboard","posthog_get_event_definition","posthog_get_experiment","posthog_get_feature_flag","posthog_get_insight","posthog_get_organization","posthog_get_person","posthog_get_project","posthog_get_property_definition","posthog_get_session_recording","posthog_get_survey","posthog_list_actions","posthog_list_annotations","posthog_list_cohorts","posthog_list_dashboards","posthog_list_event_definitions","posthog_list_experiments","posthog_list_feature_flags","posthog_list_insights","posthog_list_organizations","posthog_list_persons","posthog_list_projects","posthog_list_property_definitions","posthog_list_recording_playlists","posthog_list_session_recordings","posthog_list_surveys","posthog_query","posthog_update_cohort","posthog_update_event_definition","posthog_update_experiment","posthog_update_feature_flag","posthog_update_insight","posthog_update_property_definition","posthog_update_survey","profound_bot_logs","profound_bots_report","profound_category_assets","profound_category_personas","profound_category_prompts","profound_category_tags","profound_category_topics","profound_citation_prompts","profound_citations_report","profound_list_assets","profound_list_categories","profound_list_domains","profound_list_models","profound_list_optimizations","profound_list_personas","profound_list_regions","profound_optimization_analysis","profound_prompt_answers","profound_prompt_volume","profound_query_fanouts","profound_raw_logs","profound_referrals_report","profound_sentiment_report","profound_visibility_report","prospeo_account_information","prospeo_bulk_enrich_company","prospeo_bulk_enrich_person","prospeo_enrich_company","prospeo_enrich_person","prospeo_search_company","prospeo_search_person","prospeo_search_suggestions","pulse_parser","pulse_parser_v2","qdrant_fetch_points","qdrant_search_vector","qdrant_upsert_points","quartr_get_audio","quartr_get_company","quartr_get_event","quartr_get_event_summary","quartr_get_report","quartr_get_slide_deck","quartr_get_transcript","quartr_list_audio","quartr_list_companies","quartr_list_document_types","quartr_list_documents","quartr_list_event_types","quartr_list_events","quartr_list_live_events","quartr_list_reports","quartr_list_slide_decks","quartr_list_transcripts","quiver_image_to_svg","quiver_list_models","quiver_text_to_svg","rabbitmq_create_binding","rabbitmq_create_exchange","rabbitmq_create_policy","rabbitmq_create_queue","rabbitmq_delete_binding","rabbitmq_delete_exchange","rabbitmq_delete_policy","rabbitmq_delete_queue","rabbitmq_get_exchange","rabbitmq_get_messages","rabbitmq_get_overview","rabbitmq_get_queue","rabbitmq_health_check","rabbitmq_list_bindings","rabbitmq_list_channels","rabbitmq_list_connections","rabbitmq_list_consumers","rabbitmq_list_exchange_bindings","rabbitmq_list_exchanges","rabbitmq_list_nodes","rabbitmq_list_policies","rabbitmq_list_queues","rabbitmq_list_vhosts","rabbitmq_publish_message","rabbitmq_purge_queue","railway_create_environment","railway_create_project","railway_create_service","railway_delete_environment","railway_delete_project","railway_delete_service","railway_delete_variable","railway_deploy_service","railway_get_deployment","railway_get_deployment_logs","railway_get_project","railway_list_deployments","railway_list_project_members","railway_list_projects","railway_list_variables","railway_restart_deployment","railway_rollback_deployment","railway_transfer_project","railway_update_project","railway_upsert_variable","rb2b_credit_check","rb2b_email_to_activity","rb2b_hem_to_best_linkedin","rb2b_hem_to_business_profile","rb2b_hem_to_linkedin","rb2b_hem_to_maid","rb2b_ip_to_company","rb2b_ip_to_hem","rb2b_ip_to_maid","rb2b_linkedin_slug_search","rb2b_linkedin_to_best_personal_email","rb2b_linkedin_to_business_profile","rb2b_linkedin_to_hashed_emails","rb2b_linkedin_to_mobile_phone","rb2b_linkedin_to_personal_email","rds_delete","rds_execute","rds_insert","rds_introspect","rds_query","rds_update","reddit_delete","reddit_edit","reddit_get_comments","reddit_get_controversial","reddit_get_info","reddit_get_me","reddit_get_messages","reddit_get_posts","reddit_get_saved","reddit_get_subreddit_info","reddit_get_subreddit_rules","reddit_get_user","reddit_get_user_comments","reddit_get_user_posts","reddit_hide","reddit_hot_posts","reddit_list_my_subreddits","reddit_lock","reddit_mark_all_read","reddit_mark_read","reddit_marknsfw","reddit_mod_approve","reddit_mod_distinguish","reddit_mod_remove","reddit_mod_sticky","reddit_reply","reddit_report","reddit_save","reddit_search","reddit_search_subreddits","reddit_send_message","reddit_submit_post","reddit_subscribe","reddit_unhide","reddit_unlock","reddit_unmarknsfw","reddit_unsave","reddit_vote","redis_command","redis_delete","redis_exists","redis_expire","redis_get","redis_hdel","redis_hget","redis_hgetall","redis_hset","redis_incr","redis_incrby","redis_keys","redis_llen","redis_lpop","redis_lpush","redis_lrange","redis_persist","redis_rpop","redis_rpush","redis_set","redis_setnx","redis_ttl","reducto_parser","reducto_parser_v2","resend_cancel_email","resend_create_audience","resend_create_broadcast","resend_create_contact","resend_delete_audience","resend_delete_contact","resend_get_audience","resend_get_broadcast","resend_get_contact","resend_get_email","resend_list_audiences","resend_list_contacts","resend_list_domains","resend_send","resend_send_broadcast","resend_update_contact","revenuecat_create_purchase","revenuecat_defer_google_subscription","revenuecat_delete_customer","revenuecat_get_customer","revenuecat_grant_entitlement","revenuecat_list_offerings","revenuecat_refund_google_subscription","revenuecat_revoke_entitlement","revenuecat_revoke_google_subscription","revenuecat_update_subscriber_attributes","rippling_bulk_create_custom_object_records","rippling_bulk_delete_custom_object_records","rippling_bulk_update_custom_object_records","rippling_create_business_partner","rippling_create_business_partner_group","rippling_create_custom_app","rippling_create_custom_object","rippling_create_custom_object_field","rippling_create_custom_object_record","rippling_create_custom_page","rippling_create_custom_setting","rippling_create_department","rippling_create_draft_hires","rippling_create_object_category","rippling_create_title","rippling_create_work_location","rippling_delete_business_partner","rippling_delete_business_partner_group","rippling_delete_custom_app","rippling_delete_custom_object","rippling_delete_custom_object_field","rippling_delete_custom_object_record","rippling_delete_custom_page","rippling_delete_custom_setting","rippling_delete_object_category","rippling_delete_title","rippling_delete_work_location","rippling_get_business_partner","rippling_get_business_partner_group","rippling_get_current_user","rippling_get_custom_app","rippling_get_custom_object","rippling_get_custom_object_field","rippling_get_custom_object_record","rippling_get_custom_object_record_by_external_id","rippling_get_custom_page","rippling_get_custom_setting","rippling_get_department","rippling_get_employment_type","rippling_get_job_function","rippling_get_object_category","rippling_get_report_run","rippling_get_supergroup","rippling_get_team","rippling_get_title","rippling_get_user","rippling_get_work_location","rippling_get_worker","rippling_list_business_partner_groups","rippling_list_business_partners","rippling_list_companies","rippling_list_custom_apps","rippling_list_custom_fields","rippling_list_custom_object_fields","rippling_list_custom_object_records","rippling_list_custom_objects","rippling_list_custom_pages","rippling_list_custom_settings","rippling_list_departments","rippling_list_employment_types","rippling_list_entitlements","rippling_list_job_functions","rippling_list_object_categories","rippling_list_supergroup_exclusion_members","rippling_list_supergroup_inclusion_members","rippling_list_supergroup_members","rippling_list_supergroups","rippling_list_teams","rippling_list_titles","rippling_list_users","rippling_list_work_locations","rippling_list_workers","rippling_query_custom_object_records","rippling_trigger_report_run","rippling_update_custom_app","rippling_update_custom_object","rippling_update_custom_object_field","rippling_update_custom_object_record","rippling_update_custom_page","rippling_update_custom_setting","rippling_update_department","rippling_update_object_category","rippling_update_supergroup_exclusion_members","rippling_update_supergroup_inclusion_members","rippling_update_title","rippling_update_work_location","rocketlane_add_field_option","rocketlane_add_project_members","rocketlane_add_task_assignees","rocketlane_add_task_dependencies","rocketlane_add_task_followers","rocketlane_archive_project","rocketlane_assign_placeholders","rocketlane_create_field","rocketlane_create_phase","rocketlane_create_project","rocketlane_create_space","rocketlane_create_space_document","rocketlane_create_task","rocketlane_create_time_entry","rocketlane_create_time_off","rocketlane_delete_field","rocketlane_delete_phase","rocketlane_delete_project","rocketlane_delete_space","rocketlane_delete_space_document","rocketlane_delete_task","rocketlane_delete_time_entry","rocketlane_delete_time_off","rocketlane_get_field","rocketlane_get_invoice","rocketlane_get_invoice_line_items","rocketlane_get_invoice_payments","rocketlane_get_phase","rocketlane_get_project","rocketlane_get_space","rocketlane_get_space_document","rocketlane_get_task","rocketlane_get_time_entry","rocketlane_get_time_off","rocketlane_get_user","rocketlane_import_template","rocketlane_list_fields","rocketlane_list_invoices","rocketlane_list_phases","rocketlane_list_placeholders","rocketlane_list_projects","rocketlane_list_resource_allocations","rocketlane_list_space_documents","rocketlane_list_spaces","rocketlane_list_tasks","rocketlane_list_time_entries","rocketlane_list_time_entry_categories","rocketlane_list_time_offs","rocketlane_list_users","rocketlane_move_task_to_phase","rocketlane_remove_project_members","rocketlane_remove_task_assignees","rocketlane_remove_task_dependencies","rocketlane_remove_task_followers","rocketlane_search_time_entries","rocketlane_unassign_placeholders","rocketlane_update_field","rocketlane_update_field_option","rocketlane_update_phase","rocketlane_update_project","rocketlane_update_space","rocketlane_update_space_document","rocketlane_update_task","rocketlane_update_time_entry","rootly_acknowledge_alert","rootly_add_incident_event","rootly_add_subscribers","rootly_assign_incident_role","rootly_create_action_item","rootly_create_alert","rootly_create_incident","rootly_create_status_page_event","rootly_delete_action_item","rootly_delete_incident","rootly_escalate_alert","rootly_get_alert","rootly_get_incident","rootly_list_action_items","rootly_list_alerts","rootly_list_causes","rootly_list_environments","rootly_list_escalation_policies","rootly_list_functionalities","rootly_list_incident_events","rootly_list_incident_roles","rootly_list_incident_types","rootly_list_incidents","rootly_list_on_calls","rootly_list_playbooks","rootly_list_retrospectives","rootly_list_schedules","rootly_list_services","rootly_list_severities","rootly_list_teams","rootly_list_users","rootly_mitigate_incident","rootly_remove_subscribers","rootly_resolve_alert","rootly_resolve_incident","rootly_run_workflow","rootly_snooze_alert","rootly_unassign_incident_role","rootly_update_action_item","rootly_update_alert","rootly_update_incident","s3_copy_object","s3_create_bucket","s3_delete_bucket","s3_delete_object","s3_delete_objects","s3_get_object","s3_head_object","s3_list_buckets","s3_list_objects","s3_presigned_url","s3_put_object","salesforce_create_account","salesforce_create_case","salesforce_create_contact","salesforce_create_custom_field","salesforce_create_custom_object","salesforce_create_lead","salesforce_create_opportunity","salesforce_create_task","salesforce_delete_account","salesforce_delete_case","salesforce_delete_contact","salesforce_delete_custom_field","salesforce_delete_lead","salesforce_delete_opportunity","salesforce_delete_task","salesforce_describe_object","salesforce_get_accounts","salesforce_get_cases","salesforce_get_contacts","salesforce_get_dashboard","salesforce_get_leads","salesforce_get_opportunities","salesforce_get_report","salesforce_get_tasks","salesforce_list_dashboards","salesforce_list_objects","salesforce_list_report_types","salesforce_list_reports","salesforce_query","salesforce_query_more","salesforce_refresh_dashboard","salesforce_run_report","salesforce_tooling_query","salesforce_update_account","salesforce_update_case","salesforce_update_contact","salesforce_update_custom_field","salesforce_update_lead","salesforce_update_opportunity","salesforce_update_task","sap_concur_approve_expense_report","sap_concur_associate_attendees","sap_concur_create_cash_advance","sap_concur_create_expected_expense","sap_concur_create_expense_report","sap_concur_create_list_item","sap_concur_create_purchase_request","sap_concur_create_quick_expense","sap_concur_create_quick_expense_with_image","sap_concur_create_report_comment","sap_concur_create_travel_request","sap_concur_create_user","sap_concur_delete_expected_expense","sap_concur_delete_expense","sap_concur_delete_expense_report","sap_concur_delete_list_item","sap_concur_delete_travel_request","sap_concur_delete_user","sap_concur_get_allocation","sap_concur_get_budget","sap_concur_get_cash_advance","sap_concur_get_expected_expense","sap_concur_get_expense","sap_concur_get_expense_report","sap_concur_get_itemizations","sap_concur_get_itinerary","sap_concur_get_list","sap_concur_get_list_item","sap_concur_get_purchase_request","sap_concur_get_receipt","sap_concur_get_receipt_status","sap_concur_get_request_cash_advance","sap_concur_get_travel_profile","sap_concur_get_travel_request","sap_concur_get_user","sap_concur_issue_cash_advance","sap_concur_list_allocations","sap_concur_list_attendee_associations","sap_concur_list_budget_categories","sap_concur_list_budgets","sap_concur_list_exceptions","sap_concur_list_expected_expenses","sap_concur_list_expense_reports","sap_concur_list_expenses","sap_concur_list_itineraries","sap_concur_list_list_items","sap_concur_list_lists","sap_concur_list_receipts","sap_concur_list_report_comments","sap_concur_list_reports_to_approve","sap_concur_list_travel_profiles_summary","sap_concur_list_travel_request_comments","sap_concur_list_travel_requests","sap_concur_list_users","sap_concur_move_travel_request","sap_concur_recall_expense_report","sap_concur_remove_all_attendees","sap_concur_search_locations","sap_concur_search_users","sap_concur_send_back_expense_report","sap_concur_submit_expense_report","sap_concur_update_allocation","sap_concur_update_expected_expense","sap_concur_update_expense","sap_concur_update_expense_report","sap_concur_update_list_item","sap_concur_update_travel_request","sap_concur_update_user","sap_concur_upload_exchange_rates","sap_concur_upload_receipt_image","sap_s4hana_create_business_partner","sap_s4hana_create_purchase_order","sap_s4hana_create_purchase_requisition","sap_s4hana_create_sales_order","sap_s4hana_delete_sales_order","sap_s4hana_get_billing_document","sap_s4hana_get_business_partner","sap_s4hana_get_customer","sap_s4hana_get_inbound_delivery","sap_s4hana_get_material_document","sap_s4hana_get_outbound_delivery","sap_s4hana_get_product","sap_s4hana_get_purchase_order","sap_s4hana_get_purchase_requisition","sap_s4hana_get_sales_order","sap_s4hana_get_supplier","sap_s4hana_get_supplier_invoice","sap_s4hana_list_billing_documents","sap_s4hana_list_business_partners","sap_s4hana_list_customers","sap_s4hana_list_inbound_deliveries","sap_s4hana_list_material_documents","sap_s4hana_list_material_stock","sap_s4hana_list_outbound_deliveries","sap_s4hana_list_products","sap_s4hana_list_purchase_orders","sap_s4hana_list_purchase_requisitions","sap_s4hana_list_sales_orders","sap_s4hana_list_supplier_invoices","sap_s4hana_list_suppliers","sap_s4hana_odata_query","sap_s4hana_update_business_partner","sap_s4hana_update_customer","sap_s4hana_update_product","sap_s4hana_update_purchase_order","sap_s4hana_update_purchase_requisition","sap_s4hana_update_sales_order","sap_s4hana_update_supplier","search_tool","secrets_manager_create_secret","secrets_manager_delete_secret","secrets_manager_describe_secret","secrets_manager_get_secret","secrets_manager_list_secrets","secrets_manager_restore_secret","secrets_manager_rotate_secret","secrets_manager_tag_resource","secrets_manager_untag_resource","secrets_manager_update_secret","semrush_backlinks","semrush_backlinks_anchors","semrush_backlinks_competitors","semrush_backlinks_geo_distribution","semrush_backlinks_indexed_pages","semrush_backlinks_overview","semrush_backlinks_tld_distribution","semrush_batch_keyword_overview","semrush_broad_match_keywords","semrush_domain_ad_copies","semrush_domain_ad_history","semrush_domain_organic_competitors","semrush_domain_organic_keywords","semrush_domain_overview","semrush_domain_overview_all","semrush_domain_overview_history","semrush_domain_paid_competitors","semrush_domain_paid_keywords","semrush_domain_pla_copies","semrush_domain_pla_keywords","semrush_domain_vs_domain","semrush_keyword_ad_history","semrush_keyword_difficulty","semrush_keyword_overview","semrush_keyword_overview_all","semrush_keyword_questions","semrush_organic_results","semrush_paid_results","semrush_referring_domains","semrush_referring_ips","semrush_related_keywords","semrush_subdomain_ad_copies","semrush_subdomain_organic_keywords","semrush_subdomain_overview","semrush_subdomain_overview_all","semrush_subdomain_overview_history","semrush_subdomain_paid_keywords","semrush_top_domains","semrush_url_organic_keywords","semrush_url_overview","semrush_url_overview_all","semrush_url_overview_history","semrush_url_paid_keywords","semrush_winners_and_losers","sendblue_evaluate_service","sendblue_get_message","sendblue_send_group_message","sendblue_send_message","sendblue_send_typing_indicator","sendgrid_add_contact","sendgrid_add_contacts_to_list","sendgrid_create_list","sendgrid_create_template","sendgrid_create_template_version","sendgrid_delete_contacts","sendgrid_delete_list","sendgrid_delete_template","sendgrid_get_contact","sendgrid_get_list","sendgrid_get_template","sendgrid_list_all_lists","sendgrid_list_templates","sendgrid_remove_contacts_from_list","sendgrid_search_contacts","sendgrid_send_mail","sentry_events_get","sentry_events_list","sentry_issues_get","sentry_issues_list","sentry_issues_update","sentry_projects_create","sentry_projects_get","sentry_projects_list","sentry_projects_update","sentry_releases_create","sentry_releases_deploy","sentry_releases_list","sentry_teams_list","serper_search","servicenow_add_incident_comment","servicenow_aggregate","servicenow_close_incident","servicenow_create_change_request","servicenow_create_incident","servicenow_create_record","servicenow_delete_record","servicenow_download_attachment","servicenow_find_user","servicenow_get_change_next_states","servicenow_get_change_request","servicenow_get_ci","servicenow_get_incident","servicenow_get_knowledge_article","servicenow_get_requested_item","servicenow_list_approvals","servicenow_list_attachments","servicenow_list_catalog_items","servicenow_list_change_requests","servicenow_list_change_tasks","servicenow_list_ci_relationships","servicenow_list_group_members","servicenow_list_incidents","servicenow_list_requested_items","servicenow_order_catalog_item","servicenow_read_record","servicenow_resolve_incident","servicenow_search_cis","servicenow_search_knowledge","servicenow_update_approval","servicenow_update_change_request","servicenow_update_change_state","servicenow_update_incident","servicenow_update_record","servicenow_upload_attachment","ses_create_configuration_set","ses_create_email_identity","ses_create_template","ses_delete_email_identity","ses_delete_suppressed_destination","ses_delete_template","ses_get_account","ses_get_email_identity","ses_get_suppressed_destination","ses_get_template","ses_list_identities","ses_list_suppressed_destinations","ses_list_templates","ses_put_suppressed_destination","ses_send_bulk_email","ses_send_custom_verification_email","ses_send_email","ses_send_templated_email","ses_update_template","sftp_delete","sftp_download","sftp_list","sftp_mkdir","sftp_upload","sharepoint_add_list_items","sharepoint_create_list","sharepoint_create_page","sharepoint_delete_file","sharepoint_delete_list_item","sharepoint_delete_page","sharepoint_download_file","sharepoint_get_drive_item","sharepoint_get_list","sharepoint_get_list_item","sharepoint_list_sites","sharepoint_publish_page","sharepoint_read_page","sharepoint_update_list","sharepoint_update_page","sharepoint_upload_file","shopify_adjust_inventory","shopify_cancel_order","shopify_create_customer","shopify_create_fulfillment","shopify_create_product","shopify_delete_customer","shopify_delete_product","shopify_get_collection","shopify_get_customer","shopify_get_inventory_level","shopify_get_order","shopify_get_product","shopify_list_collections","shopify_list_customers","shopify_list_inventory_items","shopify_list_locations","shopify_list_orders","shopify_list_products","shopify_update_customer","shopify_update_order","shopify_update_product","similarweb_bounce_rate","similarweb_page_views","similarweb_pages_per_visit","similarweb_traffic_visits","similarweb_visit_duration","similarweb_website_overview","sixtyfour_enrich_company","sixtyfour_enrich_lead","sixtyfour_find_email","sixtyfour_find_phone","slack_add_reaction","slack_archive_conversation","slack_canvas","slack_create_channel_canvas","slack_create_conversation","slack_delete_canvas","slack_delete_message","slack_delete_scheduled_message","slack_download","slack_edit_canvas","slack_ephemeral_message","slack_get_canvas","slack_get_channel_history","slack_get_channel_info","slack_get_message","slack_get_permalink","slack_get_thread","slack_get_thread_replies","slack_get_user","slack_get_user_presence","slack_invite_to_conversation","slack_list_canvases","slack_list_channels","slack_list_members","slack_list_scheduled_messages","slack_list_users","slack_lookup_canvas_sections","slack_message","slack_message_reader","slack_open_view","slack_publish_view","slack_push_view","slack_remove_reaction","slack_rename_conversation","slack_schedule_message","slack_set_conversation_purpose","slack_set_conversation_topic","slack_set_status","slack_set_suggested_prompts","slack_set_title","slack_update_message","slack_update_view","smartlead_add_email_accounts_to_campaign","smartlead_add_leads_to_campaign","smartlead_create_campaign","smartlead_create_lead_list","smartlead_delete_campaign","smartlead_delete_campaign_webhook","smartlead_delete_lead_from_campaign","smartlead_delete_lead_list","smartlead_duplicate_campaign","smartlead_export_campaign_leads","smartlead_get_campaign","smartlead_get_campaign_analytics","smartlead_get_campaign_analytics_by_date","smartlead_get_campaign_lead_statistics","smartlead_get_campaign_mailbox_statistics","smartlead_get_campaign_sequences","smartlead_get_campaign_statistics","smartlead_get_campaign_top_level_analytics_by_date","smartlead_get_campaign_webhook_summary","smartlead_get_lead_by_email","smartlead_get_lead_by_id","smartlead_get_lead_list","smartlead_get_lead_message_history","smartlead_list_campaign_email_accounts","smartlead_list_campaign_leads","smartlead_list_campaign_webhooks","smartlead_list_campaigns","smartlead_list_clients","smartlead_list_email_accounts","smartlead_list_inbox_replies","smartlead_list_lead_activities","smartlead_list_lead_categories","smartlead_list_lead_lists","smartlead_mark_lead_complete","smartlead_pause_lead","smartlead_remove_email_accounts_from_campaign","smartlead_resume_lead","smartlead_save_campaign_sequences","smartlead_unsubscribe_lead_from_campaign","smartlead_unsubscribe_lead_globally","smartlead_update_campaign_schedule","smartlead_update_campaign_settings","smartlead_update_campaign_status","smartlead_update_lead","smartlead_update_lead_category","smartlead_update_lead_list","smartlead_upsert_campaign_webhook","sms_send","smtp_send_mail","snowflake_alter_warehouse","snowflake_call_procedure","snowflake_cancel_statement","snowflake_cancel_task_run","snowflake_delete_rows","snowflake_execute_sql","snowflake_get_statement","snowflake_get_task","snowflake_get_task_run","snowflake_get_task_run_output","snowflake_get_warehouse","snowflake_insert_rows","snowflake_introspect_schema","snowflake_list_copy_history","snowflake_list_databases","snowflake_list_query_history","snowflake_list_schemas","snowflake_list_tables","snowflake_list_task_runs","snowflake_list_tasks","snowflake_list_warehouses","snowflake_load_data","snowflake_resume_task","snowflake_resume_warehouse","snowflake_run_task","snowflake_suspend_task","snowflake_suspend_warehouse","snowflake_unload_data","snowflake_update_rows","snowflake_upsert_rows","splunk_cancel_search_job","splunk_create_search_job","splunk_dispatch_saved_search","splunk_get_fired_alerts","splunk_get_saved_search","splunk_get_search_job","splunk_get_search_results","splunk_list_apps","splunk_list_fired_alerts","splunk_list_indexes","splunk_list_saved_searches","splunk_run_search","sportmonks_core_get_cities","sportmonks_core_get_city","sportmonks_core_get_continent","sportmonks_core_get_continents","sportmonks_core_get_countries","sportmonks_core_get_country","sportmonks_core_get_entity_filters","sportmonks_core_get_my_usage","sportmonks_core_get_region","sportmonks_core_get_regions","sportmonks_core_get_timezones","sportmonks_core_get_type","sportmonks_core_get_type_by_entity","sportmonks_core_get_types","sportmonks_core_search_cities","sportmonks_core_search_countries","sportmonks_core_search_regions","sportmonks_football_expected_by_player","sportmonks_football_expected_by_team","sportmonks_football_get_all_commentaries","sportmonks_football_get_all_fixtures","sportmonks_football_get_all_players","sportmonks_football_get_all_rivals","sportmonks_football_get_all_teams","sportmonks_football_get_all_transfer_rumours","sportmonks_football_get_all_transfers","sportmonks_football_get_brackets_by_season","sportmonks_football_get_coach","sportmonks_football_get_coaches","sportmonks_football_get_coaches_by_country","sportmonks_football_get_commentaries_by_fixture","sportmonks_football_get_current_leagues_by_team","sportmonks_football_get_expected_lineups_by_player","sportmonks_football_get_expected_lineups_by_team","sportmonks_football_get_extended_team_squad","sportmonks_football_get_fixture","sportmonks_football_get_fixtures_by_date","sportmonks_football_get_fixtures_by_date_range","sportmonks_football_get_fixtures_by_date_range_for_team","sportmonks_football_get_fixtures_by_ids","sportmonks_football_get_grouped_standings_by_round","sportmonks_football_get_head_to_head","sportmonks_football_get_inplay_livescores","sportmonks_football_get_latest_coaches","sportmonks_football_get_latest_fixtures","sportmonks_football_get_latest_livescores","sportmonks_football_get_latest_players","sportmonks_football_get_latest_totw","sportmonks_football_get_latest_transfers","sportmonks_football_get_league","sportmonks_football_get_leagues","sportmonks_football_get_leagues_by_country","sportmonks_football_get_leagues_by_date","sportmonks_football_get_leagues_by_team","sportmonks_football_get_live_leagues","sportmonks_football_get_live_probabilities","sportmonks_football_get_live_probabilities_by_fixture","sportmonks_football_get_live_standings_by_league","sportmonks_football_get_livescores","sportmonks_football_get_match_facts","sportmonks_football_get_match_facts_by_date_range","sportmonks_football_get_match_facts_by_fixture","sportmonks_football_get_match_facts_by_league","sportmonks_football_get_past_fixtures_by_tv_station","sportmonks_football_get_player","sportmonks_football_get_players_by_country","sportmonks_football_get_postmatch_news","sportmonks_football_get_postmatch_news_by_season","sportmonks_football_get_predictability_by_league","sportmonks_football_get_prematch_news","sportmonks_football_get_prematch_news_by_season","sportmonks_football_get_prematch_news_upcoming","sportmonks_football_get_probabilities","sportmonks_football_get_probabilities_by_fixture","sportmonks_football_get_referee","sportmonks_football_get_referees","sportmonks_football_get_referees_by_country","sportmonks_football_get_referees_by_season","sportmonks_football_get_rivals_by_team","sportmonks_football_get_round","sportmonks_football_get_round_statistics","sportmonks_football_get_rounds","sportmonks_football_get_rounds_by_season","sportmonks_football_get_schedules_by_season","sportmonks_football_get_schedules_by_season_and_team","sportmonks_football_get_schedules_by_team","sportmonks_football_get_season","sportmonks_football_get_seasons","sportmonks_football_get_seasons_by_team","sportmonks_football_get_stage","sportmonks_football_get_stage_statistics","sportmonks_football_get_stages","sportmonks_football_get_stages_by_season","sportmonks_football_get_standing_corrections_by_season","sportmonks_football_get_standings","sportmonks_football_get_standings_by_round","sportmonks_football_get_standings_by_season","sportmonks_football_get_state","sportmonks_football_get_states","sportmonks_football_get_team","sportmonks_football_get_team_rankings","sportmonks_football_get_team_rankings_by_date","sportmonks_football_get_team_rankings_by_team","sportmonks_football_get_team_squad","sportmonks_football_get_team_squad_by_season","sportmonks_football_get_teams_by_country","sportmonks_football_get_teams_by_season","sportmonks_football_get_topscorers_by_season","sportmonks_football_get_topscorers_by_stage","sportmonks_football_get_totw","sportmonks_football_get_totw_by_round","sportmonks_football_get_transfer","sportmonks_football_get_transfer_rumour","sportmonks_football_get_transfer_rumours_between_dates","sportmonks_football_get_transfer_rumours_by_player","sportmonks_football_get_transfer_rumours_by_team","sportmonks_football_get_transfers_between_dates","sportmonks_football_get_transfers_by_player","sportmonks_football_get_transfers_by_team","sportmonks_football_get_tv_station","sportmonks_football_get_tv_stations","sportmonks_football_get_tv_stations_by_fixture","sportmonks_football_get_upcoming_fixtures_by_market","sportmonks_football_get_upcoming_fixtures_by_tv_station","sportmonks_football_get_value_bets","sportmonks_football_get_value_bets_by_fixture","sportmonks_football_get_venue","sportmonks_football_get_venues","sportmonks_football_get_venues_by_season","sportmonks_football_search_coaches","sportmonks_football_search_fixtures","sportmonks_football_search_leagues","sportmonks_football_search_players","sportmonks_football_search_referees","sportmonks_football_search_rounds","sportmonks_football_search_seasons","sportmonks_football_search_stages","sportmonks_football_search_teams","sportmonks_football_search_venues","sportmonks_motorsport_get_all_fixtures","sportmonks_motorsport_get_current_leagues_by_team","sportmonks_motorsport_get_driver","sportmonks_motorsport_get_driver_standings","sportmonks_motorsport_get_driver_standings_by_season","sportmonks_motorsport_get_drivers","sportmonks_motorsport_get_drivers_by_country","sportmonks_motorsport_get_drivers_by_season","sportmonks_motorsport_get_fixture","sportmonks_motorsport_get_fixtures_by_date","sportmonks_motorsport_get_fixtures_by_date_range","sportmonks_motorsport_get_fixtures_by_ids","sportmonks_motorsport_get_laps_by_fixture","sportmonks_motorsport_get_laps_by_fixture_and_driver","sportmonks_motorsport_get_laps_by_fixture_and_lap","sportmonks_motorsport_get_latest_laps_by_fixture","sportmonks_motorsport_get_latest_pitstops_by_fixture","sportmonks_motorsport_get_latest_stints_by_fixture","sportmonks_motorsport_get_latest_updated_drivers","sportmonks_motorsport_get_latest_updated_fixtures","sportmonks_motorsport_get_league","sportmonks_motorsport_get_leagues","sportmonks_motorsport_get_leagues_by_country","sportmonks_motorsport_get_leagues_by_date","sportmonks_motorsport_get_leagues_by_live","sportmonks_motorsport_get_leagues_by_team","sportmonks_motorsport_get_livescores","sportmonks_motorsport_get_pitstops_by_fixture","sportmonks_motorsport_get_pitstops_by_fixture_and_driver","sportmonks_motorsport_get_pitstops_by_fixture_and_lap","sportmonks_motorsport_get_race_results_by_season_and_driver","sportmonks_motorsport_get_race_results_by_season_and_team","sportmonks_motorsport_get_schedules_by_season","sportmonks_motorsport_get_season","sportmonks_motorsport_get_seasons","sportmonks_motorsport_get_stage","sportmonks_motorsport_get_stages","sportmonks_motorsport_get_stages_by_season","sportmonks_motorsport_get_state","sportmonks_motorsport_get_states","sportmonks_motorsport_get_stints_by_fixture","sportmonks_motorsport_get_stints_by_fixture_and_driver","sportmonks_motorsport_get_stints_by_fixture_and_stint","sportmonks_motorsport_get_team","sportmonks_motorsport_get_team_standings","sportmonks_motorsport_get_team_standings_by_season","sportmonks_motorsport_get_teams","sportmonks_motorsport_get_teams_by_country","sportmonks_motorsport_get_teams_by_season","sportmonks_motorsport_get_venue","sportmonks_motorsport_get_venues","sportmonks_motorsport_get_venues_by_season","sportmonks_motorsport_search_drivers","sportmonks_motorsport_search_leagues","sportmonks_motorsport_search_stages","sportmonks_motorsport_search_teams","sportmonks_motorsport_search_venues","sportmonks_odds_get_all_historical_odds","sportmonks_odds_get_all_inplay_odds","sportmonks_odds_get_all_pre_match_odds","sportmonks_odds_get_all_premium_odds","sportmonks_odds_get_bookmaker","sportmonks_odds_get_bookmaker_event_ids_by_fixture","sportmonks_odds_get_bookmakers","sportmonks_odds_get_bookmakers_by_fixture","sportmonks_odds_get_inplay_odds_by_fixture","sportmonks_odds_get_inplay_odds_by_fixture_and_bookmaker","sportmonks_odds_get_inplay_odds_by_fixture_and_market","sportmonks_odds_get_last_updated_inplay_odds","sportmonks_odds_get_last_updated_pre_match_odds","sportmonks_odds_get_market","sportmonks_odds_get_markets","sportmonks_odds_get_pre_match_odds_by_fixture","sportmonks_odds_get_pre_match_odds_by_fixture_and_bookmaker","sportmonks_odds_get_pre_match_odds_by_fixture_and_market","sportmonks_odds_get_premium_odds_by_fixture","sportmonks_odds_get_premium_odds_by_fixture_and_bookmaker","sportmonks_odds_get_premium_odds_by_fixture_and_market","sportmonks_odds_get_updated_historical_odds_between","sportmonks_odds_get_updated_premium_odds_between","sportmonks_odds_search_bookmakers","sportmonks_odds_search_markets","spotify_add_playlist_cover","spotify_add_to_queue","spotify_add_tracks_to_playlist","spotify_check_following","spotify_check_playlist_followers","spotify_check_saved_albums","spotify_check_saved_audiobooks","spotify_check_saved_episodes","spotify_check_saved_shows","spotify_check_saved_tracks","spotify_create_playlist","spotify_follow_artists","spotify_follow_playlist","spotify_get_album","spotify_get_album_tracks","spotify_get_albums","spotify_get_artist","spotify_get_artist_albums","spotify_get_artist_top_tracks","spotify_get_artists","spotify_get_audiobook","spotify_get_audiobook_chapters","spotify_get_audiobooks","spotify_get_categories","spotify_get_current_user","spotify_get_currently_playing","spotify_get_devices","spotify_get_episode","spotify_get_episodes","spotify_get_followed_artists","spotify_get_markets","spotify_get_new_releases","spotify_get_playback_state","spotify_get_playlist","spotify_get_playlist_cover","spotify_get_playlist_tracks","spotify_get_queue","spotify_get_recently_played","spotify_get_saved_albums","spotify_get_saved_audiobooks","spotify_get_saved_episodes","spotify_get_saved_shows","spotify_get_saved_tracks","spotify_get_show","spotify_get_show_episodes","spotify_get_shows","spotify_get_top_artists","spotify_get_top_tracks","spotify_get_track","spotify_get_tracks","spotify_get_user_playlists","spotify_get_user_profile","spotify_pause","spotify_play","spotify_remove_saved_albums","spotify_remove_saved_audiobooks","spotify_remove_saved_episodes","spotify_remove_saved_shows","spotify_remove_saved_tracks","spotify_remove_tracks_from_playlist","spotify_reorder_playlist_items","spotify_replace_playlist_items","spotify_save_albums","spotify_save_audiobooks","spotify_save_episodes","spotify_save_shows","spotify_save_tracks","spotify_search","spotify_seek","spotify_set_repeat","spotify_set_shuffle","spotify_set_volume","spotify_skip_next","spotify_skip_previous","spotify_transfer_playback","spotify_unfollow_artists","spotify_unfollow_playlist","spotify_update_playlist","sqs_send","square_batch_retrieve_inventory_counts","square_cancel_invoice","square_cancel_payment","square_complete_payment","square_create_catalog_image","square_create_customer","square_create_invoice","square_create_order","square_create_payment","square_delete_catalog_object","square_delete_customer","square_delete_invoice","square_get_catalog_object","square_get_customer","square_get_invoice","square_get_location","square_get_order","square_get_payment","square_get_refund","square_list_catalog","square_list_customers","square_list_invoices","square_list_locations","square_list_payments","square_list_refunds","square_pay_order","square_publish_invoice","square_refund_payment","square_search_catalog_objects","square_search_customers","square_search_invoices","square_search_orders","square_update_customer","square_upsert_catalog_object","ssh_check_command_exists","ssh_check_file_exists","ssh_create_directory","ssh_delete_file","ssh_download_file","ssh_execute_command","ssh_execute_script","ssh_get_system_info","ssh_list_directory","ssh_move_rename","ssh_read_file_content","ssh_upload_file","ssh_write_file_content","stagehand_agent","stagehand_extract","stripe_cancel_payment_intent","stripe_cancel_subscription","stripe_capture_charge","stripe_capture_payment_intent","stripe_confirm_payment_intent","stripe_create_charge","stripe_create_customer","stripe_create_invoice","stripe_create_payment_intent","stripe_create_price","stripe_create_product","stripe_create_subscription","stripe_delete_customer","stripe_delete_invoice","stripe_delete_product","stripe_finalize_invoice","stripe_list_charges","stripe_list_customers","stripe_list_events","stripe_list_invoices","stripe_list_payment_intents","stripe_list_prices","stripe_list_products","stripe_list_subscriptions","stripe_pay_invoice","stripe_resume_subscription","stripe_retrieve_charge","stripe_retrieve_customer","stripe_retrieve_event","stripe_retrieve_invoice","stripe_retrieve_payment_intent","stripe_retrieve_price","stripe_retrieve_product","stripe_retrieve_subscription","stripe_search_charges","stripe_search_customers","stripe_search_invoices","stripe_search_payment_intents","stripe_search_prices","stripe_search_products","stripe_search_subscriptions","stripe_send_invoice","stripe_update_charge","stripe_update_customer","stripe_update_invoice","stripe_update_payment_intent","stripe_update_price","stripe_update_product","stripe_update_subscription","stripe_void_invoice","sts_assume_role","sts_assume_role_with_saml","sts_assume_role_with_web_identity","sts_get_access_key_info","sts_get_caller_identity","sts_get_session_token","stt_assemblyai","stt_assemblyai_v2","stt_deepgram","stt_deepgram_v2","stt_elevenlabs","stt_elevenlabs_v2","stt_gemini","stt_gemini_v2","stt_whisper","stt_whisper_v2","supabase_count","supabase_delete","supabase_get_row","supabase_insert","supabase_introspect","supabase_invoke_function","supabase_query","supabase_rpc","supabase_storage_copy","supabase_storage_create_bucket","supabase_storage_create_signed_upload_url","supabase_storage_create_signed_url","supabase_storage_delete","supabase_storage_delete_bucket","supabase_storage_download","supabase_storage_empty_bucket","supabase_storage_get_public_url","supabase_storage_list","supabase_storage_list_buckets","supabase_storage_move","supabase_storage_update_bucket","supabase_storage_upload","supabase_text_search","supabase_update","supabase_upsert","supabase_vector_search","table_batch_insert_rows","table_create","table_delete_row","table_delete_rows_by_filter","table_get_row","table_get_schema","table_insert_row","table_list","table_query_rows","table_query_rows_v2","table_update_row","table_update_rows_by_filter","table_upsert_row","tailscale_authorize_device","tailscale_create_auth_key","tailscale_delete_auth_key","tailscale_delete_device","tailscale_delete_user","tailscale_expire_device_key","tailscale_get_acl","tailscale_get_auth_key","tailscale_get_device","tailscale_get_device_routes","tailscale_get_dns_preferences","tailscale_get_dns_searchpaths","tailscale_list_auth_keys","tailscale_list_devices","tailscale_list_dns_nameservers","tailscale_list_users","tailscale_set_acl","tailscale_set_device_routes","tailscale_set_device_tags","tailscale_set_dns_nameservers","tailscale_set_dns_preferences","tailscale_set_dns_searchpaths","tailscale_suspend_user","tailscale_update_device_key","tavily_crawl","tavily_extract","tavily_map","tavily_search","telegram_copy_message","telegram_delete_message","telegram_edit_message_text","telegram_forward_message","telegram_get_chat","telegram_get_chat_member","telegram_message","telegram_pin_message","telegram_send_animation","telegram_send_audio","telegram_send_chat_action","telegram_send_contact","telegram_send_document","telegram_send_location","telegram_send_photo","telegram_send_poll","telegram_send_video","telegram_set_message_reaction","telegram_unpin_message","temporal_cancel_workflow","temporal_count_workflows","temporal_create_schedule","temporal_delete_schedule","temporal_describe_schedule","temporal_describe_task_queue","temporal_describe_workflow","temporal_get_workflow_history","temporal_list_schedules","temporal_list_workflows","temporal_pause_schedule","temporal_query_workflow","temporal_reset_workflow","temporal_signal_with_start","temporal_signal_workflow","temporal_start_workflow","temporal_terminate_workflow","temporal_trigger_schedule","temporal_unpause_schedule","temporal_update_workflow","textract_analyze_expense","textract_analyze_id","textract_parser","textract_parser_v2","thinking_tool","thrive_add_audience_managers","thrive_add_audience_members","thrive_add_user_tags","thrive_create_assignment","thrive_create_audience","thrive_create_completion","thrive_create_user","thrive_delete_assignment","thrive_delete_audience","thrive_delete_user","thrive_get_activity","thrive_get_assignment","thrive_get_audience","thrive_get_completion","thrive_get_content","thrive_get_cpd_category","thrive_get_cpd_entry","thrive_get_cpd_requirement","thrive_get_enrolment","thrive_get_skill_levels","thrive_get_tag","thrive_get_user_by_id","thrive_get_user_by_ref","thrive_list_assignments","thrive_list_audience_managers","thrive_list_audience_members","thrive_list_audiences","thrive_list_completions","thrive_list_enrolments","thrive_list_tags","thrive_query_activities","thrive_query_content","thrive_query_cpd_categories","thrive_query_cpd_entries","thrive_query_cpd_requirements","thrive_query_cpd_user_summaries","thrive_remove_audience_manager","thrive_remove_audience_member","thrive_remove_user_tags","thrive_replace_audience_managers","thrive_replace_audience_members","thrive_search_users","thrive_suspend_user","thrive_update_assignment","thrive_update_audience","thrive_update_user","thrive_update_user_skills","tiktok_get_post_status","tiktok_get_user","tiktok_list_videos","tiktok_query_videos","tiktok_upload_video_draft","tinybird_append_datasource","tinybird_delete_datasource_rows","tinybird_events","tinybird_get_job","tinybird_query","tinybird_query_pipe","tinybird_truncate_datasource","tinyfish_cancel_run","tinyfish_fetch","tinyfish_get_run","tinyfish_list_runs","tinyfish_list_vault_items","tinyfish_run","tinyfish_run_async","tinyfish_search","trello_add_checklist","trello_add_checklist_item","trello_add_comment","trello_add_label","trello_add_member","trello_create_board","trello_create_card","trello_create_list","trello_delete_card","trello_get_actions","trello_get_board","trello_get_card","trello_list_cards","trello_list_lists","trello_list_members","trello_remove_label","trello_remove_member","trello_search","trello_update_card","trello_update_checklist_item","trello_update_list","trigger_dev_activate_schedule","trigger_dev_add_run_tags","trigger_dev_batch_trigger_task","trigger_dev_cancel_run","trigger_dev_complete_waitpoint_token","trigger_dev_create_env_var","trigger_dev_create_schedule","trigger_dev_create_waitpoint_token","trigger_dev_deactivate_schedule","trigger_dev_delete_env_var","trigger_dev_delete_schedule","trigger_dev_execute_query","trigger_dev_get_batch","trigger_dev_get_batch_results","trigger_dev_get_deployment","trigger_dev_get_env_var","trigger_dev_get_latest_deployment","trigger_dev_get_query_schema","trigger_dev_get_queue","trigger_dev_get_run","trigger_dev_get_run_events","trigger_dev_get_run_result","trigger_dev_get_run_trace","trigger_dev_get_schedule","trigger_dev_get_waitpoint_token","trigger_dev_import_env_vars","trigger_dev_list_deployments","trigger_dev_list_env_vars","trigger_dev_list_queues","trigger_dev_list_runs","trigger_dev_list_schedules","trigger_dev_list_timezones","trigger_dev_list_waitpoint_tokens","trigger_dev_override_queue_concurrency","trigger_dev_pause_queue","trigger_dev_promote_deployment","trigger_dev_replay_run","trigger_dev_reschedule_run","trigger_dev_reset_queue_concurrency","trigger_dev_resume_queue","trigger_dev_trigger_task","trigger_dev_update_env_var","trigger_dev_update_run_metadata","trigger_dev_update_schedule","tts_azure","tts_cartesia","tts_deepgram","tts_elevenlabs","tts_google","tts_openai","tts_playht","twilio_send_sms","twilio_voice_get_recording","twilio_voice_list_calls","twilio_voice_make_call","typeform_create_form","typeform_delete_form","typeform_files","typeform_get_form","typeform_insights","typeform_list_forms","typeform_responses","typeform_update_form","upstash_redis_command","upstash_redis_delete","upstash_redis_exists","upstash_redis_expire","upstash_redis_get","upstash_redis_hget","upstash_redis_hgetall","upstash_redis_hset","upstash_redis_incr","upstash_redis_incrby","upstash_redis_keys","upstash_redis_lpush","upstash_redis_lrange","upstash_redis_set","upstash_redis_setnx","upstash_redis_ttl","uptimerobot_create_alert_contact","uptimerobot_create_maintenance_window","uptimerobot_create_monitor","uptimerobot_create_psp","uptimerobot_delete_alert_contact","uptimerobot_delete_maintenance_window","uptimerobot_delete_monitor","uptimerobot_delete_psp","uptimerobot_get_account","uptimerobot_get_alert_contact","uptimerobot_get_incident","uptimerobot_get_maintenance_window","uptimerobot_get_monitor","uptimerobot_get_psp","uptimerobot_list_alert_contacts","uptimerobot_list_incidents","uptimerobot_list_maintenance_windows","uptimerobot_list_monitors","uptimerobot_list_psps","uptimerobot_pause_monitor","uptimerobot_start_monitor","uptimerobot_update_maintenance_window","uptimerobot_update_monitor","uptimerobot_update_psp","vanta_download_document_file","vanta_get_control","vanta_get_document","vanta_get_framework","vanta_get_person","vanta_get_policy","vanta_get_risk_scenario","vanta_get_test","vanta_get_vendor","vanta_get_vulnerable_asset","vanta_list_control_documents","vanta_list_control_tests","vanta_list_controls","vanta_list_document_uploads","vanta_list_documents","vanta_list_framework_controls","vanta_list_frameworks","vanta_list_monitored_computers","vanta_list_people","vanta_list_policies","vanta_list_risk_scenarios","vanta_list_test_entities","vanta_list_tests","vanta_list_vendors","vanta_list_vulnerabilities","vanta_list_vulnerability_remediations","vanta_list_vulnerable_assets","vanta_submit_document","vanta_upload_document_file","vercel_add_domain","vercel_add_project_domain","vercel_cancel_deployment","vercel_create_alias","vercel_create_check","vercel_create_deployment","vercel_create_dns_record","vercel_create_edge_config","vercel_create_env_var","vercel_create_project","vercel_create_webhook","vercel_delete_alias","vercel_delete_deployment","vercel_delete_dns_record","vercel_delete_domain","vercel_delete_edge_config","vercel_delete_env_var","vercel_delete_project","vercel_delete_webhook","vercel_get_alias","vercel_get_check","vercel_get_deployment","vercel_get_deployment_events","vercel_get_domain","vercel_get_domain_config","vercel_get_edge_config","vercel_get_edge_config_items","vercel_get_env_vars","vercel_get_project","vercel_get_team","vercel_get_user","vercel_get_webhook","vercel_list_aliases","vercel_list_checks","vercel_list_deployment_files","vercel_list_deployments","vercel_list_dns_records","vercel_list_domains","vercel_list_edge_configs","vercel_list_project_domains","vercel_list_projects","vercel_list_team_members","vercel_list_teams","vercel_list_webhooks","vercel_pause_project","vercel_promote_deployment","vercel_remove_project_domain","vercel_rerequest_check","vercel_unpause_project","vercel_update_check","vercel_update_dns_record","vercel_update_edge_config_items","vercel_update_env_var","vercel_update_project","vercel_update_project_domain","vercel_verify_project_domain","video_falai","video_luma","video_minimax","video_runway","video_veo","vision_tool","vision_tool_v2","wealthbox_read_contact","wealthbox_read_note","wealthbox_read_task","wealthbox_write_contact","wealthbox_write_note","wealthbox_write_task","webflow_create_item","webflow_delete_item","webflow_get_item","webflow_list_items","webflow_update_item","webhook_request","whatsapp_get_media","whatsapp_mark_read","whatsapp_send_interactive","whatsapp_send_media","whatsapp_send_message","whatsapp_send_reaction","whatsapp_send_template","whatsapp_upload_media","wikipedia_content","wikipedia_random","wikipedia_search","wikipedia_summary","windchill_check_in_document","windchill_check_in_documents","windchill_check_out_document","windchill_check_out_documents","windchill_create_document","windchill_create_documents","windchill_delete_document","windchill_delete_documents","windchill_download_attachment","windchill_download_primary_content","windchill_get_document","windchill_get_document_structure","windchill_get_primary_content","windchill_get_valid_state_transitions","windchill_list_attachments","windchill_list_documents","windchill_revise_document","windchill_revise_documents","windchill_set_lifecycle_state","windchill_undo_check_out_document","windchill_undo_check_out_documents","windchill_update_common_properties","windchill_update_document","windchill_update_document_security_labels","windchill_update_documents","windchill_upload_attachments","windchill_upload_primary_content","wiza_company_enrichment","wiza_get_credits","wiza_individual_reveal","wiza_prospect_search","wordpress_create_category","wordpress_create_comment","wordpress_create_page","wordpress_create_post","wordpress_create_tag","wordpress_delete_category","wordpress_delete_comment","wordpress_delete_media","wordpress_delete_page","wordpress_delete_post","wordpress_delete_tag","wordpress_get_category","wordpress_get_current_user","wordpress_get_media","wordpress_get_page","wordpress_get_post","wordpress_get_tag","wordpress_get_user","wordpress_list_categories","wordpress_list_comments","wordpress_list_media","wordpress_list_pages","wordpress_list_posts","wordpress_list_tags","wordpress_list_users","wordpress_search_content","wordpress_update_category","wordpress_update_comment","wordpress_update_page","wordpress_update_post","wordpress_update_tag","wordpress_upload_media","workday_assign_onboarding","workday_change_job","workday_create_prehire","workday_get_compensation","workday_get_organizations","workday_get_worker","workday_hire_employee","workday_list_workers","workday_terminate_worker","workday_update_worker","workflow_executor","x_create_bookmark","x_create_tweet","x_delete_bookmark","x_delete_tweet","x_get_blocking","x_get_bookmarks","x_get_followers","x_get_following","x_get_liked_tweets","x_get_liking_users","x_get_me","x_get_personalized_trends","x_get_quote_tweets","x_get_retweeted_by","x_get_trends_by_woeid","x_get_tweets_by_ids","x_get_usage","x_get_user_mentions","x_get_user_timeline","x_get_user_tweets","x_hide_reply","x_manage_block","x_manage_follow","x_manage_like","x_manage_mute","x_manage_retweet","x_read","x_search","x_search_tweets","x_search_users","x_user","x_write","youtube_channel_info","youtube_channel_playlists","youtube_channel_videos","youtube_comments","youtube_playlist_items","youtube_search","youtube_trending","youtube_video_categories","youtube_video_details","zendesk_autocomplete_organizations","zendesk_create_organization","zendesk_create_organizations_bulk","zendesk_create_ticket","zendesk_create_tickets_bulk","zendesk_create_user","zendesk_create_users_bulk","zendesk_delete_organization","zendesk_delete_ticket","zendesk_delete_user","zendesk_get_current_user","zendesk_get_organization","zendesk_get_organizations","zendesk_get_ticket","zendesk_get_tickets","zendesk_get_user","zendesk_get_users","zendesk_merge_tickets","zendesk_search","zendesk_search_count","zendesk_search_users","zendesk_update_organization","zendesk_update_ticket","zendesk_update_tickets_bulk","zendesk_update_user","zendesk_update_users_bulk","zep_add_messages","zep_add_user","zep_create_thread","zep_delete_thread","zep_get_context","zep_get_messages","zep_get_threads","zep_get_user","zep_get_user_threads","zerobounce_get_credits","zerobounce_verify_email","zoho_desk_add_comment","zoho_desk_get_attachment","zoho_desk_get_contact","zoho_desk_get_thread","zoho_desk_get_ticket","zoho_desk_list_comments","zoho_desk_list_organizations","zoho_desk_list_threads","zoho_desk_list_tickets","zoho_desk_update_ticket","zoom_create_meeting","zoom_delete_meeting","zoom_delete_recording","zoom_get_meeting","zoom_get_meeting_invitation","zoom_get_meeting_recordings","zoom_list_meetings","zoom_list_past_participants","zoom_list_recordings","zoom_update_meeting","zoominfo_enrich_companies","zoominfo_enrich_contacts","zoominfo_search_companies","zoominfo_search_contacts","zoominfo_search_intent","zoominfo_search_news"]' ) export default toolIds diff --git a/apps/sim/tools/generated/tool-metadata.ts b/apps/sim/tools/generated/tool-metadata.ts index 2b7e717cda1..f9e4168c1b3 100644 --- a/apps/sim/tools/generated/tool-metadata.ts +++ b/apps/sim/tools/generated/tool-metadata.ts @@ -3,7 +3,7 @@ /** Serializable metadata for every built-in tool, keyed by tool id. */ const toolMetadata: Record = JSON.parse( - '{"a2a_cancel_task":{"id":"a2a_cancel_task","name":"A2A Cancel Task","description":"Request cancellation of an in-progress A2A task.","version":"1.0.0","params":{"agentUrl":{"type":"string","required":true,"visibility":"user-only","description":"The A2A agent endpoint URL"},"taskId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The task ID to cancel"},"apiKey":{"type":"string","required":false,"visibility":"user-only","description":"API key for authentication (if required)"}},"hostedApiKey":"none"},"a2a_get_agent_card":{"id":"a2a_get_agent_card","name":"A2A Get Agent Card","description":"Fetch the Agent Card (discovery document) for an external A2A agent.","version":"1.0.0","params":{"agentUrl":{"type":"string","required":true,"visibility":"user-only","description":"The A2A agent endpoint URL"},"apiKey":{"type":"string","required":false,"visibility":"user-only","description":"API key for authentication (if required)"}},"hostedApiKey":"none"},"a2a_get_task":{"id":"a2a_get_task","name":"A2A Get Task","description":"Retrieve the current state and result of an A2A task.","version":"1.0.0","params":{"agentUrl":{"type":"string","required":true,"visibility":"user-only","description":"The A2A agent endpoint URL"},"taskId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The task ID to retrieve"},"historyLength":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of history messages to include"},"apiKey":{"type":"string","required":false,"visibility":"user-only","description":"API key for authentication (if required)"}},"hostedApiKey":"none"},"a2a_send_message":{"id":"a2a_send_message","name":"A2A Send Message","description":"Send a message to an external A2A agent and return its response.","version":"1.0.0","params":{"agentUrl":{"type":"string","required":true,"visibility":"user-only","description":"The A2A agent endpoint URL"},"message":{"type":"string","required":true,"visibility":"user-or-llm","description":"The message text to send"},"data":{"type":"json","required":false,"visibility":"user-or-llm","description":"Optional structured JSON data to attach"},"files":{"type":"json","required":false,"visibility":"user-or-llm","description":"Optional files to attach"},"taskId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Existing task ID to continue"},"contextId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Conversation context ID to continue"},"apiKey":{"type":"string","required":false,"visibility":"user-only","description":"API key for authentication (if required)"}},"hostedApiKey":"none"},"affinity_batch_update_entity_fields":{"id":"affinity_batch_update_entity_fields","name":"Affinity Batch Update Entity Fields","description":"Write up to 100 non-list field values on one company or person in a single request.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Which entity to write the fields on: companies or persons"},"entityId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of that company or person"},"updates":{"type":"json","required":true,"visibility":"user-or-llm","description":"Up to 100 field updates as [{\\"id\\":\\"\\",\\"value\\":{\\"type\\":\\"…\\",\\"data\\":…}}], using the same value shapes as a single field update"}},"hostedApiKey":"none"},"affinity_batch_update_list_entry_fields":{"id":"affinity_batch_update_list_entry_fields","name":"Affinity Batch Update List Entry Fields","description":"Write up to 100 field values on one list row in a single request. Requires the \\"Export data from Lists\\" permission.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"listEntryId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list entry ID"},"updates":{"type":"json","required":true,"visibility":"user-or-llm","description":"Up to 100 field updates as [{\\"id\\":\\"\\",\\"value\\":{\\"type\\":\\"…\\",\\"data\\":…}}], using the same value shapes as a single field update"}},"hostedApiKey":"none"},"affinity_create_list":{"id":"affinity_create_list","name":"Affinity Create List","description":"Create a list. Its type fixes which entities it can hold, and the API key holder becomes its creator and owner.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"name":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the new list"},"type":{"type":"string","required":true,"visibility":"user-or-llm","description":"Entity kind the list holds: company, opportunity, or person"},"isPublic":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Whether everyone in the organization can see the list"}},"hostedApiKey":"none"},"affinity_create_list_field_dropdown_option":{"id":"affinity_create_list_field_dropdown_option","name":"Affinity Create List Field Dropdown Option","description":"Add a selectable option to a dropdown field on a list. A ranked or status option also needs a rank and a color.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"fieldId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The dropdown field ID on that list"},"type":{"type":"string","required":true,"visibility":"user-or-llm","description":"Kind of option to create, matching the field. dropdown takes only a label; ranked-dropdown also requires rank and color; status-dropdown additionally requires a status category. Sending a field the kind does not accept is rejected"},"text":{"type":"string","required":true,"visibility":"user-or-llm","description":"The option label"},"rank":{"type":"number","required":false,"visibility":"user-or-llm","description":"Sort order. Required on a ranked-dropdown or status-dropdown option"},"color":{"type":"string","required":false,"visibility":"user-or-llm","description":"Option color: white, gray, blue, green, purple, orange, or red. Required on a ranked-dropdown or status-dropdown option"},"statusCategory":{"type":"string","required":false,"visibility":"user-or-llm","description":"Pipeline meaning of the option: open, won, lost, or on-hold. Status-dropdown options only"},"winRate":{"type":"number","required":false,"visibility":"user-or-llm","description":"Expected win rate of the status. Status-dropdown options only"}},"hostedApiKey":"none"},"affinity_create_merge":{"id":"affinity_create_merge","name":"Affinity Create Merge","description":"Fold a duplicate company or person into the record you are keeping. The merge runs asynchronously — poll the returned task to see it finish. Requires the \\"Manage duplicates\\" permission and an admin role.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"What to merge: companies or persons"},"primaryId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the record to keep"},"duplicateId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the duplicate record to fold in"}},"hostedApiKey":"none"},"affinity_create_note":{"id":"affinity_create_note","name":"Affinity Create Note","description":"Write a note — attached to companies, persons, and opportunities, anchored to a meeting, call, or chat message, or posted as a reply to an existing note.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"type":{"type":"string","required":true,"visibility":"user-or-llm","description":"Note shape: entities to attach it to records, interaction to anchor it to a meeting, call, or chat message, or user-reply to reply to a note"},"html":{"type":"string","required":true,"visibility":"user-or-llm","description":"The note body as HTML"},"companyIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Companies to attach the note to, e.g. [1, 2]. Not used on a reply"},"personIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Persons to attach the note to, e.g. [1, 2]. Not used on a reply"},"opportunityIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Opportunities to attach the note to, e.g. [1, 2]. Not used on a reply"},"interactionId":{"type":"string","required":false,"visibility":"user-or-llm","description":"The interaction to anchor the note to. Required for an interaction note"},"interactionType":{"type":"string","required":false,"visibility":"user-or-llm","description":"Kind of the anchoring interaction: meeting, call, or chat-message. Required for an interaction note"},"parentId":{"type":"string","required":false,"visibility":"user-or-llm","description":"The note being replied to. Required for a user-reply note"},"creatorId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Attribute the note to another internal person. Defaults to the API key holder"},"createdAt":{"type":"string","required":false,"visibility":"user-or-llm","description":"Backdate the note to this ISO 8601 timestamp"}},"hostedApiKey":"none"},"affinity_create_reminder":{"id":"affinity_create_reminder","name":"Affinity Create Reminder","description":"Create a reminder on one company, person, or opportunity. A recurring reminder resets whenever the chosen signal happens instead of firing once.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"type":{"type":"string","required":true,"visibility":"user-or-llm","description":"one-time to fire once, or recurring to reset on a signal"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"What the reminder is about: company, person, or opportunity"},"entityId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of that company, person, or opportunity"},"dueDate":{"type":"string","required":false,"visibility":"user-or-llm","description":"When the reminder is due, as an ISO 8601 timestamp. Required for a one-time reminder; on a recurring one Affinity computes it from the period when omitted"},"content":{"type":"string","required":false,"visibility":"user-or-llm","description":"What the reminder says"},"ownerId":{"type":"string","required":true,"visibility":"user-or-llm","description":"User the reminder is assigned to. Must be an internal user. The API key holder is recorded as the creator, which is a separate field"},"resetTrigger":{"type":"string","required":false,"visibility":"user-or-llm","description":"What restarts a recurring reminder: interaction, email, or event. Required when the type is recurring"},"periodDays":{"type":"number","required":false,"visibility":"user-or-llm","description":"Days between firings of a recurring reminder. Required when the type is recurring"}},"hostedApiKey":"none"},"affinity_delete_list_field_dropdown_option":{"id":"affinity_delete_list_field_dropdown_option","name":"Affinity Delete List Field Dropdown Option","description":"Permanently delete a dropdown option on a list field. Every list entry currently set to it is cleared, and those values cannot be recovered.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"fieldId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The dropdown field ID on that list"},"dropdownOptionId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The dropdown option ID to delete"}},"hostedApiKey":"none"},"affinity_delete_note":{"id":"affinity_delete_note","name":"Affinity Delete Note","description":"Delete a note you created. Deleting a root note also deletes its replies; deleting a reply removes only that reply.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"noteId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The note ID to delete"}},"hostedApiKey":"none"},"affinity_get_company":{"id":"affinity_get_company","name":"Affinity Get Company","description":"Look up one company by ID. Field data is returned only for the Field IDs or Field Types asked for.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"companyId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The company ID"},"fieldIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field IDs to return values for, e.g. [\\"affinity-data-location\\"]. Mutually exclusive with Field Types"},"fieldTypes":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field categories to return values for: enriched, global, or relationship-intelligence. Mutually exclusive with Field IDs"}},"hostedApiKey":"none"},"affinity_get_current_user":{"id":"affinity_get_current_user","name":"Affinity Get Current User","description":"Verify an Affinity API key and return the tenant, the user behind the key, and the scopes the grant carries.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"}},"hostedApiKey":"none"},"affinity_get_entity_field_value":{"id":"affinity_get_entity_field_value","name":"Affinity Get Entity Field Value","description":"Read one non-list field value from a company or person.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Which entity to read the field from: companies or persons"},"entityId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of that company or person"},"fieldId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The field ID to read"}},"hostedApiKey":"none"},"affinity_get_list":{"id":"affinity_get_list","name":"Affinity Get List","description":"Read one list — its name, type, owner, and privacy setting.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"}},"hostedApiKey":"none"},"affinity_get_list_entry":{"id":"affinity_get_list_entry","name":"Affinity Get List Entry","description":"Read one row of a list with its entity. Field data is returned only for the Field IDs or Field Types asked for.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"listEntryId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list entry ID"},"fieldIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field IDs to return values for, e.g. [\\"affinity-data-location\\"]. Mutually exclusive with Field Types"},"fieldTypes":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field categories to return values for: enriched, global, list, or relationship-intelligence. Mutually exclusive with Field IDs"}},"hostedApiKey":"none"},"affinity_get_list_entry_field":{"id":"affinity_get_list_entry_field","name":"Affinity Get List Entry Field","description":"Read one field value on a list row.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"listEntryId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list entry ID"},"fieldId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The field ID to read"}},"hostedApiKey":"none"},"affinity_get_list_field_dropdown_option":{"id":"affinity_get_list_field_dropdown_option","name":"Affinity Get List Field Dropdown Option","description":"Read one dropdown option on a list field.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"fieldId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The dropdown field ID on that list"},"dropdownOptionId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The dropdown option ID"}},"hostedApiKey":"none"},"affinity_get_merge":{"id":"affinity_get_merge","name":"Affinity Get Merge","description":"Read the status of one company or person merge, including why it failed if it did.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Which merge to read: companies or persons"},"mergeId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The merge ID"}},"hostedApiKey":"none"},"affinity_get_merge_task":{"id":"affinity_get_merge_task","name":"Affinity Get Merge Task","description":"Read one merge task and how its merges are progressing. Poll this after starting a merge.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Which merge task to read: companies or persons"},"taskId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The merge task ID"}},"hostedApiKey":"none"},"affinity_get_note":{"id":"affinity_get_note","name":"Affinity Get Note","description":"Read one note with its body, author, mentions, and attached records.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"noteId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The note ID"},"includes":{"type":"json","required":false,"visibility":"user-or-llm","description":"Extra properties to return, e.g. [\\"repliesCount\\",\\"personsPreview\\",\\"companiesPreview\\",\\"opportunitiesPreview\\"]. Those four fields are omitted unless requested here"}},"hostedApiKey":"none"},"affinity_get_opportunity":{"id":"affinity_get_opportunity","name":"Affinity Get Opportunity","description":"Read one opportunity and the list it belongs to. Its field data lives on the list entry.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"opportunityId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The opportunity ID"}},"hostedApiKey":"none"},"affinity_get_person":{"id":"affinity_get_person","name":"Affinity Get Person","description":"Look up one person by ID. Field data is returned only for the Field IDs or Field Types asked for.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"personId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The person ID"},"fieldIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field IDs to return values for, e.g. [\\"affinity-data-location\\"]. Mutually exclusive with Field Types"},"fieldTypes":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field categories to return values for: enriched, global, or relationship-intelligence. Mutually exclusive with Field IDs"}},"hostedApiKey":"none"},"affinity_get_saved_view":{"id":"affinity_get_saved_view","name":"Affinity Get Saved View","description":"Read one saved view — its name, kind, and creation date.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"viewId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The saved view ID"}},"hostedApiKey":"none"},"affinity_get_transcript":{"id":"affinity_get_transcript","name":"Affinity Get Transcript","description":"Read one transcript with its first 100 fragments. Page the fragments endpoint for a longer meeting.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"transcriptId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The transcript ID"}},"hostedApiKey":"none"},"affinity_get_user":{"id":"affinity_get_user","name":"Affinity Get User","description":"Read one internal user. A user and their person record share the same numeric ID, so a person ID works here.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"userId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The user ID, which is also their person ID"}},"hostedApiKey":"none"},"affinity_list_calls":{"id":"affinity_list_calls","name":"Affinity List Calls","description":"Page through logged calls and their participants. Only calls the API key holder can see are returned.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression, e.g. \\"createdAt>=2026-01-01\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_chat_messages":{"id":"affinity_list_chat_messages","name":"Affinity List Chat Messages","description":"Page through logged chat messages and their participants. Only messages the API key holder can see are returned.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression, e.g. \\"createdAt>=2026-01-01\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_companies":{"id":"affinity_list_companies","name":"Affinity List Companies","description":"Page through companies. Companies come back without field data unless Field IDs or Field Types asks for it.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"ids":{"type":"json","required":false,"visibility":"user-or-llm","description":"Restrict the page to these company IDs, e.g. [1, 2, 3]"},"fieldIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field IDs to return values for, e.g. [\\"affinity-data-location\\"]. Mutually exclusive with Field Types"},"fieldTypes":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field categories to return values for: enriched, global, or relationship-intelligence. Mutually exclusive with Field IDs"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_coworker_connections":{"id":"affinity_list_coworker_connections","name":"Affinity List Coworker Connections","description":"Find warm paths into a company through shared work history: who in your Affinity data once worked alongside the people you want to reach. Grouped by target, strongest first.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"filter":{"type":"string","required":true,"visibility":"user-or-llm","description":"Required scope. The only supported filter is target.currentCompany.id, e.g. \\"target.currentCompany.id=123\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of targets to return per page, 1-50. Defaults to 20"},"totalCount":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the total size of the collection. Costs an extra query"}},"hostedApiKey":"none"},"affinity_list_emails":{"id":"affinity_list_emails","name":"Affinity List Emails","description":"Page through email metadata — subject, participants, and timestamps. Affinity never exposes email bodies through the API.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression, e.g. \\"createdAt>=2026-01-01\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_entity_field_values":{"id":"affinity_list_entity_field_values","name":"Affinity List Entity Field Values","description":"Page through a company\'s or person\'s non-list field values. List fields are not returned here — read those through the list entry.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Which entity to read field values from: companies or persons"},"entityId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of that company or person"},"ids":{"type":"json","required":false,"visibility":"user-or-llm","description":"Restrict to these field IDs. Mutually exclusive with Field Types"},"types":{"type":"json","required":false,"visibility":"user-or-llm","description":"Restrict to these field categories: enriched, global, relationship-intelligence. Mutually exclusive with Field IDs"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 20"}},"hostedApiKey":"none"},"affinity_list_entity_list_entries":{"id":"affinity_list_entity_list_entries","name":"Affinity List Entity List Entries","description":"Page through a company\'s or person\'s rows across every list, each carrying that list\'s field values and when the entity was added.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Which entity to look up the rows of: companies or persons"},"entityId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of that company or person"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_entity_lists":{"id":"affinity_list_entity_lists","name":"Affinity List Entity Lists","description":"List every list a company or person appears on that the caller can view.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Which entity to look up the lists of: companies or persons"},"entityId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of that company or person"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_entity_notes":{"id":"affinity_list_entity_notes","name":"Affinity List Entity Notes","description":"List the notes relevant to one company, person, or opportunity — directly attached notes plus notes reaching it through its people and meetings.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Which entity the notes hang off: companies, persons, or opportunities"},"entityId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of that company, person, or opportunity"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression, e.g. \\"createdAt>=2026-01-01\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"},"totalCount":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the total size of the collection. Costs an extra query"}},"hostedApiKey":"none"},"affinity_list_entity_relationships":{"id":"affinity_list_entity_relationships","name":"Affinity List Entity Relationships","description":"List who knows a company or person, scored 0.0 to 1.0 by how much the two actually interact. Strongest first by default.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Which entity to look up relationships for: companies or persons"},"entityId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of that company or person"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression. This endpoint filters on interactionScore only, e.g. \\"interactionScore>=0.5\\""},"orderBy":{"type":"json","required":false,"visibility":"user-or-llm","description":"Sort order: [\\"interactionScore\\"] for weakest first, [\\"-interactionScore\\"] for strongest first (the default)"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"},"totalCount":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the total size of the collection. Costs an extra query"}},"hostedApiKey":"none"},"affinity_list_field_dropdown_options":{"id":"affinity_list_field_dropdown_options","name":"Affinity List Field Dropdown Options","description":"List the selectable options on a dropdown or ranked-dropdown company or person field. Writing such a field needs the option ID, not its text.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Which field family the field belongs to: companies or persons"},"fieldId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The dropdown or ranked-dropdown field ID"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_field_metadata":{"id":"affinity_list_field_metadata","name":"Affinity List Field Metadata","description":"List the non-list company or person fields, with the value type, filter operators, and sort support of each. Start here to find the Field IDs the read and write tools take.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Which fields to describe: companies or persons"},"includes":{"type":"json","required":false,"visibility":"user-or-llm","description":"Extra properties to return: [\\"filterability\\",\\"sortability\\"]. Both are omitted unless requested here"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression. This endpoint filters on name only, e.g. \\"name=~Status\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_field_value_changes":{"id":"affinity_list_field_value_changes","name":"Affinity List Field Value Changes","description":"Page through field value changes across the whole workspace. Built for delta sync: follow nextCursor to the end of a run, then resume from the last cursor next time.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression over field.id, listEntry.id, changer.id, changedAt, or actionType. Resume a sync with e.g. \\"changedAt>2026-06-01T12:00:00Z\\""},"orderBy":{"type":"json","required":false,"visibility":"user-or-llm","description":"Sort order: [\\"changedAt\\"] for oldest first (the default), [\\"-changedAt\\"] for newest first"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_investor_executive_connections":{"id":"affinity_list_investor_executive_connections","name":"Affinity List Investor Executive Connections","description":"Find warm paths into a company through investment history: which investors in your Affinity data backed a company the people you want to reach once led. Grouped by target, strongest first.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"filter":{"type":"string","required":true,"visibility":"user-or-llm","description":"Required scope. The only supported filter is target.currentCompany.id, e.g. \\"target.currentCompany.id=123\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of targets to return per page, 1-50. Defaults to 20"},"totalCount":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the total size of the collection. Costs an extra query"}},"hostedApiKey":"none"},"affinity_list_list_entries":{"id":"affinity_list_list_entries","name":"Affinity List List Entries","description":"Page through the rows of a list. Rows come back without field data unless Field IDs or Field Types asks for it.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"fieldIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field IDs to return values for, e.g. [\\"affinity-data-location\\"]. Mutually exclusive with Field Types"},"fieldTypes":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field categories to return values for: enriched, global, list, or relationship-intelligence. Mutually exclusive with Field IDs"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_list_entry_field_value_changes":{"id":"affinity_list_list_entry_field_value_changes","name":"Affinity List List Entry Field Value Changes","description":"Page through the history of one list row — who changed which field, when, and to what.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"listEntryId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list entry ID"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression over field.id, changer.id, changedAt, or actionType, e.g. \\"field.id=field-1234\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_list_entry_fields":{"id":"affinity_list_list_entry_fields","name":"Affinity List List Entry Fields","description":"Page through every field value on one list row, including the list-specific columns. All fields are returned unless narrowed.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"listEntryId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list entry ID"},"ids":{"type":"json","required":false,"visibility":"user-or-llm","description":"Restrict to these field IDs. Mutually exclusive with Field Types"},"types":{"type":"json","required":false,"visibility":"user-or-llm","description":"Restrict to these field categories: enriched, global, list, relationship-intelligence. Mutually exclusive with Field IDs"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 20"}},"hostedApiKey":"none"},"affinity_list_list_field_dropdown_options":{"id":"affinity_list_list_field_dropdown_options","name":"Affinity List List Field Dropdown Options","description":"List the selectable options on a dropdown, ranked-dropdown, or status-dropdown field of a list.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"fieldId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The dropdown field ID on that list"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_list_fields":{"id":"affinity_list_list_fields","name":"Affinity List List Fields","description":"List the fields available on one list, including its list-specific columns. Use these Field IDs when reading or writing list entries.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"includes":{"type":"json","required":false,"visibility":"user-or-llm","description":"Extra properties to return: [\\"filterability\\",\\"sortability\\"]. Both are omitted unless requested here"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression. This endpoint filters on name only, e.g. \\"name=~Stage\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_lists":{"id":"affinity_list_lists","name":"Affinity List Lists","description":"Page through the lists in the organization that the caller can view.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"term":{"type":"string","required":false,"visibility":"user-or-llm","description":"Case-insensitive substring match on the list name"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_meetings":{"id":"affinity_list_meetings","name":"Affinity List Meetings","description":"Page through past and upcoming meetings with their organizer and attendees.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression, e.g. \\"createdAt>=2026-01-01\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_merge_tasks":{"id":"affinity_list_merge_tasks","name":"Affinity List Merge Tasks","description":"Page through merge tasks, each summarizing how many of its merges are in progress, succeeded, or failed.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Which merge tasks to list: companies or persons"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression. This endpoint filters on status only, e.g. \\"status=in-progress\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_merges":{"id":"affinity_list_merges","name":"Affinity List Merges","description":"Page through the company or person merges the organization has run, with the status and the records involved in each.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Which merges to list: companies or persons"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression over status or taskId, e.g. \\"status=failed\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_note_attached_companies":{"id":"affinity_list_note_attached_companies","name":"Affinity List Note Attached Companies","description":"List the companies directly attached to one note.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"noteId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The note ID"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"},"totalCount":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the total size of the collection. Costs an extra query"}},"hostedApiKey":"none"},"affinity_list_note_attached_opportunities":{"id":"affinity_list_note_attached_opportunities","name":"Affinity List Note Attached Opportunities","description":"List the opportunities directly attached to one note.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"noteId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The note ID"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"},"totalCount":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the total size of the collection. Costs an extra query"}},"hostedApiKey":"none"},"affinity_list_note_attached_persons":{"id":"affinity_list_note_attached_persons","name":"Affinity List Note Attached Persons","description":"List the persons directly attached to one note.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"noteId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The note ID"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"},"totalCount":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the total size of the collection. Costs an extra query"}},"hostedApiKey":"none"},"affinity_list_note_replies":{"id":"affinity_list_note_replies","name":"Affinity List Note Replies","description":"Page through the replies on one note, including AI Notetaker replies.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"noteId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The note ID whose replies to read"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression, e.g. \\"createdAt>=2026-01-01\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"},"totalCount":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the total size of the collection. Costs an extra query"}},"hostedApiKey":"none"},"affinity_list_notes":{"id":"affinity_list_notes","name":"Affinity List Notes","description":"Page through every note the caller can see. Replies are excluded.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"includes":{"type":"json","required":false,"visibility":"user-or-llm","description":"Extra properties to return, e.g. [\\"repliesCount\\",\\"personsPreview\\",\\"companiesPreview\\",\\"opportunitiesPreview\\"]. Those four fields are omitted unless requested here"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression, e.g. \\"createdAt>=2026-01-01\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"},"totalCount":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the total size of the collection. Costs an extra query"}},"hostedApiKey":"none"},"affinity_list_opportunities":{"id":"affinity_list_opportunities","name":"Affinity List Opportunities","description":"Page through opportunities. Field data lives on the list entry, not here — read it through the list or saved view the opportunity belongs to.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"ids":{"type":"json","required":false,"visibility":"user-or-llm","description":"Restrict the page to these opportunity IDs, e.g. [1, 2, 3]"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_persons":{"id":"affinity_list_persons","name":"Affinity List Persons","description":"Page through persons. Persons come back without field data unless Field IDs or Field Types asks for it.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"ids":{"type":"json","required":false,"visibility":"user-or-llm","description":"Restrict the page to these person IDs, e.g. [1, 2, 3]"},"fieldIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field IDs to return values for, e.g. [\\"affinity-data-location\\"]. Mutually exclusive with Field Types"},"fieldTypes":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field categories to return values for: enriched, global, or relationship-intelligence. Mutually exclusive with Field IDs"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_reminders":{"id":"affinity_list_reminders","name":"Affinity List Reminders","description":"Page through the reminders the caller can see. Filter by status to surface what is overdue.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression, e.g. \\"createdAt>=2026-01-01\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"},"totalCount":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the total size of the collection. Costs an extra query"}},"hostedApiKey":"none"},"affinity_list_saved_view_entries":{"id":"affinity_list_saved_view_entries","name":"Affinity List Saved View Entries","description":"Page through the rows of a saved view. The view\'s own filters and columns decide which rows and which field data come back.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"viewId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The saved view ID"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_saved_views":{"id":"affinity_list_saved_views","name":"Affinity List Saved Views","description":"List the saved views on a list that the caller can view.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_transcript_fragments":{"id":"affinity_list_transcript_fragments","name":"Affinity List Transcript Fragments","description":"Page through everything said in a meeting, segment by segment with the speaker.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"transcriptId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The transcript ID"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"},"totalCount":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the total size of the collection. Costs an extra query"}},"hostedApiKey":"none"},"affinity_list_transcripts":{"id":"affinity_list_transcripts","name":"Affinity List Transcripts","description":"Page through meeting transcript metadata. Read one transcript to get what was actually said.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression, e.g. \\"createdAt>=2026-01-01\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"},"totalCount":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the total size of the collection. Costs an extra query"}},"hostedApiKey":"none"},"affinity_list_users":{"id":"affinity_list_users","name":"Affinity List Users","description":"Page through the internal users in the organization. Email addresses and roles are returned only to callers with the \\"Manage Users\\" permission.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"term":{"type":"string","required":false,"visibility":"user-or-llm","description":"Case-insensitive match across first name, last name, and primary email"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression over id or status, e.g. \\"status=active\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_search_companies":{"id":"affinity_search_companies","name":"Affinity Search Companies","description":"Search companies by filters, sorts, and a free-text term. Requires the \\"Export All Organizations directory\\" permission.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"filters":{"type":"json","required":false,"visibility":"user-or-llm","description":"Filter group as {operator: \\"and\\"|\\"or\\", filters: [...]}, at most 50 leaves. Each leaf is {valueType, fieldId, operator, value}, and a leaf may itself be a nested group"},"searchTerm":{"type":"string","required":false,"visibility":"user-or-llm","description":"Free-text term matched against the searchable fields. At least 3 characters"},"searchFieldIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field IDs the search term is matched against. Defaults to the searchable fields"},"sorts":{"type":"json","required":false,"visibility":"user-or-llm","description":"Sort order as [{fieldId, direction: \\"asc\\"|\\"desc\\", attributeId?}], up to 5, applied in order"},"fieldIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field IDs to return values for, e.g. [\\"affinity-data-location\\"]. Mutually exclusive with Field Types"},"fieldTypes":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field categories to return values for: enriched, global, or relationship-intelligence. Mutually exclusive with Field IDs"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"},"totalCount":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the total size of the collection. Costs an extra query"}},"hostedApiKey":"none"},"affinity_search_files":{"id":"affinity_search_files","name":"Affinity Search Files","description":"Search files by keyword, ordered by relevance. Narrow to specific files or to one company, or leave both unset to search the whole account.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"prompt":{"type":"string","required":true,"visibility":"user-or-llm","description":"What to search for. Between 3 and 500 characters"},"ids":{"type":"json","required":false,"visibility":"user-or-llm","description":"Restrict the search to these file IDs. Cannot be combined with Company ID"},"companyId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Restrict the search to one company\'s files. Cannot be combined with file IDs"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of files to return, 1-100. Defaults to 20"}},"hostedApiKey":"none"},"affinity_search_list_entries":{"id":"affinity_search_list_entries","name":"Affinity Search List Entries","description":"Search the rows of one list by filters, sorts, and a free-text term. Requires the \\"Export data from Lists\\" permission.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID to search"},"filters":{"type":"json","required":false,"visibility":"user-or-llm","description":"Filter group as {operator: \\"and\\"|\\"or\\", filters: [...]}, at most 50 leaves. Each leaf is {valueType, fieldId, operator, value}, and a leaf may itself be a nested group"},"searchTerm":{"type":"string","required":false,"visibility":"user-or-llm","description":"Free-text term matched against the searchable fields. At least 3 characters"},"searchFieldIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field IDs the search term is matched against. Defaults to the searchable fields"},"sorts":{"type":"json","required":false,"visibility":"user-or-llm","description":"Sort order as [{fieldId, direction: \\"asc\\"|\\"desc\\", attributeId?}], up to 5, applied in order"},"fieldIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field IDs to return values for, e.g. [\\"affinity-data-location\\"]. Mutually exclusive with Field Types"},"fieldTypes":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field categories to return values for: enriched, global, list, or relationship-intelligence. Mutually exclusive with Field IDs"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"},"totalCount":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the total size of the collection. Costs an extra query"}},"hostedApiKey":"none"},"affinity_search_notes":{"id":"affinity_search_notes","name":"Affinity Search Notes","description":"Search notes by keyword, ordered by relevance. Narrow to specific notes or to one company, or leave both unset to search the whole account.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"prompt":{"type":"string","required":true,"visibility":"user-or-llm","description":"What to search for. Between 3 and 500 characters"},"ids":{"type":"json","required":false,"visibility":"user-or-llm","description":"Restrict the search to these note IDs. Cannot be combined with Company ID"},"companyId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Restrict the search to one company\'s notes. Cannot be combined with note IDs"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of notes to return, 1-100. Defaults to 20"}},"hostedApiKey":"none"},"affinity_search_persons":{"id":"affinity_search_persons","name":"Affinity Search Persons","description":"Search persons by filters, sorts, and a free-text term. Requires the \\"Export All People directory\\" permission.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"filters":{"type":"json","required":false,"visibility":"user-or-llm","description":"Filter group as {operator: \\"and\\"|\\"or\\", filters: [...]}, at most 50 leaves. Each leaf is {valueType, fieldId, operator, value}, and a leaf may itself be a nested group"},"searchTerm":{"type":"string","required":false,"visibility":"user-or-llm","description":"Free-text term matched against the searchable fields. At least 3 characters"},"searchFieldIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field IDs the search term is matched against. Defaults to the searchable fields"},"sorts":{"type":"json","required":false,"visibility":"user-or-llm","description":"Sort order as [{fieldId, direction: \\"asc\\"|\\"desc\\", attributeId?}], up to 5, applied in order"},"fieldIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field IDs to return values for, e.g. [\\"affinity-data-location\\"]. Mutually exclusive with Field Types"},"fieldTypes":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field categories to return values for: enriched, global, or relationship-intelligence. Mutually exclusive with Field IDs"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"},"totalCount":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the total size of the collection. Costs an extra query"}},"hostedApiKey":"none"},"affinity_semantic_search":{"id":"affinity_semantic_search","name":"Affinity Semantic Search","description":"Find companies from a description in plain language — industry, technology, stage, or business model. Currently searches companies only.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"prompt":{"type":"string","required":true,"visibility":"user-or-llm","description":"What to look for, in plain language, e.g. \\"climate tech companies in our pipeline\\". Up to 500 characters"},"listIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Restrict the search to companies on these lists, e.g. [1, 2]"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of companies to return, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_update_entity_field_value":{"id":"affinity_update_entity_field_value","name":"Affinity Update Entity Field Value","description":"Write one non-list field value on a company or person. The value type must match how the field is defined.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Which entity to write the field on: companies or persons"},"entityId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of that company or person"},"fieldId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The field ID to write"},"value":{"type":"json","required":true,"visibility":"user-or-llm","description":"The new value as {type, data}, where type matches the field\'s value type. Examples: {\\"type\\":\\"text\\",\\"data\\":\\"Series B\\"}, {\\"type\\":\\"number\\",\\"data\\":42}, {\\"type\\":\\"dropdown\\",\\"data\\":{\\"dropdownOptionId\\":7}}, {\\"type\\":\\"person\\",\\"data\\":{\\"id\\":123}}, {\\"type\\":\\"person-multi\\",\\"data\\":[{\\"id\\":123}]}. Pass data as null to clear the field"}},"hostedApiKey":"none"},"affinity_update_list_entry_field":{"id":"affinity_update_list_entry_field","name":"Affinity Update List Entry Field","description":"Write one field value on a list row. Requires the \\"Export data from Lists\\" permission.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"listEntryId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list entry ID"},"fieldId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The field ID to write"},"value":{"type":"json","required":true,"visibility":"user-or-llm","description":"The new value as {type, data}, where type matches the field\'s value type. Examples: {\\"type\\":\\"text\\",\\"data\\":\\"Series B\\"}, {\\"type\\":\\"number\\",\\"data\\":42}, {\\"type\\":\\"dropdown\\",\\"data\\":{\\"dropdownOptionId\\":7}}, {\\"type\\":\\"person\\",\\"data\\":{\\"id\\":123}}, {\\"type\\":\\"person-multi\\",\\"data\\":[{\\"id\\":123}]}. Pass data as null to clear the field"}},"hostedApiKey":"none"},"affinity_update_list_field_dropdown_option":{"id":"affinity_update_list_field_dropdown_option","name":"Affinity Update List Field Dropdown Option","description":"Change a dropdown option on a list field. Every field is optional — supply only what should change, and only fields the option\'s kind actually has.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"fieldId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The dropdown field ID on that list"},"dropdownOptionId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The dropdown option ID to update"},"text":{"type":"string","required":false,"visibility":"user-or-llm","description":"Replacement option label. Supply at least one field to change"},"rank":{"type":"number","required":false,"visibility":"user-or-llm","description":"Sort order. Required on a ranked-dropdown or status-dropdown option"},"color":{"type":"string","required":false,"visibility":"user-or-llm","description":"Option color: white, gray, blue, green, purple, orange, or red. Required on a ranked-dropdown or status-dropdown option"},"statusCategory":{"type":"string","required":false,"visibility":"user-or-llm","description":"Pipeline meaning of the option: open, won, lost, or on-hold. Status-dropdown options only"},"winRate":{"type":"number","required":false,"visibility":"user-or-llm","description":"Expected win rate of the status. Status-dropdown options only"}},"hostedApiKey":"none"},"affinity_update_note":{"id":"affinity_update_note","name":"Affinity Update Note","description":"Rewrite a note\'s body or replace which records it is attached to. Each list of IDs replaces that association wholesale, an empty list clears it, and omitting one leaves it untouched. A note\'s type cannot be changed.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"noteId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The note ID to update"},"html":{"type":"string","required":false,"visibility":"user-or-llm","description":"Replacement note body as HTML"},"companyIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Replacement set of attached companies, e.g. [1, 2]. Send [] to detach every company; omit to leave them unchanged"},"personIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Replacement set of attached persons, e.g. [1, 2]. Send [] to detach every person; omit to leave them unchanged"},"opportunityIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Replacement set of attached opportunities, e.g. [1, 2]. Send [] to detach every opportunity; omit to leave them unchanged"}},"hostedApiKey":"none"},"agentmail_create_draft":{"id":"agentmail_create_draft","name":"Create Draft","description":"Create a new email draft in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox to create the draft in"},"to":{"type":"string","required":false,"visibility":"user-or-llm","description":"Recipient email addresses (comma-separated)"},"subject":{"type":"string","required":false,"visibility":"user-or-llm","description":"Draft subject line"},"text":{"type":"string","required":false,"visibility":"user-or-llm","description":"Plain text draft body"},"html":{"type":"string","required":false,"visibility":"user-or-llm","description":"HTML draft body"},"cc":{"type":"string","required":false,"visibility":"user-or-llm","description":"CC recipient email addresses (comma-separated)"},"bcc":{"type":"string","required":false,"visibility":"user-or-llm","description":"BCC recipient email addresses (comma-separated)"},"inReplyTo":{"type":"string","required":false,"visibility":"user-or-llm","description":"ID of message being replied to"},"sendAt":{"type":"string","required":false,"visibility":"user-or-llm","description":"ISO 8601 timestamp to schedule sending"}},"hostedApiKey":"none"},"agentmail_create_inbox":{"id":"agentmail_create_inbox","name":"Create Inbox","description":"Create a new email inbox with AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"username":{"type":"string","required":false,"visibility":"user-or-llm","description":"Username for the inbox email address"},"domain":{"type":"string","required":false,"visibility":"user-or-llm","description":"Domain for the inbox email address"},"displayName":{"type":"string","required":false,"visibility":"user-or-llm","description":"Display name for the inbox"}},"hostedApiKey":"none"},"agentmail_delete_draft":{"id":"agentmail_delete_draft","name":"Delete Draft","description":"Delete an email draft in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox containing the draft"},"draftId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the draft to delete"}},"hostedApiKey":"none"},"agentmail_delete_inbox":{"id":"agentmail_delete_inbox","name":"Delete Inbox","description":"Delete an email inbox in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox to delete"}},"hostedApiKey":"none"},"agentmail_delete_thread":{"id":"agentmail_delete_thread","name":"Delete Thread","description":"Delete an email thread in AgentMail (moves to trash, or permanently deletes if already in trash)","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox containing the thread"},"threadId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the thread to delete"},"permanent":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Force permanent deletion instead of moving to trash"}},"hostedApiKey":"none"},"agentmail_forward_message":{"id":"agentmail_forward_message","name":"Forward Message","description":"Forward an email message to new recipients in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox containing the message"},"messageId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the message to forward"},"to":{"type":"string","required":true,"visibility":"user-or-llm","description":"Recipient email addresses (comma-separated)"},"subject":{"type":"string","required":false,"visibility":"user-or-llm","description":"Override subject line"},"text":{"type":"string","required":false,"visibility":"user-or-llm","description":"Additional plain text to prepend"},"html":{"type":"string","required":false,"visibility":"user-or-llm","description":"Additional HTML to prepend"},"cc":{"type":"string","required":false,"visibility":"user-or-llm","description":"CC recipient email addresses (comma-separated)"},"bcc":{"type":"string","required":false,"visibility":"user-or-llm","description":"BCC recipient email addresses (comma-separated)"}},"hostedApiKey":"none"},"agentmail_get_draft":{"id":"agentmail_get_draft","name":"Get Draft","description":"Get details of a specific email draft in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox the draft belongs to"},"draftId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the draft to retrieve"}},"hostedApiKey":"none"},"agentmail_get_inbox":{"id":"agentmail_get_inbox","name":"Get Inbox","description":"Get details of a specific email inbox in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox to retrieve"}},"hostedApiKey":"none"},"agentmail_get_message":{"id":"agentmail_get_message","name":"Get Message","description":"Get details of a specific email message in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox containing the message"},"messageId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the message to retrieve"}},"hostedApiKey":"none"},"agentmail_get_thread":{"id":"agentmail_get_thread","name":"Get Thread","description":"Get details of a specific email thread including messages in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox containing the thread"},"threadId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the thread to retrieve"}},"hostedApiKey":"none"},"agentmail_list_drafts":{"id":"agentmail_list_drafts","name":"List Drafts","description":"List email drafts in an inbox in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox to list drafts from"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of drafts to return"},"pageToken":{"type":"string","required":false,"visibility":"user-or-llm","description":"Pagination token for next page of results"}},"hostedApiKey":"none"},"agentmail_list_inboxes":{"id":"agentmail_list_inboxes","name":"List Inboxes","description":"List all email inboxes in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of inboxes to return"},"pageToken":{"type":"string","required":false,"visibility":"user-or-llm","description":"Pagination token for next page of results"}},"hostedApiKey":"none"},"agentmail_list_messages":{"id":"agentmail_list_messages","name":"List Messages","description":"List messages in an inbox in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox to list messages from"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of messages to return"},"pageToken":{"type":"string","required":false,"visibility":"user-or-llm","description":"Pagination token for next page of results"}},"hostedApiKey":"none"},"agentmail_list_threads":{"id":"agentmail_list_threads","name":"List Threads","description":"List email threads in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox to list threads from"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of threads to return"},"pageToken":{"type":"string","required":false,"visibility":"user-or-llm","description":"Pagination token for next page of results"},"labels":{"type":"string","required":false,"visibility":"user-or-llm","description":"Comma-separated labels to filter threads by"},"before":{"type":"string","required":false,"visibility":"user-or-llm","description":"Filter threads before this ISO 8601 timestamp"},"after":{"type":"string","required":false,"visibility":"user-or-llm","description":"Filter threads after this ISO 8601 timestamp"}},"hostedApiKey":"none"},"agentmail_reply_message":{"id":"agentmail_reply_message","name":"Reply to Message","description":"Reply to an existing email message in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox to reply from"},"messageId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the message to reply to"},"text":{"type":"string","required":false,"visibility":"user-or-llm","description":"Plain text reply body"},"html":{"type":"string","required":false,"visibility":"user-or-llm","description":"HTML reply body"},"to":{"type":"string","required":false,"visibility":"user-or-llm","description":"Override recipient email addresses (comma-separated)"},"cc":{"type":"string","required":false,"visibility":"user-or-llm","description":"CC email addresses (comma-separated)"},"bcc":{"type":"string","required":false,"visibility":"user-or-llm","description":"BCC email addresses (comma-separated)"},"replyAll":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Reply to all recipients of the original message"}},"hostedApiKey":"none"},"agentmail_send_draft":{"id":"agentmail_send_draft","name":"Send Draft","description":"Send an existing email draft in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox containing the draft"},"draftId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the draft to send"}},"hostedApiKey":"none"},"agentmail_send_message":{"id":"agentmail_send_message","name":"Send Message","description":"Send an email message from an AgentMail inbox","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox to send from"},"to":{"type":"string","required":true,"visibility":"user-or-llm","description":"Recipient email address (comma-separated for multiple)"},"subject":{"type":"string","required":true,"visibility":"user-or-llm","description":"Email subject line"},"text":{"type":"string","required":false,"visibility":"user-or-llm","description":"Plain text email body"},"html":{"type":"string","required":false,"visibility":"user-or-llm","description":"HTML email body"},"cc":{"type":"string","required":false,"visibility":"user-or-llm","description":"CC recipient email addresses (comma-separated)"},"bcc":{"type":"string","required":false,"visibility":"user-or-llm","description":"BCC recipient email addresses (comma-separated)"}},"hostedApiKey":"none"},"agentmail_update_draft":{"id":"agentmail_update_draft","name":"Update Draft","description":"Update an existing email draft in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox containing the draft"},"draftId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the draft to update"},"to":{"type":"string","required":false,"visibility":"user-or-llm","description":"Recipient email addresses (comma-separated)"},"subject":{"type":"string","required":false,"visibility":"user-or-llm","description":"Draft subject line"},"text":{"type":"string","required":false,"visibility":"user-or-llm","description":"Plain text draft body"},"html":{"type":"string","required":false,"visibility":"user-or-llm","description":"HTML draft body"},"cc":{"type":"string","required":false,"visibility":"user-or-llm","description":"CC recipient email addresses (comma-separated)"},"bcc":{"type":"string","required":false,"visibility":"user-or-llm","description":"BCC recipient email addresses (comma-separated)"},"sendAt":{"type":"string","required":false,"visibility":"user-or-llm","description":"ISO 8601 timestamp to schedule sending"}},"hostedApiKey":"none"},"agentmail_update_inbox":{"id":"agentmail_update_inbox","name":"Update Inbox","description":"Update the display name of an email inbox in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox to update"},"displayName":{"type":"string","required":true,"visibility":"user-or-llm","description":"New display name for the inbox"}},"hostedApiKey":"none"},"agentmail_update_message":{"id":"agentmail_update_message","name":"Update Message","description":"Add or remove labels on an email message in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox containing the message"},"messageId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the message to update"},"addLabels":{"type":"string","required":false,"visibility":"user-or-llm","description":"Comma-separated labels to add to the message"},"removeLabels":{"type":"string","required":false,"visibility":"user-or-llm","description":"Comma-separated labels to remove from the message"}},"hostedApiKey":"none"},"agentmail_update_thread":{"id":"agentmail_update_thread","name":"Update Thread Labels","description":"Add or remove labels on an email thread in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox containing the thread"},"threadId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the thread to update"},"addLabels":{"type":"string","required":false,"visibility":"user-or-llm","description":"Comma-separated labels to add to the thread"},"removeLabels":{"type":"string","required":false,"visibility":"user-or-llm","description":"Comma-separated labels to remove from the thread"}},"hostedApiKey":"none"},"agentphone_create_call":{"id":"agentphone_create_call","name":"Create Outbound Call","description":"Initiate an outbound voice call from an AgentPhone agent","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"agentId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Agent that will handle the call"},"toNumber":{"type":"string","required":true,"visibility":"user-or-llm","description":"Phone number to call in E.164 format (e.g. +14155551234)"},"fromNumberId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Phone number ID to use as caller ID. Must belong to the agent. If omitted, the agent\'s first assigned number is used."},"initialGreeting":{"type":"string","required":false,"visibility":"user-or-llm","description":"Optional greeting spoken when the recipient answers"},"voice":{"type":"string","required":false,"visibility":"user-or-llm","description":"Voice ID override for this call (defaults to the agent\'s configured voice)"},"systemPrompt":{"type":"string","required":false,"visibility":"user-or-llm","description":"When provided, uses a built-in LLM for the conversation instead of forwarding to your webhook"}},"hostedApiKey":"none"},"agentphone_create_contact":{"id":"agentphone_create_contact","name":"Create Contact","description":"Create a new contact in AgentPhone","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"phoneNumber":{"type":"string","required":true,"visibility":"user-or-llm","description":"Phone number in E.164 format (e.g. +14155551234)"},"name":{"type":"string","required":true,"visibility":"user-or-llm","description":"Contact\'s full name"},"email":{"type":"string","required":false,"visibility":"user-or-llm","description":"Contact\'s email address"},"notes":{"type":"string","required":false,"visibility":"user-or-llm","description":"Freeform notes stored on the contact"}},"hostedApiKey":"none"},"agentphone_create_number":{"id":"agentphone_create_number","name":"Create Phone Number","description":"Provision a new SMS- and voice-enabled phone number","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"country":{"type":"string","required":false,"visibility":"user-or-llm","description":"Two-letter country code (e.g. US, CA). Defaults to US."},"areaCode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Preferred area code (US/CA only, e.g. \\"415\\"). Best-effort — may be ignored if unavailable."},"agentId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Optionally attach the number to an agent immediately"}},"hostedApiKey":"none"},"agentphone_delete_contact":{"id":"agentphone_delete_contact","name":"Delete Contact","description":"Delete a contact by ID","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"contactId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Contact ID"}},"hostedApiKey":"none"},"agentphone_get_call":{"id":"agentphone_get_call","name":"Get Call","description":"Fetch a call and its full transcript","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"callId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the call to retrieve"}},"hostedApiKey":"none"},"agentphone_get_call_transcript":{"id":"agentphone_get_call_transcript","name":"Get Call Transcript","description":"Get the full ordered transcript for a call","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"callId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the call to retrieve the transcript for"}},"hostedApiKey":"none"},"agentphone_get_contact":{"id":"agentphone_get_contact","name":"Get Contact","description":"Fetch a single contact by ID","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"contactId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Contact ID"}},"hostedApiKey":"none"},"agentphone_get_conversation":{"id":"agentphone_get_conversation","name":"Get Conversation","description":"Get a conversation along with its recent messages","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"conversationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Conversation ID"},"messageLimit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of recent messages to include (default 50, max 100)"}},"hostedApiKey":"none"},"agentphone_get_conversation_messages":{"id":"agentphone_get_conversation_messages","name":"Get Conversation Messages","description":"Get paginated messages for a conversation","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"conversationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Conversation ID"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of messages to return (default 50, max 200)"},"before":{"type":"string","required":false,"visibility":"user-or-llm","description":"Return messages received before this ISO 8601 timestamp"},"after":{"type":"string","required":false,"visibility":"user-or-llm","description":"Return messages received after this ISO 8601 timestamp"}},"hostedApiKey":"none"},"agentphone_get_number_messages":{"id":"agentphone_get_number_messages","name":"Get Phone Number Messages","description":"Fetch messages received on a specific phone number","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"numberId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the phone number"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of messages to return (default 50, max 200)"},"before":{"type":"string","required":false,"visibility":"user-or-llm","description":"Return messages received before this ISO 8601 timestamp"},"after":{"type":"string","required":false,"visibility":"user-or-llm","description":"Return messages received after this ISO 8601 timestamp"}},"hostedApiKey":"none"},"agentphone_get_usage":{"id":"agentphone_get_usage","name":"Get Usage","description":"Retrieve current usage statistics for the AgentPhone account","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"}},"hostedApiKey":"none"},"agentphone_get_usage_daily":{"id":"agentphone_get_usage_daily","name":"Get Daily Usage","description":"Get a daily breakdown of usage (messages, calls, webhooks) for the last N days","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"days":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of days to return (1-365, default 30)"}},"hostedApiKey":"none"},"agentphone_get_usage_monthly":{"id":"agentphone_get_usage_monthly","name":"Get Monthly Usage","description":"Get monthly usage aggregation (messages, calls, webhooks) for the last N months","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"months":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of months to return (1-24, default 6)"}},"hostedApiKey":"none"},"agentphone_list_calls":{"id":"agentphone_list_calls","name":"List Calls","description":"List voice calls for this AgentPhone account","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of results to return (default 20, max 100)"},"offset":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of results to skip (min 0)"},"status":{"type":"string","required":false,"visibility":"user-or-llm","description":"Filter by status (completed, in-progress, failed)"},"direction":{"type":"string","required":false,"visibility":"user-or-llm","description":"Filter by direction (inbound, outbound)"},"type":{"type":"string","required":false,"visibility":"user-or-llm","description":"Filter by call type (pstn, web)"},"search":{"type":"string","required":false,"visibility":"user-or-llm","description":"Search by phone number (matches fromNumber or toNumber)"}},"hostedApiKey":"none"},"agentphone_list_contacts":{"id":"agentphone_list_contacts","name":"List Contacts","description":"List contacts for this AgentPhone account","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"search":{"type":"string","required":false,"visibility":"user-or-llm","description":"Filter by name or phone number (case-insensitive contains)"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of results to return (default 50, max 200)"},"offset":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of results to skip (min 0)"}},"hostedApiKey":"none"},"agentphone_list_conversations":{"id":"agentphone_list_conversations","name":"List Conversations","description":"List conversations (message threads) for this AgentPhone account","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of results to return (default 20, max 100)"},"offset":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of results to skip (min 0)"}},"hostedApiKey":"none"},"agentphone_list_numbers":{"id":"agentphone_list_numbers","name":"List Phone Numbers","description":"List all phone numbers provisioned for this AgentPhone account","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of results to return (default 20, max 100)"},"offset":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of results to skip (min 0)"}},"hostedApiKey":"none"},"agentphone_react_to_message":{"id":"agentphone_react_to_message","name":"React to Message","description":"Send an iMessage tapback reaction to a message (iMessage only)","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"messageId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the message to react to"},"reaction":{"type":"string","required":true,"visibility":"user-or-llm","description":"Reaction type: love, like, dislike, laugh, emphasize, or question"}},"hostedApiKey":"none"},"agentphone_release_number":{"id":"agentphone_release_number","name":"Release Phone Number","description":"Release (delete) a phone number. This action is irreversible.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"numberId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the phone number to release"}},"hostedApiKey":"none"},"agentphone_send_message":{"id":"agentphone_send_message","name":"Send Message","description":"Send an outbound SMS or iMessage from an AgentPhone agent","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"agentId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Agent sending the message"},"toNumber":{"type":"string","required":true,"visibility":"user-or-llm","description":"Recipient phone number in E.164 format (e.g. +14155551234)"},"body":{"type":"string","required":true,"visibility":"user-or-llm","description":"Message text to send"},"mediaUrl":{"type":"string","required":false,"visibility":"user-or-llm","description":"Optional URL of an image, video, or file to attach"},"numberId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Phone number ID to send from. If omitted, the agent\'s first assigned number is used."}},"hostedApiKey":"none"},"agentphone_update_contact":{"id":"agentphone_update_contact","name":"Update Contact","description":"Update a contact\'s fields","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"contactId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Contact ID"},"phoneNumber":{"type":"string","required":false,"visibility":"user-or-llm","description":"New phone number in E.164 format"},"name":{"type":"string","required":false,"visibility":"user-or-llm","description":"New contact name"},"email":{"type":"string","required":false,"visibility":"user-or-llm","description":"New email address"},"notes":{"type":"string","required":false,"visibility":"user-or-llm","description":"New freeform notes"}},"hostedApiKey":"none"},"agentphone_update_conversation":{"id":"agentphone_update_conversation","name":"Update Conversation","description":"Update conversation metadata (stored state). Pass null to clear existing metadata.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"conversationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Conversation ID"},"metadata":{"type":"json","required":false,"visibility":"user-or-llm","description":"Custom key-value metadata to store on the conversation. Pass null to clear existing metadata."}},"hostedApiKey":"none"},"agiloft_async_status":{"id":"agiloft_async_status","name":"Agiloft Async Status","description":"Check whether an asynchronous Agiloft call, such as a run action button, has completed.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table the asynchronous call was made against"},"callbackId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Callback ID returned by the asynchronous call, e.g. from Run Action Button"}},"hostedApiKey":"none"},"agiloft_attach_file":{"id":"agiloft_attach_file","name":"Agiloft Attach File","description":"Attach a file to a field in an Agiloft record.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table name (e.g., \\"contracts\\")"},"recordId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the record to attach the file to"},"fieldName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the attachment field"},"file":{"type":"file","required":true,"visibility":"user-or-llm","description":"File to attach"},"fileName":{"type":"string","required":false,"visibility":"user-or-llm","description":"Name to assign to the file (defaults to original file name)"},"overwrite":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Replace the contents of the field instead of adding another file to it"}},"hostedApiKey":"none"},"agiloft_attachment_info":{"id":"agiloft_attachment_info","name":"Agiloft Attachment Info","description":"Get information about file attachments on a record field.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table name (e.g., \\"contracts\\")"},"recordId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the record to check attachments on"},"fieldName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the attachment field to inspect"}},"hostedApiKey":"none"},"agiloft_create_record":{"id":"agiloft_create_record","name":"Agiloft Create Record","description":"Create a new record in an Agiloft table.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table name (e.g., \\"contracts\\", \\"contacts.employees\\")"},"data":{"type":"string","required":true,"visibility":"user-or-llm","description":"Record field values as a JSON object (e.g., {\\"first_name\\": \\"John\\", \\"status\\": \\"Active\\"})"}},"hostedApiKey":"none"},"agiloft_delete_record":{"id":"agiloft_delete_record","name":"Agiloft Delete Record","description":"Delete a record from an Agiloft table.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table name (e.g., \\"contracts\\", \\"contacts.employees\\")"},"recordId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the record to delete"},"substituteIds":{"type":"string","required":false,"visibility":"user-or-llm","description":"Comma-separated IDs of records that adopt the dependants of the deleted record. Read only when the delete rule is REPLACE_WITH_ANOTHER."},"deleteRule":{"type":"string","required":false,"visibility":"user-or-llm","description":"How to treat records that depend on this one: ERROR_IF_DEPENDANTS (default — fails rather than cascading), APPLY_DELETE_WHERE_POSSIBLE, DELETE_WHERE_POSSIBLE_OTHERWISE_UNLINK, APPLY_UNLINK, UNLINK_WHERE_POSSIBLE_OTHERWISE_DELETE, or REPLACE_WITH_ANOTHER"}},"hostedApiKey":"none"},"agiloft_get_choice_line_id":{"id":"agiloft_get_choice_line_id","name":"Agiloft Get Choice Line ID","description":"Resolve the internal numeric ID of a choice-list value, for use in EWSelect WHERE clauses against choice fields.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table name (e.g., \\"case\\", \\"contracts\\")"},"fieldName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Choice field name (e.g., \\"priority\\", \\"status\\")"},"value":{"type":"string","required":true,"visibility":"user-or-llm","description":"Choice display value to resolve (e.g., \\"High\\", \\"Active\\")"}},"hostedApiKey":"none"},"agiloft_list_tables":{"id":"agiloft_list_tables","name":"Agiloft List Tables","description":"List the tables and fields in an Agiloft knowledge base, to discover the logical names other operations need.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":false,"visibility":"user-or-llm","description":"Logical name of a single table to describe (e.g., \\"contacts\\"). Leave empty to list every table in the knowledge base."},"includeLinkedInfo":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the source table and column behind each linked field"},"skipColumnsInfo":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Return table names only, omitting field details, for a much smaller response"}},"hostedApiKey":"none"},"agiloft_lock_record":{"id":"agiloft_lock_record","name":"Agiloft Lock Record","description":"Lock, unlock, or check the lock status of an Agiloft record.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table name (e.g., \\"contracts\\")"},"recordId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the record to lock, unlock, or check"},"lockAction":{"type":"string","required":true,"visibility":"user-or-llm","description":"Action to perform: \\"lock\\", \\"unlock\\", or \\"check\\""},"force":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Unlock only: release a lock held by another user."}},"hostedApiKey":"none"},"agiloft_nlp_search":{"id":"agiloft_nlp_search","name":"Agiloft Natural Language Search","description":"Search Agiloft records by describing what you want in plain language, such as \\"active NDAs submitted last month\\".","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"nlpQuery":{"type":"string","required":true,"visibility":"user-or-llm","description":"The request in plain language, e.g. \\"Show me open, high-priority contracts\\". Structured field filters are not accepted — use Search Records for those."},"fields":{"type":"string","required":true,"visibility":"user-or-llm","description":"Comma-separated field names to return, e.g. \\"id, contract_title1, company_name\\""},"page":{"type":"string","required":false,"visibility":"user-or-llm","description":"Page number, starting from 0"},"limit":{"type":"string","required":false,"visibility":"user-or-llm","description":"Records per page"}},"hostedApiKey":"none"},"agiloft_read_record":{"id":"agiloft_read_record","name":"Agiloft Read Record","description":"Read a record by ID from an Agiloft table.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table name (e.g., \\"contracts\\", \\"contacts.employees\\")"},"recordId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the record to read"},"fields":{"type":"string","required":false,"visibility":"user-or-llm","description":"Comma-separated list of field names to include in the response"}},"hostedApiKey":"none"},"agiloft_remove_attachment":{"id":"agiloft_remove_attachment","name":"Agiloft Remove Attachment","description":"Remove an attached file from a field in an Agiloft record.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table name (e.g., \\"contracts\\")"},"recordId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the record containing the attachment"},"fieldName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the attachment field"},"position":{"type":"string","required":true,"visibility":"user-or-llm","description":"Position index of the file to remove (starting from 0)"}},"hostedApiKey":"none"},"agiloft_retrieve_attachment":{"id":"agiloft_retrieve_attachment","name":"Agiloft Retrieve Attachment","description":"Download an attached file from an Agiloft record field.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table name (e.g., \\"contracts\\")"},"recordId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the record containing the attachment"},"fieldName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the attachment field"},"position":{"type":"string","required":true,"visibility":"user-or-llm","description":"Position index of the file in the field (starting from 0)"}},"hostedApiKey":"none"},"agiloft_run_action_button":{"id":"agiloft_run_action_button","name":"Agiloft Run Action Button","description":"Run an action button on an Agiloft record, such as an approval or send-for-signature step.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table name (e.g., \\"contracts\\", \\"case\\")"},"recordId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the record to run the action button on"},"actionButtonField":{"type":"string","required":true,"visibility":"user-or-llm","description":"Logical name of the field holding the action button (e.g., \\"ab_field\\")"}},"hostedApiKey":"none"},"agiloft_saved_search":{"id":"agiloft_saved_search","name":"Agiloft Saved Search","description":"List the saved searches defined for an Agiloft table.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Logical table name to list saved searches for (e.g., \\"contract\\")"}},"hostedApiKey":"none"},"agiloft_search_records":{"id":"agiloft_search_records","name":"Agiloft Search Records","description":"Search for records in an Agiloft table using a query.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table name to search in (e.g., \\"contracts\\", \\"contacts.employees\\")"},"query":{"type":"string","required":false,"visibility":"user-or-llm","description":"Ad hoc EWSearch query. Combine conditions with && (and) or || (or) and quote every value — e.g. \\"summary~=\'test\'&&priority=\'High\'\\". Required unless a saved search is given."},"search":{"type":"string","required":false,"visibility":"user-or-llm","description":"Label of a saved search defined on the table (e.g., \\"C: Status is Closed\\"). Can be combined with a query to narrow it further."},"fields":{"type":"string","required":false,"visibility":"user-or-llm","description":"Comma-separated list of field names to include in the results"},"page":{"type":"string","required":false,"visibility":"user-or-llm","description":"Page number for paginated results (starting from 0)"},"limit":{"type":"string","required":false,"visibility":"user-or-llm","description":"Maximum number of records to return per page. Agiloft treats 0 as \\"all records\\", so leave it unset or use a positive value to keep result sizes bounded."}},"hostedApiKey":"none"},"agiloft_select_records":{"id":"agiloft_select_records","name":"Agiloft Select Records","description":"Select record IDs matching a SQL WHERE clause from an Agiloft table.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table name (e.g., \\"contracts\\", \\"contacts.employees\\")"},"where":{"type":"string","required":true,"visibility":"user-or-llm","description":"SQL WHERE clause using database column names (e.g., \\"summary like \'%new%\'\\" or \\"assigned_person=\'John Doe\'\\"). EWSelect has no page size and returns every matching ID, so append a database limit such as \\"limit 0,200\\" to bound the result."}},"hostedApiKey":"none"},"agiloft_update_record":{"id":"agiloft_update_record","name":"Agiloft Update Record","description":"Update an existing record in an Agiloft table.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table name (e.g., \\"contracts\\", \\"contacts.employees\\")"},"recordId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the record to update"},"data":{"type":"string","required":true,"visibility":"user-or-llm","description":"Updated field values as a JSON object (e.g., {\\"status\\": \\"Active\\", \\"priority\\": \\"High\\"})"}},"hostedApiKey":"none"},"agiloft_upsert_record":{"id":"agiloft_upsert_record","name":"Agiloft Upsert Record","description":"Create an Agiloft record, or update it when a record already matches the given fields.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table name (e.g., \\"contracts\\", \\"contacts.employees\\")"},"match":{"type":"string","required":true,"visibility":"user-or-llm","description":"Field used to find an existing record (e.g., \\"ext_id\\"). Pick something that identifies a record uniquely — if more than one record matches, Agiloft writes nothing and returns a conflict."},"async":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Queue the write instead of waiting for it. Returns a callback ID instead of a record ID; pass that to Async Status to poll the result."},"data":{"type":"string","required":true,"visibility":"user-or-llm","description":"Field values as a JSON object. On create these populate the new record; on update only the supplied fields change."}},"hostedApiKey":"none"},"ahrefs_anchors":{"id":"ahrefs_anchors","name":"Ahrefs Anchors","description":"Get the anchor text distribution for a target domain or URL\'s backlinks, showing how many links and referring domains use each anchor text.","version":"1.0.0","params":{"target":{"type":"string","required":true,"visibility":"user-or-llm","description":"The target domain or URL to analyze. Example: \\"example.com\\" or \\"https://example.com/page\\""},"mode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Analysis mode: domain (entire domain), prefix (URL prefix), subdomains (include all subdomains, default), exact (exact URL match)"},"history":{"type":"string","required":false,"visibility":"user-or-llm","description":"Historical scope: \\"live\\" (currently live), \\"all_time\\" (default, includes lost backlinks), or \\"since:YYYY-MM-DD\\" (backlinks found since a date)"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of results to return. Example: 50 (default: 1000)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_backlinks":{"id":"ahrefs_backlinks","name":"Ahrefs Backlinks","description":"Get a list of backlinks pointing to a target domain or URL. Returns details about each backlink including source URL, anchor text, and domain rating.","version":"1.0.0","params":{"target":{"type":"string","required":true,"visibility":"user-or-llm","description":"The target domain or URL to analyze. Example: \\"example.com\\" or \\"https://example.com/page\\""},"mode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Analysis mode: domain (entire domain), prefix (URL prefix), subdomains (include all subdomains, default), exact (exact URL match). Example: \\"domain\\""},"history":{"type":"string","required":false,"visibility":"user-or-llm","description":"Historical scope: \\"live\\" (currently live backlinks), \\"all_time\\" (default, includes lost backlinks), or \\"since:YYYY-MM-DD\\" (backlinks found since a date)."},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of results to return. Example: 50 (default: 1000)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_backlinks_stats":{"id":"ahrefs_backlinks_stats","name":"Ahrefs Backlinks Stats","description":"Get backlink and referring domain totals for a target domain or URL, both currently live and across all time.","version":"1.0.0","params":{"target":{"type":"string","required":true,"visibility":"user-or-llm","description":"The target domain or URL to analyze. Example: \\"example.com\\" or \\"https://example.com/page\\""},"mode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Analysis mode: domain (entire domain), prefix (URL prefix), subdomains (include all subdomains, default), exact (exact URL match). Example: \\"domain\\""},"date":{"type":"string","required":false,"visibility":"user-only","description":"Date to report metrics on, in YYYY-MM-DD format (defaults to today)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_batch_analysis":{"id":"ahrefs_batch_analysis","name":"Ahrefs Batch Analysis","description":"Get bulk SEO metrics (Domain Rating, backlinks, referring domains, organic traffic, and more) for multiple domains or URLs in a single request. Useful for comparing many competitors at once.","version":"1.0.0","params":{"targets":{"type":"string","required":true,"visibility":"user-or-llm","description":"Comma-separated list of domains or URLs to analyze. Example: \\"example.com,competitor.com\\""},"mode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Analysis mode applied to every target: domain (entire domain), prefix (URL prefix), subdomains (include all subdomains, default), exact (exact URL match)"},"protocol":{"type":"string","required":false,"visibility":"user-or-llm","description":"Protocol applied to every target: \\"both\\" (default), \\"http\\", or \\"https\\""},"country":{"type":"string","required":false,"visibility":"user-or-llm","description":"Country code for traffic data. Example: \\"us\\", \\"gb\\", \\"de\\" (default: \\"us\\")"},"volumeMode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Search volume calculation: \\"monthly\\" or \\"average\\" (default: \\"monthly\\")"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_broken_backlinks":{"id":"ahrefs_broken_backlinks","name":"Ahrefs Broken Backlinks","description":"Get a list of broken backlinks pointing to a target domain or URL. Useful for identifying link reclamation opportunities.","version":"1.0.0","params":{"target":{"type":"string","required":true,"visibility":"user-or-llm","description":"The target domain or URL to analyze. Example: \\"example.com\\" or \\"https://example.com/page\\""},"mode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Analysis mode: domain (entire domain), prefix (URL prefix), subdomains (include all subdomains, default), exact (exact URL match). Example: \\"domain\\""},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of results to return. Example: 50 (default: 1000)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_domain_rating":{"id":"ahrefs_domain_rating","name":"Ahrefs Domain Rating","description":"Get the Domain Rating (DR) and Ahrefs Rank for a target domain. Domain Rating shows the strength of a website\'s backlink profile on a scale from 0 to 100.","version":"1.0.0","params":{"target":{"type":"string","required":true,"visibility":"user-or-llm","description":"The target domain to analyze (e.g., example.com)"},"date":{"type":"string","required":false,"visibility":"user-only","description":"Date for historical data in YYYY-MM-DD format (defaults to today)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_domain_rating_history":{"id":"ahrefs_domain_rating_history","name":"Ahrefs Domain Rating History","description":"Get the historical Domain Rating (DR) trend for a target domain or URL over a date range, grouped daily, weekly, or monthly.","version":"1.0.0","params":{"target":{"type":"string","required":true,"visibility":"user-or-llm","description":"The target domain or URL to analyze. Example: \\"example.com\\""},"dateFrom":{"type":"string","required":true,"visibility":"user-only","description":"Start date of the historical period, in YYYY-MM-DD format"},"dateTo":{"type":"string","required":false,"visibility":"user-only","description":"End date of the historical period, in YYYY-MM-DD format (defaults to today)"},"historyGrouping":{"type":"string","required":false,"visibility":"user-or-llm","description":"Time interval for grouping data points: \\"daily\\", \\"weekly\\", or \\"monthly\\" (default: \\"monthly\\")"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_keyword_overview":{"id":"ahrefs_keyword_overview","name":"Ahrefs Keyword Overview","description":"Get detailed metrics for a keyword including search volume, keyword difficulty, CPC, clicks, and traffic potential.","version":"1.0.0","params":{"keyword":{"type":"string","required":true,"visibility":"user-or-llm","description":"The keyword to analyze"},"country":{"type":"string","required":false,"visibility":"user-or-llm","description":"Country code for keyword data. Example: \\"us\\", \\"gb\\", \\"de\\" (default: \\"us\\")"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_keywords_history":{"id":"ahrefs_keywords_history","name":"Ahrefs Keywords History","description":"Get the historical organic keyword ranking distribution for a target domain or URL over a date range: how many keywords rank in each position bucket at each point in time.","version":"1.0.0","params":{"target":{"type":"string","required":true,"visibility":"user-or-llm","description":"The target domain or URL to analyze. Example: \\"example.com\\""},"dateFrom":{"type":"string","required":true,"visibility":"user-only","description":"Start date of the historical period, in YYYY-MM-DD format"},"dateTo":{"type":"string","required":false,"visibility":"user-only","description":"End date of the historical period, in YYYY-MM-DD format (defaults to today)"},"historyGrouping":{"type":"string","required":false,"visibility":"user-or-llm","description":"Time interval for grouping data points: \\"daily\\", \\"weekly\\", or \\"monthly\\" (default: \\"monthly\\")"},"country":{"type":"string","required":false,"visibility":"user-or-llm","description":"Country code for search results. Example: \\"us\\", \\"gb\\", \\"de\\" (default: \\"us\\")"},"mode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Analysis mode: domain (entire domain), prefix (URL prefix), subdomains (include all subdomains, default), exact (exact URL match)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_metrics":{"id":"ahrefs_metrics","name":"Ahrefs Metrics","description":"Get a one-call organic and paid search overview for a target domain or URL: organic traffic, organic keywords, paid traffic, paid keywords, and estimated traffic cost.","version":"1.0.0","params":{"target":{"type":"string","required":true,"visibility":"user-or-llm","description":"The target domain or URL to analyze. Example: \\"example.com\\""},"country":{"type":"string","required":false,"visibility":"user-or-llm","description":"Country code for traffic data. Example: \\"us\\", \\"gb\\", \\"de\\""},"mode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Analysis mode: domain (entire domain), prefix (URL prefix), subdomains (include all subdomains, default), exact (exact URL match). Example: \\"domain\\""},"date":{"type":"string","required":false,"visibility":"user-only","description":"Date to report metrics on, in YYYY-MM-DD format (defaults to today)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_metrics_history":{"id":"ahrefs_metrics_history","name":"Ahrefs Metrics History","description":"Get the historical organic and paid traffic trend for a target domain or URL over a date range: organic traffic/cost and paid traffic/cost at each point in time.","version":"1.0.0","params":{"target":{"type":"string","required":true,"visibility":"user-or-llm","description":"The target domain or URL to analyze. Example: \\"example.com\\""},"dateFrom":{"type":"string","required":true,"visibility":"user-only","description":"Start date of the historical period, in YYYY-MM-DD format"},"dateTo":{"type":"string","required":false,"visibility":"user-only","description":"End date of the historical period, in YYYY-MM-DD format (defaults to today)"},"volumeMode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Search volume calculation: \\"monthly\\" or \\"average\\" (default: \\"monthly\\")"},"historyGrouping":{"type":"string","required":false,"visibility":"user-or-llm","description":"Time interval for grouping data points: \\"daily\\", \\"weekly\\", or \\"monthly\\" (default: \\"monthly\\")"},"country":{"type":"string","required":false,"visibility":"user-or-llm","description":"Country code for traffic data. Example: \\"us\\", \\"gb\\", \\"de\\" (default: \\"us\\")"},"mode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Analysis mode: domain (entire domain), prefix (URL prefix), subdomains (include all subdomains, default), exact (exact URL match)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_organic_competitors":{"id":"ahrefs_organic_competitors","name":"Ahrefs Organic Competitors","description":"Get domains that compete with a target domain or URL for the same organic keywords, ranked by keyword overlap.","version":"1.0.0","params":{"target":{"type":"string","required":true,"visibility":"user-or-llm","description":"The target domain or URL to analyze. Example: \\"example.com\\""},"country":{"type":"string","required":false,"visibility":"user-or-llm","description":"Country code for search results. Example: \\"us\\", \\"gb\\", \\"de\\" (default: \\"us\\")"},"mode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Analysis mode: domain (entire domain), prefix (URL prefix), subdomains (include all subdomains, default), exact (exact URL match). Example: \\"domain\\""},"date":{"type":"string","required":false,"visibility":"user-only","description":"Date to report metrics on, in YYYY-MM-DD format (defaults to today)"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of results to return. Example: 50 (default: 1000)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_organic_keywords":{"id":"ahrefs_organic_keywords","name":"Ahrefs Organic Keywords","description":"Get organic keywords that a target domain or URL ranks for in Google search results. Returns keyword details including search volume, ranking position, and estimated traffic.","version":"1.0.0","params":{"target":{"type":"string","required":true,"visibility":"user-or-llm","description":"The target domain or URL to analyze. Example: \\"example.com\\" or \\"https://example.com/page\\""},"country":{"type":"string","required":false,"visibility":"user-or-llm","description":"Country code for search results. Example: \\"us\\", \\"gb\\", \\"de\\" (default: \\"us\\")"},"mode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Analysis mode: domain (entire domain), prefix (URL prefix), subdomains (include all subdomains, default), exact (exact URL match). Example: \\"domain\\""},"date":{"type":"string","required":false,"visibility":"user-only","description":"Date to report metrics on, in YYYY-MM-DD format (defaults to today)"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of results to return. Example: 50 (default: 1000)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_paid_pages":{"id":"ahrefs_paid_pages","name":"Ahrefs Paid Pages","description":"Get a target domain\'s pages that receive paid search traffic, sorted by estimated paid traffic. Returns page URLs with their paid traffic, keyword counts, and estimated spend.","version":"1.0.0","params":{"target":{"type":"string","required":true,"visibility":"user-or-llm","description":"The target domain or URL to analyze. Example: \\"example.com\\""},"country":{"type":"string","required":false,"visibility":"user-or-llm","description":"Country code for traffic data. Example: \\"us\\", \\"gb\\", \\"de\\" (default: \\"us\\")"},"mode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Analysis mode: domain (entire domain), prefix (URL prefix), subdomains (include all subdomains, default), exact (exact URL match)"},"date":{"type":"string","required":false,"visibility":"user-only","description":"Date to report metrics on, in YYYY-MM-DD format (defaults to today)"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of results to return. Example: 50 (default: 1000)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_rank_tracker_competitors_overview":{"id":"ahrefs_rank_tracker_competitors_overview","name":"Ahrefs Rank Tracker Competitors Overview","description":"Get competitor rankings for the keywords tracked in an Ahrefs Rank Tracker project: each tracked keyword\'s volume and difficulty alongside every competitor\'s position, traffic, and traffic value. This endpoint is free and does not consume API units.","version":"1.0.0","params":{"projectId":{"type":"number","required":true,"visibility":"user-or-llm","description":"The Rank Tracker project ID (found in the project URL in Ahrefs)"},"date":{"type":"string","required":true,"visibility":"user-only","description":"Date to report rankings for, in YYYY-MM-DD format"},"device":{"type":"string","required":true,"visibility":"user-or-llm","description":"Rankings device type: \\"desktop\\" or \\"mobile\\""},"dateCompared":{"type":"string","required":false,"visibility":"user-only","description":"Comparison date in YYYY-MM-DD format, to compute position/traffic deltas"},"volumeMode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Search volume calculation: \\"monthly\\" or \\"average\\" (default: \\"monthly\\")"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of results to return. Example: 50 (default: 1000)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_rank_tracker_competitors_stats":{"id":"ahrefs_rank_tracker_competitors_stats","name":"Ahrefs Rank Tracker Competitors Stats","description":"Get aggregate competitor stats for an Ahrefs Rank Tracker project: each competitor\'s traffic, traffic value, average position, and share of voice across all tracked keywords. This endpoint is free and does not consume API units.","version":"1.0.0","params":{"projectId":{"type":"number","required":true,"visibility":"user-or-llm","description":"The Rank Tracker project ID (found in the project URL in Ahrefs)"},"date":{"type":"string","required":true,"visibility":"user-only","description":"Date to report metrics for, in YYYY-MM-DD format"},"device":{"type":"string","required":true,"visibility":"user-or-llm","description":"Rankings device type: \\"desktop\\" or \\"mobile\\""},"volumeMode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Search volume calculation: \\"monthly\\" or \\"average\\" (default: \\"monthly\\")"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_rank_tracker_overview":{"id":"ahrefs_rank_tracker_overview","name":"Ahrefs Rank Tracker Overview","description":"Get ranking overview metrics for the keywords tracked in an Ahrefs Rank Tracker project: position, search volume, keyword difficulty, and estimated traffic. This endpoint is free and does not consume API units.","version":"1.0.0","params":{"projectId":{"type":"number","required":true,"visibility":"user-or-llm","description":"The Rank Tracker project ID (found in the project URL in Ahrefs)"},"date":{"type":"string","required":true,"visibility":"user-only","description":"Date to report rankings for, in YYYY-MM-DD format"},"device":{"type":"string","required":true,"visibility":"user-or-llm","description":"Rankings device type: \\"desktop\\" or \\"mobile\\""},"dateCompared":{"type":"string","required":false,"visibility":"user-only","description":"Comparison date in YYYY-MM-DD format, to compute position/traffic deltas"},"volumeMode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Search volume calculation: \\"monthly\\" or \\"average\\" (default: \\"monthly\\")"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of results to return. Example: 50 (default: 1000)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_rank_tracker_serp_overview":{"id":"ahrefs_rank_tracker_serp_overview","name":"Ahrefs Rank Tracker SERP Overview","description":"Get the full SERP (search engine results page) for a keyword tracked in an Ahrefs Rank Tracker project, including every ranking URL with its position, title, and authority metrics. This endpoint is free and does not consume API units.","version":"1.0.0","params":{"projectId":{"type":"number","required":true,"visibility":"user-or-llm","description":"The Rank Tracker project ID (found in the project URL in Ahrefs)"},"keyword":{"type":"string","required":true,"visibility":"user-or-llm","description":"The tracked keyword to retrieve SERP data for"},"country":{"type":"string","required":true,"visibility":"user-or-llm","description":"Country code for the tracked keyword. Example: \\"us\\", \\"gb\\", \\"de\\""},"device":{"type":"string","required":true,"visibility":"user-or-llm","description":"Rankings device type: \\"desktop\\" or \\"mobile\\""},"topPositions":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of top organic positions to return (defaults to all available)"},"date":{"type":"string","required":false,"visibility":"user-only","description":"Timestamp to return the last available SERP Overview at, in YYYY-MM-DDThh:mm:ss format"},"locationId":{"type":"number","required":false,"visibility":"user-or-llm","description":"Location ID of the tracked keyword, if tracked at a specific location"},"languageCode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Language code of the tracked keyword"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_refdomains_history":{"id":"ahrefs_refdomains_history","name":"Ahrefs Referring Domains History","description":"Get the historical referring domains trend for a target domain or URL over a date range, grouped daily, weekly, or monthly.","version":"1.0.0","params":{"target":{"type":"string","required":true,"visibility":"user-or-llm","description":"The target domain or URL to analyze. Example: \\"example.com\\""},"dateFrom":{"type":"string","required":true,"visibility":"user-only","description":"Start date of the historical period, in YYYY-MM-DD format"},"dateTo":{"type":"string","required":false,"visibility":"user-only","description":"End date of the historical period, in YYYY-MM-DD format (defaults to today)"},"historyGrouping":{"type":"string","required":false,"visibility":"user-or-llm","description":"Time interval for grouping data points: \\"daily\\", \\"weekly\\", or \\"monthly\\" (default: \\"monthly\\")"},"mode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Analysis mode: domain (entire domain), prefix (URL prefix), subdomains (include all subdomains, default), exact (exact URL match)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_referring_domains":{"id":"ahrefs_referring_domains","name":"Ahrefs Referring Domains","description":"Get a list of domains that link to a target domain or URL. Returns unique referring domains with their domain rating, backlink counts, and discovery dates.","version":"1.0.0","params":{"target":{"type":"string","required":true,"visibility":"user-or-llm","description":"The target domain or URL to analyze. Example: \\"example.com\\" or \\"https://example.com/page\\""},"mode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Analysis mode: domain (entire domain), prefix (URL prefix), subdomains (include all subdomains, default), exact (exact URL match). Example: \\"domain\\""},"history":{"type":"string","required":false,"visibility":"user-or-llm","description":"Historical scope: \\"live\\" (currently live), \\"all_time\\" (default, includes lost domains), or \\"since:YYYY-MM-DD\\" (domains found since a date)."},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of results to return. Example: 50 (default: 1000)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_related_terms":{"id":"ahrefs_related_terms","name":"Ahrefs Related Terms","description":"Get keyword ideas related to a seed keyword: terms the same top-ranking pages also rank for (\\"also rank for\\") or also discuss (\\"also talk about\\"), with volume, difficulty, and CPC.","version":"1.0.0","params":{"keyword":{"type":"string","required":true,"visibility":"user-or-llm","description":"The seed keyword to find related terms for"},"country":{"type":"string","required":false,"visibility":"user-or-llm","description":"Country code for keyword data. Example: \\"us\\", \\"gb\\", \\"de\\" (default: \\"us\\")"},"terms":{"type":"string","required":false,"visibility":"user-or-llm","description":"Type of related keywords to return: \\"also_rank_for\\", \\"also_talk_about\\", or \\"all\\" (default: \\"all\\")"},"viewFor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Whether to derive related terms from the top 10 or top 100 ranking pages (default: \\"top_10\\")"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of results to return. Example: 50 (default: 1000)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_site_audit_page_explorer":{"id":"ahrefs_site_audit_page_explorer","name":"Ahrefs Site Audit Page Explorer","description":"Get crawled pages from an Ahrefs Site Audit project with health and SEO metrics: HTTP status, title, link counts, backlinks, indexability, and traffic. Optionally filter to pages affected by a specific issue.","version":"1.0.0","params":{"projectId":{"type":"number","required":true,"visibility":"user-or-llm","description":"The Site Audit project ID (found in the project URL in Ahrefs)"},"date":{"type":"string","required":false,"visibility":"user-only","description":"Crawl date in YYYY-MM-DDThh:mm:ss format (defaults to the most recent crawl)"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of results to return. Example: 50 (default: 1000)"},"offset":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of results to skip, for pagination"},"issueId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Only return pages affected by this issue ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_top_pages":{"id":"ahrefs_top_pages","name":"Ahrefs Top Pages","description":"Get the top pages of a target domain sorted by organic traffic. Returns page URLs with their traffic, keyword counts, and estimated traffic value.","version":"1.0.0","params":{"target":{"type":"string","required":true,"visibility":"user-or-llm","description":"The target domain to analyze. Example: \\"example.com\\""},"country":{"type":"string","required":false,"visibility":"user-or-llm","description":"Country code for traffic data. Example: \\"us\\", \\"gb\\", \\"de\\" (default: \\"us\\")"},"mode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Analysis mode: domain (entire domain), prefix (URL prefix), subdomains (include all subdomains, default), exact (exact URL match). Example: \\"domain\\""},"date":{"type":"string","required":false,"visibility":"user-only","description":"Date to report metrics on, in YYYY-MM-DD format (defaults to today)"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of results to return. Example: 50 (default: 1000)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"airtable_create_records":{"id":"airtable_create_records","name":"Airtable Create Records","description":"Write new records to an Airtable table","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token"},"baseId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Airtable base ID (starts with \\"app\\", e.g., \\"appXXXXXXXXXXXXXX\\")"},"tableId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table ID (starts with \\"tbl\\") or table name"},"records":{"type":"json","required":true,"visibility":"user-or-llm","description":"Array of records to create, each with a `fields` object"},"typecast":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"When true, Airtable automatically converts string values to the field type"}},"oauth":{"required":true,"provider":"airtable"},"hostedApiKey":"none"},"airtable_delete_records":{"id":"airtable_delete_records","name":"Airtable Delete Records","description":"Delete one or more records from an Airtable table by ID","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token"},"baseId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Airtable base ID (starts with \\"app\\", e.g., \\"appXXXXXXXXXXXXXX\\")"},"tableId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table ID (starts with \\"tbl\\") or table name"},"recordIds":{"type":"json","required":true,"visibility":"user-or-llm","description":"Array of record IDs to delete (each starts with \\"rec\\", e.g., [\\"recXXXXXXXXXXXXXX\\"]). Pass a single-element array to delete one record."}},"oauth":{"required":true,"provider":"airtable"},"hostedApiKey":"none"},"airtable_get_base_schema":{"id":"airtable_get_base_schema","name":"Airtable Get Base Schema","description":"Get the schema of all tables, fields, and views in an Airtable base","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token"},"baseId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Airtable base ID (starts with \\"app\\", e.g., \\"appXXXXXXXXXXXXXX\\")"}},"oauth":{"required":true,"provider":"airtable"},"hostedApiKey":"none"},"airtable_get_record":{"id":"airtable_get_record","name":"Airtable Get Record","description":"Retrieve a single record from an Airtable table by its ID","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token"},"baseId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Airtable base ID (starts with \\"app\\", e.g., \\"appXXXXXXXXXXXXXX\\")"},"tableId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table ID (starts with \\"tbl\\") or table name"},"recordId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Record ID to retrieve (starts with \\"rec\\", e.g., \\"recXXXXXXXXXXXXXX\\")"}},"oauth":{"required":true,"provider":"airtable"},"hostedApiKey":"none"},"airtable_list_bases":{"id":"airtable_list_bases","name":"Airtable List Bases","description":"List all bases the authenticated user has access to","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token"},"offset":{"type":"string","required":false,"visibility":"user-or-llm","description":"Pagination offset for retrieving additional bases"}},"oauth":{"required":true,"provider":"airtable"},"hostedApiKey":"none"},"airtable_list_records":{"id":"airtable_list_records","name":"Airtable List Records","description":"Read records from an Airtable table","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token"},"baseId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Airtable base ID (starts with \\"app\\", e.g., \\"appXXXXXXXXXXXXXX\\")"},"tableId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table ID (starts with \\"tbl\\") or table name"},"maxRecords":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of records to return (default: all records)"},"filterFormula":{"type":"string","required":false,"visibility":"user-or-llm","description":"Formula to filter records (e.g., \\"({Field Name} = \'Value\')\\")"}},"oauth":{"required":true,"provider":"airtable"},"hostedApiKey":"none"},"airtable_list_tables":{"id":"airtable_list_tables","name":"Airtable List Tables","description":"List all tables and their schema in an Airtable base","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token"},"baseId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Airtable base ID (starts with \\"app\\", e.g., \\"appXXXXXXXXXXXXXX\\")"}},"oauth":{"required":true,"provider":"airtable"},"hostedApiKey":"none"},"airtable_update_multiple_records":{"id":"airtable_update_multiple_records","name":"Airtable Update Multiple Records","description":"Update multiple existing records in an Airtable table","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token"},"baseId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Airtable base ID (starts with \\"app\\", e.g., \\"appXXXXXXXXXXXXXX\\")"},"tableId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table ID (starts with \\"tbl\\") or table name"},"records":{"type":"json","required":true,"visibility":"user-or-llm","description":"Array of records to update, each with an `id` and a `fields` object"},"typecast":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"When true, Airtable automatically converts string values to the field type"}},"oauth":{"required":true,"provider":"airtable"},"hostedApiKey":"none"},"airtable_update_record":{"id":"airtable_update_record","name":"Airtable Update Record","description":"Update an existing record in an Airtable table by ID","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token"},"baseId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Airtable base ID (starts with \\"app\\", e.g., \\"appXXXXXXXXXXXXXX\\")"},"tableId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table ID (starts with \\"tbl\\") or table name"},"recordId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Record ID to update (starts with \\"rec\\", e.g., \\"recXXXXXXXXXXXXXX\\")"},"fields":{"type":"json","required":true,"visibility":"user-or-llm","description":"An object containing the field names and their new values"},"typecast":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"When true, Airtable automatically converts string values to the field type"}},"oauth":{"required":true,"provider":"airtable"},"hostedApiKey":"none"},"airtable_upsert_records":{"id":"airtable_upsert_records","name":"Airtable Upsert Records","description":"Update existing records or create new ones in an Airtable table, matching on the specified merge fields","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token"},"baseId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Airtable base ID (starts with \\"app\\", e.g., \\"appXXXXXXXXXXXXXX\\")"},"tableId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table ID (starts with \\"tbl\\") or table name"},"records":{"type":"json","required":true,"visibility":"user-or-llm","description":"Array of records to upsert, each with a `fields` object"},"fieldsToMergeOn":{"type":"json","required":true,"visibility":"user-or-llm","description":"Array of field names used to match existing records (max 3). A record is updated when all merge fields match, otherwise it is created. Example: [\\"Name\\"]"},"typecast":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"When true, Airtable automatically converts string values to the field type"}},"oauth":{"required":true,"provider":"airtable"},"hostedApiKey":"none"},"airweave_search":{"id":"airweave_search","name":"Airweave Search","description":"Search your synced data collections using Airweave. Supports semantic search with hybrid, neural, or keyword retrieval strategies. Optionally generate AI-powered answers from search results.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Airweave API Key for authentication"},"collectionId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The readable ID of the collection to search"},"query":{"type":"string","required":true,"visibility":"user-or-llm","description":"The search query text"},"limit":{"type":"number","required":false,"visibility":"user-only","description":"Maximum number of results to return (default: 100)"},"retrievalStrategy":{"type":"string","required":false,"visibility":"user-or-llm","description":"Retrieval strategy: hybrid (default), neural, or keyword"},"expandQuery":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Generate query variations to improve recall"},"rerank":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Reorder results for improved relevance using LLM"},"generateAnswer":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Generate a natural-language answer to the query"}},"hostedApiKey":"none"},"algolia_add_record":{"id":"algolia_add_record","name":"Algolia Add Record","description":"Add or replace a record in an Algolia index","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Admin API Key"},"indexName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the Algolia index"},"objectID":{"type":"string","required":false,"visibility":"user-or-llm","description":"Object ID for the record (auto-generated if not provided)"},"record":{"type":"json","required":true,"visibility":"user-or-llm","description":"JSON object representing the record to add"}},"hostedApiKey":"none"},"algolia_batch_operations":{"id":"algolia_batch_operations","name":"Algolia Batch Operations","description":"Perform batch add, update, partial update, or delete operations on records in an Algolia index","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Admin API Key"},"indexName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the Algolia index"},"requests":{"type":"json","required":true,"visibility":"user-or-llm","description":"Array of batch operations. Each item has \\"action\\" (addObject, updateObject, partialUpdateObject, partialUpdateObjectNoCreate, deleteObject, delete, clear) and \\"body\\" (the record data; must include objectID for update/delete; use an empty object {} for the index-level delete/clear actions)"}},"hostedApiKey":"none"},"algolia_browse_records":{"id":"algolia_browse_records","name":"Algolia Browse Records","description":"Browse and iterate over all records in an Algolia index using cursor pagination","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia API Key (must have browse ACL)"},"indexName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the Algolia index to browse"},"query":{"type":"string","required":false,"visibility":"user-or-llm","description":"Search query to filter browsed records"},"filters":{"type":"string","required":false,"visibility":"user-or-llm","description":"Filter string to narrow down results"},"attributesToRetrieve":{"type":"string","required":false,"visibility":"user-or-llm","description":"Comma-separated list of attributes to retrieve"},"hitsPerPage":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of hits per page (default: 1000, max: 1000)"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous browse response for pagination"},"aroundLatLng":{"type":"string","required":false,"visibility":"user-or-llm","description":"Coordinates for geo-search (e.g., \\"40.71,-74.01\\")"},"aroundRadius":{"type":"string","required":false,"visibility":"user-or-llm","description":"Maximum radius in meters for geo-search, or \\"all\\" for unlimited"},"insideBoundingBox":{"type":"json","required":false,"visibility":"user-or-llm","description":"Bounding box coordinates as [[lat1, lng1, lat2, lng2]] for geo-search"},"insidePolygon":{"type":"json","required":false,"visibility":"user-or-llm","description":"Polygon coordinates as [[lat1, lng1, lat2, lng2, lat3, lng3, ...]] for geo-search"}},"hostedApiKey":"none"},"algolia_clear_records":{"id":"algolia_clear_records","name":"Algolia Clear Records","description":"Clear all records from an Algolia index while keeping settings, synonyms, and rules","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Admin API Key (must have deleteIndex ACL)"},"indexName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the Algolia index to clear"}},"hostedApiKey":"none"},"algolia_copy_move_index":{"id":"algolia_copy_move_index","name":"Algolia Copy/Move Index","description":"Copy or move an Algolia index to a new destination","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Admin API Key"},"indexName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the source index"},"operation":{"type":"string","required":true,"visibility":"user-or-llm","description":"Operation to perform: \\"copy\\" or \\"move\\""},"destination":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the destination index"},"scope":{"type":"json","required":false,"visibility":"user-or-llm","description":"Array of scopes to copy (only for \\"copy\\" operation): [\\"settings\\", \\"synonyms\\", \\"rules\\"]. Omit to copy everything including records."}},"hostedApiKey":"none"},"algolia_delete_by_filter":{"id":"algolia_delete_by_filter","name":"Algolia Delete By Filter","description":"Delete all records matching a filter from an Algolia index","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Admin API Key (must have deleteIndex ACL)"},"indexName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the Algolia index"},"filters":{"type":"string","required":false,"visibility":"user-or-llm","description":"Filter expression to match records for deletion (e.g., \\"category:outdated\\")"},"facetFilters":{"type":"json","required":false,"visibility":"user-or-llm","description":"Array of facet filters (e.g., [\\"brand:Acme\\"])"},"numericFilters":{"type":"json","required":false,"visibility":"user-or-llm","description":"Array of numeric filters (e.g., [\\"price > 100\\"])"},"tagFilters":{"type":"json","required":false,"visibility":"user-or-llm","description":"Array of tag filters using the _tags attribute (e.g., [\\"published\\"])"},"aroundLatLng":{"type":"string","required":false,"visibility":"user-or-llm","description":"Coordinates for geo-search filter (e.g., \\"40.71,-74.01\\")"},"aroundRadius":{"type":"string","required":false,"visibility":"user-or-llm","description":"Maximum radius in meters for geo-search, or \\"all\\" for unlimited"},"insideBoundingBox":{"type":"json","required":false,"visibility":"user-or-llm","description":"Bounding box coordinates as [[lat1, lng1, lat2, lng2]] for geo-search filter"},"insidePolygon":{"type":"json","required":false,"visibility":"user-or-llm","description":"Polygon coordinates as [[lat1, lng1, lat2, lng2, lat3, lng3, ...]] for geo-search filter"}},"hostedApiKey":"none"},"algolia_delete_index":{"id":"algolia_delete_index","name":"Algolia Delete Index","description":"Delete an entire Algolia index and all its records","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Admin API Key (must have deleteIndex ACL)"},"indexName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the Algolia index to delete"}},"hostedApiKey":"none"},"algolia_delete_record":{"id":"algolia_delete_record","name":"Algolia Delete Record","description":"Delete a record by objectID from an Algolia index","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Admin API Key"},"indexName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the Algolia index"},"objectID":{"type":"string","required":true,"visibility":"user-or-llm","description":"The objectID of the record to delete"}},"hostedApiKey":"none"},"algolia_get_record":{"id":"algolia_get_record","name":"Algolia Get Record","description":"Get a record by objectID from an Algolia index","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia API Key"},"indexName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the Algolia index"},"objectID":{"type":"string","required":true,"visibility":"user-or-llm","description":"The objectID of the record to retrieve"},"attributesToRetrieve":{"type":"string","required":false,"visibility":"user-or-llm","description":"Comma-separated list of attributes to retrieve"}},"hostedApiKey":"none"},"algolia_get_records":{"id":"algolia_get_records","name":"Algolia Get Records","description":"Retrieve multiple records by objectID from one or more Algolia indices","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia API Key"},"indexName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Default index name for all requests"},"requests":{"type":"json","required":true,"visibility":"user-or-llm","description":"Array of objects specifying records to retrieve. Each must have \\"objectID\\" and optionally \\"indexName\\" and \\"attributesToRetrieve\\"."}},"hostedApiKey":"none"},"algolia_get_settings":{"id":"algolia_get_settings","name":"Algolia Get Settings","description":"Retrieve the settings of an Algolia index","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia API Key"},"indexName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the Algolia index"}},"hostedApiKey":"none"},"algolia_get_task_status":{"id":"algolia_get_task_status","name":"Algolia Get Task Status","description":"Check whether an Algolia indexing task has finished publishing","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia API Key"},"indexName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the Algolia index the task ran against"},"taskID":{"type":"number","required":true,"visibility":"user-or-llm","description":"The taskID returned by a previous write operation"}},"hostedApiKey":"none"},"algolia_list_indices":{"id":"algolia_list_indices","name":"Algolia List Indices","description":"List all indices in an Algolia application","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia API Key"},"page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Page number for paginating indices (default: not paginated)"},"hitsPerPage":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of indices per page (default: 100)"}},"hostedApiKey":"none"},"algolia_partial_update_record":{"id":"algolia_partial_update_record","name":"Algolia Partial Update Record","description":"Partially update a record in an Algolia index without replacing it entirely","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Admin API Key"},"indexName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the Algolia index"},"objectID":{"type":"string","required":true,"visibility":"user-or-llm","description":"The objectID of the record to update"},"attributes":{"type":"json","required":true,"visibility":"user-or-llm","description":"JSON object with attributes to update. Supports built-in operations like {\\"stock\\": {\\"_operation\\": \\"Decrement\\", \\"value\\": 1}}"},"createIfNotExists":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Whether to create the record if it does not exist (default: true)"}},"hostedApiKey":"none"},"algolia_search":{"id":"algolia_search","name":"Algolia Search","description":"Search an Algolia index","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia API Key"},"indexName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the Algolia index to search"},"query":{"type":"string","required":true,"visibility":"user-or-llm","description":"Search query text"},"hitsPerPage":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of hits per page (default: 20)"},"page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Page number to retrieve (default: 0)"},"filters":{"type":"string","required":false,"visibility":"user-or-llm","description":"Filter string (e.g., \\"category:electronics AND price < 100\\")"},"attributesToRetrieve":{"type":"string","required":false,"visibility":"user-or-llm","description":"Comma-separated list of attributes to retrieve"},"facets":{"type":"string","required":false,"visibility":"user-or-llm","description":"Comma-separated list of facet attribute names to retrieve counts for (use \\"*\\" for all)"},"getRankingInfo":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Whether to include detailed ranking information in each hit"},"aroundLatLng":{"type":"string","required":false,"visibility":"user-or-llm","description":"Coordinates for geo-search (e.g., \\"40.71,-74.01\\")"},"aroundRadius":{"type":"string","required":false,"visibility":"user-or-llm","description":"Maximum radius in meters for geo-search, or \\"all\\" for unlimited"},"insideBoundingBox":{"type":"json","required":false,"visibility":"user-or-llm","description":"Bounding box coordinates as [[lat1, lng1, lat2, lng2]] for geo-search"},"insidePolygon":{"type":"json","required":false,"visibility":"user-or-llm","description":"Polygon coordinates as [[lat1, lng1, lat2, lng2, lat3, lng3, ...]] for geo-search"}},"hostedApiKey":"none"},"algolia_update_settings":{"id":"algolia_update_settings","name":"Algolia Update Settings","description":"Update the settings of an Algolia index","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Admin API Key (must have editSettings ACL)"},"indexName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the Algolia index"},"settings":{"type":"json","required":true,"visibility":"user-or-llm","description":"JSON object with settings to update (e.g., {\\"searchableAttributes\\": [\\"name\\", \\"description\\"], \\"customRanking\\": [\\"desc(popularity)\\"]})"},"forwardToReplicas":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Whether to apply changes to replica indices (default: false)"}},"hostedApiKey":"none"},"amplitude_event_segmentation":{"id":"amplitude_event_segmentation","name":"Amplitude Event Segmentation","description":"Query event analytics data with segmentation. Get event counts, uniques, averages, and more.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude API Key"},"secretKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude Secret Key"},"eventType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Event type name to analyze"},"start":{"type":"string","required":true,"visibility":"user-or-llm","description":"Start date in YYYYMMDD format"},"end":{"type":"string","required":true,"visibility":"user-or-llm","description":"End date in YYYYMMDD format"},"metric":{"type":"string","required":false,"visibility":"user-or-llm","description":"Metric type: uniques, totals, pct_dau, average, histogram, sums, value_avg, or formula (default: uniques)"},"interval":{"type":"string","required":false,"visibility":"user-or-llm","description":"Time interval: 1 (daily), 7 (weekly), or 30 (monthly)"},"groupBy":{"type":"string","required":false,"visibility":"user-or-llm","description":"Property name to group by (prefix custom user properties with \\"gp:\\")"},"groupBy2":{"type":"string","required":false,"visibility":"user-or-llm","description":"Second property name to group by (prefix custom user properties with \\"gp:\\")"},"limit":{"type":"string","required":false,"visibility":"user-or-llm","description":"Maximum number of group-by values (max 1000)"},"filters":{"type":"string","required":false,"visibility":"user-or-llm","description":"JSON array of filter objects applied to the event, e.g. [{\\"subprop_type\\":\\"event\\",\\"subprop_key\\":\\"city\\",\\"subprop_op\\":\\"is\\",\\"subprop_value\\":[\\"San Francisco\\"]}]"},"formula":{"type":"string","required":false,"visibility":"user-or-llm","description":"Required when metric is \\"formula\\", e.g. \\"UNIQUES(A)/UNIQUES(B)\\""},"segment":{"type":"string","required":false,"visibility":"user-or-llm","description":"JSON segment definition(s) applied to the query"},"dataResidency":{"type":"string","required":false,"visibility":"user-or-llm","description":"Data residency region: \\"us\\" (default) or \\"eu\\""}},"hostedApiKey":"none"},"amplitude_funnels":{"id":"amplitude_funnels","name":"Amplitude Funnels","description":"Analyze conversion rates and drop-off between a sequence of events.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude API Key"},"secretKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude Secret Key"},"events":{"type":"string","required":true,"visibility":"user-or-llm","description":"JSON array of event objects, one per funnel step in order, e.g. [{\\"event_type\\":\\"signup\\"},{\\"event_type\\":\\"purchase\\"}]"},"start":{"type":"string","required":true,"visibility":"user-or-llm","description":"Start date in YYYYMMDD format"},"end":{"type":"string","required":true,"visibility":"user-or-llm","description":"End date in YYYYMMDD format"},"mode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Funnel ordering: \\"ordered\\", \\"unordered\\", or \\"sequential\\" (default: ordered)"},"userType":{"type":"string","required":false,"visibility":"user-or-llm","description":"User type: \\"new\\" or \\"active\\" (default: active)"},"interval":{"type":"string","required":false,"visibility":"user-or-llm","description":"Time interval: -300000 (real-time), -3600000 (hourly), 1 (daily), 7 (weekly), or 30 (monthly)"},"conversionWindowSeconds":{"type":"string","required":false,"visibility":"user-or-llm","description":"Conversion window in seconds (default: 2592000, i.e. 30 days)"},"groupBy":{"type":"string","required":false,"visibility":"user-or-llm","description":"Property to group by (limit: one; prefix custom properties with \\"gp:\\")"},"limit":{"type":"string","required":false,"visibility":"user-or-llm","description":"Maximum number of group-by values (default: 100, max: 1000)"},"segment":{"type":"string","required":false,"visibility":"user-or-llm","description":"JSON segment definition(s) applied to the query"},"dataResidency":{"type":"string","required":false,"visibility":"user-or-llm","description":"Data residency region: \\"us\\" (default) or \\"eu\\""}},"hostedApiKey":"none"},"amplitude_get_active_users":{"id":"amplitude_get_active_users","name":"Amplitude Get Active Users","description":"Get active or new user counts over a date range from the Dashboard REST API.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude API Key"},"secretKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude Secret Key"},"start":{"type":"string","required":true,"visibility":"user-or-llm","description":"Start date in YYYYMMDD format"},"end":{"type":"string","required":true,"visibility":"user-or-llm","description":"End date in YYYYMMDD format"},"metric":{"type":"string","required":false,"visibility":"user-or-llm","description":"Metric type: \\"active\\" or \\"new\\" (default: active)"},"interval":{"type":"string","required":false,"visibility":"user-or-llm","description":"Time interval: 1 (daily), 7 (weekly), or 30 (monthly)"},"groupBy":{"type":"string","required":false,"visibility":"user-or-llm","description":"Property name to group by"},"segment":{"type":"string","required":false,"visibility":"user-or-llm","description":"JSON segment definition(s) applied to the query"},"dataResidency":{"type":"string","required":false,"visibility":"user-or-llm","description":"Data residency region: \\"us\\" (default) or \\"eu\\""}},"hostedApiKey":"none"},"amplitude_get_revenue":{"id":"amplitude_get_revenue","name":"Amplitude Get Revenue","description":"Get revenue LTV data including ARPU, ARPPU, total revenue, and paying user counts.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude API Key"},"secretKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude Secret Key"},"start":{"type":"string","required":true,"visibility":"user-or-llm","description":"Start date in YYYYMMDD format"},"end":{"type":"string","required":true,"visibility":"user-or-llm","description":"End date in YYYYMMDD format"},"metric":{"type":"string","required":false,"visibility":"user-or-llm","description":"Metric: 0 (ARPU), 1 (ARPPU), 2 (Total Revenue), 3 (Paying Users)"},"interval":{"type":"string","required":false,"visibility":"user-or-llm","description":"Time interval: 1 (daily), 7 (weekly), or 30 (monthly)"},"groupBy":{"type":"string","required":false,"visibility":"user-or-llm","description":"Property name to group by (limit: one)"},"segment":{"type":"string","required":false,"visibility":"user-or-llm","description":"JSON segment definition(s) applied to the query"},"dataResidency":{"type":"string","required":false,"visibility":"user-or-llm","description":"Data residency region: \\"us\\" (default) or \\"eu\\""}},"hostedApiKey":"none"},"amplitude_group_identify":{"id":"amplitude_group_identify","name":"Amplitude Group Identify","description":"Set group-level properties in Amplitude. Supports $set, $setOnce, $add, $append, $unset operations.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude API Key"},"groupType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Group classification (e.g., \\"company\\", \\"org_id\\")"},"groupValue":{"type":"string","required":true,"visibility":"user-or-llm","description":"Specific group identifier (e.g., \\"Acme Corp\\")"},"groupProperties":{"type":"string","required":true,"visibility":"user-or-llm","description":"JSON object of group properties. Use operations like $set, $setOnce, $add, $append, $unset."},"dataResidency":{"type":"string","required":false,"visibility":"user-or-llm","description":"Data residency region: \\"us\\" (default) or \\"eu\\""}},"hostedApiKey":"none"},"amplitude_identify_user":{"id":"amplitude_identify_user","name":"Amplitude Identify User","description":"Set user properties in Amplitude using the Identify API. Supports $set, $setOnce, $add, $append, $unset operations.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude API Key"},"userId":{"type":"string","required":false,"visibility":"user-or-llm","description":"User ID (required if no device_id)"},"deviceId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Device ID (required if no user_id)"},"userProperties":{"type":"string","required":true,"visibility":"user-or-llm","description":"JSON object of user properties. Use operations like $set, $setOnce, $add, $append, $unset."},"dataResidency":{"type":"string","required":false,"visibility":"user-or-llm","description":"Data residency region: \\"us\\" (default) or \\"eu\\""}},"hostedApiKey":"none"},"amplitude_list_events":{"id":"amplitude_list_events","name":"Amplitude List Events","description":"List all event types in the Amplitude project with their weekly totals and unique counts.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude API Key"},"secretKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude Secret Key"},"dataResidency":{"type":"string","required":false,"visibility":"user-or-llm","description":"Data residency region: \\"us\\" (default) or \\"eu\\""}},"hostedApiKey":"none"},"amplitude_realtime_active_users":{"id":"amplitude_realtime_active_users","name":"Amplitude Real-time Active Users","description":"Get real-time active user counts at 5-minute granularity for the last 2 days.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude API Key"},"secretKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude Secret Key"},"dataResidency":{"type":"string","required":false,"visibility":"user-or-llm","description":"Data residency region: \\"us\\" (default) or \\"eu\\""}},"hostedApiKey":"none"},"amplitude_retention":{"id":"amplitude_retention","name":"Amplitude Retention","description":"Measure how many users return to perform an action after a starting action.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude API Key"},"secretKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude Secret Key"},"startEvent":{"type":"string","required":true,"visibility":"user-or-llm","description":"JSON starting event object, e.g. {\\"event_type\\":\\"_new\\"} or {\\"event_type\\":\\"_active\\"}"},"returnEvent":{"type":"string","required":true,"visibility":"user-or-llm","description":"JSON returning event object, e.g. {\\"event_type\\":\\"_all\\"} or {\\"event_type\\":\\"_active\\"}"},"start":{"type":"string","required":true,"visibility":"user-or-llm","description":"Start date in YYYYMMDD format"},"end":{"type":"string","required":true,"visibility":"user-or-llm","description":"End date in YYYYMMDD format"},"retentionMode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Retention type: \\"bracket\\", \\"rolling\\", or \\"n-day\\" (default: n-day)"},"retentionBrackets":{"type":"string","required":false,"visibility":"user-or-llm","description":"Required when Retention Mode is \\"bracket\\". Day ranges, e.g. [[0,4]]"},"interval":{"type":"string","required":false,"visibility":"user-or-llm","description":"Time interval: 1 (daily), 7 (weekly), or 30 (monthly)"},"groupBy":{"type":"string","required":false,"visibility":"user-or-llm","description":"Property to group by (limit: one; prefix custom properties with \\"gp:\\")"},"segment":{"type":"string","required":false,"visibility":"user-or-llm","description":"JSON segment definition(s) applied to the query"},"dataResidency":{"type":"string","required":false,"visibility":"user-or-llm","description":"Data residency region: \\"us\\" (default) or \\"eu\\""}},"hostedApiKey":"none"},"amplitude_send_event":{"id":"amplitude_send_event","name":"Amplitude Send Event","description":"Track an event in Amplitude using the HTTP V2 API.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude API Key"},"userId":{"type":"string","required":false,"visibility":"user-or-llm","description":"User ID (required if no device_id)"},"deviceId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Device ID (required if no user_id)"},"eventType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the event (e.g., \\"page_view\\", \\"purchase\\")"},"eventProperties":{"type":"string","required":false,"visibility":"user-or-llm","description":"JSON object of custom event properties"},"userProperties":{"type":"string","required":false,"visibility":"user-or-llm","description":"JSON object of user properties to set (supports $set, $setOnce, $add, $append, $unset)"},"time":{"type":"string","required":false,"visibility":"user-or-llm","description":"Event timestamp in milliseconds since epoch"},"sessionId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Session start time in milliseconds since epoch"},"insertId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Unique ID for deduplication (within 7-day window)"},"appVersion":{"type":"string","required":false,"visibility":"user-or-llm","description":"Application version string"},"platform":{"type":"string","required":false,"visibility":"user-or-llm","description":"Platform (e.g., \\"Web\\", \\"iOS\\", \\"Android\\")"},"country":{"type":"string","required":false,"visibility":"user-or-llm","description":"Two-letter country code"},"language":{"type":"string","required":false,"visibility":"user-or-llm","description":"Language code (e.g., \\"en\\")"},"ip":{"type":"string","required":false,"visibility":"user-or-llm","description":"IP address for geo-location"},"price":{"type":"string","required":false,"visibility":"user-or-llm","description":"Price of the item purchased"},"quantity":{"type":"string","required":false,"visibility":"user-or-llm","description":"Quantity of items purchased"},"revenue":{"type":"string","required":false,"visibility":"user-or-llm","description":"Revenue amount"},"productId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Product identifier"},"revenueType":{"type":"string","required":false,"visibility":"user-or-llm","description":"Revenue type (e.g., \\"purchase\\", \\"refund\\")"},"dataResidency":{"type":"string","required":false,"visibility":"user-or-llm","description":"Data residency region: \\"us\\" (default) or \\"eu\\""}},"hostedApiKey":"none"},"amplitude_user_activity":{"id":"amplitude_user_activity","name":"Amplitude User Activity","description":"Get the event stream for a specific user by their Amplitude ID.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude API Key"},"secretKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude Secret Key"},"amplitudeId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Amplitude internal user ID"},"offset":{"type":"string","required":false,"visibility":"user-or-llm","description":"Offset for pagination (default 0)"},"limit":{"type":"string","required":false,"visibility":"user-or-llm","description":"Maximum number of events to return (default 1000, max 1000)"},"direction":{"type":"string","required":false,"visibility":"user-or-llm","description":"Sort direction: \\"latest\\" or \\"earliest\\" (default: latest)"},"dataResidency":{"type":"string","required":false,"visibility":"user-or-llm","description":"Data residency region: \\"us\\" (default) or \\"eu\\""}},"hostedApiKey":"none"},"amplitude_user_profile":{"id":"amplitude_user_profile","name":"Amplitude User Profile","description":"Get a user profile including properties, cohort memberships, and computed properties. Not available for EU data-residency projects.","version":"1.0.0","params":{"secretKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude Secret Key"},"userId":{"type":"string","required":false,"visibility":"user-or-llm","description":"External user ID (required if no device_id)"},"deviceId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Device ID (required if no user_id)"},"getAmpProps":{"type":"string","required":false,"visibility":"user-or-llm","description":"Include Amplitude user properties (true/false, default: false)"},"getCohortIds":{"type":"string","required":false,"visibility":"user-or-llm","description":"Include cohort IDs the user belongs to (true/false, default: false)"},"getComputations":{"type":"string","required":false,"visibility":"user-or-llm","description":"Include computed user properties (true/false, default: false)"}},"hostedApiKey":"none"},"amplitude_user_search":{"id":"amplitude_user_search","name":"Amplitude User Search","description":"Search for a user by User ID, Device ID, or Amplitude ID using the Dashboard REST API.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude API Key"},"secretKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude Secret Key"},"user":{"type":"string","required":true,"visibility":"user-or-llm","description":"User ID, Device ID, or Amplitude ID to search for"},"dataResidency":{"type":"string","required":false,"visibility":"user-or-llm","description":"Data residency region: \\"us\\" (default) or \\"eu\\""}},"hostedApiKey":"none"},"apify_get_dataset_items":{"id":"apify_get_dataset_items","name":"APIFY Get Dataset Items","description":"Retrieve items stored in an APIFY dataset","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"APIFY API token from console.apify.com/account#/integrations"},"datasetId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Dataset ID to read items from. Example: \\"9RnD3Pql2vGZkc5H5\\""},"itemLimit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Max items to return (1-250000). Default: all items. Example: 500"},"offset":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to skip at the start. Default: 0"},"fields":{"type":"string","required":false,"visibility":"user-or-llm","description":"Comma-separated list of fields to include. Example: \\"title,url,price\\""}},"hostedApiKey":"none"},"apify_get_run":{"id":"apify_get_run","name":"APIFY Get Run","description":"Get the status and details of an APIFY actor run","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"APIFY API token from console.apify.com/account#/integrations"},"runId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Actor run ID to fetch. Example: \\"HG7ML7M8z78YcAPEB\\""}},"hostedApiKey":"none"},"apify_run_actor_async":{"id":"apify_run_actor_async","name":"APIFY Run Actor (Async)","description":"Run an APIFY actor asynchronously with polling for long-running tasks","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"APIFY API token from console.apify.com/account#/integrations"},"actorId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Actor ID or username/actor-name. Examples: \\"apify/web-scraper\\", \\"janedoe/my-actor\\", \\"moJRLRc85AitArpNN\\""},"input":{"type":"string","required":false,"visibility":"user-or-llm","description":"Actor input as JSON string. Example: {\\"startUrls\\": [{\\"url\\": \\"https://example.com\\"}], \\"maxPages\\": 10}"},"waitForFinish":{"type":"number","required":false,"visibility":"user-or-llm","description":"Initial wait time in seconds (0-60) before polling starts. Example: 30"},"itemLimit":{"type":"number","required":false,"default":100,"visibility":"user-or-llm","description":"Max dataset items to fetch (1-250000). Default: 100. Example: 500"},"memory":{"type":"number","required":false,"visibility":"user-or-llm","description":"Memory in megabytes allocated for the actor run (128-32768). Example: 1024 for 1GB, 2048 for 2GB"},"timeout":{"type":"number","required":false,"visibility":"user-or-llm","description":"Timeout in seconds for the actor run. Example: 300 for 5 minutes, 3600 for 1 hour"},"build":{"type":"string","required":false,"visibility":"user-or-llm","description":"Actor build to run. Examples: \\"latest\\", \\"beta\\", \\"1.2.3\\", \\"build-tag-name\\""}},"hostedApiKey":"none"},"apify_run_actor_sync":{"id":"apify_run_actor_sync","name":"APIFY Run Actor (Sync)","description":"Run an APIFY actor synchronously and get results (max 5 minutes)","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"APIFY API token from console.apify.com/account#/integrations"},"actorId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Actor ID or username/actor-name. Examples: \\"apify/web-scraper\\", \\"janedoe/my-actor\\", \\"moJRLRc85AitArpNN\\""},"input":{"type":"string","required":false,"visibility":"user-or-llm","description":"Actor input as JSON string. Example: {\\"startUrls\\": [{\\"url\\": \\"https://example.com\\"}], \\"maxPages\\": 10}"},"memory":{"type":"number","required":false,"visibility":"user-or-llm","description":"Memory in megabytes allocated for the actor run (128-32768). Example: 1024 for 1GB, 2048 for 2GB"},"timeout":{"type":"number","required":false,"visibility":"user-or-llm","description":"Timeout in seconds for the actor run. Example: 300 for 5 minutes, 3600 for 1 hour"},"build":{"type":"string","required":false,"visibility":"user-or-llm","description":"Actor build to run. Examples: \\"latest\\", \\"beta\\", \\"1.2.3\\", \\"build-tag-name\\""}},"hostedApiKey":"none"},"apify_run_task":{"id":"apify_run_task","name":"APIFY Run Task","description":"Run a saved APIFY actor task synchronously and get dataset items (max 5 minutes)","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"APIFY API token from console.apify.com/account#/integrations"},"taskId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Task ID or username/task-name. Examples: \\"janedoe/my-task\\", \\"moJRLRc85AitArpNN\\""},"input":{"type":"string","required":false,"visibility":"user-or-llm","description":"JSON string that overrides the task\'s saved input. Example: {\\"startUrls\\": [{\\"url\\": \\"https://example.com\\"}]}"},"itemLimit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Max dataset items to return (1-250000). Example: 500"},"memory":{"type":"number","required":false,"visibility":"user-or-llm","description":"Memory in megabytes allocated for the run (128-32768). Example: 1024 for 1GB"},"timeout":{"type":"number","required":false,"visibility":"user-or-llm","description":"Timeout in seconds for the run. Example: 300 for 5 minutes"},"build":{"type":"string","required":false,"visibility":"user-or-llm","description":"Actor build to run. Examples: \\"latest\\", \\"beta\\", \\"1.2.3\\""}},"hostedApiKey":"none"},"apollo_account_bulk_create":{"id":"apollo_account_bulk_create","name":"Apollo Bulk Create Accounts","description":"Create up to 100 accounts at once in your Apollo database. Set run_dedupe=true to deduplicate by domain, organization_id, and name. Master key required.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key (master key required)"},"accounts":{"type":"array","required":true,"visibility":"user-or-llm","description":"Array of accounts to create (max 100). Each account should include a name, and may optionally include domain, phone, phone_status_cd, raw_address, owner_id, linkedin_url, facebook_url, twitter_url, salesforce_id, and hubspot_id."},"append_label_names":{"type":"array","required":false,"visibility":"user-only","description":"Array of label names to add to ALL accounts in this request"},"run_dedupe":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"When true, performs aggressive deduplication by domain, organization_id, and name (defaults to false)"}},"hostedApiKey":"none"},"apollo_account_bulk_update":{"id":"apollo_account_bulk_update","name":"Apollo Bulk Update Accounts","description":"Update up to 1000 existing accounts at once in your Apollo database (higher limit than contacts!). Each account must include an id field. Master key required.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key (master key required)"},"account_ids":{"type":"array","required":false,"visibility":"user-or-llm","description":"Array of account IDs to update with the same values (max 1000). Use with name/owner_id for uniform updates. Use either this OR account_attributes."},"name":{"type":"string","required":false,"visibility":"user-or-llm","description":"When using account_ids, apply this name to all accounts"},"owner_id":{"type":"string","required":false,"visibility":"user-or-llm","description":"When using account_ids, apply this owner to all accounts"},"account_stage_id":{"type":"string","required":false,"visibility":"user-or-llm","description":"When using account_ids, apply this account stage to all accounts"},"account_attributes":{"type":"json","required":false,"visibility":"user-or-llm","description":"Array of account objects with individual updates (each must include id). Example: [{\\"id\\": \\"acc1\\", \\"name\\": \\"Acme\\", \\"owner_id\\": \\"u1\\", \\"account_stage_id\\": \\"s1\\", \\"typed_custom_fields\\": {\\"field_id\\": \\"value\\"}}]"},"async":{"type":"boolean","required":false,"visibility":"user-only","description":"When true, processes the update asynchronously. Only supported when using account_ids; returns 422 if used with account_attributes."}},"hostedApiKey":"none"},"apollo_account_create":{"id":"apollo_account_create","name":"Apollo Create Account","description":"Create a new account (company) in your Apollo database","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key (master key required)"},"name":{"type":"string","required":true,"visibility":"user-or-llm","description":"Company name (e.g., \\"Acme Corporation\\")"},"domain":{"type":"string","required":false,"visibility":"user-or-llm","description":"Company domain without www. prefix (e.g., \\"acme.com\\")"},"phone":{"type":"string","required":false,"visibility":"user-or-llm","description":"Primary phone number for the account"},"owner_id":{"type":"string","required":false,"visibility":"user-only","description":"Apollo user ID of the account owner"},"account_stage_id":{"type":"string","required":false,"visibility":"user-only","description":"Apollo ID for the account stage to assign this account to"},"raw_address":{"type":"string","required":false,"visibility":"user-or-llm","description":"Corporate location (e.g., \\"San Francisco, CA, USA\\")"},"typed_custom_fields":{"type":"json","required":false,"visibility":"user-only","description":"Custom field values as { custom_field_id: value } map"}},"hostedApiKey":"none"},"apollo_account_search":{"id":"apollo_account_search","name":"Apollo Search Accounts","description":"Search your team\'s accounts in Apollo. Display limit: 50,000 records (100 records per page, 500 pages max). Use filters to narrow results. Master key required.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key (master key required)"},"q_organization_name":{"type":"string","required":false,"visibility":"user-or-llm","description":"Filter accounts by organization name (partial-match search)"},"account_stage_ids":{"type":"array","required":false,"visibility":"user-only","description":"Filter by account stage IDs"},"account_label_ids":{"type":"array","required":false,"visibility":"user-only","description":"Filter by account label IDs"},"sort_by_field":{"type":"string","required":false,"visibility":"user-or-llm","description":"Sort field: \\"account_last_activity_date\\", \\"account_created_at\\", or \\"account_updated_at\\""},"sort_ascending":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Sort ascending when true. Defaults to descending."},"page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Page number for pagination (e.g., 1, 2, 3)"},"per_page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Results per page, max 100 (e.g., 25, 50, 100)"}},"hostedApiKey":"none"},"apollo_account_update":{"id":"apollo_account_update","name":"Apollo Update Account","description":"Update an existing account in your Apollo database","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key"},"account_id":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the account to update (e.g., \\"acc_abc123\\")"},"name":{"type":"string","required":false,"visibility":"user-or-llm","description":"Company name (e.g., \\"Acme Corporation\\")"},"domain":{"type":"string","required":false,"visibility":"user-or-llm","description":"Company domain (e.g., \\"acme.com\\")"},"phone":{"type":"string","required":false,"visibility":"user-or-llm","description":"Company phone number"},"owner_id":{"type":"string","required":false,"visibility":"user-only","description":"Apollo user ID of the account owner"},"account_stage_id":{"type":"string","required":false,"visibility":"user-only","description":"Apollo ID for the account stage to assign this account to"},"raw_address":{"type":"string","required":false,"visibility":"user-or-llm","description":"Corporate location (e.g., \\"San Francisco, CA, USA\\")"},"typed_custom_fields":{"type":"json","required":false,"visibility":"user-only","description":"Custom field values as { custom_field_id: value } map"}},"hostedApiKey":"none"},"apollo_contact_bulk_create":{"id":"apollo_contact_bulk_create","name":"Apollo Bulk Create Contacts","description":"Create up to 100 contacts at once in your Apollo database. Supports deduplication to prevent creating duplicate contacts. Master key required.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key (master key required)"},"contacts":{"type":"array","required":true,"visibility":"user-or-llm","description":"Array of contacts to create (max 100). Each contact may include first_name, last_name, email, title, organization_name, account_id, owner_id, contact_stage_id, linkedin_url, phone (single string) or phone_numbers (array of {raw_number, position}), contact_emails, typed_custom_fields, and CRM IDs (salesforce_contact_id, hubspot_id, team_id) for cross-system matching"},"append_label_names":{"type":"array","required":false,"visibility":"user-or-llm","description":"Label names to add to all contacts in this request (e.g., [\\"Hot Lead\\"])"},"run_dedupe":{"type":"boolean","required":false,"visibility":"user-only","description":"Enable deduplication to prevent creating duplicate contacts. When true, existing contacts are returned without modification"}},"hostedApiKey":"none"},"apollo_contact_bulk_update":{"id":"apollo_contact_bulk_update","name":"Apollo Bulk Update Contacts","description":"Update up to 100 existing contacts at once in your Apollo database. Each contact must include an id field. Master key required.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key (master key required)"},"contact_ids":{"type":"array","required":false,"visibility":"user-or-llm","description":"Array of contact IDs to update. Must be paired with an object-form contact_attributes specifying the fields to apply uniformly to all listed contacts."},"contact_attributes":{"type":"json","required":false,"visibility":"user-or-llm","description":"Required. Either an array of per-contact updates (each with id) — used standalone — or a single object of attributes to apply to all contact_ids. Supported fields: owner_id, email, organization_name, title, first_name, last_name, account_id, present_raw_address, linkedin_url, typed_custom_fields"},"async":{"type":"boolean","required":false,"visibility":"user-only","description":"Force asynchronous processing. Automatically enabled for >100 contacts"}},"hostedApiKey":"none"},"apollo_contact_create":{"id":"apollo_contact_create","name":"Apollo Create Contact","description":"Create a new contact in your Apollo database","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key"},"first_name":{"type":"string","required":true,"visibility":"user-or-llm","description":"First name of the contact"},"last_name":{"type":"string","required":true,"visibility":"user-or-llm","description":"Last name of the contact"},"email":{"type":"string","required":false,"visibility":"user-or-llm","description":"Email address of the contact"},"title":{"type":"string","required":false,"visibility":"user-or-llm","description":"Job title (e.g., \\"VP of Sales\\", \\"Software Engineer\\")"},"account_id":{"type":"string","required":false,"visibility":"user-or-llm","description":"Apollo account ID to associate with (e.g., \\"acc_abc123\\")"},"owner_id":{"type":"string","required":false,"visibility":"user-only","description":"User ID of the contact owner (accepted by Apollo but not officially documented for POST /contacts)"},"organization_name":{"type":"string","required":false,"visibility":"user-or-llm","description":"Name of the contact\'s employer (e.g., \\"Apollo\\")"},"website_url":{"type":"string","required":false,"visibility":"user-or-llm","description":"Corporate website URL (e.g., \\"https://www.apollo.io/\\")"},"label_names":{"type":"array","required":false,"visibility":"user-or-llm","description":"Lists/labels to add the contact to (e.g., [\\"Prospects\\"])"},"contact_stage_id":{"type":"string","required":false,"visibility":"user-or-llm","description":"Apollo ID for the contact stage"},"present_raw_address":{"type":"string","required":false,"visibility":"user-or-llm","description":"Personal location for the contact (e.g., \\"Atlanta, United States\\")"},"direct_phone":{"type":"string","required":false,"visibility":"user-or-llm","description":"Primary phone number"},"corporate_phone":{"type":"string","required":false,"visibility":"user-or-llm","description":"Work/office phone number"},"mobile_phone":{"type":"string","required":false,"visibility":"user-or-llm","description":"Mobile phone number"},"home_phone":{"type":"string","required":false,"visibility":"user-or-llm","description":"Home phone number"},"other_phone":{"type":"string","required":false,"visibility":"user-or-llm","description":"Alternative phone number"},"typed_custom_fields":{"type":"json","required":false,"visibility":"user-or-llm","description":"Custom field values keyed by custom field ID"},"run_dedupe":{"type":"boolean","required":false,"visibility":"user-only","description":"When true, Apollo deduplicates against existing contacts"}},"hostedApiKey":"none"},"apollo_contact_search":{"id":"apollo_contact_search","name":"Apollo Search Contacts","description":"Search your team\'s contacts in Apollo","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key"},"q_keywords":{"type":"string","required":false,"visibility":"user-or-llm","description":"Keywords to search for"},"contact_stage_ids":{"type":"array","required":false,"visibility":"user-only","description":"Filter by contact stage IDs"},"contact_label_ids":{"type":"array","required":false,"visibility":"user-only","description":"Filter by Apollo label IDs (lists)"},"sort_by_field":{"type":"string","required":false,"visibility":"user-only","description":"Sort field: contact_last_activity_date, contact_email_last_opened_at, contact_email_last_clicked_at, contact_created_at, or contact_updated_at"},"sort_ascending":{"type":"boolean","required":false,"visibility":"user-only","description":"When true, sort ascending. Must be used together with sort_by_field"},"page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Page number for pagination (e.g., 1, 2, 3)"},"per_page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Results per page, max 100 (e.g., 25, 50, 100)"}},"hostedApiKey":"none"},"apollo_contact_update":{"id":"apollo_contact_update","name":"Apollo Update Contact","description":"Update an existing contact in your Apollo database","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key"},"contact_id":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the contact to update (e.g., \\"con_abc123\\")"},"first_name":{"type":"string","required":false,"visibility":"user-or-llm","description":"First name of the contact"},"last_name":{"type":"string","required":false,"visibility":"user-or-llm","description":"Last name of the contact"},"email":{"type":"string","required":false,"visibility":"user-or-llm","description":"Email address"},"title":{"type":"string","required":false,"visibility":"user-or-llm","description":"Job title (e.g., \\"VP of Sales\\", \\"Software Engineer\\")"},"account_id":{"type":"string","required":false,"visibility":"user-or-llm","description":"Apollo account ID (e.g., \\"acc_abc123\\")"},"owner_id":{"type":"string","required":false,"visibility":"user-only","description":"User ID of the contact owner (accepted by Apollo but not officially documented for PATCH /contacts/{id})"},"organization_name":{"type":"string","required":false,"visibility":"user-or-llm","description":"Name of the contact\'s employer (e.g., \\"Apollo\\")"},"website_url":{"type":"string","required":false,"visibility":"user-or-llm","description":"Corporate website URL (e.g., \\"https://www.apollo.io/\\")"},"label_names":{"type":"array","required":false,"visibility":"user-or-llm","description":"Lists/labels to add the contact to (e.g., [\\"Prospects\\"])"},"contact_stage_id":{"type":"string","required":false,"visibility":"user-or-llm","description":"Apollo ID for the contact stage"},"present_raw_address":{"type":"string","required":false,"visibility":"user-or-llm","description":"Personal location for the contact (e.g., \\"Atlanta, United States\\")"},"direct_phone":{"type":"string","required":false,"visibility":"user-or-llm","description":"Primary phone number"},"corporate_phone":{"type":"string","required":false,"visibility":"user-or-llm","description":"Work/office phone number"},"mobile_phone":{"type":"string","required":false,"visibility":"user-or-llm","description":"Mobile phone number"},"home_phone":{"type":"string","required":false,"visibility":"user-or-llm","description":"Home phone number"},"other_phone":{"type":"string","required":false,"visibility":"user-or-llm","description":"Alternative phone number"},"typed_custom_fields":{"type":"json","required":false,"visibility":"user-or-llm","description":"Custom field values keyed by custom field ID"}},"hostedApiKey":"none"},"apollo_email_accounts":{"id":"apollo_email_accounts","name":"Apollo Get Email Accounts","description":"Get list of team\'s linked email accounts in Apollo","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key (master key required)"}},"hostedApiKey":"none"},"apollo_opportunity_create":{"id":"apollo_opportunity_create","name":"Apollo Create Opportunity","description":"Create a new deal for an account in your Apollo database (master key required)","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key (master key required)"},"name":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the opportunity/deal (e.g., \\"Enterprise License - Q1\\")"},"account_id":{"type":"string","required":false,"visibility":"user-or-llm","description":"ID of the account this opportunity belongs to (e.g., \\"acc_abc123\\")"},"amount":{"type":"string","required":false,"visibility":"user-or-llm","description":"Monetary value as a plain number string with no commas or currency symbols"},"opportunity_stage_id":{"type":"string","required":false,"visibility":"user-only","description":"ID of the opportunity stage"},"owner_id":{"type":"string","required":false,"visibility":"user-only","description":"User ID of the opportunity owner"},"closed_date":{"type":"string","required":false,"visibility":"user-or-llm","description":"Expected close date in YYYY-MM-DD format"},"typed_custom_fields":{"type":"json","required":false,"visibility":"user-only","description":"Custom field values as { custom_field_id: value } map"}},"hostedApiKey":"none"},"apollo_opportunity_get":{"id":"apollo_opportunity_get","name":"Apollo Get Opportunity","description":"Retrieve complete details of a specific deal/opportunity by ID","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key"},"opportunity_id":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the opportunity to retrieve (e.g., \\"opp_abc123\\")"}},"hostedApiKey":"none"},"apollo_opportunity_search":{"id":"apollo_opportunity_search","name":"Apollo Search Opportunities","description":"Search and list all deals/opportunities in your team\'s Apollo account","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key"},"sort_by_field":{"type":"string","required":false,"visibility":"user-or-llm","description":"Sort field: \\"amount\\", \\"is_closed\\", or \\"is_won\\""},"page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Page number for pagination (e.g., 1, 2, 3)"},"per_page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Results per page, max 100 (e.g., 25, 50, 100)"}},"hostedApiKey":"none"},"apollo_opportunity_update":{"id":"apollo_opportunity_update","name":"Apollo Update Opportunity","description":"Update an existing deal/opportunity in your Apollo database","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key"},"opportunity_id":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the opportunity to update (e.g., \\"opp_abc123\\")"},"name":{"type":"string","required":false,"visibility":"user-or-llm","description":"Name of the opportunity/deal (e.g., \\"Enterprise License - Q1\\")"},"amount":{"type":"string","required":false,"visibility":"user-or-llm","description":"Monetary value as a plain number string with no commas or currency symbols"},"opportunity_stage_id":{"type":"string","required":false,"visibility":"user-only","description":"ID of the opportunity stage"},"owner_id":{"type":"string","required":false,"visibility":"user-only","description":"User ID of the opportunity owner"},"closed_date":{"type":"string","required":false,"visibility":"user-or-llm","description":"Expected close date in YYYY-MM-DD format"},"typed_custom_fields":{"type":"json","required":false,"visibility":"user-only","description":"Custom field values as { custom_field_id: value } map"}},"hostedApiKey":"none"},"apollo_organization_bulk_enrich":{"id":"apollo_organization_bulk_enrich","name":"Apollo Bulk Organization Enrichment","description":"Enrich data for up to 10 organizations at once using Apollo","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key"},"domains":{"type":"array","required":true,"visibility":"user-or-llm","description":"Array of company domains to enrich (max 10, no www. or @, e.g., [\\"apollo.io\\", \\"stripe.com\\"])"}},"hostedApiKey":"none"},"apollo_organization_enrich":{"id":"apollo_organization_enrich","name":"Apollo Organization Enrichment","description":"Enrich data for a single organization using Apollo","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key"},"domain":{"type":"string","required":true,"visibility":"user-or-llm","description":"Company domain (e.g., \\"apollo.io\\", \\"acme.com\\")"}},"hostedApiKey":"none"},"apollo_organization_search":{"id":"apollo_organization_search","name":"Apollo Organization Search","description":"Search Apollo\'s database for companies using filters","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key"},"organization_locations":{"type":"array","required":false,"visibility":"user-or-llm","description":"Company HQ locations (cities, US states, or countries)"},"organization_not_locations":{"type":"array","required":false,"visibility":"user-or-llm","description":"Exclude companies whose HQ is in these locations"},"organization_num_employees_ranges":{"type":"array","required":false,"visibility":"user-or-llm","description":"Employee count ranges as \\"min,max\\" strings (e.g., [\\"1,10\\", \\"250,500\\", \\"10000,20000\\"])"},"q_organization_keyword_tags":{"type":"array","required":false,"visibility":"user-or-llm","description":"Industry or keyword tags"},"q_organization_name":{"type":"string","required":false,"visibility":"user-or-llm","description":"Organization name to search for (e.g., \\"Acme\\", \\"TechCorp\\")"},"organization_ids":{"type":"array","required":false,"visibility":"user-or-llm","description":"Apollo organization IDs to include (e.g., [\\"5e66b6381e05b4008c8331b8\\"])"},"q_organization_domains_list":{"type":"array","required":false,"visibility":"user-or-llm","description":"Domain names to filter by (no www. or @, up to 1,000)"},"page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Page number for pagination (e.g., 1, 2, 3)"},"per_page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Results per page, max 100 (e.g., 25, 50, 100)"}},"hostedApiKey":"none"},"apollo_people_bulk_enrich":{"id":"apollo_people_bulk_enrich","name":"Apollo Bulk People Enrichment","description":"Enrich data for up to 10 people at once using Apollo","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key"},"people":{"type":"array","required":true,"visibility":"user-or-llm","description":"Array of people to enrich (max 10)"},"reveal_personal_emails":{"type":"boolean","required":false,"visibility":"user-only","description":"Reveal personal email addresses (uses credits)"},"reveal_phone_number":{"type":"boolean","required":false,"visibility":"user-only","description":"Reveal phone numbers (uses credits, requires webhook_url)"},"webhook_url":{"type":"string","required":false,"visibility":"user-only","description":"Webhook URL for async phone number delivery (required when reveal_phone_number is true)"}},"hostedApiKey":"none"},"apollo_people_enrich":{"id":"apollo_people_enrich","name":"Apollo People Enrichment","description":"Enrich data for a single person using Apollo","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key"},"first_name":{"type":"string","required":false,"visibility":"user-or-llm","description":"First name of the person"},"last_name":{"type":"string","required":false,"visibility":"user-or-llm","description":"Last name of the person"},"name":{"type":"string","required":false,"visibility":"user-or-llm","description":"Full name of the person (alternative to first_name/last_name)"},"id":{"type":"string","required":false,"visibility":"user-or-llm","description":"Apollo ID for the person"},"hashed_email":{"type":"string","required":false,"visibility":"user-or-llm","description":"MD5 or SHA-256 hashed email"},"email":{"type":"string","required":false,"visibility":"user-or-llm","description":"Email address of the person"},"organization_name":{"type":"string","required":false,"visibility":"user-or-llm","description":"Company name where the person works"},"domain":{"type":"string","required":false,"visibility":"user-or-llm","description":"Company domain (e.g., \\"apollo.io\\", \\"acme.com\\")"},"linkedin_url":{"type":"string","required":false,"visibility":"user-or-llm","description":"LinkedIn profile URL"},"reveal_personal_emails":{"type":"boolean","required":false,"visibility":"user-only","description":"Reveal personal email addresses (uses credits)"},"reveal_phone_number":{"type":"boolean","required":false,"visibility":"user-only","description":"Reveal phone numbers (uses credits, requires webhook_url)"},"webhook_url":{"type":"string","required":false,"visibility":"user-only","description":"Webhook URL for async phone number delivery (required when reveal_phone_number is true)"}},"hostedApiKey":"none"},"apollo_people_search":{"id":"apollo_people_search","name":"Apollo People Search","description":"Search Apollo\'s database for people using demographic filters","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key"},"person_titles":{"type":"array","required":false,"visibility":"user-or-llm","description":"Job titles to search for (e.g., [\\"CEO\\", \\"VP of Sales\\"])"},"include_similar_titles":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Whether to return people with job titles similar to person_titles"},"person_locations":{"type":"array","required":false,"visibility":"user-or-llm","description":"Locations to search in (e.g., [\\"San Francisco, CA\\", \\"New York, NY\\"])"},"person_seniorities":{"type":"array","required":false,"visibility":"user-or-llm","description":"Seniority levels (one of: owner, founder, c_suite, partner, vp, head, director, manager, senior, entry, intern)"},"organization_ids":{"type":"array","required":false,"visibility":"user-or-llm","description":"Apollo organization IDs to filter by (e.g., [\\"5e66b6381e05b4008c8331b8\\"])"},"organization_names":{"type":"array","required":false,"visibility":"user-or-llm","description":"Company names to search within (legacy filter)"},"organization_locations":{"type":"array","required":false,"visibility":"user-or-llm","description":"Headquarters locations of the people\'s current employer (e.g., [\'texas\', \'tokyo\', \'spain\'])"},"q_organization_domains_list":{"type":"array","required":false,"visibility":"user-or-llm","description":"Employer domain names (e.g., [\\"apollo.io\\", \\"microsoft.com\\"]) — up to 1,000, no www. or @"},"organization_num_employees_ranges":{"type":"array","required":false,"visibility":"user-or-llm","description":"Employee count ranges for the person\'s current employer. Each entry is \\"min,max\\" (e.g., [\\"1,10\\", \\"250,500\\", \\"10000,20000\\"])"},"contact_email_status":{"type":"array","required":false,"visibility":"user-or-llm","description":"Email statuses to filter by: \\"verified\\", \\"unverified\\", \\"likely to engage\\", \\"unavailable\\""},"q_keywords":{"type":"string","required":false,"visibility":"user-or-llm","description":"Keywords to search for"},"page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Page number for pagination, default 1 (e.g., 1, 2, 3)"},"per_page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Results per page, default 25, max 100 (e.g., 25, 50, 100)"}},"hostedApiKey":"none"},"apollo_sequence_add_contacts":{"id":"apollo_sequence_add_contacts","name":"Apollo Add Contacts to Sequence","description":"Add contacts to an Apollo sequence","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key (master key required)"},"sequence_id":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the sequence to add contacts to (e.g., \\"seq_abc123\\")"},"contact_ids":{"type":"array","required":false,"visibility":"user-or-llm","description":"Array of contact IDs to add to the sequence (e.g., [\\"con_abc123\\", \\"con_def456\\"]). Either contact_ids or label_names must be provided."},"label_names":{"type":"array","required":false,"visibility":"user-or-llm","description":"Array of label names to identify contacts to add to the sequence. Either contact_ids or label_names must be provided."},"send_email_from_email_account_id":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the email account to send from. Use the Get Email Accounts operation to look this up."},"send_email_from_email_address":{"type":"string","required":false,"visibility":"user-only","description":"Specific email address to send from within the email account."},"sequence_no_email":{"type":"boolean","required":false,"visibility":"user-only","description":"Add contacts even if they have no email address"},"sequence_unverified_email":{"type":"boolean","required":false,"visibility":"user-only","description":"Add contacts with unverified email addresses"},"sequence_job_change":{"type":"boolean","required":false,"visibility":"user-only","description":"Add contacts who recently changed jobs"},"sequence_active_in_other_campaigns":{"type":"boolean","required":false,"visibility":"user-only","description":"Add contacts active in other campaigns"},"sequence_finished_in_other_campaigns":{"type":"boolean","required":false,"visibility":"user-only","description":"Add contacts who finished other campaigns"},"sequence_same_company_in_same_campaign":{"type":"boolean","required":false,"visibility":"user-only","description":"Add contacts even if others from the same company are in the sequence"},"contacts_without_ownership_permission":{"type":"boolean","required":false,"visibility":"user-only","description":"Add contacts without ownership permission"},"add_if_in_queue":{"type":"boolean","required":false,"visibility":"user-only","description":"Add contacts even if they are in the queue"},"contact_verification_skipped":{"type":"boolean","required":false,"visibility":"user-only","description":"Skip contact verification when adding"},"user_id":{"type":"string","required":false,"visibility":"user-only","description":"ID of the user performing the action"},"status":{"type":"string","required":false,"visibility":"user-only","description":"Initial status for added contacts: \\"active\\" or \\"paused\\""},"auto_unpause_at":{"type":"string","required":false,"visibility":"user-only","description":"ISO 8601 datetime to automatically unpause contacts"}},"hostedApiKey":"none"},"apollo_sequence_search":{"id":"apollo_sequence_search","name":"Apollo Search Sequences","description":"Search for sequences/campaigns in your team\'s Apollo account (master key required)","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key (master key required)"},"q_name":{"type":"string","required":false,"visibility":"user-or-llm","description":"Search sequences by name (e.g., \\"Outbound Q1\\", \\"Follow-up\\")"},"page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Page number for pagination (e.g., 1, 2, 3)"},"per_page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Results per page, max 100 (e.g., 25, 50, 100)"}},"hostedApiKey":"none"},"apollo_task_create":{"id":"apollo_task_create","name":"Apollo Create Task","description":"Create one or more tasks in Apollo (one task per contact_id, master key required)","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key (master key required)"},"user_id":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the Apollo user the task is assigned to"},"contact_ids":{"type":"array","required":true,"visibility":"user-or-llm","description":"Array of contact IDs. One task is created per contact."},"priority":{"type":"string","required":false,"visibility":"user-or-llm","description":"Task priority: \\"high\\", \\"medium\\", or \\"low\\" (defaults to \\"medium\\")"},"due_at":{"type":"string","required":true,"visibility":"user-or-llm","description":"Due date/time in ISO 8601 format (e.g., \\"2024-12-31T23:59:59Z\\")"},"type":{"type":"string","required":true,"visibility":"user-or-llm","description":"Task type: \\"call\\", \\"outreach_manual_email\\", \\"linkedin_step_connect\\", \\"linkedin_step_message\\", \\"linkedin_step_view_profile\\", \\"linkedin_step_interact_post\\", or \\"action_item\\""},"status":{"type":"string","required":true,"visibility":"user-or-llm","description":"Task status: \\"scheduled\\", \\"completed\\", or \\"skipped\\""},"note":{"type":"string","required":false,"visibility":"user-or-llm","description":"Free-form note providing context for the task"}},"hostedApiKey":"none"},"apollo_task_search":{"id":"apollo_task_search","name":"Apollo Search Tasks","description":"Search for tasks in Apollo","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key (master key required)"},"sort_by_field":{"type":"string","required":false,"visibility":"user-or-llm","description":"Sort field: \\"task_due_at\\" or \\"task_priority\\""},"open_factor_names":{"type":"array","required":false,"visibility":"user-or-llm","description":"Filter by status. Common values: [\\"task_types\\"] for open tasks, [\\"task_completed_at\\"] for completed tasks."},"page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Page number for pagination (e.g., 1, 2, 3)"},"per_page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Results per page, max 100 (e.g., 25, 50, 100)"}},"hostedApiKey":"none"},"appconfig_create_application":{"id":"appconfig_create_application","name":"AppConfig Create Application","description":"Create an application in AWS AppConfig","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"name":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the application to create"},"description":{"type":"string","required":false,"visibility":"user-or-llm","description":"Description of the application"}},"hostedApiKey":"none"},"appconfig_create_configuration_profile":{"id":"appconfig_create_configuration_profile","name":"AppConfig Create Configuration Profile","description":"Create a configuration profile in an AWS AppConfig application","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID to create the configuration profile in"},"name":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the configuration profile"},"locationUri":{"type":"string","required":true,"visibility":"user-or-llm","description":"Where the configuration is stored. Use \\"hosted\\" for AppConfig-hosted configurations, or an SSM/S3 URI"},"description":{"type":"string","required":false,"visibility":"user-or-llm","description":"Description of the configuration profile"},"retrievalRoleArn":{"type":"string","required":false,"visibility":"user-or-llm","description":"ARN of an IAM role to retrieve the configuration (required for non-hosted URIs)"},"type":{"type":"string","required":false,"visibility":"user-or-llm","description":"Profile type: AWS.Freeform (default) or AWS.AppConfig.FeatureFlags"}},"hostedApiKey":"none"},"appconfig_create_environment":{"id":"appconfig_create_environment","name":"AppConfig Create Environment","description":"Create an environment for an AWS AppConfig application","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID to create the environment in"},"name":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the environment to create"},"description":{"type":"string","required":false,"visibility":"user-or-llm","description":"Description of the environment"}},"hostedApiKey":"none"},"appconfig_create_hosted_configuration_version":{"id":"appconfig_create_hosted_configuration_version","name":"AppConfig Create Hosted Configuration Version","description":"Create a new hosted configuration version for an AppConfig configuration profile","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID that owns the configuration profile"},"configurationProfileId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The configuration profile ID to add the version to"},"content":{"type":"string","required":true,"visibility":"user-or-llm","description":"The configuration content (e.g., a JSON or YAML document)"},"contentType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Content type of the configuration (e.g., application/json, text/plain)"},"description":{"type":"string","required":false,"visibility":"user-or-llm","description":"Description of the configuration version"},"latestVersionNumber":{"type":"number","required":false,"visibility":"user-or-llm","description":"The version number of the latest version, used for optimistic concurrency"},"versionLabel":{"type":"string","required":false,"visibility":"user-or-llm","description":"A user-defined label for the configuration version"}},"hostedApiKey":"none"},"appconfig_delete_application":{"id":"appconfig_delete_application","name":"AppConfig Delete Application","description":"Delete an AWS AppConfig application","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID to delete"}},"hostedApiKey":"none"},"appconfig_delete_configuration_profile":{"id":"appconfig_delete_configuration_profile","name":"AppConfig Delete Configuration Profile","description":"Delete an AWS AppConfig configuration profile","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID that owns the configuration profile"},"configurationProfileId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The configuration profile ID to delete"}},"hostedApiKey":"none"},"appconfig_delete_environment":{"id":"appconfig_delete_environment","name":"AppConfig Delete Environment","description":"Delete an AWS AppConfig environment","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID that owns the environment"},"environmentId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The environment ID to delete"}},"hostedApiKey":"none"},"appconfig_delete_hosted_configuration_version":{"id":"appconfig_delete_hosted_configuration_version","name":"AppConfig Delete Hosted Configuration Version","description":"Delete a specific hosted configuration version from an AppConfig profile","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID that owns the configuration profile"},"configurationProfileId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The configuration profile ID that owns the version"},"versionNumber":{"type":"number","required":true,"visibility":"user-or-llm","description":"The version number to delete"}},"hostedApiKey":"none"},"appconfig_get_application":{"id":"appconfig_get_application","name":"AppConfig Get Application","description":"Get details about a single AWS AppConfig application","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID to retrieve"}},"hostedApiKey":"none"},"appconfig_get_configuration":{"id":"appconfig_get_configuration","name":"AppConfig Get Configuration","description":"Retrieve the latest deployed configuration for an AppConfig application, environment, and profile","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID or name to retrieve configuration for"},"environmentId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The environment ID or name to retrieve configuration for"},"configurationProfileId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The configuration profile ID or name to retrieve"}},"hostedApiKey":"none"},"appconfig_get_configuration_profile":{"id":"appconfig_get_configuration_profile","name":"AppConfig Get Configuration Profile","description":"Get details about a single AWS AppConfig configuration profile","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID that owns the configuration profile"},"configurationProfileId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The configuration profile ID to retrieve"}},"hostedApiKey":"none"},"appconfig_get_deployment":{"id":"appconfig_get_deployment","name":"AppConfig Get Deployment","description":"Get details about a specific AWS AppConfig deployment","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID of the deployment"},"environmentId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The environment ID of the deployment"},"deploymentNumber":{"type":"number","required":true,"visibility":"user-or-llm","description":"The sequence number of the deployment"}},"hostedApiKey":"none"},"appconfig_get_environment":{"id":"appconfig_get_environment","name":"AppConfig Get Environment","description":"Get details about a single AWS AppConfig environment","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID that owns the environment"},"environmentId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The environment ID to retrieve"}},"hostedApiKey":"none"},"appconfig_get_hosted_configuration_version":{"id":"appconfig_get_hosted_configuration_version","name":"AppConfig Get Hosted Configuration Version","description":"Retrieve a specific hosted configuration version from an AppConfig profile","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID that owns the configuration profile"},"configurationProfileId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The configuration profile ID to read the version from"},"versionNumber":{"type":"number","required":true,"visibility":"user-or-llm","description":"The version number to retrieve"}},"hostedApiKey":"none"},"appconfig_list_applications":{"id":"appconfig_list_applications","name":"AppConfig List Applications","description":"List applications in AWS AppConfig","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"maxResults":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of applications to return (1-50)"},"nextToken":{"type":"string","required":false,"visibility":"user-or-llm","description":"Pagination token from a previous response"}},"hostedApiKey":"none"},"appconfig_list_configuration_profiles":{"id":"appconfig_list_configuration_profiles","name":"AppConfig List Configuration Profiles","description":"List configuration profiles for an AWS AppConfig application","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID that owns the configuration profiles"},"maxResults":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of configuration profiles to return (1-50)"},"nextToken":{"type":"string","required":false,"visibility":"user-or-llm","description":"Pagination token from a previous response"}},"hostedApiKey":"none"},"appconfig_list_deployment_strategies":{"id":"appconfig_list_deployment_strategies","name":"AppConfig List Deployment Strategies","description":"List deployment strategies available in AWS AppConfig","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"maxResults":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of deployment strategies to return (1-50)"},"nextToken":{"type":"string","required":false,"visibility":"user-or-llm","description":"Pagination token from a previous response"}},"hostedApiKey":"none"},"appconfig_list_deployments":{"id":"appconfig_list_deployments","name":"AppConfig List Deployments","description":"List deployments for an AWS AppConfig environment","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID of the deployments"},"environmentId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The environment ID of the deployments"},"maxResults":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of deployments to return (1-50)"},"nextToken":{"type":"string","required":false,"visibility":"user-or-llm","description":"Pagination token from a previous response"}},"hostedApiKey":"none"},"appconfig_list_environments":{"id":"appconfig_list_environments","name":"AppConfig List Environments","description":"List environments for an AWS AppConfig application","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID that owns the environments"},"maxResults":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of environments to return (1-50)"},"nextToken":{"type":"string","required":false,"visibility":"user-or-llm","description":"Pagination token from a previous response"}},"hostedApiKey":"none"},"appconfig_list_hosted_configuration_versions":{"id":"appconfig_list_hosted_configuration_versions","name":"AppConfig List Hosted Configuration Versions","description":"List hosted configuration versions for an AWS AppConfig configuration profile","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID that owns the configuration profile"},"configurationProfileId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The configuration profile ID to list versions for"},"maxResults":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of versions to return (1-50)"},"nextToken":{"type":"string","required":false,"visibility":"user-or-llm","description":"Pagination token from a previous response"}},"hostedApiKey":"none"},"appconfig_start_deployment":{"id":"appconfig_start_deployment","name":"AppConfig Start Deployment","description":"Start deploying a configuration version to an AWS AppConfig environment","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID to deploy in"},"environmentId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The environment ID to deploy to"},"deploymentStrategyId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The deployment strategy ID to use"},"configurationProfileId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The configuration profile ID to deploy"},"configurationVersion":{"type":"string","required":true,"visibility":"user-or-llm","description":"The configuration version to deploy"},"description":{"type":"string","required":false,"visibility":"user-or-llm","description":"Description of the deployment"}},"hostedApiKey":"none"},"appconfig_stop_deployment":{"id":"appconfig_stop_deployment","name":"AppConfig Stop Deployment","description":"Stop an in-progress AWS AppConfig deployment","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID of the deployment"},"environmentId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The environment ID of the deployment"},"deploymentNumber":{"type":"number","required":true,"visibility":"user-or-llm","description":"The sequence number of the deployment to stop"}},"hostedApiKey":"none"},"appconfig_update_application":{"id":"appconfig_update_application","name":"AppConfig Update Application","description":"Update the name or description of an AWS AppConfig application","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID to update"},"name":{"type":"string","required":false,"visibility":"user-or-llm","description":"New name for the application"},"description":{"type":"string","required":false,"visibility":"user-or-llm","description":"New description for the application"}},"hostedApiKey":"none"},"appconfig_update_configuration_profile":{"id":"appconfig_update_configuration_profile","name":"AppConfig Update Configuration Profile","description":"Update the name, description, or retrieval role of an AppConfig configuration profile","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID that owns the configuration profile"},"configurationProfileId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The configuration profile ID to update"},"name":{"type":"string","required":false,"visibility":"user-or-llm","description":"New name for the configuration profile"},"description":{"type":"string","required":false,"visibility":"user-or-llm","description":"New description for the configuration profile"},"retrievalRoleArn":{"type":"string","required":false,"visibility":"user-or-llm","description":"New ARN of the IAM role used to retrieve the configuration"}},"hostedApiKey":"none"},"appconfig_update_environment":{"id":"appconfig_update_environment","name":"AppConfig Update Environment","description":"Update the name or description of an AWS AppConfig environment","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID that owns the environment"},"environmentId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The environment ID to update"},"name":{"type":"string","required":false,"visibility":"user-or-llm","description":"New name for the environment"},"description":{"type":"string","required":false,"visibility":"user-or-llm","description":"New description for the environment"}},"hostedApiKey":"none"},"arxiv_get_author_papers":{"id":"arxiv_get_author_papers","name":"ArXiv Get Author Papers","description":"Search for papers by a specific author on ArXiv.","version":"1.0.0","params":{"authorName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Author name to search for"},"maxResults":{"type":"number","required":false,"visibility":"user-only","description":"Maximum number of results to return (default: 10, max: 2000)"}},"hostedApiKey":"none"},"arxiv_get_paper":{"id":"arxiv_get_paper","name":"ArXiv Get Paper","description":"Get detailed information about a specific ArXiv paper by its ID.","version":"1.0.0","params":{"paperId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ArXiv paper ID (e.g., \\"1706.03762\\")"}},"hostedApiKey":"none"},"arxiv_search":{"id":"arxiv_search","name":"ArXiv Search","description":"Search for academic papers on ArXiv by keywords, authors, titles, or other fields.","version":"1.0.0","params":{"searchQuery":{"type":"string","required":true,"visibility":"user-or-llm","description":"The search query to execute"},"searchField":{"type":"string","required":false,"visibility":"user-only","description":"Field to search in: all, ti (title), au (author), abs (abstract), co (comment), jr (journal), cat (category), rn (report number)"},"maxResults":{"type":"number","required":false,"visibility":"user-only","description":"Maximum number of results to return (default: 10, max: 2000)"},"sortBy":{"type":"string","required":false,"visibility":"user-only","description":"Sort by: relevance, lastUpdatedDate, submittedDate (default: relevance)"},"sortOrder":{"type":"string","required":false,"visibility":"user-only","description":"Sort order: ascending, descending (default: descending)"}},"hostedApiKey":"none"},"asana_add_comment":{"id":"asana_add_comment","name":"Asana Add Comment","description":"Add a comment (story) to an Asana task","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token for Asana"},"taskGid":{"type":"string","required":true,"visibility":"user-or-llm","description":"Asana task GID (numeric string)"},"text":{"type":"string","required":true,"visibility":"user-or-llm","description":"The text content of the comment"}},"oauth":{"required":true,"provider":"asana"},"hostedApiKey":"none"},"asana_add_followers":{"id":"asana_add_followers","name":"Asana Add Followers","description":"Add one or more followers to an Asana task","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token for Asana"},"taskGid":{"type":"string","required":true,"visibility":"user-or-llm","description":"GID of the Asana task (numeric string)"},"followers":{"type":"array","required":true,"visibility":"user-or-llm","description":"Array of user GIDs to add as followers to the task"}},"oauth":{"required":true,"provider":"asana"},"hostedApiKey":"none"},"asana_create_project":{"id":"asana_create_project","name":"Asana Create Project","description":"Create a new project in an Asana workspace","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token for Asana"},"workspace":{"type":"string","required":true,"visibility":"user-or-llm","description":"Asana workspace GID (numeric string) where the project will be created"},"name":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the project"},"notes":{"type":"string","required":false,"visibility":"user-or-llm","description":"Notes or description for the project"}},"oauth":{"required":true,"provider":"asana"},"hostedApiKey":"none"},"asana_create_section":{"id":"asana_create_section","name":"Asana Create Section","description":"Create a new section in an Asana project","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token for Asana"},"projectGid":{"type":"string","required":true,"visibility":"user-or-llm","description":"GID of the Asana project (numeric string) to add the section to"},"name":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the section"}},"oauth":{"required":true,"provider":"asana"},"hostedApiKey":"none"},"asana_create_subtask":{"id":"asana_create_subtask","name":"Asana Create Subtask","description":"Create a subtask under an existing Asana task","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token for Asana"},"taskGid":{"type":"string","required":true,"visibility":"user-or-llm","description":"GID of the parent Asana task (numeric string)"},"name":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the subtask"},"notes":{"type":"string","required":false,"visibility":"user-or-llm","description":"Notes or description for the subtask"},"assignee":{"type":"string","required":false,"visibility":"user-or-llm","description":"User GID to assign the subtask to"},"due_on":{"type":"string","required":false,"visibility":"user-or-llm","description":"Due date in YYYY-MM-DD format"}},"oauth":{"required":true,"provider":"asana"},"hostedApiKey":"none"},"asana_create_task":{"id":"asana_create_task","name":"Asana Create Task","description":"Create a new task in Asana","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token for Asana"},"workspace":{"type":"string","required":true,"visibility":"user-or-llm","description":"Asana workspace GID (numeric string) where the task will be created"},"name":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the task"},"notes":{"type":"string","required":false,"visibility":"user-or-llm","description":"Notes or description for the task"},"assignee":{"type":"string","required":false,"visibility":"user-or-llm","description":"User GID to assign the task to"},"due_on":{"type":"string","required":false,"visibility":"user-or-llm","description":"Due date in YYYY-MM-DD format"}},"oauth":{"required":true,"provider":"asana"},"hostedApiKey":"none"},"asana_delete_task":{"id":"asana_delete_task","name":"Asana Delete Task","description":"Delete an Asana task by its GID (moves it to the trash)","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token for Asana"},"taskGid":{"type":"string","required":true,"visibility":"user-or-llm","description":"GID of the Asana task to delete (numeric string)"}},"oauth":{"required":true,"provider":"asana"},"hostedApiKey":"none"},"asana_get_project":{"id":"asana_get_project","name":"Asana Get Project","description":"Retrieve a single Asana project by its GID","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token for Asana"},"projectGid":{"type":"string","required":true,"visibility":"user-or-llm","description":"Asana project GID (numeric string) to retrieve"}},"oauth":{"required":true,"provider":"asana"},"hostedApiKey":"none"},"asana_get_projects":{"id":"asana_get_projects","name":"Asana Get Projects","description":"Retrieve all projects from an Asana workspace","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token for Asana"},"workspace":{"type":"string","required":true,"visibility":"user-or-llm","description":"Asana workspace GID (numeric string) to retrieve projects from"}},"oauth":{"required":true,"provider":"asana"},"hostedApiKey":"none"},"asana_get_task":{"id":"asana_get_task","name":"Asana Get Task","description":"Retrieve a single task by GID or get multiple tasks with filters","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token for Asana"},"taskGid":{"type":"string","required":false,"visibility":"user-or-llm","description":"The globally unique identifier (GID) of the task. If not provided, will get multiple tasks."},"workspace":{"type":"string","required":false,"visibility":"user-or-llm","description":"Asana workspace GID (numeric string) to filter tasks (required when not using taskGid)"},"project":{"type":"string","required":false,"visibility":"user-or-llm","description":"Asana project GID (numeric string) to filter tasks"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of tasks to return (default: 50)"}},"oauth":{"required":true,"provider":"asana"},"hostedApiKey":"none"},"asana_list_sections":{"id":"asana_list_sections","name":"Asana List Sections","description":"List all sections in an Asana project","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token for Asana"},"projectGid":{"type":"string","required":true,"visibility":"user-or-llm","description":"GID of the Asana project (numeric string) to list sections from"}},"oauth":{"required":true,"provider":"asana"},"hostedApiKey":"none"},"asana_list_workspaces":{"id":"asana_list_workspaces","name":"Asana List Workspaces","description":"List all Asana workspaces and organizations the authenticated user belongs to","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token for Asana"}},"oauth":{"required":true,"provider":"asana"},"hostedApiKey":"none"},"asana_search_tasks":{"id":"asana_search_tasks","name":"Asana Search Tasks","description":"Search for tasks in an Asana workspace","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token for Asana"},"workspace":{"type":"string","required":true,"visibility":"user-or-llm","description":"Asana workspace GID (numeric string) to search tasks in"},"text":{"type":"string","required":false,"visibility":"user-or-llm","description":"Text to search for in task names"},"assignee":{"type":"string","required":false,"visibility":"user-or-llm","description":"Filter tasks by assignee user GID"},"projects":{"type":"array","required":false,"visibility":"user-or-llm","description":"Array of Asana project GIDs (numeric strings) to filter tasks by"},"completed":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Filter by completion status"}},"oauth":{"required":true,"provider":"asana"},"hostedApiKey":"none"},"asana_update_task":{"id":"asana_update_task","name":"Asana Update Task","description":"Update an existing task in Asana","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token for Asana"},"taskGid":{"type":"string","required":true,"visibility":"user-or-llm","description":"Asana task GID (numeric string) of the task to update"},"name":{"type":"string","required":false,"visibility":"user-or-llm","description":"Updated name for the task"},"notes":{"type":"string","required":false,"visibility":"user-or-llm","description":"Updated notes or description for the task"},"assignee":{"type":"string","required":false,"visibility":"user-or-llm","description":"Updated assignee user GID"},"completed":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Mark task as completed or not completed"},"due_on":{"type":"string","required":false,"visibility":"user-or-llm","description":"Updated due date in YYYY-MM-DD format"}},"oauth":{"required":true,"provider":"asana"},"hostedApiKey":"none"},"ashby_add_candidate_tag":{"id":"ashby_add_candidate_tag","name":"Ashby Add Candidate Tag","description":"Adds a tag to a candidate in Ashby and returns the updated candidate.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ashby API Key"},"candidateId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The UUID of the candidate to add the tag to"},"tagId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The UUID of the tag to add"}},"hostedApiKey":"none"},"ashby_anonymize_candidate":{"id":"ashby_anonymize_candidate","name":"Ashby Anonymize Candidate","description":"Strips personally identifiable information from a candidate in Ashby. This does not delete the candidate - the record and its applications remain, with the PII removed. Ashby exposes no candidate deletion endpoint; true deletion is UI-only, restricted by role, and limited to a 10-day window. Requires the candidatesWrite permission.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ashby API Key"},"candidateId":{"type":"string","required":true,"visibility":"user-or-llm","description":"UUID of the candidate to anonymize"}},"hostedApiKey":"none"},"ashby_change_application_source":{"id":"ashby_change_application_source","name":"Ashby Change Application Source","description":"Changes the source attributed to an existing application, so programmatically created applications report correctly on the recruiting side. Requires the candidatesWrite permission.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ashby API Key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"UUID of the application whose source should change"},"sourceId":{"type":"string","required":false,"visibility":"user-or-llm","description":"UUID of the source to attribute the application to, as returned by List Sources. Omit only when unsetSource is true."},"unsetSource":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Set true to deliberately clear the application source. Required to unset, so that a missing or empty sourceId cannot wipe attribution by accident."}},"hostedApiKey":"none"},"ashby_change_application_stage":{"id":"ashby_change_application_stage","name":"Ashby Change Application Stage","description":"Moves an application to a different interview stage. Requires an archive reason when moving to an Archived stage.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ashby API Key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The UUID of the application to update the stage of"},"interviewStageId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The UUID of the interview stage to move the application to"},"archiveReasonId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Archive reason UUID. Required when moving to an Archived stage, ignored otherwise"}},"hostedApiKey":"none"},"ashby_create_application":{"id":"ashby_create_application","name":"Ashby Create Application","description":"Creates a new application for a candidate on a job. Optionally specify interview plan, stage, source, and credited user.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ashby API Key"},"candidateId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The UUID of the candidate to consider for the job"},"jobId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The UUID of the job to consider the candidate for"},"interviewPlanId":{"type":"string","required":false,"visibility":"user-or-llm","description":"UUID of the interview plan to use (defaults to the job default plan)"},"interviewStageId":{"type":"string","required":false,"visibility":"user-or-llm","description":"UUID of the interview stage to place the application in (defaults to first Lead stage)"},"sourceId":{"type":"string","required":false,"visibility":"user-or-llm","description":"UUID of the source to set on the application"},"creditedToUserId":{"type":"string","required":false,"visibility":"user-or-llm","description":"UUID of the user the application is credited to"},"createdAt":{"type":"string","required":false,"visibility":"user-or-llm","description":"ISO 8601 timestamp to set as the application creation date (defaults to now)"}},"hostedApiKey":"none"},"ashby_create_candidate":{"id":"ashby_create_candidate","name":"Ashby Create Candidate","description":"Creates a new candidate record in Ashby.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ashby API Key"},"name":{"type":"string","required":true,"visibility":"user-or-llm","description":"The candidate full name"},"email":{"type":"string","required":false,"visibility":"user-or-llm","description":"Primary email address for the candidate"},"phoneNumber":{"type":"string","required":false,"visibility":"user-or-llm","description":"Primary phone number for the candidate"},"linkedInUrl":{"type":"string","required":false,"visibility":"user-or-llm","description":"LinkedIn profile URL"},"githubUrl":{"type":"string","required":false,"visibility":"user-or-llm","description":"GitHub profile URL"},"website":{"type":"string","required":false,"visibility":"user-or-llm","description":"Personal website URL"},"sourceId":{"type":"string","required":false,"visibility":"user-or-llm","description":"UUID of the source to attribute the candidate to"},"creditedToUserId":{"type":"string","required":false,"visibility":"user-or-llm","description":"UUID of the Ashby user to credit with sourcing this candidate"},"createdAt":{"type":"string","required":false,"visibility":"user-or-llm","description":"Backdated creation timestamp in ISO 8601 (e.g. 2024-01-01T00:00:00Z). Defaults to now."},"alternateEmailAddresses":{"type":"json","required":false,"visibility":"user-or-llm","description":"Array of additional email address strings to add to the candidate, e.g. [\\"a@x.com\\",\\"b@y.com\\"]"}},"hostedApiKey":"none"},"ashby_create_note":{"id":"ashby_create_note","name":"Ashby Create Note","description":"Creates a note on a candidate in Ashby. Supports plain text and HTML content (bold, italic, underline, links, lists, code).","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ashby API Key"},"candidateId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The UUID of the candidate to add the note to"},"note":{"type":"string","required":true,"visibility":"user-or-llm","description":"The note content. If noteType is text/html, supports: , , , ,