Developer Platform

Connect organization logs to Snowflake

Set up, verify, operate and troubleshoot organization log export with Snowpipe Streaming.

BuildWithHQ can copy selected log metadata from your organization to a Snowflake account you control. Your app continues to use its own database; Snowflake is an analytics destination. The setup uses eight high-performance Snowpipe Streaming pipes, a dedicated service user and role, and a private key generated and encrypted on the BuildWithHQ server.

Who manages the connection?

Each tenant organization has its own Snowflake destination. Its active owner manages it in Admin → Snowflake. Builders intentionally have support authority: a Builder-managed live API credential can manage the connection when it has explicit Snowflake scopes and is bound to that organization's active owner. A delegated tenant-user credential may use the same routes. Ordinary organization members cannot administer the connection.

The server derives app, organization and user identity from the verified session or credential, then checks current ownership through secured stored procedures. An organization ID in a request body does not grant access. Builder support must select the correct organization principal before calling these routes.

What the Snowflake administrator needs

1. A Snowflake account and permission to create a dedicated service user, role, database, schema and streaming pipes.
2. The preferred organization-account identifier, without a URL. This release supports public Snowflake endpoints.
3. A database, schema, dedicated role and dedicated service-user name. Use simple unquoted identifiers.

Save the destination to generate its key pair on the server. Copy the returned setup SQL into a Snowflake worksheet for an administrator to review and run. The script creates eight tables and pipes and grants USAGE on the database/schema, INSERT on the tables and OPERATE on the pipes. It creates a new SERVICE user and does not replace an existing user. Choose names that are owned by your organization; do not run the script against an unrelated database. No ingestion warehouse or staging bucket is required; your analytics queries use your own query role and warehouse. Snowflake charges apply separately.

Only the public key reaches the browser or API response. Private keys are encrypted on the server. Do not paste private keys, passwords or tokens into configuration fields. For key replacement, save with generateKey: true, run the returned keyRotationSql, and test again.

Step-by-step setup in the tenant app

1. Sign in as an active organization owner and open Admin → Snowflake. If the page says server setup is required, ask your BuildWithHQ operator to install the shared key-protection secret before you continue.
2. In Snowflake Account Details, copy the organization-account identifier, such as exampleco-analytics. Enter only that identifier in BuildWithHQ, without https://, a region suffix, or a browser fragment. Choose a dedicated database, schema, role and service-user name; the form proposes BWHQ_LOGS, ANALYTICS, BWHQ_LOG_WRITER and BWHQ_LOG_SERVICE.
3. Select Save and generate connection key. The destination remains paused. Save the public-key setup SQL locally and have your Snowflake administrator review its database, schema, object names and grants. The downloadable offline checker can confirm that all eight expected tables and pipes are present; it does not execute SQL or authenticate to Snowflake.
4. Run the reviewed SQL in the intended Snowflake account. If a service user with the chosen name already exists, stop and select a new dedicated name or review a controlled migration. Do not grant broad account privileges just to make the connection test pass.
5. Back in BuildWithHQ, select Test connection. Look for Connection test passed; a green HTTP response alone is not proof. The test authenticates and opens a separate empty test channel on each of the eight pipes.
6. Select Enable export and confirm. You can select Export next batch for a manual run; repeat until caught up. For automatic export, confirm that the page reports the scheduled host is available, select a cadence under Automatic export, then select Enable schedule and confirm. The destination must remain verified and enabled. Each completed run shows a safe result and correlation reference in recent history.

What is exported?

Each table stores a source app ID, organization ID, source event ID, a structured RECORD value and export time. The export manifest explicitly excludes passwords, session tokens, prompt/response text, raw URL, email, IP address, record/file contents, raw diagnostic payloads and error-message text. Snowflake retention and access policy are your responsibility after delivery.

Verify delivery and recover a retry

In Recent export history, check each dataset result, batch row count and correlation reference. An empty dataset can complete with zero rows; that does not indicate a failed pipe. A real approved account passed all eight test channels and hosted append checks. A forced local checkpoint failure after Snowflake committed a row left the checkpoint unchanged; the next run recovered the remote offset and completed without duplicating that row. Temporary test rows were removed. This is integration evidence, not a guarantee that every future batch will succeed.

The toolkit includes verify_counts.sql for an authorized read role. Its eight row counts provide a simple destination check; a service user with only streaming permissions need not have SELECT. For exact progress, compare the channel's last_committed_offset_token with BuildWithHQ's source checkpoint. A request acknowledgement is not the commit fence. If a request times out, refresh the page and retry: the stable channel is reopened, its committed offset is read, and only the uncommitted suffix is sent. Do not reset, drop or recreate a pipe/channel while a checkpoint exists without a reviewed recovery plan.

Pause, rotate, or troubleshoot

Pause schedule stops future automatic runs while leaving manual export available. Pause export stops new batches entirely; both preserve confirmed checkpoints, completed history and data already in Snowflake. A run already in progress finishes under its lease before pause takes effect. To rotate the key, use Replace the connection key, run the returned public-key update command on the same dedicated service user, then test again before enabling. Protect the BuildWithHQ server's encryption master key separately from the Logs database; losing that key requires a new connection key.

Give support the correlation reference, dataset and safe result code. Do not send passwords, private keys, JWTs, scoped tokens, browser cookies or entire raw log rows. If an HTTP request succeeds but destination.lastResult is not connected or export_completed, treat the operation as failed.

Public routes and scopes

All routes start with /v1/apps/{saasAppId}/organization/snowflake. The app path is checked against the credential; the organization comes from its verified principal.

Save accepts accountIdentifier, databaseName, schemaName, roleName, principalName, generateKey and, for updates, expectedRowVersion. Creating a destination requires key generation. Other mutations require the latest returned expectedRowVersion; enable/pause also requires an explicit boolean isEnabled. Read again after HTTP 409. Unknown body fields are rejected.

The generated BuildWithHQModuleClient exposes snowflakeGet, snowflakeSave, snowflakeTest, snowflakeSetEnabled, snowflakeSetSchedule and snowflakeExport. Pass mutation payloads under body. Scheduling takes isEnabled, cadenceMinutes (5–1440) and the current schedule.rowVersion when updating an existing schedule. Read again after HTTP 409. schedulingAvailable reports whether a hosted worker has checked in recently; it does not say the schedule is enabled.

Test, enable and export

A saved or rotated connection stays paused until its connection test succeeds. Saving settings invalidates earlier verification. Test/export responses contain safe destination.lastResult codes; HTTP 200 alone does not prove a provider test succeeded. Require connected after testing or export_completed after exporting. HTTP errors use a safe explanation, stable code and correlation ID.

Each export handles up to 100 entries per dataset. Run another batch to continue. Stable channels and committed offsets recover an interrupted batch before advancing local checkpoints. A busy operation, stale revision, rejected rows or regressed remote offset stops progress safely. The history displays the latest 25 completed dataset runs. Pausing preserves history and data already in Snowflake.

Included datasets cover AI usage, login outcomes, page views, record changes, field access/changes, file access and automation job metadata. Passwords, session tokens, prompts, responses, record/file contents and raw diagnostic payloads are excluded. Export does not extend local diagnostic retention or create staged log files.

Current availability and cost

Manual and hosted scheduled export have been exercised against a real approved Snowflake account through a route-owned disposable tenant. Scheduling is available only while a configured worker has checked in recently; if the worker stops, the API reports schedulingAvailable: false and new enable requests are rejected. A schedule does not start when you merely enable the destination. The hosted worker sends at most 100 rows per dataset each cycle, checks the route and lease, and retries from Snowflake's confirmed offset. Private connectivity, managed encryption-master-key rotation and destination moves after checkpoints require a separate operator review.

Snowflake bills ingestion/storage and any queries under its own account terms. BuildWithHQ does not set or control Snowflake prices; review your Snowflake usage and credit limits in that account. A query warehouse is needed when you run analytics queries, even though the ingestion path itself does not require a dedicated warehouse.

Downloads and Developer API Kit

The Snowflake setup toolkit includes this workflow as a README, an offline check_setup.py script and read-only verify_counts.sql. It contains no credentials and never contacts Snowflake. The same files are under snowflake/ in Developer API Kit 1.20.76, alongside the generated OpenAPI contract and typed SDK. Verify the adjacent SHA-256 files before distributing either ZIP.

If protected server key storage is unavailable, ask the operator to install the shared encryption key on the serving API instances. The operator must back it up separately from the Logs database; losing it requires replacing tenant connection keys.

References: account identifiers, key-pair authentication, Streaming REST tutorial, and access control.

Capability review: 2026-09-14. See current features and setup requirements, the generated API Map and first-class module inventory.