Bring your systems in, or query them where they are.
Connectors incrementally pull files and mail from the systems your teams already use into the lakehouse, extracting structure as they go. External databases can instead be registered as native tables and queried in place.
Where content comes from #
Each source syncs incrementally and per workspace, so an initial import is followed by cheap catch-up rather than a full re-crawl.
- Google Drive, SharePoint, OneDrive, Dropbox
- Document libraries synced with their structure intact, ready for search and retrieval.
- S3-compatible object storage
- Buckets you already own, read into the lakehouse or used directly as its backing store.
- Gmail and IMAP
- Mail brought in as searchable, queryable content.
- High-throughput events
- A staged, manifest-committed path for ingestion well beyond ordinary batched writes.
The best migration is often the one you do not do. Register the database and query it where it is.
What happens on the way in #
Unstructured files are turned into structured, retrievable content before they land. Nothing has to be prepared by hand first.
- PDF layout, tables, figures and OCR, plus Office, spreadsheet and image formats
- Content split into overlapping, de-duplicated segments sized for retrieval
- Segments embedded and indexed for semantic search, with bounded retries
- Keyword and vector indexes built on demand and folded into results as data lands
Databases you do not want to move #
That choice matters more than it sounds: documents usually have to be processed and indexed before search or an agent can use them, while systems of record usually should not be copied at all. Register an external relational database and its tables become queryable as if they were native. Work is pushed down to the source system where it can be, so a single query can span the lakehouse and your systems of record without a copy or a nightly job.
-- 'contracts' is a governed lakehouse table.
-- 'erp' is a Postgres database nobody migrated, registered as a source;
-- its tables resolve as an ordinary three-part reference.
SELECT c.customer_id,
c.signed_at,
a.owner
FROM contracts AS c
JOIN "erp"."public"."accounts" AS a USING (customer_id)
WHERE c.signed_at >= now() - interval '90 days'
ORDER BY c.signed_at DESC; One planner, two systems. The predicate is pushed down to the external database in its own SQL dialect, the external side stays read-only, and the row filters and column masks that apply to the lakehouse table apply to the whole query.
Where the data rests #
Tables are held as open, columnar and vector-native datasets over object storage: the platform's own sovereign store by default, or your own S3-compatible backend. Compaction and old-version cleanup run in the background without ever taking data offline, and everything above resolves as a table in the same session.
- Open columnar and vector datasets, versioned and queryable in place
- Fresh writes buffered hot and flushed in the background behind a visibility barrier
- Every change published as a live feed, so search indexes and subscriptions stay current
- Scheduled off-site backup and multi-zone replication for durability
$ psql "postgresql://[email protected]:5432/operations"
=> SELECT site,
count(*) AS open_findings
FROM inspections
JOIN findings USING (inspection_id)
WHERE findings.status = 'open'
GROUP BY site
ORDER BY open_findings DESC
LIMIT 20; Documents, structured records and synced external systems are all queryable as tables. Your dashboard tool connects with the same string.
Keep going #
APIs and protocols
Query everything you have connected, through the protocol whichever client you are holding already speaks.
Open →Integrations
Identity federation, directory sync, business intelligence, automation and object storage in your existing estate.
Open →Platform
Atom in full: the lakehouse, hybrid search, data intelligence and the governance applied on every read.
Open →