Warehouse & Integration
The "last mile" of the deployment — pushing clean platform output into the customer's Snowflake / BigQuery, integrating with their ERP and BI, getting the auth right, owning refresh cadence and backfills.
Why integration is half the deployment
Extraction quality can be perfect, the data model can be elegant, the SQL can be clean — and the deployment still fails because the customer can't get the data into the system where they actually do their work. The "last mile" — warehouse, ERP, BI, identity — is where deployments get stuck, and it's where FDE-DE earns most of their pay.
The work spans three integration zones:
- Warehouse — push clean data into the customer's Snowflake / BigQuery / Databricks / Redshift so their analytics team can build on it.
- System of record — write back to their ERP (SAP, Oracle, Workday, NetSuite) or CLM (Ironclad, DocuSign CLM) so action lives where their process already does.
- BI / consumption — surface insights through their BI tool (Looker, Power BI, Tableau, Hex) so business users see them in the place they already look.
Warehouse delivery
The most common pattern. Push the platform's output tables into the customer's warehouse on a refresh schedule.
Delivery mechanisms
- Direct write — your pipeline writes directly into the customer's warehouse, using credentials they provide. Easiest to set up; some customers' security policies forbid it.
- S3 / GCS staging — your pipeline writes Parquet files to a shared bucket; the customer's pipeline picks up and loads. Decoupled, customer owns the load step.
- Snowflake / BigQuery shares — use the warehouse's native data-sharing feature. Read-only, real-time, no copy. Cleanest when both sides are on the same warehouse.
- Reverse ETL tools — Hightouch, Census. Customer-owned, mid-tier flexibility.
What to put in the warehouse
Three layers, mirroring standard analytics-engineering practice:
- Raw extracted — platform output as-is, JSON sidecar columns for everything not yet promoted.
- Cleaned / typed — typed columns, normalized values, foreign keys resolved.
- Marts — business-friendly tables (e.g.,
agreements_with_renewal_alerts,off_contract_spend_by_supplier) that the customer's analysts can query directly.
Build the marts in dbt models inside the customer's warehouse. The customer's team can extend them after handoff without needing platform engineering involvement.
ERP integration
Often optional but high-value. Writing back to the customer's system of record means the insight lives where their process happens, not just where their dashboard lives.
The major systems and their quirks
| System | What you'll write back | Watch out for |
|---|---|---|
| SAP (ECC / S/4HANA) | Supplier master updates, PO matching, contract refs | BAPI/IDocs are arcane; customer's SAP team often gatekeeps writes |
| Oracle ERP | Supplier hierarchy, contract metadata | Custom flexfields per customer; expect schema variance |
| Coupa / Ariba | Supplier risk scores, contract refs, renewal flags | Modern REST APIs but rate-limited |
| Workday Financials | Supplier setup, contract metadata, accruals | SOAP web services; iframe-style integrations |
| NetSuite | Vendor records, contract IDs, custom fields | Custom record types per customer; SuiteScript quirks |
| Ironclad / DocuSign CLM | Contract metadata, status flags | Modern APIs but customer's contract workflow can break if you push too aggressively |
ERP write-back has political weight. Procurement and IT teams have policies about what touches the system of record. Always go through their change-management process — even if you have credentials and could push tomorrow. Earning trust by respecting their controls pays off when you later need them to move fast.
BI delivery
The customer's BI tool is where business users actually look at the data. The deployment usually delivers three things:
- A semantic layer — LookML, dbt semantic models, or Power BI dataset — that defines the platform's outputs in business terms.
- A starter dashboard — 3–5 dashboards covering the customer's named priorities: renewal pipeline, off-contract spend, supplier consolidation.
- Self-serve enablement — the customer's analysts should be able to extend the dashboards. Document the columns; document the joins.
The BI tool tradeoffs you'll meet
- Looker — strong semantic layer, expensive, slow to iterate on. Common at large enterprises.
- Power BI — ubiquitous in Microsoft-shop enterprises; dataset modeling is real.
- Tableau — strong viz, weaker semantic layer; common in finance/operations teams.
- Hex / Mode — notebook-friendly, faster iteration, less common as the enterprise standard.
- Embedded BI — sometimes the platform embeds its own UI; integration is shallower.
Match the customer's standard. Resist the urge to introduce a tool they don't have.
Auth & identity
The most-debugged part of an enterprise integration. Patterns you'll meet:
- Service accounts — IT issues a service principal scoped to specific operations. Common, secure, requires customer setup.
- OAuth / OIDC — for connector access to customer apps. Manage refresh tokens carefully.
- SAML SSO — for the platform UI; customer's IdP authorizes platform users.
- SCIM provisioning — for user lifecycle. The customer's IdP creates / disables platform users automatically.
- PrivateLink / VPC peering — network-level connectivity to keep traffic off the public internet. Common for regulated industries.
- BYO-key encryption — customer holds the encryption key; platform encrypts at rest with it. Higher-tier security requirement.
The FDE move
Spend a half-day in week one with the customer's IT/security team mapping their auth requirements. Skipping this conversation is how week-six deployments stall.
Refresh cadence & backfills
Refresh cadence
- Daily batch — the default. Cheap, predictable, fits most use cases.
- Hourly — when the customer needs intraday updates (renewal alerts approaching deadlines, real-time spend dashboards).
- Real-time push — webhook-driven, only when actually needed. More expensive to operate.
Backfills
Customers will ask for backfills. Several flavors:
- Historical extraction backfill — extract all their historical contracts up to N years ago. Often the bulk of the first deployment.
- Re-extraction backfill — new model version, re-run extraction across historical corpus. Triggered by platform updates.
- Schema-evolution backfill — new column added, populate it for existing rows.
- Reconciliation backfill — customer's ERP has an authoritative supplier master that contradicts the platform's; reconcile.
Each is operationally distinct. Idempotent pipelines (chapter 6) are what makes backfills routine instead of terrifying.
Interview probes
Show probe 1: "How would you deliver platform output to a customer's Snowflake?"
Four mechanisms. (1) Direct write with credentials they provide — easiest, sometimes forbidden by security policy. (2) S3 staging — write Parquet files; customer loads. Decoupled. (3) Snowflake data share — read-only, real-time, no copy; cleanest when both sides are Snowflake. (4) Reverse-ETL tool like Hightouch — customer-owned. Pick by their security posture and existing patterns. Within the warehouse, ship three layers: raw, cleaned/typed, and business-mart tables in dbt for the customer's analysts to extend.
Show probe 2: "Customer wants writeback to their SAP. How do you approach it?"
Carefully. (1) Spend a full meeting with their SAP team mapping what they'll let you write — BAPI/IDoc, which fields, what record types. (2) Don't write into the ERP without explicit sign-off; the system of record has political weight and IT gatekeeps. (3) Start with read-only — pull supplier master and contract refs out of SAP into the platform; defer writeback until the customer has internal alignment. (4) Plan for SAP-specific quirks: BAPI verbosity, custom Z-tables, IDoc latency. (5) Always go through their change-management process — even if you have credentials.
Show probe 3: "Customer's analysts will own the dashboards after handoff. How do you set them up to succeed?"
Three pieces. (1) Build a semantic layer (LookML / dbt models) that defines the platform's outputs in business terms, not raw column names. (2) Ship 3–5 starter dashboards covering their named priorities, not a generic templated set. (3) Document the joins, the SLA boundaries, and the known gotchas. (4) Pair-session with their analysts during the last two weeks of deployment so they extend dashboards while you're still around to answer questions. Senior signal: not just delivering dashboards but designing the model so they can self-serve.
Show probe 4: "What's the difference between Snowflake data share and S3 staging?"
Data share is read-only access to your tables — no copy, real-time, governed by Snowflake's RBAC. Works when both sides are on Snowflake. S3 staging writes Parquet to a shared bucket; customer loads into their warehouse on their schedule. More flexible (works across warehouses), introduces a copy and a load step (cost, delay). Use data share when both sides are Snowflake and security policy allows; staging when they're on different warehouses or want the load step in their pipeline.
Show probe 5: "Customer asks for real-time updates. How do you respond?"
Push back first. "What decision is real-time enabling that hourly wouldn't?" 80% of "real-time" asks turn out to be "fresh enough to make a daily decision." If genuine real-time is needed (alerts approaching deadlines, fraud-flavored signals), evaluate event-push with webhooks vs streaming infrastructure. The cost is operational complexity — more on-call burden, more monitoring, more failure modes. Make the cost visible to the customer before committing.