Data Engineer Data Architect Platform Team Architecture decision guide for OneLake, platform topology, workload selection, medallion, and event-driven solutions.
OneLake Architecture
Designing a unified, governed data foundation for analytics and AI in Microsoft Fabric.
Architecture Overview
Microsoft Fabric combines specialized analytics workloads with OneLake, a tenant-wide logical data lake. A Fabric deployment follows four levels: tenant → capacity → workspace → item. OneLake provides the shared data foundation beneath these levels, while domains, the OneLake catalog, security, and governance support distributed ownership.
Platform Hierarchy
Identity, administration, governance, and organizational boundary.
Regional compute, performance isolation, and billing boundary.
Ownership, access, collaboration, and application lifecycle boundary.
Deployable workload artifact such as a lakehouse, warehouse, or report.
OneLake is the tenant-wide logical data foundation. Capacities provide compute, while workspaces and items provide ownership and lifecycle boundaries over the data and experiences that use it.
Prefer virtualization through shortcuts when data can remain at its source. Use mirroring when a synchronized analytics-ready copy is needed, and use pipelines or other movement tools when orchestration, transformation, streaming, or unsupported sources require them.
OneLake Architecture Patterns
These patterns are complementary building blocks rather than mutually exclusive choices. Most enterprise implementations combine several of them.
Unified Access, Minimal Replication
Expose data across clouds, external lakes, catalogs, and operational systems through shortcuts or mirroring before creating custom movement pipelines.
Medallion Architecture
Move data through raw, enriched, and curated quality layers with explicit contracts and ownership between each stage.
Domain-Oriented Data Mesh
Let domains own discoverable data products while OneLake, shared policies, and the catalog provide a governed platform foundation.
Analytics and AI Consolidation
Use open OneLake data across engineering, warehousing, BI, data science, and AI workloads instead of maintaining engine-specific copies.
External Data Sharing
Share governed OneLake data with external organizations without relying on unmanaged exports and duplicate files.
Fabric Deployment Patterns
Choosing tenant, capacity, and workspace boundaries for governance, isolation, lifecycle, and cost.
Start with the simplest topology that satisfies your security, performance, geography, and lifecycle requirements. Add boundaries deliberately: every additional capacity or workspace improves some forms of isolation but also increases administration and cost-management complexity.
Use for proofs of concept or small, tightly coupled workloads.
Lowest overhead; weakest lifecycle and workload isolation.Add access, ownership, Git, and deployment boundaries while sharing compute.
Better governance; workloads still contend for capacity.Separate criticality, performance, geography, billing, or operational responsibility.
Stronger isolation; higher cost and administration.Reserve for legal, sovereignty, merger, or hard identity-separation requirements.
Maximum isolation; fragmented governance and collaboration.Define the tenant boundary first, place capacities according to region and isolation needs, use workspaces for ownership and application lifecycle, then organize related workspaces into domains. Avoid using separate tenants or capacities as substitutes for a clear governance model.
Workload Architecture Choices
Selecting the primary Fabric item for each data access and processing pattern.
Choose the primary item by the workload's dominant responsibility. Then combine item types where the solution needs different write, query, semantic, or latency characteristics.
Open-format engineering, data science, and mixed structured or unstructured data.
- Writes
- Batch, streaming, notebooks, pipelines
- Users
- Engineers and data scientists
- Interfaces
- Spark, SQL endpoint, OneLake APIs
T-SQL-first dimensional modeling, relational serving, and governed BI engineering.
- Writes
- Set-based SQL and pipelines
- Users
- SQL developers and BI teams
- Interfaces
- T-SQL and Direct Lake
Transactional applications that need operational writes and integrated analytics.
- Writes
- Row-level application transactions
- Users
- Application developers
- Interfaces
- T-SQL application access
Continuously arriving logs, events, telemetry, and time-series analysis.
- Writes
- Append-heavy event ingestion
- Users
- Operations and real-time analysts
- Interfaces
- KQL and supported T-SQL access
Reusable business measures, governed BI semantics, and report consumption.
- Writes
- Model metadata and refresh operations
- Users
- Analysts and business consumers
- Interfaces
- DAX, Power BI, XMLA
Business concepts and operational context that help people and agents reason in business terms.
- Foundation
- OneLake and semantic models
- Capabilities
- Ontology, graph, planning, and agents
- Users
- Business operations and AI solutions
Common Solution Combinations
Medallion Architecture
The proven data organization pattern for building reliable and scalable lakehouses and warehouses.
What is the Medallion Architecture?
The medallion architecture (also known as the "multi-hop" architecture) is a data design pattern that organizes data into three logical layers: Bronze, Silver, and Gold. Each layer represents an increasing level of data quality and business readiness.
Choose a Layer Topology
Use a separate Lakehouse for Bronze, Silver, and Gold. This keeps open Delta data available across Spark, SQL, Direct Lake, and OneLake APIs.
Use Lakehouses for Bronze and Silver, then a Warehouse for Gold when the serving team prefers T-SQL, dimensional modeling, and warehouse semantics.
Layers can share a workspace, but separate workspaces provide stronger ownership, access, deployment, and governance boundaries. Choose the boundary deliberately based on team structure and lifecycle requirements.
Layer Contracts
Preserve source fidelity and ingestion metadata. Keep data append-oriented and immutable by default, while honoring retention, privacy, and legal deletion requirements.
- Prefer shortcuts when source data can remain in place
- Record source, batch, and ingestion time
- Support replay and recovery
Validate, standardize, deduplicate, enrich, and conform data into reusable domain datasets.
- Publish data-quality outcomes
- Apply common keys and business rules
- Provide stable contracts for downstream products
Shape trusted data for specific analytical and operational consumers.
- Build marts, aggregates, and semantic models
- Optimize for access patterns and Direct Lake
- Apply final security and endorsement
Implementation guidance: Fabric Data Warehouse operating rules
1. Batch the writes
Favor set-based operations over row-by-row processing. Use COPY INTO or Fabric pipelines for Bronze ingestion and CTAS, INSERT...SELECT, or MERGE for transformations. When practical, consolidate source files toward roughly 100 MB to 1 GB each.
2. Make Silver rerunnable
Design transformations to be idempotent so the same input can be processed again without duplicates or corruption. Use staging for complex logic and record quality failures, rejected rows, and rejection reasons.
3. Design Gold for consumption
Shape Gold around business access patterns: star schemas, data marts, wide tables, or aggregations. Build it from trusted Silver data; direct Gold-to-Bronze dependencies should be exceptional.
4. Start with Fabric defaults
Do not carry traditional warehouse tuning habits forward automatically. First optimize data layout, set-based transformations, data types, and table design, then tune further only when workload metrics justify it.
5. Monitor by layer
Separate ingestion, transformation and quality, and serving signals. Combine pipeline monitoring with Query Insights and Warehouse monitoring views to isolate failures and performance regressions quickly.
If Power BI still needs to clean source data, Silver is incomplete. If Gold becomes a staging area or routinely reads Bronze directly, layer responsibilities have drifted and should be corrected before the pipeline grows.
Reference: naming conventions
Lakehouses: lh_bronze — Raw ingestion lakehouse lh_silver — Cleansed and conformed data lh_gold — Business-ready consumption layer Tables (Bronze): bronze_crm_customers — Source: CRM, Table: customers bronze_erp_sales_orders — Source: ERP, Table: sales_orders Tables (Silver): silver_dim_customer — Conformed customer dimension silver_fact_sales — Conformed sales fact Tables (Gold): gold_sales_summary_daily — Daily sales aggregation gold_customer_360 — Customer 360 view
Don't skip the Silver layer. Going directly from Bronze to Gold creates brittle pipelines and makes it harder to add new consumers later. The Silver layer provides a stable contract between producers and consumers.
How Medallion Combines with Other Patterns
Use domains to own data products and apply medallion layers inside each domain where quality stages are useful.
Use shortcuts or mirroring as Bronze inputs so source data can be reused without unnecessary custom copies.
Use Eventhouse for live analysis and actions, then retain or expose historical data through OneLake for Silver, Gold, AI, and BI.
Real-Time Intelligence
Event-driven analytics and action patterns for continuously arriving data in Microsoft Fabric.
Use Real-Time Intelligence when data arrives continuously and the business benefits from discovering, analyzing, visualizing, or acting on events as they happen. The deciding factor is the event-driven operating model, not only extreme volume or sub-second latency. Use batch or medallion pipelines when scheduled processing is sufficient.
Design the Event Lifecycle
Find, govern, and connect to event sources and streams available to the organization.
Ingest, filter, enrich, aggregate, and route supported event sources without custom plumbing.
Store append-heavy event data and analyze time-series, logs, telemetry, and operational signals.
Deliver live operational views and governed analytical experiences for different audiences.
Evaluate conditions and trigger notifications, workflows, or business actions from changing data.
Reference Pattern: Live and Historical Paths
Use the Paths Together
Optimize for fast ingestion, time-based exploration, operational dashboards, and event-driven actions.
- Append-heavy events, logs, and telemetry
- KQL time-series and pattern analysis
- Live monitoring and Activator rules
Optimize for cross-domain reuse, longer-term transformation, relational serving, data science, and AI.
- Lakehouse and Warehouse processing
- Semantic models and governed BI
- Notebooks, machine learning, and agents
Treat Eventhouse and OneLake analytics as complementary paths. Keep live dashboards and actions connected to the event path, and use OneLake availability for cross-engine historical analytics rather than creating an unrelated duplicate pipeline.