Lakehouse vs. Warehouse

Fabric offers two primary storage artifacts. Choose based on your workload:

Notebook Best Practices

• Parameterize notebooks: Use Fabric's built-in parameters for environment-specific values (file paths, dates, flags)
• Modular design: Break complex logic into reusable functions or separate notebooks using %run
• Error handling: Use try/catch blocks and log errors to a monitoring table
• Avoid hardcoded paths: Use relative paths within OneLake or parameterized paths
• Version control: Connect notebooks to Git for versioning and code review

Spark Optimization

• V-Order optimization: Enable V-Order for write-optimized Delta tables — significantly faster reads for Power BI and SQL
• Optimize file sizes: Target 128 MB – 1 GB files. Use OPTIMIZE (bin-compaction) to merge small files
• Partition wisely: Partition by high-cardinality columns used in filters (e.g., year, region). Avoid over-partitioning
• Use Delta caching: SSD caching for repeated reads on the same data
• Right-size sessions: Don't default to large Spark clusters — start small and scale up based on the Spark UI metrics

File Format Guidance

✅ Delta (Parquet + transaction log)  — Default for all Fabric tables
✅ Parquet                             — Good for external/read-only data
⚠️  CSV                                — Only for Bronze ingestion, convert to Delta ASAP
❌ JSON (nested)                       — Flatten before storing in tables
❌ Excel/XML                           — Convert during ingestion pipeline

Shortcuts Strategy

Shortcuts are pointers to data in external locations — they appear as folders in your lakehouse but don't copy data. Use them to:

• Access data in other lakehouses or workspaces without duplication
• Connect to external ADLS Gen2, AWS S3, or Google Cloud Storage
• Enable cross-domain data sharing without moving data
• Create a virtual data mesh where each domain manages its data but shares via shortcuts

Overview

Fabric's Real-Time Intelligence experience is designed for high-volume streaming and time-series data. It's powered by the same engine behind Azure Data Explorer (Kusto) and provides sub-second query latency on billions of records.

Key Components

Streaming Ingestion Patterns

When to Use Real-Time vs. Batch

• Real-Time Intelligence: IoT telemetry, application logs, clickstream analytics, fraud detection, operational monitoring
• Batch (Lakehouse/Warehouse): Historical reporting, data warehousing, regulatory reporting, large-scale transformations
• Hybrid: Ingest in real-time to Eventhouse for operational use, then copy to Lakehouse for long-term analytics

Direct Lake Mode

Direct Lake is a game-changing connectivity mode exclusive to Fabric. It combines the performance of Import mode with the freshness of DirectQuery — without copying data into the semantic model.

• Reads data directly from Delta tables in OneLake
• No data refresh needed — reports always show the latest data
• Performance comparable to Import mode for most scenarios
• Falls back to DirectQuery if data exceeds memory (configurable behavior)

Semantic Model Best Practices

• Build star schemas in your Gold layer — separate facts and dimensions
• Use surrogate keys (integer) instead of natural keys for relationships
• Minimize calculated columns — push computations to the Lakehouse/Warehouse layer
• Use calculation groups for reusable time intelligence (YTD, MTD, YoY)
• Enable automatic aggregations for large tables to speed up high-level queries

Report Governance

• Centralize semantic models: Create shared models that multiple reports connect to (avoid dataset proliferation)
• Use endorsement: Certify production-ready models and promote development models
• Apply RLS: Configure row-level security on semantic models for multi-tenant reports
• Monitor usage: Use the usage metrics to track report adoption

Performance Optimization