Beyond Monitoring: AI-Driven Spark Optimization in Microsoft Fabric

This session is primarily about data engineering & architecture within the Microsoft Fabric and SQL ecosystem. Join this demo by Microsoft AI and Spark experts to see how Fabric takes Spark observability to the next level! Learn how to stream Spark logs to Eventhouse using Diagnostic Emitter. See Fabric AI/ML tooling in action to detect bottlenecks, predict resource impact, and suggest code improvements to optimize performance and cost. Visualize the insights and recommendations in Real Time Dashboard.