Paul Andrew

In this session, we will explore the administration of Microsoft Fabric, with a focus on the organisation and management of data storage/compute through the configuration of capacities, domains, data products, environments and workspaces. We will discuss the application of data mesh and data fabric concepts in the context of Microsoft Fabric capabilities, including organising data products for effective delivery to business users. Also, considering how Microsoft Fabric compares to the practices and technical standards we’ve applied in Microsoft Azure over the years, making that shift from PaaS to SaaS, in theory and maybe in practical. Additionally, we will explore how to use domains and separate workspaces to serve reports to business users, providing them with the information they need to make informed decisions. Aligning industry governance standards to Microsoft Fabric features and access controls. Join us to learn how to effectively administer Microsoft Fabric beyond the simple Workspaces inherited from Power BI

How has advancements in highly scalable cloud native technology influenced the design principles we apply when building data platform solutions? Are we designing for just speed and batch layers or do we what more from our platforms, and who says these patterns must be delivered exclusively?

In today’s data-driven world, fast and efficient data platform delivery is crucial for staying ahead of the competition. Join me for a dynamic session that demonstrates how to build a metadata-driven Lakehouse with Microsoft cloud native technologies.

Discover the evolution of the data warehouse to the cloud using Delta.io, an open-source standard by Databricks, in Azure and Fabric. Learn how cloud-native Delta Lake technology provides ACID properties and distributed storage/compute for next-gen cloud data warehousing.

Explore scalable architectures in Azure & Microsoft Fabric for high throughput data ingestion and real-time analytics. Use Event Hub, Stream Analytics, SQL, and KQL to blend stream and batch data for rapid insights and competitive advantage.

The velocity of data is getting faster across many industries, fuelled by the business demand to gain insights and value from sources in near real-time. This necessity is then allowing decision makers to pivot and ultimately stay ahead of the competition.

In this session we will roll our sleeves up and get hands on, demonstrating how we can handle real-time data feeds in Microsoft Fabric for a popular cycling & adventure shop! Ingesting data and having it flow into the Real-time Intelligence Hub, then downstream to a variety of different Experience features, including other Fabric Item triggering.

The velocity of data is getting faster across many industries, fuelled by the business demand to gain insights and value from sources in near real-time. This necessity is then allowing decision makers to pivot and ultimately stay ahead of the competition.

The resources on offer in Azure are constantly changing, which means as data professionals we need to constantly change too. Updating knowledge and learning new skills.

This pre-conference workshop will begin by defining 'big data' and clarifying various data architecture concepts to establish a solid foundation of understanding before delving into specific data architectures. Topics to be covered include relational data warehouses, data lakes, data marts, data virtualization, and…

As a platform, we’ll use Azure Event Hub and Azure Stream Analytics to ingest and handle that initial data stream. Before applying the same patterns to other resources in Microsoft Fabric and Azure Data Explorer. Understanding the patterns to apply as an architect vs the tooling available for...

Start this full day of training with the fundamentals of data pipelines before building the skills needed to orchestrate an entire data platform solution. We’ll focus Azure Data Factory as the mature resource, but also cover Synapse Analytics and Microsoft Fabric pipelines.

How has advancements in highly scalable cloud technology influenced the design principals we apply when building data platform solutions? Are we designing for just speed and batch layers or do we what more from our platforms, and who says these patterns must be delivered exclusively?

Within a typical Azure data platform solution for any enterprise grade data analytics or data science workload an umbrella resource is needed to trigger, monitor, and handle the control flow for transforming datasets. Those requirements are met by deploying Azure Data Integration pipelines, delivered using Synapse Analytics or Data Factory.

The principals of a data mesh architecture have been around for a while now, but we still don’t have a clear way to deliver such a solution in Azure. Are the concepts so abstract that it’s hard to translate the principals into real world requirements and maybe even harder to think about what technology you might need…

Based on real world experience let’s think about just how far the breadth of our knowledge now needs to reach when starting from nothing and building a complete Microsoft Azure Data Analytics solution.

In this full day of training, we’ll start with the very basics and learn how to orchestrate your Azure data platform from start to finish. You will learn how to build out Azure control flow and data flow components as processing pipelines using Azure Data Factory and Azure Synapse Analytics. We’ll start by covering the fundamentals within the resources and together build out pipelines that ingest data from local source systems, transform and serve it to consumers. We’ll then continue taking an end-to-end look at our Azure integration pipeline tools within highly scalable cloud native architectures, dealing with triggering, monitoring, dynamic pipeline content as well as CI/CD practices. Start the day knowing nothing about Azure Data Integration pipelines and leave with the knowledge, slides, demos, and code to apply these resources in your role as a data engineering professional.

The resources on offer in Azure are constantly changing, which means as data professionals we need to constantly change too. Updating knowledge and learning new skills. No longer can we rely on products matured over a decade to deliver all our solution requirements. Today, data platform architectures designed in Azure with best intentions and known design patterns can go out of date within months. That said, is there now a set of core components we can utilise in the Microsoft cloud to ingest, curation and deliver insights from our data? When does ETL become ELT? When is IaaS better than PaaS? Do we need to consider scaling up or scaling out? And should we start making cost the primary factor for choosing certain technologies? In this session we'll explore the answers to all these questions and more from an architect’s viewpoint. Based on real world experience let’s think about just how far the breadth of our knowledge now needs to reach when starting from nothing and building a complete Microsoft Azure Data Analytics solution.

The Microsoft abstraction machine is at it again with this latest veneer over what we had come to understand as the ‘modern data warehouse’. Or is it?! When creating an Azure PaaS data platform/analytics solution we would typically use a set of core Azure services; Data Factory, Data Lake, Databricks and maybe SQL Data Warehouse. Now with the latest round of enhancements from the MPP team and others it seems in the third generation of the Azure SQLDW offering we can access all these core services as a bundle. We might even call it a Kappa architecture! Ok, so what? Well, this is a reasonable starting point in our understanding of what Azure Synapse Analytics is, but it is also far from the whole story. In this session we will go deeper into the evolution of our SQLDW to complete our knowledge on why Synapse Analytics is a game changer for various data platform architectures. We’ll discover what Synapse has to offer with its Data Virtualisation layer, flexible storage, and variety of compute engines. A simple veneer of things, this new resource is not. In this introduction to Synapse we will cover the what, that why and importantly the how for this emerging bundle of exciting technology. Finally, we’ll touch on Microsoft’s latest thinking for a HTAP environment with direct links into our transactional data stores.

In this full day of training, we’ll start with the very basics and learn how to orchestrate your Azure data platform from start to finish. You will learn how to build out Azure control flow and data flow components as processing pipelines using Azure Data Factory and Azure Synapse Analytics. We’ll start by covering the fundamentals within the resources and together build out pipelines that ingest data from local source systems, transform and serve it to consumers. We’ll then continue taking an end-to-end look at our Azure integration pipeline tools within highly scalable cloud native architectures, dealing with triggering, monitoring, dynamic pipeline content as well as CI/CD practices. Start the day knowing nothing about Azure Data Integration pipelines and leave with the knowledge, slides, demos, and code to apply these resources in your role as a data engineering professional.

We'll explore delivering this framework within an enterprise and consider an architect’s perspective on a wider platform of ingestion/transformation workloads with multiple batches and execution stages.

In this full-day workshop, you will learn the skills you need to build and orchestrate hybrid, complex and scalable data pipelines using Azure Data Factory (ADF).

What happens when you combine a cloud orchestration service with a Spark cluster?! The answer is a feature rich, graphical, scalable data flow environment to rival any ETL tech we’ve previously had available in Azure.

DevOps as a concept does not always translate to the technology when implemented. In this session we'll explore that problem when working with Azure Data Factory and what the different cloud only CI/CD options are.

What happens when you combine a cloud orchestration service with a Spark cluster?! The answer is a feature rich, graphical, scalable data flow environment to rival any ETL tech we’ve previously had available in Azure. In this session we’ll look at Azure Data Factory v2 and how it integrates with Azure Data Bricks to produce a powerful abstraction over the Apache Spark analytics ecosystem. Now we can transform data in Azure using Data Bricks but without the need to write a single line of Scala or Python! If you haven’t used either service yet, don’t worry, you’ll get a quick introduction to both before we go deeper into the new ADF Data Flow feature.