Senior Data Engineer - Microsoft Partner Practice

What You'll Be Doing

End-to-End Platform Architecture

• Lead the design and build of end-to-end data platforms spanning ingestion, cleaning, preparation, transformation, analytics, and activation, using Azure Data Factory, Microsoft Fabric (Lakehouse, Warehouse, OneLake), and/or Azure Databricks with PySpark, Python, and SQL.
• Architect Medallion (Bronze, Silver, Gold) lakehouse patterns using OneLake/Delta Lake, balancing performance, reliability, and cost.
• Own data modeling for client engagements - dimensional, normalized, and semantic layers - so that data is trustworthy, reusable, and analytics-ready across domains.
• Design batch and streaming ingestion patterns, including Fabric Data Factory/pipelines, Azure Databricks Structured Streaming, and event-driven architectures (e.g., Event Hubs).
• Identify and resolve architectural risk and technical debt, and optimize pipelines for performance and cost.

APIs, Data Agents & Activation

• Design and build APIs and data services that activate governed data for client analytics, reporting, and downstream applications.
• Design MCP (Model Context Protocol) and API-based integrations that let front-line business systems, Fabric/Copilot Data Agents, and other agentic AI interfaces securely query and exchange data with the platform.
• Partner with AI/Architecture teams to expose enterprise data to Data Agents and Generative AI applications (e.g., Fabric Data Agents, Azure AI Foundry) in a governed, reusable way.
• Implement data governance patterns using Microsoft Purview and/or Unity Catalog, including access control, metadata, and lineage.
• Support production operations and incident response for critical pipelines and activation services on client engagements.

Technical Leadership & Client Engagement

• Mentor and provide technical direction to Data Engineers, reviewing code, data models, and designs to raise delivery quality.
• Act as a trusted technical resource to client stakeholders, explaining architecture trade-offs and design decisions in clear terms.
• Contribute to reusable frameworks, accelerators, and engineering standards used across engagements.
• Support presales activities where needed, including technical input on estimates, proposals, and solution design.
• Evaluate new capabilities across Fabric and Azure Databricks and recommend where they add value for client platforms.

What You'll Bring to the Role

• A Bachelor's or Master's degree in Computer Science, Engineering, Mathematics, Statistics, or a related technical discipline.
• 5-8 years of experience in data engineering, with strong ownership of production data pipelines on Azure.
• Expert-level proficiency in Python, SQL, and PySpark for building distributed data pipelines at scale.
• Deep hands-on experience with the Microsoft Azure Data & AI stack (Azure Data Factory, Microsoft Fabric, Synapse, ADLS Gen2) and/or Azure Databricks, including Delta Lake/OneLake and Medallion architecture patterns.
• Strong background in data modeling, schema design, and enterprise data architecture across the full lifecycle from ingestion to activation.
• Experience designing APIs and MCP-based integrations that exchange data with front-line systems, Data Agents, and other agentic AI interfaces.
• Experience with orchestration and CI/CD (Fabric pipelines, Azure Data Factory, Databricks Workflows, Azure DevOps).
• Strong knowledge of data governance and security tooling, including Microsoft Purview and/or Unity Catalog.
• Demonstrated ability to mentor engineers and lead technical design discussions on client-facing engagements.
• Strong problem-solving skills and ability to troubleshoot complex, large-scale data issues independently.
• Excellent communication skills, with the ability to explain technical trade-offs to both technical and client audiences.
• Relevant Microsoft certifications (e.g., Azure Data Engineer Associate, Fabric Analytics Engineer Associate) and/or Databricks certifications preferred.