Data Engineering Consultant

Role Overview

You will work across the full data lifecycle — from ingestion and transformation through to governance and performance optimization — enabling clients to derive value from their data assets reliably and efficiently.

• Design, build, and maintain scalable, end-to-end data solutions on Microsoft Azure tailored to business data requirements.
• Develop and manage data pipelines for ingesting, transforming, and integrating structured and unstructured data from diverse sources using Apache Spark.
• Implement robust data security, governance, and privacy controls across Azure data platforms in line with organizational standards.
• Monitor, troubleshoot, and continuously optimize data workflows to ensure high reliability, performance, and cost efficiency.
• Apply Azure best practices to optimize data storage and processing solutions across the platform.
• Collaborate with stakeholders to understand data requirements and translate them into technically sound, scalable architectures.

• Bachelor's degree in Computer Science, a related field, or equivalent practical experience with a minimum of 4+ years in the domain.
• At least 3 years of consulting or client-facing service delivery experience in Azure-based Microsoft data engineering.
• Minimum 1 year of hands-on experience building data ingestion, processing, and analytics pipelines using SQL Server, Azure Synapse Analytics, Azure Databricks, and Microsoft Fabric.
• Practical experience with ETL and data processing using Azure Data Factory, Azure Functions, Azure Data Lake Storage (ADLS), OneLake, and Microsoft Fabric services.
• At least 1 year working with Azure and Big Data technologies including Microsoft Fabric, Databricks, Python, SQL, ADLS/Blob Storage, Spark, and Spark SQL.
• Minimum 1 year of hands-on experience with RDBMS technologies.
• Familiarity with big data file formats and compression techniques.
• Proficiency with developer tools including Azure DevOps, Git, and Visual Studio Team Services (VSTS).

• Experience designing and deploying ETL solutions on Azure Cloud using Azure Data Factory, Synapse Analytics, Azure Functions, and Notebooks.
• Hands-on experience with Microsoft Fabric, including Medallion Architecture, metadata-driven ingestion frameworks, data quality frameworks, and SCD Type 1 & Type 2 transformations via Fabric Notebooks or Azure Databricks.
• Exposure to Microsoft Copilot, Generative AI fundamentals, and industry-relevant AI use cases is an advantage.
• Relevant Microsoft role-based certifications are preferred (e.g., DP-600, DP-700, DP-203, DP-900, AI-102, AI-900).
• Working knowledge of Microsoft Power BI for developing reports, dashboards, and delivering actionable business insights.
• Understanding of Azure RBAC and IAM, including access control principles and cloud security concepts.
• Genuine interest in Microsoft's technology vision and roadmap, with a drive to adopt emerging tools and capabilities.

• Graduate or postgraduate degree in Computer Science, Engineering, or a related specialization.

• Minimum 4 years of hands-on experience in software development, with a significant focus on Azure Data Engineering.