Azure Data Engineer

JOB SUMMARY :

The ideal candidate will possess a strong proficiency in Azure Data Platform, along with extensive knowledge of Azure cloud services and relevant certifications.

DUTIES AND RESPONSIBILITIES:

• Develop end-to-end implementation of our solution on Azure Data Platform, including design, development, deployment, and ongoing maintenance. Implementation will be carried out by an external provider.
• Collaborates with cross-functional teams, including data architects, tech developers, data scientists and other data engineers too, to define and refine the solution architecture.
• Provides technical expertise and guidance to ensure that the solution is implemented efficiently and effectively by our supplier, leveraging the capabilities of the Azure Data Platform.
• Works closely with Data Ambassadors and Data Center of Excellence (CoE) to gather requirements, define project scope, and manage expectations throughout the project lifecycle.
• Develops and maintains a deep understanding of Azure cloud services, with a focus on Azure Data Platform, including Azure Data Lake, Azure Databricks, Azure Machine Learning, Microsoft Fabric, Azure Data Factory, Azure Cognitive Services, Azure Cosmos DB (or other NoSQ DB), and other relevant technologies.
• Extensive experience in data integration with various sources such as SAP, Salesforce, DBs, SCADA, PIMS, etc.
• Stays up to date with the latest advancements in Azure services, best practices, and industry trends, and apply this knowledge to enhance the solution.
• Monitors and optimizes the performance and scalability of the solution on Azure Data Platform, making recommendations for improvements as needed.
• Collaborates with the operations team to ensure the availability, security, and compliance of the solution in the Azure environment.
• Provides technical leadership and mentorship to junior team members, fostering a culture of continuous learning and growth.
• Responsible for overseeing AI implementations. This includes understanding AI models developed by data scientists and ensuring their successful deployment on Azure. You’ll also monitor these models for performance and make necessary adjustments.
• Designs and implements innovative solutions and prototypes to stay ahead of the market trends in collaboration with other department.
• Develops and manages data warehousing systems to ensure the availability and accessibility of data for other teams.
• Designs and manages orchestration workflows to automate and streamline data processes.
• Develops and manages a data Lakehouse to provide a unified platform for all types of data.
• Implements real-time data streaming solutions to provide timely insights and analytics.
• Manages data effectively to ensure its quality, security, and privacy.
• Designs and implements data consumption solutions to make data easily accessible and usable for end users.

JOB SPECIFICATIONS:

• At least 4-8 years Proven experience as Data Engineer or similar role, with a focus on implementing solutions on Azure Data Platform.
• Higher education at Bachelor's or Bachelor's level, as well as a high level of English.
• Master`s degree in Business Administration or Information Technology, advanced studies in Data Science/Analytics is desirable but not mandatory.
• At least one certification between DP-203 Azure Data Engineer Associate and AZ-305 Designing Microsoft Azure Infrastructure Solutions is a must.
• Proven knowhow in Continuous Integration/Continuous Deployment and how deployments work between environment using pipelines
• Expertise on data management tools / IT architecture models (Data Warehouse, Datalake, Lakehouse, etc.)
• Strong proficiency in Azure Data Platform, including Azure Data Lake, Azure Databricks, Azure Machine Learning, Microsoft Fabric, Azure Data Factory, Azure Cognitive Services, Azure Cosmos DB (or other NoSQ DB), and other relevant technologies.
• Demonstrated expertise in designing and implementing scalable and efficient solutions on Azure.
• Extensive knowledge of Azure cloud services, infrastructure, and architecture, with relevant certifications (e.g., DP-203 Azure Data Engineer Associate or AZ-305 Designing Microsoft Azure Infrastructure Solutions).
• In-depth understanding of data management principles, data lakehouse architecture, and data integration concepts.
• Experience working in an Agile development environment, with a solid understanding of Agile methodologies.
• Knowledge and experience of DAMA Capabilities and the different disciplines of Data Management
• Strong attention to details and technicalities.
• Excellent problem-solving and analytical skills, with the ability to troubleshoot complex technical issues.
• Strong communication and collaboration skills, with the ability to effectively interact with stakeholders at various levels of the organization.
• Self-motivated, proactive, and able to work independently with minimal supervision.
• Ability to work remotely and collaborate effectively with a geographically dispersed team.
• Proficiency in English, both written and verbal, is required for this position for effective communication and collaboration with team members and stakeholders. Spanish is nice to have.