Kafka Cloud Engineer

Job Description

Role Overview

We are looking for a skilled Cloud Data Engineer with strong hands‑on experience in Azure, Kafka, and Kubernetes (AKS) to design, develop, and manage scalable data streaming and cloud‑native systems.

The ideal candidate will have expertise in event‑driven architectures, distributed systems, and modern data platforms, along with strong debugging and stakeholder communication capabilities in enterprise environments.

Key Responsibilities

• Design, develop, and maintain event‑driven data pipelines using Confluent Kafka
• Work extensively with KsqlDB, Kafka Connect, Kafka Streams, and Schema Registry
• Implement and manage real‑time data streaming solutions using Kafka and cloud-native services (e.g., Azure Event Hubs, Confluent Cloud)

• Design and deploy solutions on Microsoft Azure, leveraging cloud‑native services
• Work on Kubernetes (AKS) for containerized application deployment and orchestration
• Manage and optimize CI/CD pipelines using GitHub Actions and DevOps practices
• Implement Infrastructure automation and scripting using Bash, PowerShell, and Python

• Implement secrets management using tools such as HashiCorp Vault
• Configure monitoring and observability using tools like Grafana and Prometheus
• Apply strong authentication and security frameworks including:
• OAuth
• LDAP
• SAML
• SSO

• Work with SQL and NoSQL databases such as Oracle, MySQL, PostgreSQL, and MongoDB
• Configure Change Data Capture (CDC) and integrate APIs across systems
• Support integration of microservices and enterprise systems using event-driven architectures

• Troubleshoot and resolve complex issues in Kubernetes and distributed systems
• Collaborate with global stakeholders to design and deliver solutions
• Ensure compliance with security, governance, and data standards (especially in regulated environments like banking)

• 4–5 years of overall IT experience with at least 1+ year of hands‑on Azure experience
• Strong experience with:
• Kubernetes (AKS)
• Confluent Kafka ecosystem (Kafka, KsqlDB, Connect, Streams)
• CI/CD (GitHub Actions, DevOps practices)
• Proficiency in Java and Python programming
• Strong scripting skills (Bash, PowerShell, Python)
• Experience with monitoring tools (Grafana, Prometheus)
• Strong problem‑solving and troubleshooting skills

• Azure certifications and Core Java certification
• Experience with Azure Event Hubs or Confluent Cloud (SaaS platforms)
• Exposure to banking / regulated environments
• Knowledge of HashiCorp Vault or secrets management tools