[ref. f1829524] CITCO - Data Engineer (Databricks and AWS)
Citco Manila Full-time
Position: Data Engineer (Databricks & AWS)
Company Overview Citco is a global leader in financial services, delivering innovative solutions to some of the world's largest institutional clients. We harness the power of data to drive operational efficiency and informed decision-making. We are looking for a Data Engineer with strong Databricks expertise and AWS experience to contribute to mission-critical data initiatives.
Role Summary as a Data Engineer, you will be responsible for developing and maintaining end-to-end data solutions on Databricks (Spark, Delta Lake, MLflow, etc.) while working with core AWS services (S3, Glue, Lambda, etc.). You will work within a technical team, implementing best practices in performance, security, and scalability.This role requires solid understanding of Databricks and experience with cloud-based data platforms.
Key Responsibilities- Databricks Platform & Development
- Implement Databricks Lakehouse solutions using Delta Lake for ACID transactions and data versioning
- Utilize Databricks SQL Analytics for querying and report generation
- Develop structured streaming pipelines for data ingestion and processing
- Use Databricks Repos, notebooks, and job scheduling for development workflows
- AWS Cloud Integration
- Work with Databricks and AWS S3 integration for data lake storage
- Build ETL/ELT pipelines using AWS Glue catalog, AWS Lambda, and AWS Step Functions
- Configure networking settings for secure data access
- Support infrastructure deployment using AWS CloudFormation or Terraform
- Data Pipeline & Workflow Development
- Create scalable ETL frameworks using Spark (Python/Scala)
- Participate in workflow orchestration and CI/CD implementation
- Develop Delta Live Tables for data ingestion and transformations
- Support MLflow integration for data lineage and reproducibility
- Performance & Optimization
- Implement Spark job optimizations (caching, partitioning, joins)
- Support cluster configuration for optimal performance
- Optimize data processing for large-scale datasets
- Security & Governance
- Apply Unity Catalog features for governance and access control
- Follow compliance requirements and security policies
- Implement IAM best practices
- Team Collaboration
- Participate in code reviews and knowledge-sharing sessions
- Work within Agile/Scrum development framework
- Collaborate with team members and stakeholders
- Monitoring & Maintenance
- Help implement monitoring solutions for pipeline performance
- Support alert system setup and maintenance
- Ensure data quality and reliability standards
- Educational Background
- Bachelor's degree in Computer Science, Data Science, Engineering, or equivalent experience
- Technical Experience
- Databricks Experience: 2+ years of hands-on Databricks (Spark) experience
- AWS Knowledge: Experience with AWS S3, Glue, Lambda, and basic security practices
- Programming Skills: Strong proficiency in Python (PySpark) and SQL
- Data Warehousing: Understanding of RDBMS and data modeling concepts
- Infrastructure: Familiarity with infrastructure as code concepts
IBMManila
based skills and technical expertise to drive innovation and adoption of new technology.
Your role and responsibilities
A Data Engineer with expertise in Google's data platforms is responsible for designing, building, and maintaining data engineering...
CreaThink Solutions.IncManila
About the Role
We are seeking a highly skilled Data Engineer with a strong DevOps mindset to help build a reusable Microsoft Fabric accelerator solution. This role is ideal for someone with a solid software and backend engineering background...
WeSource Management Consultancy FirmManila
Job Description
Data Engineer - Ortigas | Salary is up to 85K based on experience
Onsite - Dayshift Schedule
This is a direct employment - 12 months contract (subject to extension or absorption)
Qualifications:
At least 3 years of work...