Senior Data Engineer

As a Senior Data Engineer, you will be a key driver in our transition from an on-premises data infrastructure to a modern, scalable cloud lake house architecture. Your expertise in SQL querying, query optimization, stored procedures, data modelling, and streaming will support current operations while enabling the transformation to a cloud-first strategy. You will play a critical role in designing and implementing a future-ready data platform, ensuring seamless integration of real-time and batch processing with the latest technologies. Additionally, you will mentor and train junior team members, fostering their development and ensuring the team’s success in this transformative journey.

• Designing, developing, and optimizing robust data pipelines for on-premises and hybrid cloud environments.
• Leading efforts to migrate data and systems from on-premises databases to a cloud lake house architecture.
• Developing scalable data models for cloud-based storage and processing.
• Implementing and managing streaming data solutions using platforms like Kafka or RabbitMQ.
• Collaborating with cross-functional teams to integrate data from multiple sources into a unified data platform.
• Optimizing query performance and database structures to meet evolving business needs.
• Building and maintaining robust ETL/ELT processes to support cloud-native architectures.
• Driving the adoption of modern data lake house technologies such as Delta Lake, Apache Iceberg, or Apache Hudi.
• Mentoring and training junior team members to build their expertise in cloud data engineering.
• Staying updated on emerging cloud and lake house technologies to guide the data architecture evolution.

• Bachelor’s degree in Computer Science, Data Engineering, Information Systems, or a related field (Master’s preferred). (required)
• 5+ years of experience in data engineering or related roles. (required)
• Relevant certifications (e.g., AWS Certified Data Analytics Specialty, Microsoft Certified: Azure Data Engineer, or Databricks Certified Data Engineer). (preferred)
• Strong expertise in SQL, including querying, optimization, and writing stored procedures. (required)
• Proven experience in cloud-based data platforms such as AWS Redshift, Azure Synapse, or Google BigQuery. (required)
• Knowledge of cloud lake house technologies (e.g., Delta Lake, Apache Iceberg, or Apache Hudi). (required)
• Hands-on experience with streaming platforms such as Kafka or RabbitMQ. (required)
• Expertise in data modelling for both on-premises and cloud-based systems. (required)
• Proficiency in Python, Java, or Scala for data engineering tasks. (required)
• Experience with CI/CD practices for data pipelines in a cloud environment. (required)
• Familiarity with data governance and security best practices in cloud environments. (required)
• Strong leadership and mentoring skills, with experience guiding junior team members. (required)
• Experience with big data technologies such as Hadoop, Spark, or Flink, especially in a cloud context. (nice-to-have)
• Knowledge of containerization tools like Docker and orchestration platforms such as Kubernetes. (nice-to-have)
• Expertise in monitoring and observability tools for cloud data pipelines. (nice-to-have)
• Certification in cloud platforms or lake house technologies (e.g., AWS Data Analytics Specialty, Databricks Certified Data Engineer). (nice-to-have)
• Background in implementing real-time analytics and event-driven architectures in the cloud. (nice-to-have)
• Understanding of cost optimization strategies for cloud-based data infrastructure. (nice-to-have)
• Experience in managing hybrid environments during migration from on-prem to cloud. (nice-to-have)
• Hands-on experience with serverless computing in cloud such as Lambda Functions. (nice-to-have)
• Hands-on experience with cloud analytics platforms such as Big Query or Databricks. (nice-to-have)
• Proficiency in C# for general engineering tasks. (nice-to-have)
• Hashicorp Terraform for IaC workloads. (nice-to-have)
• Automation using shell scripts (Powershell, Bash etc). (nice-to-have)

• Comprehensive learning and development programmes.
• Innovative Performance Tool for regular, constructive feedback.
• Employee Assistance programme for you and your family.
• Free Daily Meal.
• Free Massages On-site.
• Free On-Site Gym.
• Group Life Cover.
• Funeral Fund Benefit.
• Financial Services Assistance.
• Employee Assistance Programme.
• Curro School Fees Benefit.
• Income Continuation Benefit.
• Leadership Training.
• Referral Bonus.
• Medical Aid Subsidy.
• Free Sleep Coaching.
• On-site Barista.
• Retirement Annuity Subsidy.
• Team socials.