Data Engineer
We are on the lookout for a skilled Data Engineer to become an integral part of our Data Engineering Team. You will not only maintain and optimize our data infrastructure but also spearhead its evolution. Built predominantly on AWS, and utilizing technologies like Pyspark and Iceberg, our infrastructure is designed for scalability, robustness, and efficiency. You will be part of developing sophisticated data integrations with various platforms, developing real-time data solutions, improving automation, and enabling crucial business intelligence.
- Design and Build Data Pipelines: Take ownership of designing, developing, and maintaining robust data pipelines and ETL processes using AWS services such as AWS Glue, AWS Lambda, and AWS S3.
- Contribute to Cloud Migration: Actively participate in and support the migration of our existing Data Warehouse from SQL Server to AWS, with a focus on implementing best practices using S3 and Redshift.
- Ensure Data Quality and Integrity: Develop, implement, and maintain data quality checks and validation procedures to ensure the accuracy and reliability of our data.
- Implement Architectural Solutions: Contribute to the design and implementation of data lakehouse architectures and data warehousing solutions.
- Collaborate and Deliver: Work closely with data scientists and analysts to support the deployment and operationalization of machine learning and advanced analytical solutions.
- Maintain and Optimize: Develop and maintain data documentation and operational procedures, and proactively investigate and resolve data quality issues and performance bottlenecks.
- Stay Current and Innovate: Stay up-to-date with the latest data technologies and industry best practices and apply this knowledge to improve our platform.
- Provide Technical Support: Offer technical guidance and support to junior data engineers and other team members when needed.
- Strong AWS Proficiency: Solid understanding and practical experience with AWS services such as AWS Glue, AWS Lambda, AWS S3, AWS Redshift, and Amazon EMR. (required)
- Core Technical Skills: Proven proficiency in Python, SQL, Pipeline Orchestration, PySpark and a deep understanding of data warehousing concepts. (required)
- Problem-Solving: Demonstrated ability to independently diagnose and resolve complex data issues and performance bottlenecks. (required)
- Communication: Ability to effectively communicate technical concepts and project updates to both technical and non-technical stakeholders. (required)
- Experience with Data Architectures: Hands-on experience with data lakehouse architectures and data warehousing solutions. (required)
- Agile Experience: Experience with Agile development methodologies. (required)
- Security and Best Practices: A solid understanding of data security and privacy best practices. (required)
- A competitive rate of pay and pension contribution ( £45,000 to £60,000) (required)
- Generous discretionary bonus schemes, incentives and competitions (required)
- An annual leave entitlement that increases with length of service (required)
- Access to an online GP 24/7, 365 days a year for you and your immediate family. (required)
- Employee wellbeing support through our Employee Assistance Programme (required)
- Enhanced Maternity & Paternity Pay (required)
- Long Service Recognition (required)
- Access to a pay day savings scheme, financial coach and up to 40% of your earned wage ahead of payday, through Wagestream. (required)
Betfred is a leading betting and gaming company that has been at the forefront of the industry for over 50 years. With over 1300 shops and 9500 employees, we are constantly pushing ourselves to innovate and improve, and we are always on the lookout for the best talent to join our team. In 2019, we entered the US sports betting market as Betfred Sportsbook, and we now operate in 10 states. We have also recently established a new presence in South Africa, with over 50 betting halls, and we own a majority stake in Lottostar, South Africa's largest online betting company. At Betfred, our mission is to provide our customers with the best possible service, products and experience while ensuring responsible gambling. We believe our employees are our greatest asset and we are committed to providing them with the training and development they need to succeed. We offer a range of career opportunities, from retail positions in our shops to roles in our head office. We are also dedicated to giving back to the communities in which we operate. Through our Betfred Supports initiative, we support a range of charities and community initiatives.