Skip to main content

Lead Data Engineer

Posted: 11 Nov 2020 Closes: 11 Dec 2020
London, England, United Kingdom Competitive
Permanent & Full-time -1487744383


A Lead Data Engineer will join the Data and Analytics team to lead the Data Engineering function. The Lead Data Engineer will be pivotal in further enhancing the Data Lake and implementing streaming technologies, Data services (data pipelines) and other Big Data platforms within AWS and Databricks.

The role will be focused on continued building of further data pipelines within the Data Lake to start with and delivered in an iterative fashion. The Lead will directly manage several Data Engineers and is responsible for continuing a high engineering and development standard ensuring correct use of tools, sustaining a secure platform, and ensuing logging, monitoring, TDD, BDD are further improved upon. The Lead will work closely with Data Science focused projects and other Analytics efforts including working with the Business Intelligence team.


What you'll be doing:

  • Technical ownership, Developing and supporting the Data Lake and overall data architecture within AWS and Databricks.
  • Designing, Developing and supporting ELT/ETL landscape within AWS and Databricks.
  • Designing, Developing and supporting Streaming/Event sourcing services.
  • Designing, Developing and supporting Big Data platforms with Databricks within AWS.
  • Designing, developing and supporting data services within AWS.
  • Assisting the Data Science and Analytics functions where required.
  • Line management duties, leadership and championing the Data Engineering function within Tech and across the business.

About you:
  • You are a passionate and expert Data Engineer having excellent technical skills and a natural aptitude for data. Expert Data Engineer proficient with Big Data Technologies, Data Lake and streaming fundamentals.
  • Extensive experience within Spark
  • Extensive experience with AWS cloud services
  • Experience with wider distributed computing, Big Data technologies (e.g. Hadoop, NoSQL dbs, Kafka, etc).
  • Experienced software development/engineering skills
  • Expert in Scala
  • Proficiency with Python, Java, extensive experience in database development across a variety of platforms.
  • Containerisation expertise - Docker, Kubernetes
  • Experience with streaming technologies and concepts.
  • Extensive experience with REST web services and APIs
  • Extensive experience in CI/CD principles and DevOps methods
  • Extensive experience and knowledge of Test automation including TDD and BDD

We lead the way, moving our industry forward, and we are always looking for talented individuals to come in and inspire the world with us. is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for our people.


Share Job