• Location: London, England
  • Salary: Up to £80000 per annum Fully Remote
  • Technology: Microsoft Jobs
  • Job Type: Permanent
  • Date Posted: 11th Aug, 2021
  • Reference: SDE220138JFI

Senior Data Engineer - FULLY REMOTE

Up to £80k DOE

My client are seeking a talented individual to help increase their delivery, managing a continuous integration and deployment tools, helping to define infrastructure as code, and with estimating system requirements, as well as monitoring smooth operations.

  • You will work with external and internal stakeholders as well as team members when capturing and documenting IT requirements.

  • You will show attention to detail and show strong problem solving skills when helping to spec out projects and infrastructure.

  • You will help with the development of data pipelines, setting up data lake and data warehousing infrastructure.

  • You will help creating new integrations with third party providers and external data sources.

  • You will help with the development of dashboards and other reporting tools.

  • You will be comfortable working to Agile principles and development methodologies.

Your Skills


  • Excellent knowledge of Python

  • You will know common data query languages for relational and document databases.

  • Experience working with version control tools, understand the importance of code reviews and coding standards).

  • Experience with Messaging queues and event driven development (eg. Rabbit MQ, SQS).

  • Experience setting up data pipelines, data extraction, transformation and load processes.

  • Experience developing REST APIs.

  • Understanding of common cloud metrics and alerting within AWS.


  • Experience with Apache Spark and Databricks.

  • Experience using the AWS eco system, especially VPC, EC2, RDS and Redshift and the AWS Cloud Development Kit, Lambda.

  • Experience with workflow management tools (eg. Airflow, Luigi).

  • Experience working with continuous integration, CI/CD processes.

  • Experience with orchestration tools (eg. Cloud formation, Kubernetes, etc).

To apply or for more info contact j.mckelvey@nigelfrank.com or call 0191 338 7638.