• Ort: Port Washington, New York
  • Veröffentlicht am: 25th Oct, 2021
  • Reference: srnfi10252021

Senior Data Engineer - Permanent - Remote - $150k

This large market research company is expanding and seeking a senior data engineer who has hands on experience working with cloud and Big data platforms. You will be responsible for data engineering, data analysis and user support primarily focused on Azure Databricks and Cloudera Hadoop. You will have strong hands-on technical skills with programming languages such as Python and Scala, including tuning complex big data spark applications.


  • Design, develop, and maintain highly scalable cloud-based data processing applications

  • Write efficient, reusable and well documented code

  • Deliver highly complex big data projects using Databricks, Data Factory and other Azure services

  • Maintain and tune existing Hadoop/Hbase applications and migrate to Azure cloud

  • Actively participate in daily agile / scrum meetings


  • 5 years of software development

  • Experience in developing and tuning spark applications using databricks

  • Excellent understanding of spark architecture, data frames and tuning spark applications

  • Excellent with Python, Scala, SQL and shell scripting

  • Hands-on experience with Azure

  • Experience in Database Design

  • Excellent problem solving and analytical skills

  • Solid written and verbal communication skills

  • Experience with Synapse, Snowflake or Google BigQuery is a plus

  • Bachelor's degree in Computer Science or related field

If this role is of interest, please contact Shannon today at s.roberts@nigelfrank.com.