• Lieu: Port Washington, New York
  • Date de publication: 25th Oct, 2021
  • Référence: srnfi10252021

Senior Data Engineer - Permanent - Remote - $150k


This large market research company is expanding and seeking a senior data engineer who has hands on experience working with cloud and Big data platforms. You will be responsible for data engineering, data analysis and user support primarily focused on Azure Databricks and Cloudera Hadoop. You will have strong hands-on technical skills with programming languages such as Python and Scala, including tuning complex big data spark applications.


Responsibilities



  • Design, develop, and maintain highly scalable cloud-based data processing applications

  • Write efficient, reusable and well documented code

  • Deliver highly complex big data projects using Databricks, Data Factory and other Azure services

  • Maintain and tune existing Hadoop/Hbase applications and migrate to Azure cloud

  • Actively participate in daily agile / scrum meetings


Qualifications:



  • 5 years of software development

  • Experience in developing and tuning spark applications using databricks

  • Excellent understanding of spark architecture, data frames and tuning spark applications

  • Excellent with Python, Scala, SQL and shell scripting

  • Hands-on experience with Azure

  • Experience in Database Design

  • Excellent problem solving and analytical skills

  • Solid written and verbal communication skills

  • Experience with Synapse, Snowflake or Google BigQuery is a plus

  • Bachelor's degree in Computer Science or related field


If this role is of interest, please contact Shannon today at s.roberts@nigelfrank.com.

Emplois similaire

Microsoft Cloud Engineer
New York, New York

Senior & Junior Azure DevOps Consultant roles
New York, New York

Azure Senior DevOps/Release Engineer
New York, New York

Data Engineer - Permanent - Remote - $130k
Port Washington, New York

Cloud Security Engineer
New York, New York