• Ort: New York, New York
  • Gehalt: $150000 - $170000 per annum
  • Technologie: Dynamics CRM Jobs
  • Vertragsart: Permanent
  • Veröffentlicht am: 22nd Nov, 2021
  • Reference: AMAZUREDATA

Senior Data Engineering Lead:
Headquartered in Port Washington, NY with offices around the globe, The NPD Group is a leader in the collection, processing, and delivery of point-of-sale and consumer tracking data in more than 20 industries. We are expanding and seeking a senior data engineering lead who has hands on experience working with cloud and Big data platforms. You will be responsible for data engineering, data analysis and user support primarily focused on Azure Databricks and Cloudera Hadoop. You will have strong hands-on technical skills as well as being able to mentor and train other Engineers with programming languages such as Python and Scala, including tuning complex big data spark applications.


Role & Responsibilities



  • Design, develop, and maintain highly scalable cloud-based data processing applications

  • Manage and mentor a team of 3-5 data engineers

  • Write efficient, reusable and well documented code

  • Deliver highly complex big data projects using Databricks, Data Factory and other Azure services

  • Maintain and tune existing Hadoop/Hbase applications and migrate to Azure cloud

  • Actively participate in daily agile / scrum meetings



Skills & Qualifications




  • 7 years of software development and design

  • 2 years of leading a team of 3-5 software developers

  • Experience in developing and tuning spark applications using Databricks

  • Excellent understanding of spark architecture, spark SQL, data frames and tuning big data spark applications

  • Excellent with Python, Scala, SQL and shell scripting

  • Hands-on experience with Azure

  • Experience in Database Design and Data Modelling

  • Excellent problem solving and analytic skills

  • Strong diagramming skills - flowcharts, data flows, etc.

  • Solid written and verbal communication skills

  • Experience with Synapse, Snowflake or Google Big Query is a plus
    Bachelor's degree in Computer Science or related field


Benefits



  • Standard Benefits

Ähnliche Jobs

Dynamics Functional Specialist - Remote - $120k
Schenectady, New York

Dynamics 365 CRM Functional Specialist
Albany, New York

D365 CE Specialist
Buffalo, New York

D365 CE Specialist
Albany, New York

D365 CE Technical Architect
New York, New York