Job Title: Senior Data Engineering Lead
Job Type: Permanent, Full-Time
Salary: $170k Base
Combining consumer surveys, retail point-of-sale data, and cutting-edge analytics, the hiring company delivers trusted retail industry research to the world's most successful brands and retailers. We also offer careers that reward innovation and provide growth opportunities for forward-thinking problem-solvers who join our team. Speed your career growth, while working with the best and brightest in the market research industry.
Headquartered in Port Washington, NY with offices around the globe, the hiring company is a leader in collection, processing, and delivery of point-of-sale and consumer tracking data in more than 20 industries. We are expanding and seeking a senior data engineering lead who has hands on experience working with cloud and Big Data platforms. You will be responsible for data engineering, data analysis and user support primarily focused on Azure Databricks and Cloudera Hadoop. You will have strong hands-on technical skills as well as being able to mentor and train other Engineers with programming languages such as Python and Scala, including tuning complex Big Data Spark applications.
- Design, develop, and maintain highly scalable cloud-based data processing applications
- Manage and mentor a team of 3-5 data engineers
- Write efficient, reusable and well documented code
- Deliver highly complex big data projects using Databricks, Data Factory, and other Azure services
- Maintain and tune existing Hadoop/Hbase applications and migrate to Azure cloud
- Actively participate in daily agile/scrum meetings
- 7 years of software development and design
- 2 years of leading a team of 3-5 software developers
- Experience in developing and tuning Spark applications using Databricks
- Excellent understanding of Spark architecture, Spark SQL, data frames and tuning Big Data Spark applications
- Excellent with Python, Scala, SQL, and shell scripting
- Hands-on experience with Azure
- Experience in Database Design and Data Modeling
- Excellent problem solving and analytical skills
- Strong diagramming skills - flowcharts, data flows, etc.
- Solid written and verbal communication skills
- Experience with Synapse, Snowflake, or Google BigQuery is a plus
- Bachelor's degree in Computer Science or related field
- Advancement opportunities with structured career paths
- Work with some of the world's most successful brands and retailers
Please email firstname.lastname@example.org for immediate consideration!