Position: Big Data Engineer
Location: Remote - East Coast*
- Minimum 5-7 years of experience
- Hive, Spark, Hbase, Cloud Platform, Azure Cloud knowledge
- Extensive knowledge on Spark programming
- Excellent Hadoop, Hive, and Spark programming knowledge
- Excellent SQL
Summary: The Big Data Engineer (Azure) will be responsible for big data engineering, data wrangling, data analysis and user support primarily focused on the Cloudera Hadoop platform, but in future extending to Azure cloud. You will have proficient hands-on technical skill along with the ability to mentor and train other Engineers on conventional ETL and SQL skills, with programming languages such as Python and Scala, using big data techniques. This position will also play a role in defining and implementing Big Data Strategy for the organization along with driving implementation of IT solutions for the business.
- Design, develop, and tune data products, streaming applications, and integrations on large-scale data platforms (Hadoop, SQL server, Data warehousing, Big Data, Hbase/Phoenix etc.) with an emphasis on performance, reliability, scalability, and quality.
- Develop custom data models and algorithms
- Proactively analyze the business needs, profile large data sets and build custom data models and applications to drive business decision making and customers experience
- Build workflows that empower analysts to efficiently use data
- Develop and extend design patterns, processes, standards, frameworks, and reusable components for various data engineering functions areas.
- Perform requirements analysis, planning and forecasting for Hadoop data engineering/ingestion projects
- Design optimized Hadoop and Big Data solutions for data ingestion, data processing, data wrangling, and data delivery
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
- Build the infrastructure required for efficient extraction, transformation, and loading of data from a wide variety of data sources
- Build data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
- Demonstrated experience in architecture, engineering, and implementation of enterprise-grade production big data use case
- 7 years experience in developing software applications
- 7 years experience working on Big Data technologies like Hadoop and Spark
- Proficiency in at-least one of the following languages Java, Scala, Python with building and supporting distributed systems
- Experience with REST services preferred
- Fluency in Unix command line tools and Bash is preferred
- Experience with data visualization
- Databricks experiene a plus
- Strong SQL, ETL, scripting, and or programming skills with a preference towards Python, Java, Scala, shell scripting
- Proven ability to solve data engineering and data mining issues
- Expert experience performing root cause analysis on internal and external data and processes to answer specific business questions and find opportunities for improvement
- Excellent bonus/comp packages
- Growth opportunity
- Remote (based on location)
Please send your resume to firstname.lastname@example.org for additional information!
Nigel Frank International is the global leader for Microsoft Azure recruitment, advertising more Microsoft Azure jobs than any other agency. We work with both Microsoft Partners & End Users throughout North America. By specializing solely in placing candidates in the Microsoft Azure market, we've built relationships with the key employers across the nation and have an unrivalled understanding of where the best opportunities are.
We understand the need for discretion and would welcome the opportunity to speak to any Azure candidates that are considering a new career or job either now or in the future. Confidentiality is of course guaranteed.
Please call or email email@example.com / (718) 412-1203 for additional information about the Azure market and available Azure job opportunities in your area.
See www.nigelfrank.com for more fantastic Azure opportunities!
Nigel Frank International Inc. is acting as an Employment Agency in relation to this vacancy.