As a Data Engineer you will help us build this production grade cloud environment. In this role, you will work alongside our platform team to provide the technical necessities for integrating the different data sources into our main data platform. You'll be responsible for the design, implementation, test and maintenance of data products on Microsoft Azure. You will work closely together with various (internal) business stakeholders and multiple data providers. You see yourself as a senior engineer, with extensive programming experience, and have the willingness to help the team achieve its goals. Your main activities will be:
- Identify the integration requirements of several data-sources;
- Design, implement and test cloud based (Azure) data intensive applications;
- Collaborate with other Lead Engineers, Data Architects & Enter. Architects for technical solutions
- Contribute improvements to the Data Platform
- Convert data product prototypes to data products.
Skills & Qualifications
- Strong Java/Scala experience, we would prefer someone with 2+ years hands-on experience with at least, one of this two.
- Professional experience with Kafka (or Azure EventHub)
- Professional experience with Spark, implementing both batch and streaming pipelines
- You recognize the importance of logging and monitoring (we currently use DataDog, Splunk, sentry)
- You have at least one year experience with Microsoft Azure (AWS, GCP are fine too)
- You are comfortable working with latest DevOps technologies i.e., Kubernetes, Docker;
- Professional experience with Scrum
- You have a BSC or MsC in a relevant field (e.g. Computer Science, Mathematics etc.)
- You have at least 4 year of experience running data driven solutions, including deployment and management of data-pipelines in production.
- You are fluent in English. Knowledge of the Dutch language is a plus.