My clients are creating a global Big Data, Microservices SAAS platform that will very likely be as popular as the likes of Spotify & UBER (I appreciate that's some claim but these are big players). creating the Worlds largest and most transparent marketplace for transportation, led by a leadership team that boasts a track record of taking technology based solutions from conception through to IPO or acquisition, in deals that made headline news.
Due to amazing growth and cntinued success they are looking to engage the services of a Senior Data Engineer.
As part of the Data team, you will thrive in taking the lead developing cutting-edge production grade microservices for our Machine Learning models, developing Machine Learning rendezvous architecture and building real-time robust data and feature pipelines, in conjunction with Data Scientists, Site Reliability Engineers, Exchange Backend Developers, etc.
Core duties will include:
● Design and build production-grade microservices for our ML models which solve a range of problems in the platform, e.g. ETA, Final fare prediction, etc.
● Develop, deploy and maintain real-time robust data pipelines, often utilizing external dynamic data sources (e.g. weather, traffic, etc.) and geospatial feature aggregation using Redis data stores
● Work closely with Data Scientists and SRE to ensure ML algorithms are scalable, maintainable, robust and easily deployable into production, utilizing a ML rendezvous architecture (to be designed)
● Understand and improve data quality across data pipelines, microservices and data stores, with quality designed in
● Troubleshoot complex bugs in distributed systems
● Design and build tools and dashboards to enable people to consume and understand data faster, often using geospatial libraries, frameworks or visualizations
● Make architecture, technology, tool, and language choices
● Continuous drive improvement, excellence and best practice in Data Engineering within the company Experience/Competence
● 7+ years of industry experience working with Big Data in a senior engineering role
● 4+ years of solid experience writing solid code in programming languages such as Go, Scala, Java or Python) and data stores (MySQL, Redis, etc)
● Strong computer science fundamentals & programming skills. You know which algorithms and data structures to apply when
● Hands-on experience building data pipelines and microservices from scratch
● Experience with large-scale distributed and database systems (SQL or NoSQL)
● An advanced degree in computer science or equivalent
● Working with cloud technologies, ideally GCP; DataFlow, BigQuery, Pub/Sub, BigTable, Cloud ML, TensorFlow, etc.
● Detailed experience deploying, serving and managing ML models, including canary, warming up models, rollbacks, etc.
● Experience working with CI/CD tools and technologies; Github, Jenkins, Terraform
● Hands-on experience with Kubernetes (or similar container orchestration) and Docker
● Experience of Agile, preferably using Jira
● Experience of working with geospatial data and tools
● Demonstrable ability to self-manage and be organised with a strong work ethic
● Proactive, self-starter, who is able to manage time effectively and independently complete tasks within agreed timeframes
● Good technical communication skills & excellent attention to detail; to quality check own work and assess source data quality
● Clear & confident technical communication
● Ability to thrive in a fast-paced, collaborative, agile environment
● Strong commercial awareness, with an understanding of business priorities
- Six Figure Basic Salary (DOE)
- Full time employee shares (actual shares, not options, that will be life changing if you can deliver)
- Great office space and perks (free beer on tap!)
- Private medical and travel insurance
- Dental insurance
- Life assurance - £150,000 per life
To be considered for this life-changing opportunity please apply ASAP with your CV, including any examples of your great skills via GitHub, BitBucket etc.
Edison Hill Limited are operating and advertising as an Employment Agency for permanent positions and as an Employment Business for interim / contract / temporary positions. Edison Hill Limited are an Equal Opportunities employer and we encourage applicants from all backgrounds.PLEASE ENSURE THAT THE JOB REFERENCE NUMBER APPEARS IN THE SUBJECT BOX.