Data Engineer
Data Engineer
Salary: £70,000 - £80,000
Contract: 6 month fixed term contract
Location: City of London / hybrid
We are supporting a leading organisation within the specialty insurance market that is investing heavily in a major transformation programme across its data, technology & operating model.
As part of this programme, they are looking for an experienced Data Engineer to join on a 6 month fixed term contract & play a key role in delivering a modern cloud based data capability.
This is a hands on delivery role, suited to someone who can take ownership of a defined piece of work, build high quality data pipelines, & help create reliable datasets that support reporting, analytics & business decision making.
The successful candidate will be working with modern Azure data technologies, helping to improve how data is ingested, transformed, validated & made available to the business.
Key responsibilities
- Design & build scalable data pipelines using Azure Databricks
- Transform raw data into clean, structured & reliable datasets for reporting & analytics
- Develop Databricks notebooks, jobs & workflows to support end to end data processing
- Implement data quality checks, validation rules & monitoring processes
- Troubleshoot pipeline issues & improve reliability, performance & maintainability
- Work across development & production environments using CI/CD principles
- Collaborate with data, technology & business stakeholders as part of a wider transformation programme
- Produce clear technical documentation to support future ownership & handover
Experience required
- Strong commercial experience as a Data Engineer
- Hands on experience with Azure Databricks
- Strong Python, PySpark & Spark SQL skills
- Experience building & optimising data pipelines
- Good understanding of modern cloud data platforms & Lakehouse architecture
- Experience working with Delta Lake or similar technologies
- Understanding of data quality, validation, monitoring & pipeline performance
- Experience with CI/CD processes, ideally using Azure DevOps
- Ability to work independently & deliver within a project focused environment
Desirable experience
- Experience supporting reporting or BI platforms such as Power BI
- Experience within insurance, financial services or another regulated environment
- Exposure to enterprise transformation or data modernisation programmes
This is a strong opportunity for a Data Engineer to join a business undergoing meaningful transformation & take ownership of a visible data delivery project using modern Azure technologies. Apply following the link
Data & Application Notice
We process personal data submitted as part of your application for recruitment purposes in line with UK GDPR. All applications are reviewed by a member of our team, we do not use automated decision-making in our selection process.
Your application will be assessed against the requirements of this role.
By applying to this advert, you consent to receiving future relevant roles & industry news & insights from us. To opt out of this, please contact: compliance@pioneer-search.com.
For more information on how we use your data, including how long we retain it & your rights, please refer to our Privacy Policy: www.pioneer-search.com/privacy-policy.
