Data Engineer- Databricks & Treasury Data Lake en Málaga, España

SET Europa provides near shore, recruitment and outsourcing services and solutions for immediate project needs.

Our client is a recruitment consultancy specialising in IT, Finance, HR, and Digital Technology. Our client is seeking an experienced Data Engineer specializing in Databricks and Treasury Data Lake environments to join a fully remote project.

The successful candidate will be responsible for designing, developing, maintaining, and optimizing end-to-end data pipelines within a modern Lakehouse architecture, ensuring reliable, scalable, and high-performance data solutions. This role requires strong hands‑on experience with Databricks, DBT, advanced SQL, and enterprise data integration, alongside a solid understanding of data governance, automation, and CI/CD practices.

RESPONSIBILITIES

• Design, build, and maintain end‑to‑end data ingestion pipelines integrating multiple source systems.
• Develop and optimise data transformations across bronze, silver and gold layers within a Databricks Lakehouse architecture.
• Implement automated data quality controls, monitoring frameworks and alerting mechanisms to ensure data integrity and reliability.
• Design and maintain data models and transformation logic using DBT and advanced SQL.
• Leverage Databricks Lakehouse Federation and Lakeflow Connect to support efficient data integration and accessibility.
• Develop and maintain CI/CD pipelines using GitHub and Azure DevOps, ensuring controlled deployments, testing and version management.
• Identify and resolve performance bottlenecks, improving scalability, maintainability and efficiency.
• Collaborate with business and technical stakeholders to translate requirements into robust data engineering solutions.
• Support data governance standards, documentation, troubleshooting and release management activities.
• Contribute to analytics and reporting capabilities by delivering reliable, high‑quality data products.

Requirements

• Proven professional experience as a Data Engineer in enterprise data environments.
• Strong hands‑on expertise with Databricks and modern Data Lake/Lakehouse architecture.
• Practical experience developing data transformations using DBT (Data Build Tool) .
• Advanced SQL skills, including query optimisation and complex data transformations.
• Experience designing and managing data pipelines across bronze, silver and gold layers.
• Knowledge of Databricks Lakehouse Federation and Lakeflow Connect .
• Experience implementing data quality frameworks, monitoring and automated alerting.