Importante consultora en tecnología busca un Data Engineer con mínimo 3 años de experiencia como desarrollador y mínimo 1 año de experiencia trabajando en proyectos de Big Data.
Requisitos técnicos
Experiencia con las tecnologías Spark, YARN, Hadoop, Hive.
Experiencia con bases de datos relacionales (MySQL).
Lenguajes de programación.
Programaciónfuncional.
Experiencia con Linux y shell scripting.
Familiarizado con entornos cloud, particularmente Google Cloud Platform (se valorarán conocimientos de AWS o Azure).
Familiarizado con Agile, GitHub, JIRA/Redmine, ContinuousIntegration).
Responsabilidades
Participará en el desarrollo y mantenimiento de procesos de extracción, transformación y carga (ETL) sobre datos provenientes de las distintas áreas de negocio.
Análisis y diseño de los procesos ETL en base a los requerimientos funcionales, teniendo en cuenta también los no funcionales como el rendimiento o la escalabilidad.
Desarrollo y pruebas unitarias.
Participación en revisiones de código.
Automatización de procesos y soporte a IT en el despliegue y mantenimiento.
What you’ll do - Build the data-serving layer: curated datasets, marts, and product-ready tables - Develop incremental / micro-batch pipelines and support CDC near-real-time ingestion (AWS DMS) - Design BI-friendly data models (star schema) and manage schemas - Build ETL/ELT in Python (Polars) and serve/query via Athena ...
... are: - Migrate the existing Hadoop infrastructure to cloud infrastructure on Kubernetes Engine, COS, Spark as a service, and Airflow as a service. - Implement data transformation and quality to ensure data consistency and accuracy. Utilize programming languages such as Scala and SQL and tools like Spark for data transformation ...
... the team alongside you, and grow into more of the platform as you go. You'll also ship with agentic AI tools as a real part of how we work — this is AI-native data engineering. If you like owning data end to end, from a Pub/Sub event to a Looker explore, and want scale you can't get just anywhere (hundreds of millions of ...
We are looking for an experienced Data Engineer with a strong technical background in data engineering, data architecture, and data pipeline development. The successful candidate will work within an Agile team to design, develop, test, and deploy data solutions that meet business requirements while adhering to the client's ...
... capacidades: - Experiencia mínima de 3 años como Data Engineer o en roles similares. - Conocimiento sólido del ecosistema de datos Microsoft Azure y Microsoft Fabric: Data Factory, Synapse, Databricks, Fabric (Lakehouse/Warehouse), Power BI y Purview. - Experiencia con servicios de almacenamiento en Azure: Data Lake Storage Gen2, ...
... cloud-based data services (AWS, Azure, or GCP). - Solid understanding of Data Lakes, Data Warehouses, and modern data architectures. - Experience with Python data engineering and analytics libraries, such as Pandas, NumPy, and PySpark and Spark. - Hands-on experience with Databricks, including data engineering, analytics, ...
... importance of our data is growing fast, and we're looking for a Data Engineer to build the infrastructure that keeps up. Your profile You'll design and build the data infrastructure that carries Bookboost from hundreds of gigabytes to terabytes of guest data, moving, transforming and making it available to our frontend application ...
Misión del rol El rol de Data Engineer es eminentemente técnico y de ejecución. Buscamos a alguien que disfrute escribiendo código, construyendo pipelines robustos y resolviendo con velocidad las necesidades cambiantes del negocio. El Data Engineer será el propietario técnico de Azure Data Factory (ADF) y dbt — las herramientas ...
... commitments. Qualifications Skills and Experience Required 3+ years in data engineering or related data processing roles. Strong experience working with varied data formats and sources (JSON, CSV, Parquet, APIs, multiple DB engines). Hands-on experience with data modelling in Big Data platforms (e.g. Databricks, Glue, Snowflake). ...
... Titan OS is looking for a Data Engineer to join our talented Data team in Barcelona. In this role, you will play a crucial part in building and maintaining the data infrastructure that empowers our operating system to transform data into actionable insights. If you are passionate about data, have a strong technical skillset, ...