AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards.
WHY JOIN US
If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you!
ABOUT THE ROLE
We are looking for a Data Engineer to execute large-scale data platform migrations to Snowflake and support reliable Airflow workflows.
MUST HAVES
- Strong software engineering fundamentals.
- 3+ years of hands-on Python development experience.
- Advanced SQL , including complex joins, CTEs, window functions, and performance tuning.
- Snowflake data modeling.
- Apache Airflow — developing and testing pipelines in Airflow and Python.
- Testing strategy experience, with the ability to hands-on debug and test code.
- At least 2 years of industry experience working with large-scale datasets, including Airflow and Spark.
- Overlap until 12 PM Pacific Time.
NICE TO HAVES
- Hands-on experience with Snowflake migrations (data warehouse modernization, performance and cost optimization).
- Experience with Apache Airflow migrations (version upgrades, executor changes, or platform migrations).
WHAT YOU WILL DO
- Work embedded within the data engineering function, partnering closely with the data platform team to support a large-scale data platform modernization effort.
- Execute the migration of approximately 1,000 datasets to Snowflake.
- Adapt existing pipelines to Snowflake, following the core architecture and technical approach defined by the internal team.
- Ensure data quality and reliability throughout the migration process.
- Support workflow scheduling and monitoring in Apache Airflow 3.
- Execute time-sensitive migrations, including Airflow 2 to 3, Confluent region migration, and the move to Iceberg.
- Help accelerate the migration initiatives targeted for completion by the end of March 2027.
PERKS AND BENEFITS
- Professional growth : Mentorship, TechTalks, and personalized growth roadmaps.
- Competitive compensation : USD-based pay with education, fitness, and team activity budgets.
- Exciting projects : Modern solutions with Fortune 500 and top product companies.
- Flextime : Flexible schedule with remote and office options.
Meet Our Recruitment Process
Each step helps us understand your skills and overall fit.
... (version upgrades, executor changes, or platform migrations). WHAT YOU WILL DO - - Work embedded within the data engineering function, partnering closely with the data platform team to support a large-scale data platform modernization effort. - - Execute the migration of approximately 1,000 datasets to Snowflake. - - Adapt existing ...
Data Engineer - Málaga | KRELL Consulting Buscamos incorporar un/a Data Engineer para participar en un proyecto de integración de plataformas, trabajando en el desarrollo y mantenimiento de soluciones de datos escalables y eficientes. Buscamos un perfil con experiencia en Python, Spark y Apache Airflow, valorándose especialmente ...
What you’ll do - Build the data-serving layer: curated datasets, marts, and product-ready tables - Develop incremental / micro-batch pipelines and support CDC near-real-time ingestion (AWS DMS) - Design BI-friendly data models (star schema) and manage schemas - Build ETL/ELT in Python (Polars) and serve/query via Athena ...
... are: - Migrate the existing Hadoop infrastructure to cloud infrastructure on Kubernetes Engine, COS, Spark as a service, and Airflow as a service. - Implement data transformation and quality to ensure data consistency and accuracy. Utilize programming languages such as Scala and SQL and tools like Spark for data transformation ...
Overview Importante consultora en tecnología busca un Data Engineer con mínimo 3 años de experiencia como desarrollador y mínimo 1 año de experiencia trabajando en proyectos de Big Data. Requisitos técnicos - Experiencia con las tecnologías Spark, YARN, Hadoop, Hive. - Experiencia con bases de datos relacionales (MySQL). ...
... the team alongside you, and grow into more of the platform as you go. You'll also ship with agentic AI tools as a real part of how we work — this is AI-native data engineering. If you like owning data end to end, from a Pub/Sub event to a Looker explore, and want scale you can't get just anywhere (hundreds of millions of ...
We are looking for an experienced Data Engineer with a strong technical background in data engineering, data architecture, and data pipeline development. The successful candidate will work within an Agile team to design, develop, test, and deploy data solutions that meet business requirements while adhering to the client's ...
... capacidades: - Experiencia mínima de 3 años como Data Engineer o en roles similares. - Conocimiento sólido del ecosistema de datos Microsoft Azure y Microsoft Fabric: Data Factory, Synapse, Databricks, Fabric (Lakehouse/Warehouse), Power BI y Purview. - Experiencia con servicios de almacenamiento en Azure: Data Lake Storage Gen2, ...
... cloud-based data services (AWS, Azure, or GCP). - Solid understanding of Data Lakes, Data Warehouses, and modern data architectures. - Experience with Python data engineering and analytics libraries, such as Pandas, NumPy, and PySpark and Spark. - Hands-on experience with Databricks, including data engineering, analytics, ...
... importance of our data is growing fast, and we're looking for a Data Engineer to build the infrastructure that keeps up. Your profile You'll design and build the data infrastructure that carries Bookboost from hundreds of gigabytes to terabytes of guest data, moving, transforming and making it available to our frontend application ...