Descripción de la vacante
Spain, España
Senior ML / Evaluation Engineer en España, España is listed on Jobeax. Browse 80,000+ vacancies available.
Senior ML / Evaluation Engineer
Location: Remote from Spain (an indefinite Spanish employment contract)
We are looking for a Senior ML / Evaluation Engineer to help define and implement quality standards for enterprise-grade AI agents and LLM-powered applications. In this role, you will design evaluation frameworks, build custom evaluation pipelines, and establish automated quality gates across the AI delivery lifecycle. You will work closely with AI Platform Engineers, ML Engineers, and DevOps teams to ensure reliable, measurable, and production-ready AI systems through scalable evaluation, observability, and governance practices.
Our customer is a multinational corporation with more than a century of history and offices in over 180 countries. Their most ambitious goal at the time is to introduce a range of Reduced-Risk Products (RRPs). The target audience is more than 1 billion consumers around the globe. IT platform hosts 700+ applications.
Intellia's mission is to help the client with the engineering of a comprehensive software ecosystem for a game-changing IoT product on the margin of innovative consumer experience and cutting-edge technology. Our teams are involved in the engineering of core platform components for best-in-class eCommerce, Digital Marketing and IoT solutions. As an Engineer, you will become a part of Core Architecture Team and be responsible for the architecture, implementation of best practices in our Digital Engineering Enterprise Platform.
The Platform is a set of services and internet applications that accelerate the development and delivery of software applications by taking care of common SDLC challenges. The Platform provides access and consumption for engineering teams to a set of services, technologies, practices for their development and for operating their application, ensuring a set of compliance and best practices.
Requirements:
• 5+ years ML engineering or AI platform engineering
• LLM evaluation framework design and implementation
• Custom evaluator implementation for deterministic quality checks
• CI/CD deployment gate design for ML model or agent quality
• AWS AgentCore Evaluation (on-demand mode for CI/CD gates, online mode for production sampling)
• LLM-as-judge evaluator design (built-in AgentCore evaluators — helpfulness, correctness)
• Custom code-based Lambda evaluators (Python — deterministic checks)
• Evaluation levels (TRACE for per-response, TOOL_CALL for per-invocation, SESSION for workflow)
• OTel spans from AWS AgentCore Observability as evaluation input
• Enterprise evaluation standard authoring (mandatory dimensions, pass/fail criteria)
Nice-to-have
• AWS AgentCore Evaluation API hands-on (CreateEvaluation, GetEvaluationResult)
• AWS Bedrock Guardrails for PII detection evaluator integration
• CloudWatch metrics output from AgentCore Evaluation for online mode
Responsibilities:
Vacantes similares
Fully remote
... consuming existing benchmarks. - Clear, precise communicator who can translate quantitative evaluation results into decisions for researchers, product teams, and senior leadership. Nice to have - Deep familiarity with open-source evaluation frameworks - Experience designing evaluations for agentic systems: tool use, multi-turn ...
Spain
Blu Selection
60k - 70k / year
... data-driven products across their international ecosystem. You'll be jumping right into the heart of their digital transformation journey!And your role in this?As the Senior AI & ML Engineer, you are the technical backbone of the end-to-end machine learning lifecycle. You’ll sit right at the core of the digital and data team, applying ...
Spain, Barcelona
Permanent contract
... solutions. Our mission is to bring happiness to life through play. We value collaboration, creativity, and a strong passion for excellence. We seek an ambitious ML/AI Engineer to join our team. You will help improve our AI capabilities and craft the future of our products. This role offers a chance to work on mature machine ...
Spain
Grid Dynamics
Grid Dynamics busca un Senior Machine Learning Engineer enfocado en sistemas LLM y evaluación para liderar proyectos de IA de última generación, incluidas técnicas de RAG, agentes y evaluación de seguridad AI. El candidato ideal tiene 5+ años en ML engineering, fuerte Python y experiencia con PyTorch, TensorFlow o JAX, ...
Spain, Castro-Urdiales
A leading AI solutions provider in Barcelona is seeking a Senior AI/ML Engineer. This role involves developing and deploying AI initiatives, focusing on the full lifecycle of machine learning solutions. The ideal candidate will have strong programming skills, expertise in MLOps, and a passion for driving real-world impact ...
Spain, Barcelona
Up to US$40 an hour
... ones, neither too strict nor too lenient - Iterate on tasks and tests based on QA feedback - review agent solutions, analyze failures, and refine until the evaluation is fair and robust - Not data labeling - Not prompt engineering - Not writing code from scratch - the agent writes most of the code; you guide and evaluate ...
Spain
de 80000 a 120000 $/año
... hypotheses into production-ready engineering solutions that improve search quality, ranking, and user experience. - Work on large-scale distributed data pipelines and ML workflows using big data technologies to support end-to-end model development and deployment. - Collaborate closely with cross-functional engineering teams to ...
Spain, Madrid
Sword Health
€60,000 - €85,000 a year
... agentic LLM systems: design multi-step workflows with tool use, retrieval, and orchestration, and make them reliable enough for clinical settings; - Treat evaluation as core engineering work: build eval sets, offline and online harnesses, LLM-as-judge pipelines with human review, and regression tests that catch quality ...
Spain
The BIG Jobsite
€270 EUR - €300 EUR
SEEKING : Senior AI/Platform Engineer We are looking for a Senior AI/Platform Engineer GOAL : - Engage with project delivery teams during planning and execution to build architecture recommendations Productize Mosaic AI into a governed, multi-tenant, self-service platform - Shipping golden paths, SDK/CLI/templates, and ...
Spain, Vitoria
€270 EUR - €300 EUR
SEEKING : Senior AI/Platform Engineer We are looking for a Senior AI/Platform Engineer GOAL : ► Engage with project delivery teams during planning and execution to build architecture recommendations Productize Mosaic AI into a governed, multi-tenant, self-service platform ► Shipping golden paths, SDK/CLI/templates, and ...
Spain, Cequelinos
€270 EUR - €300 EUR
SEEKING : Senior AI/Platform Engineer We are looking for a Senior AI/Platform Engineer GOAL : ► Engage with project delivery teams during planning and execution to build architecture recommendations Productize Mosaic AI into a governed, multi-tenant, self-service platform ► Shipping golden paths, SDK/CLI/templates, and ...
Spain
€270 EUR - €300 EUR
SEEKING : Senior AI/Platform Engineer We are looking for a Senior AI/Platform Engineer GOAL : ► Engage with project delivery teams during planning and execution to build architecture recommendations Productize Mosaic AI into a governed, multi-tenant, self-service platform ► Shipping golden paths, SDK/CLI/templates, and ...
Spain, Castro
Full-time hours
... plus, or strong ability to ramp up quickly. - Practical experience with LLM APIs (Anthropic, AWS Bedrock, or OpenAI) in real projects. - 2+ years of software or ML engineering experience, including some exposure to production systems. - Solid AI/ML foundations — you understand what the models do well enough to reason about ...
Spain
Full-time hours
... de Llobregat, Barcelona, Spain, Madrid, Spain Job Description Johnson & Johnson Innovative Medicine R&D Data Science and Digital Health is recruiting for a Senior Engineer - GenAI Quality Assurance. This role can be based in Madrid or Barcelona. Hybrid working. J&J Innovative Medicine develops treatments that improve the ...
Spain
All European Careers
... Castro e Marzán, Galicia, Spain Descripción del trabajo Join our dynamic international team and embrace the adventure of working abroad! We are looking for a Senior AI & NLP Engineer for the development and maintenance of software applications in the field of Natural Language Processing (NLP), Machine Learning (ML) and/or ...
Spain, Mos
Have the right to work in Spain
... processes. - You will prepare system review reports based on the collected and analyzed data. - You will coordinate follow-up groups to communicate process evaluations and trends. - You will review documentation that describes production operations and the functioning of equipment and facilities. - You will review qualification ...
Spain
Fully remote
... - Prototype and validate new AI workflows, including debugging, code understanding, refactoring, and project navigation. - Collaborate with product managers, ML engineers, QA engineers, stakeholders, and senior developers to shape and deliver the roadmap. - Work closely with JetBrains AI, ML, and product teams to help ...
Spain
Fully remote
... - Prototype and validate new AI workflows, including debugging, code understanding, refactoring, and project navigation. - Collaborate with product managers, ML engineers, QA engineers, stakeholders, and senior developers to shape and deliver the roadmap. - Work closely with JetBrains AI, ML, and product teams to help ...
Spain
US$12 an hour
... text captions and task requirements. - Voice Quality: Evaluate whether generated voices sound natural and, where applicable, match the reference audio. - Evaluation & Feedback: Assign accurate scores and identify issues according to the provided evaluation guidelines. What we're looking for - Native-level fluency in Spanish ...
Spain
Flomics Biotech
Flomics Biotech in Barcelona is seeking a Bioinformatics Senior Scientist to join our growing team. You will help develop pipelines for early cancer diagnosis using RNA-seq, apply machine learning, and contribute to publications and regulatory submissions. You should have a PhD in bioinformatics or related field with 3+ ...
Spain, Barcelona