Descripción de la vacante
Mindrift
Up to US$70,000 a year
Spain
Freelance Agent Evaluation Engineer en Spain, España is listed on Jobeax. Browse 80,000+ vacancies available.
We're building a dataset to evaluate AI coding agents - how well a model handles real-world developer tasks.
You'll create challenging tasks and evaluation criteria within realistic simulated environments:
Why this is hard:
Frontier models are already good at coding. Creating a task that genuinely challenges the best models is non-trivial. You need to deeply understand where models fail and what scenarios reveal the difference between a good and a bad solution. Tasks have many valid solutions - writing tests that accept all correct solutions and reject incorrect ones is harder than it sounds.
Candidates should have a minimum of 3 years of professional experience in related roles or domain - specifically for QA-automation/testing or cybersecurity roles
Educational qualifications
Compensation:
Paid per accepted task. Your rate depends on the qualification tier you reach and how efficiently you complete tasks — up to the equivalent of $40/hr. Because payment is per task, a faster pace raises your effective hourly rate.
part-time
remote
USD 70,000 per year
Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment.
Creator (Writer)
Please submit your CV in English and indicate your level of English proficiency.
Academic and/or Professional Experience
How it works
Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid
Vacantes similares
Up to US$40 an hour
... ones, neither too strict nor too lenient - Iterate on tasks and tests based on QA feedback - review agent solutions, analyze failures, and refine until the evaluation is fair and robust - Not data labeling - Not prompt engineering - Not writing code from scratch - the agent writes most of the code; you guide and evaluate ...
Spain
Up to US$70,000 a year
... focused on testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment. We're building a dataset to evaluate AI coding agents - how well a model handles real-world developer tasks. You'll create challenging tasks and evaluation criteria within realistic simulated environments: What ...
Spain
Up to US$70,000 a year
... focused on testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment. We're building a dataset to evaluate AI coding agents - how well a model handles real-world developer tasks. What you'll do You'll create challenging tasks and evaluation criteria within realistic simulated environments: ...
Spain
Up to US$262,500 a year
... improving AI systems. Participation is project-based, not permanent employment. What this opportunity involves: We're building a dataset to evaluate AI coding agents - how well a model handles real-world developer tasks. You'll create challenging tasks and evaluation criteria within realistic simulated environments: What ...
Spain
The BIG Jobsite
€270 EUR - €300 EUR
... self-service platform - Shipping golden paths, SDK/CLI/templates, and evaluation patterns. REQUIREMENTS : - Deep expertise in Databricks Mosaic AI platform and agent development - Experience with agentic AI architectures - autonomous agents, multi-agent systems, and orchestration - Databricks MLflow, AutoML, and Vector Search ...
Spain, Vitoria
€270 EUR - €300 EUR
... self-service platform ► Shipping golden paths, SDK/CLI/templates, and evaluation patterns. REQUIREMENTS : ■ Deep expertise in Databricks Mosaic AI platform and agent development ■ Experience with agentic AI architectures ■ autonomous agents, multi-agent systems, and orchestration ■ Databricks MLflow, AutoML, and Vector Search ...
Spain
€270 EUR - €300 EUR
... self-service platform ► Shipping golden paths, SDK/CLI/templates, and evaluation patterns. REQUIREMENTS : ■ Deep expertise in Databricks Mosaic AI platform and agent development ■ Experience with agentic AI architectures ■ autonomous agents, multi-agent systems, and orchestration ■ Databricks MLflow, AutoML, and Vector Search ...
Spain, Castro
Fully remote
... translate quantitative evaluation results into decisions for researchers, product teams, and senior leadership. Nice to have - Deep familiarity with open-source evaluation frameworks - Experience designing evaluations for agentic systems: tool use, multi-turn reasoning, environment-based benchmarks (SWE-bench, GAIA, WebArena-style), ...
Spain
Grid Dynamics
Grid Dynamics busca un Senior Machine Learning Engineer enfocado en sistemas LLM y evaluación para liderar proyectos de IA de última generación, incluidas técnicas de RAG, agentes y evaluación de seguridad AI. El candidato ideal tiene 5+ años en ML engineering, fuerte Python y experiencia con PyTorch, TensorFlow o JAX, ...
Spain, Castro-Urdiales
Up to US$70,000 a year
... ones, neither too strict nor too lenient - Iterate on tasks and tests based on QA feedback - review agent solutions, analyze failures, and refine until the evaluation is fair and robust What this is NOT - Not data labeling - Not prompt engineering - Not writing code from scratch - the agent writes most of the code; you guide ...
Spain
From €210 a day
... solid engineering foundation. - You have native-level Spanish and fluent English. Bonus points if you speak French! Education & certifications - Contract type: Freelance - Location: Madrid - Experience: > 6 months - Possible full remote - Salary: €210 /day Our team Our technical team is made up of more than 90 engineers from ...
Spain
Have the right to work in Spain
... processes. - You will prepare system review reports based on the collected and analyzed data. - You will coordinate follow-up groups to communicate process evaluations and trends. - You will review documentation that describes production operations and the functioning of equipment and facilities. - You will review qualification ...
Spain
Una compañía con equipos de IA avanzada incorpora un/a AI Runtime & Evaluation Engineer para el diseño, implementación y evaluación de arquitecturas de agentes de IA a escala productiva. Rol clave en la creación de sistemas robustos de evaluación y benchmarking de LLMs, gestión del runtime de agentes y observabilidad avanzada. ...
Spain, León
FirstIgnite
... trust. • You're based somewhere between New York time (ET) and Western European time. Italy is the furthest east we can go. You might currently be titled AI Agent Engineer · Agentic AI Engineer · Senior AI Engineer · AI Systems Engineer · Lead AI Developer · AI Solutions Architect · AI Integration Specialist · Senior Data ...
Spain, Castro
... de 35 años de experiencia en el aprovechamiento del viento, la compañía cuenta con fábricas a nivel mundial en más de 32 países. Actualmente buscamos un/a FREELANCE/AUTÓNOMO Data Engineer para incorporarse a su equipo y para diseñar y operar pipelines escalables de ingestión de datos utilizando Azure Databricks, estandarizar ...
Spain, Valencia
Ad4 Octogono
22 - 25 €/hora
En AD4 Octógono buscamos un/a Senior OSS Engineer para incorporarse a un proyecto especializado dentro del sector de Telecomunicaciones , trabajando sobre entornos OSS de alta exigencia técnica. Buscamos un perfil hands-on , con experiencia real en administración, desarrollo y mantenimiento de soluciones sobre Oracle Unified ...
Spain, Madrid
... ¡Un ReCoder es un aficionado de la tecnología, es [la clave de nuestro éxito], la clave de nuestra Fuerza!”. Actualmente estamos buscando un/a Google Cloud Engineer , para incorporarse con nosotros al proyecto de uno de nuestros clientes en modalidad remota. Estamos buscando un profesional que pueda trabajar como freelance ...
Spain, Castro
... ¡Un ReCoder es un aficionado de la tecnología, es [la clave de nuestro éxito], la clave de nuestra Fuerza!”. Actualmente estamos buscando un/a Google Cloud Engineer , para incorporarse con nosotros al proyecto de uno de nuestros clientes en modalidad remota. Estamos buscando un profesional que pueda trabajar como freelance ...
Spain, Vitoria
... ¡Un ReCoder es un aficionado de la tecnología, es [la clave de nuestro éxito], la clave de nuestra Fuerza!". Actualmente estamos buscando un/a Google Cloud Engineer, para incorporarse con nosotros al proyecto de uno de nuestros clientes en modalidad remota. Estamos buscando un profesional que pueda trabajar como freelance ...
Spain, Madrid