ML Evaluation Designer (Expert)

About the role

We are looking for an experienced Machine Learning Engineer to design and implement evaluation tasks for large language models. The role is contract‑based, remote, and requires 30+ hours per week.

Key responsibilities

• Design evaluation tasks, rubrics and metrics to assess LLM performance.
• Grade model and agent outputs, providing detailed feedback to improve evaluation quality.
• Apply training‑side judgment techniques such as supervised fine‑tuning (SFT), reinforcement learning from human feedback (RLHF) and reward modeling to shape evaluation design.
• Collaborate with AI research teams to refine evaluation signals and enhance model outputs.
• Work independently and asynchronously, meeting deadlines and driving continuous improvement of AI models.

Required profile

• 5+ years of experience as a Machine Learning Engineer with hands‑on training and evaluation work.
• Strong written communication skills.

Required skills

• Proficiency in PyTorch.
• Experience with JAX.
• Familiarity with Hugging Face libraries.
• Knowledge of SFT, RLHF and reward modeling techniques.

What we offer

• Hourly rate ranging from $45 to $140.
• Flexible remote work schedule.
• Opportunity to influence cutting‑edge AI research.