Evaluations Team Lead

<h3>Accountabilities:</h3><li>Build and maintain a shared evaluation platform that supports research, engineering, and Applied AI teams, enabling independent benchmarking and investigation of model performance.</li><li>Develop and continuously improve benchmark suites, curated datasets, and model comparisons to assess predictive quality across a broad range of business use cases.</li><li>Establish rigorous evaluation standards covering statistical metrics, validation strategies, data splits, leakage prevention, calibration, uncertainty estimation, and benchmark contamination.</li><li>Design reproducible evaluation pipelines with version-controlled inputs, datasets, and artifacts, integrating automated regression checks into research and release workflows.</li><li>Benchmark models against competing approaches using fair and consistent protocols for hyperparameter tuning, data access, computational resources, and end-to-end latency measurement.</li><li>Evaluate predictive performance, inference latency, and cost efficiency across deployment configurations, task types, and dataset characteristics.</li><li>Support customer proof-of-concept evaluations by providing reliable tooling, methodological guidance, and data-driven analysis to help demonstrate model value.</li><li>Translate evaluation findings into actionable research priorities, release recommendations, and evidence-based improvement plans for customer use cases.</li><li>Recruit, lead, and develop a small team of evaluation specialists, setting priorities, providing technical coaching, and delivering constructive performance feedback.</li><li>Remain hands-on with Python development, experimental design, statistical analysis, and evaluation methodology while fostering collaboration across research, engineering, and customer-facing teams.</li><h2>Requirements:</h2><li>Proven experience owning the evaluation of tabular machine learning systems used in production environments or high-impact customer decision-making.</li><li>Strong statistical expertise, including selecting appropriate metrics and validation schemes, estimating uncertainty, comparing models across datasets, and accounting for repeated experimentation.</li><li>Practical experience identifying and preventing data leakage involving preprocessing, feature engineering, joins, temporal dependencies, and related entities across training and evaluation splits.</li><li>Strong proficiency in Python and SQL, with familiarity with scikit-learn, gradient-boosted decision trees, and the development of reliable ML tools or platforms used by multiple teams.</li><li>Demonstrated experience designing fair and reproducible model comparisons, including hyperparameter optimization, resource allocation, and end-to-end inference latency measurement.</li><li>Previous people management experience, including hiring, technical mentoring, team development, and performance management, combined with a willingness to remain actively involved in technical work.</li><li>Excellent written and verbal communication skills, with the ability to collaborate effectively with researchers, software engineers, and customer data scientists.</li><li>A strong analytical mindset and the confidence to challenge unsupported conclusions, communicate uncertainty, and ensure that technical decisions are grounded in reliable evidence.</li><li>Experience evaluating tabular foundation models or automated machine learning (AutoML) systems is an advantage.</li><li>Familiarity with measuring the impact of model optimizations on predictive performance and inference efficiency is desirable.</li><li>Experience working with relational or multi-table datasets and data platforms such as Snowflake or Databricks is a plus.</li><li>Knowledge of automated or agent-driven machine learning workflows, including identifying failure modes that aggregate metrics may conceal, is beneficial.</li><h2>Benefits:</h2><li>Competitive compensation package combining salary and equity.</li><li>Comprehensive health coverage for employees and their dependents.</li><li>Paid parental leave for all new parents, including adoptive and surrogacy journeys.</li><li>Relocation assistance for employees moving to join the team at one of the company's office locations.</li><li>Remote working opportunities across eligible locations in Europe and Israel.</li><li>The opportunity to establish evaluation standards and build a new function from the ground up, with substantial technical ownership and influence.</li><li>A mission-driven, collaborative, and low-ego culture that values diverse perspectives, individual ownership, and proactive decision-making.</li><li>The opportunity to work on advanced AI research and help improve the reliability, predictive performance, and efficiency of enterprise AI systems.</li>