AI Evaluation Engineer
4500 - 11000 SGDTemporary
Talentvis Singapore Pte Ltd
- Develop and maintain AI evaluation frameworks, regression test suites and automated evaluation pipelines.
- Build datasets, quality dashboards and evaluation tools to improve the performance of AI agents and LLM-powered applications.
- Design evaluation methods such as LLM-as-a-Judge, multi-turn conversations, tool evaluation and agent workflow analysis.
- Integrate AI evaluation into CI/CD and model release processes.
- Analyse production issues and improve datasets, prompts, evaluation logic and AI workflows.
- Conduct experiments to optimise model accuracy, retrieval performance, latency and cost.
- Build reusable platform capabilities and developer tools to support multiple AI products.
- Prepare technical documentation and collaborate with cross-functional engineering teams.
Requirements
- Bachelor's Degree in Computer Science, Software Engineering or a related discipline.
- Minimum 2 years of relevant experience in AI Platform Engineering, LLMOps, Machine Learning Engineering or Software Engineering.
- Experience with most of the following technologies: LangSmith, LangGraph / LangChain, FastAPI, Temporal, Grafana, Redis, React / TypeScript, EvalsHub or similar AI evaluation tools
- Experience with LLM evaluation, Retrieval-Augmented Generation (RAG), semantic search, embeddings or model benchmarking is preferred.
- Strong analytical and problem-solving skills.
- Good communication and stakeholder management skills.
- Able to work independently and collaboratively in a fast-paced environment.
Interested candidates are encouraged to submit their resumes outlining their relevant experience and achievements to sherriyang(@)talentvis.com or click apply!
**We regret to inform that only shortlisted candidates would be notified**
EA License No: 04C3537
EA Personnel No: R22106683
EA Personnel Name: Yang Hui Shan, Sherri
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the AI Evaluation Engineer in Singapore vacancy
8000 - 18000 SGD
...Role Overview｜岗位概述 We are looking for a Senior AI Systems Engineer with deep experience in multi-model systems, model routing, AI evaluation, and production AI infrastructure. The role will focus on designing systems that can compare, select, and operate multiple foundation...80 - 120 SGD per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Position: Compliance / regulatory response with financial-services AI Evaluator Type: Contract Compensation: $80–$120/hour...Remote job60 SGD per hour
...Toloka AI supports frontier model post-training by building domain-specific reinforcement learning environments, tasks, and evaluation frameworks designed by real practitioners. Mindrift, powered by Toloka — a leading enterprise AI and machine learning data partner since...- ...communication systems. Responsibilities: Design and implement stress tests for network infrastructures. Simulate Byzantine attacks to evaluate network security and resilience. Collaborate with a cross-functional team to enhance the communication stack. Analyze and...
50 SGD per hour
...Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is... ...is NOT Not data labeling Not prompt engineering Not writing code from scratch - the agent writes...2000 - 3000 SGD
...Company Overview The Company specializes in vehicle evaluation in Singapore. We uphold values of integrity, pride in work, and delivering... ...of vehicle exterior, interior, undercarriage, chassis, engine bay, and boot compartments to assess condition and identify issues...5000 - 6500 SGD
...Job: AI & Data Engineering Role: • Design and maintain scalable, secure and efficient data ingestion pipelines on AWS by collaborating... ...and serverless architecture. • Develop and maintain AI evaluation metrics and guardrails on AWS deployment. • Conduct...5000 - 6000 SGD
...About the Role We are looking for an AI Engineer to design, build, and maintain production-grade AI systems across our product and data... ...observability tooling for latency, drift, and reliability · Evaluate and integrate third-party AI services, vector databases, and embedding...3500 - 5000 SGD
...Architect, build, and maintain end-to-end AI/ML pipelines (data ingestion, preprocessing, training, deployment, predictive analytics... ...Limited and our group of companies, shall be used for the purposes of evaluation, and processing in accordance with our recruitment processes and...6000 - 7000 SGD
...Job Summary We are seeking an AI Specialist/Engineer to develop innovative AI solutions by applying machine learning, deep learning, and data... ...project decisions Design and conduct experiments to evaluate AI solution effectiveness and optimize outcomes Create and...5500 - 7500 SGD
...About Fragment Works and Moment AI FRAGMENTWORKS PTE. LTD. is a Singapore-incorporated... ...are looking for a talented and driven AI Engineer to join our growing team in Singapore.... ...ingestion, preprocessing, model training, evaluation, and deployment, in close collaboration with...- ...technology company for Bitcoin mining and AI cloud. Bitdeer is committed to... ..., latency, or cost. Work with runtime engineers to identify bottlenecks and verify improvements... ...infrastructure, performance engineering, model evaluation, QA automation, or backend testing for...
5000 - 6000 SGD
...About the Role We are building an AI-powered, multi-modal RAN optimisation platform and need a technically sharp junior engineer to help design, train, and deploy language model... ...mitigation. Key Responsibilities Evaluate, benchmark, select, deploy and optimise SLMs...2700 - 4800 SGD
...automation activities · Lab operations in equipment maintenance and inventory · Develop training program/materials for assistant engineer and technician to understand test procedure and requirement · Other duties as assigned by the supervisor REQUIREMENTS: · Bachelor...6000 - 12000 SGD
...Responsibilities End-to-End AI Model Optimization Support Responsible for model... ...and characteristics. Large-Scale Data Engineering Development Design and build efficient... ...principles, training workflows, and performance evaluation metrics; Data Engineering: Experience...8000 - 10000 SGD
...Description: We are looking for an IT Manager (AI & Digital Transformation) to drive the... .../teams. Work with architects and engineers to integrate AI solutions with enterprise... ...) and coordinate validation (UAT, model evaluation where applicable) to confirm benefits before...7000 - 13000 SGD
...Love building things with AI that people actually use? We are looking for an AI Platform Engineer to build the foundations behind intelligent products: LLM applications, RAG, agents, APIs, evaluation frameworks and the tooling that takes ideas from prototype to production...16000 - 31000 SGD
...iteration, and deployment of the company’s core AI algorithms, covering areas such as Large... .... 4. Collaborate closely with Product, Engineering, and DevOps teams to productize, deploy,... ...of data processing, model training, evaluation, and deployment. Strong coding standards...6000 - 7000 SGD
...Role Overview We are looking for an Applied AI Engineer to help us explore, develop and implement practical AI solutions across the... ...capabilities into internal systems and workflows. ● Test, evaluate and improve AI solutions based on business needs, accuracy,...4500 - 5600 SGD
...About the Role: We are looking for an AI Engineer to design, develop, and deploy intelligent systems powered by machine learning and Large... ...using cloud services (AWS serverless preferred) Monitor, evaluate, and improve model outputs continuously Collaborate with cross...6000 - 8000 SGD
...exciting role at the forefront of Generative AI transformation, building Singapore’s... ...assisted development, we are looking for a AI Engineer for the role in SGTraDex Technologies Pte... ...maintainable, and secure code Design, evaluate, and refine AI-generated outputs to meet...5800 - 6400 SGD
...may disrupt services. Qualifications & Requirements A good engineering or computing degree Basic knowledge on Linux Operating... ...operations engineers to establish continuous deployment plans Evaluate new tools and infrastructures to improve efficiency of the release...5800 - 6800 SGD
...transformation by designing and managing AI/IT systems that enhance manufacturing quality... ...Collaborate with R&D and hardware engineering teams to apply AI in navigation systems such... ...maritime and industrial data standards Evaluate and pilot AI tools and platforms...5000 - 8000 SGD
...real-world systems. This role exists to embed AI directly into those operations. We are seeking a Founding AI Engineer to serve as the first dedicated software engineering... ...AI into live business workflows with proper evaluation, observability, and guardrails Architect and...5000 - 7000 SGD
...culture that integrates diverse perspectives and fosters an innovative, results-oriented environment. about the job As an Evaluator, you will occupy a pivotal Middle Management role within the department. This is a high-impact position designed for a strategic...6900 - 8100 SGD
...within a team. Job Summary: The Data Engineer designs, builds and maintains data... ...integrations and data structures that support AI, machine learning, analytics and automation... ...structure data for machine learning, model evaluation and AI solution development. • Support feature...5500 - 8500 SGD
...As an Applied AI Engineer, you will turn model capabilities into real product behavior. You will own problems end-to-end, from shaping model... ..., cost, and production reliability Develop lightweight evaluation frameworks to measure real-world performance Work closely with...8000 - 13000 SGD
...Responsibilities Model Development : Implement, fine-tune, and evaluate advanced AI architectures (e.g., LLMs, Vision-Language Models, Diffusion... ..., distillation, and compilation techniques. Data Engineering : Build robust pipelines for data curation, deduplication,...6500 - 9000 SGD
...THE ROLE We are seeking a hands-on AI Applications Developer to design, build,... ...delivery rather than Azure infrastructure engineering . The successful candidate will... ...instructions, guardrails, response formats, evaluation criteria and testing datasets to improve...15000 - 20000 SGD
...The Role We’re looking for a Head of AI Engineering to own the technical direction of Thout.ai’s core AI systems end-to-end: the multi... ...piece of the AI pipeline end-to-end; establish or improve evaluation and testing practices; have a clear point of view on architecture...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Evaluation Engineer. Be the first to apply!
