Senior LLM Inference Performance & Evaluation Engineer
Full-time
Bitdeer Technologies Group
About Bitdeer:
Bitdeer is a world-leading technology company for Bitcoin mining and AI cloud.
Bitdeer is committed to providing comprehensive Bitcoin mining solutions for its customers. Apart from designing industry-leading ASIC chips and manufacturing mining rigs, the Group handles complex processes involved in computing across the value chain. This includes equipment procurement, transport logistics, datacenter design and construction, equipment management, and network and facility operations. Bitdeer also offers advanced cloud capabilities to customers with a high demand for artificial intelligence. Headquartered in Singapore, Bitdeer operates globally with a diversified 3 GW energy portfolio, and deploys Bitcoin mining and HPC datacenters in the United States, Bhutan, Norway, Canada, Malaysia, and Ethiopia. What you will be responsible for:
- Build benchmark pipelines for TTFT, inter-token latency, output throughput, request latency, concurrency curves, error rate, and GPU utilization.
- Create model launch gates covering OpenAI/Anthropic compatibility, streaming behavior, tool calling, reasoning output, multimodal behavior, and long-context cases.
- Maintain representative workloads using synthetic, replayed, and customer-like traffic patterns for both steady-state and burst scenarios.
- Compare model/runtime/provider options and publish clear recommendations for routing, fallback, pricing, and capacity decisions.
- Automate regression detection in CI/CD and staging so model, runtime, or configuration changes do not silently degrade quality, latency, or cost.
- Work with runtime engineers to identify bottlenecks and verify improvements; work with SRE to convert benchmark results into SLOs and alert thresholds.
How you will stand out:
- 5+ years in ML infrastructure, performance engineering, model evaluation, QA automation, or backend testing for production systems.
- Experience with LLM serving metrics such as TTFT, TPOT/ITL, request latency, token throughput, concurrency, and GPU utilization.
- Strong Python skills for benchmark/evaluation automation; Go experience is a plus for integrating with platform services.
- Familiarity with OpenAI/Anthropic APIs, vLLM/Dynamo/SGLang/Triton-style servers, and Kubernetes-based test environments.
- Able to design statistically meaningful tests and communicate tradeoffs between model quality, latency, reliability, cost, and customer experience.
- Experience building dashboards, reports, and release gates for engineering, product, and business stakeholders.
What you will experience working with us:
- A culture that values authenticity and diversity of thoughts and backgrounds;
- An inclusive and respectable environment with open workspaces and exciting start-up spirit;
- Fast-growing company with the chance to network with industrial pioneers and enthusiasts;
- Ability to contribute directly and make an impact on the future of the digital asset industry;
- Involvement in new projects, developing processes/systems;
- Personal accountability, autonomy, fast growth, and learning opportunities;
- Attractive welfare benefits and developmental opportunities such as training and mentoring.
--------------------------------------------------------------------
Bitdeer is committed to providing equal employment opportunities in accordance with country, state, and local laws. Bitdeer does not discriminate against employees or applicants based on conditions such as race, colour, gender identity and/or expression, sexual orientation, marital and/or parental status, religion, political opinion, nationality, ethnic background or social origin, social status, disability, age, indigenous status, and union.
#LI-ST1
Vacancy posted a month ago
Similar jobs that could be interesting for youBased on the Senior LLM Inference Performance & Evaluation Engineer in Singapore vacancy
- ...The role You'll own the cost and performance of our inference stack. Your work will determine how efficiently... .... You'll work closely with the engineers operating the serving fleet while... ...engines such as vLLM, SGLang, and TensorRT-LLM, going below the framework when needed...
11000 - 14000 SGD
...users. We are looking for a Lead AI Research Engineer to establish the scientific and technical foundations for evaluating our LLM and agent systems. This role operates at... ...differences between offline evaluation and production performance. - Translate research findings into...6000 - 12000 SGD
...Summary We are seeking an AI Evaluation Engineer to help understand, analyse, and improve the performance of advanced AI systems. This role combines data-driven analysis... ...Frameworks: Develop deterministic, heuristic, and LLM-assisted evaluation frameworks (e.g., LLM-as-a-...5000 - 10000 SGD
...Summary We are looking for an ML Evaluation Engineer/Data Scientist to bring statistical rigor... ...excellence to the evaluation of AI and LLM systems. This role will focus on ensuring... ...critical questions such as: "Is this performance improvement real?", "Are our evaluators...- ...in the United States, Bhutan, Norway, Canada, Malaysia, and Ethiopia. About the team We are seeking a Senior Inference Runtime Engineer to own the performance-critical serving layer of the MaaS platform. This role focuses on making self-hosted LLMs faster, cheaper, and...
7000 - 10500 SGD
...Responsibilities: Design, develop, and execute performance, load, stress, endurance, and... ...in Performance Testing and Performance Engineering. ~ Strong hands-on experience with tools... ...companies based in any country for purposes of evaluating suitability for employment, conducting...6000 - 9000 SGD
...AI Engineer/ /Senior Developer (LLM) • Configure and deploy AI agents • Design and build AI-powered workflows, automation pipelines, and agentic systems using AI-native platforms and APIs • Evaluate AI outputs, identify failure modes • Integrate AI capabilities into...3000 - 4000 SGD
...Build, iterate, andmaintain LLM-powered applications — including chatbots, document processingpipelines... ...— maintaining prompt libraries, evaluating promptperformance, and implementing... ...to improve domainaccuracy and reduce inference costs Integrate LLM capabilitiesinto YCH...10000 - 20000 SGD
...What you will be doing Build LLM-powered applications... ...business lines Build automated evaluation, monitoring, and alerting systems... ...to continuously track model performance across accuracy, latency, and... ...in production AI application engineering Proficient in Python or...- ...About the Team The Cloud Performance Engineering team is responsible for building cloud native ClickHouse Cloud Platform that will transform the OLAP space. Our team is looking for exceptional performance engineers with a proven track record of understanding the performance...
15000 - 24000 SGD
...mission is to grow vLLM as the world's AI inference engine and accelerate AI progress by making... ...push the boundaries of what's possible in LLM and diffusion model serving. Models grow... ...Demonstrate the ability to contribute performant and maintainable code and debug in complex...6000 - 9500 SGD
...managing, and allocating budget and manpower resources as well as evaluating budget requests for various programmes; • Funding Policy... ...strengthen CET grants governance for WOG and SWDA. The Manager/Senior Manager is an individual contributor role who reports to the Team...12000 - 15000 SGD
...Engineering Leader: Healthcare AI (People Management + Technical Architecture)... ...-grade generative AI and advanced LLM applications. Direct the creation... ...engineering standards across model evaluation, continuous testing, CI/CD, and inference optimization. Partner closely...35 SGD per hour
...leading tech companies, focused on testing, evaluating, and improving AI systems.... ...Not in scope Data labeling, prompt engineering. Production code to ship — you design... ...hours. Higher volume available for top performers. You choose when and how to contribute...8000 - 18000 SGD
...Role Overview｜岗位概述 We are looking for a Senior AI Systems Engineer with deep experience in multi-model systems, model routing, AI evaluation, and production AI infrastructure. The... ...策略和决策标准。 Continuously improve routing performance using evaluation results, production...3400 - 8000 SGD
...for Singaporeans. In this role, you will support the Team Lead in evaluating programmes, strengthening data administrative practices, and... ...consistency, accuracy, and timeliness across divisions and for senior management · Working with internal teams to track programme impact...10000 - 18000 SGD
...models, agentic AI, and modern AI engineering. We are looking for an... ...building production-ready LLM applications, AI infrastructure... ...Responsibilities Lead and grow a high-performing Machine Learning Engineering... ...workflows covering model evaluation, experimentation, deployment,...50 SGD per hour
...opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based, not... ...robust What this is NOT Not data labeling Not prompt engineering Not writing code from scratch - the agent writes most of the...4800 - 5200 SGD
...Deliver professional live vocal performances at licensed entertainment venues, including KTV lounges Deliver engaging stage performances to enhance customer entertainment experience Curate and manage a dynamic song repertoire aligned with venue atmosphere and audience...14000 - 25000 SGD
...every level understand and improve their performance. From Major League Baseball pitchers to... ...-device.” You will develop and optimize inference pipelines, integrate vendor runtimes on... ...target hardware). Establish and uphold engineering standards: code quality, test strategy,...4000 - 6000 SGD
...for large enterprises, provides an exciting opportunity for Performance engineers to join hands with highly talented product development teams... ...Candidates with 5 or more years of experience will be considered for senior or lead roles. Education & Experience A bachelor's or...- ...infrastructure, and AI platform engineering About Our Client Our... ...artificial intelligence, high-performance computing, and digital transformation... .... Develop AI agents and LLM-powered applications. Improve GPU training and inference performance. Design scalable...
5000 - 7000 SGD
...architecture. • Develop and maintain AI evaluation metrics and guardrails for AWS... ...optimise data processing, retrieval, and inference workflows for performance and accuracy. • Ensure compliance... ...: • Degree in Computer Science, Engineering, or related field, or equivalent...7000 - 8000 SGD
...Role: Senior AI Engineer (LLM, GenAI & Model Training) Location: Central, Singapore Employment Type: Perm About the role We're looking... ...into products and business systems using Python and APIs Evaluate model performance and continuously improve accuracy and...12000 - 15000 SGD
...Research, develop, and optimize AI/ML solutions with a focus on LLM-powered applications. Design and implement Retrieval-Augmented... ...Applicant Degree in Computer Science, Mathematics, Engineering, or a related discipline. Hands-on experience building and deploying...15416 - 16250 SGD
...everything we do. This role is based in Four Seasons APAC Corporate Office in Singapore, reporting to the Global Corporate Senior Director, Engineering Performance. Summary: The Area Director of Engineering Performance (ADEP) – APAC is a regional performance leadership...5000 - 8000 SGD
...Fencing Singapore (FS), the High Performance Sport Institute (HPSI), and the Singapore Sports School(SSP) are seeking a dynamic and experienced... ...of the NFP and High-Performance Athlete Pathway across Senior, Junior and Cadet squads. · Work with National Coaches and National...6000 - 8000 SGD
...Role: AI Developer (LLM & GenAI Solutions) Location: Central... ...reliable solutions through prompt engineering, RAG architectures, agent... ...workflows to optimize AI performance Integrate AI capabilities... ...and Python-based automation Evaluate AI outputs, troubleshoot model...5750 - 8500 SGD
...perceptions and communications. Specifically, the RF will work on the evaluation of risk communication and resilience interventions. Evaluations... ...primary data for research projects Design experiments Perform qualitative and quantitative data analysis Write scholarly...- ...improve and enhance our fragrances. Active participation in the Evaluation Sessions (actual projects, fragrance stability in actual product... ...assigned. Good project management working and learning with Senior Evaluator and other team members. - Develop olfactive...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior LLM Inference Performance & Evaluation Engineer. Be the first to apply!
