Senior Manager-Data Platforms #sn

OSOTto Databricks Migration & Platform Ownership

· Lead the end-to-end migrationof the current OSOT solution (data lakes, data warehouse, data marts, andTalend/SQL/Python pipelines) onto the Databricks Data Intelligence Platform,defining the target lakehouse architecture, migration roadmap, sequencing, andcutover strategy with minimal disruption to production.

· Re-architect OSOT into anAPAC-specific Enterprise Data Platform on Databricks using Delta Lake, themedallion (bronze/silver/gold) architecture, Unity Catalog for governance andlineage, and Lakeflow / Delta Live Tables and Workflows for orchestration.

· Migrate and modernize ETL/ELTfrom existing tooling (Talend, SQL, Python) to Databricks-native pipelines(PySpark, Spark SQL, Databricks SQL), establishing reusable frameworks foringestion, transformation, data quality, and metadata/dependency management.

· Build data pipelines,architectures, and curated data sets from raw, loosely structured data, andstand up Databricks SQL warehouses to serve BI and self-service analytics forAPAC businesses.

· Own a modern data engineeringoperating model that follows DevOps/DataOps principles – CI/CD for notebooksand jobs (Databricks Asset Bundles, Git integration), automated testing, andinfrastructure-as-code.

AIPlatform Integration (Claude, OpenAI) on Databricks

· Define and own how Databricksworks with external AI platforms such as Anthropic Claude and OpenAI –establishing secure, governed integration patterns via Databricks Mosaic AI,Model Serving, and AI Gateway (external model endpoints) so LLMs can be consumedcentrally with rate limiting, logging, and cost controls.

· Use Unity Catalog andDatabricks governance to control which enterprise data is exposed to AI models,ensuring PII protection, access policies, and auditability before any datareaches Claude or OpenAI.

· Enable Retrieval-AugmentedGeneration (RAG) on APAC enterprise data using Databricks Vector Search andfeature/embedding pipelines, grounding Claude and OpenAI responses in trustedOSOT data for use cases such as analyst copilots, document intelligence, andnatural-language querying of the lakehouse.

· Operationalize GenAI use casesend-to-end – prompt management, evaluation, monitoring, and MLOps – usingDatabricks Mosaic AI (MLflow, Model Serving, AI Gateway, and agent/evaluationframeworks).

· Partner with Global IT andSecurity to define responsible-AI guardrails for LLM usage (data residency,prompt/response logging, content safety, and acceptable-use policies for Claudeand OpenAI).

DataModeling, Analytics & Stakeholder Delivery

· Interface with cross-functionalteams (Finance, Risk, SCM, and Product) to design and build data models on thelakehouse that provide actionable insights into key business performancemetrics and support the commercial team.

· Leverage cloud-based lakehousearchitecture and Databricks ML capabilities to deliver optimized ML and AImodels at scale.

· Work closely with leadersacross data architecture, enterprise architecture, data science/analytics, anddomain experts to build and maintain roadmaps aligned to the IT strategy.

· Act as a strategic thinker witha holistic vision, with specific focus on identifying and automating existingmanual processes – including AI-assisted automation – to drive key businessperformance.

· Participate in project workinggroups and assist in tracking project milestones and deliverables.

Critical Competencies / Critical SuccessFactors

· Bachelor’s degree required;master’s degree (or equivalent) in computer science, engineering (allbranches), mathematics, or a related field preferred.

· 8+ years of experience withpublic and private cloud solutions, including building and maintaining a dataecosystem that includes an ERP environment.

· 8+ years of experiencedesigning and building data-intensive solutions using distributed computing.

· Hands-on experience with theDatabricks Data Intelligence Platform – Delta Lake, Unity Catalog, DatabricksSQL, Lakeflow / Delta Live Tables, Workflows, and the medallion architecture;experience leading a migration onto Databricks is strongly preferred.

· Experience integrating andoperationalizing LLM/GenAI platforms (e.g., Anthropic Claude, OpenAI) –including Databricks Mosaic AI, Model Serving, AI Gateway, Vector Search, andRAG patterns – with appropriate governance and security.

· 6+ years of programmingexperience with SQL, Python, Scala, and Java.

· Experience with distributeddata streaming frameworks such as Spark Structured Streaming, Apache Flink, andKafka.

· OLAP experience (Cubes, MSSQL)and data warehouse usage/optimization experience (Databricks SQL, Redshift,Hive, Snowflake).

· Experience building datavisualizations or analytics (e.g., Power BI, Databricks dashboards, SSRS).

· Experience building machinelearning models; familiarity with MLflow and modern MLOps practices.

· Strong analytic skills andunderstanding of statistical methodologies.

· Strong knowledge of datagovernance, security, and compliance, including responsible/secure use of AIand LLMs.

· Ability to communicateeffectively with external clients and internal teams and manage expectations.

· Demonstrated ability to work asan effective team member – sharing knowledge and helping others meet teampriorities – while also working independently as an effective decision-maker.

· Good verbal, written, andpresentation skills; able to engage with offshore-based development and supportteams.

· An effective troubleshooter,able to analyze problems and develop/recommend solutions, with a willingness tolearn and adapt to strategic initiatives in the pipeline.

If you’re interested in this role, click ‘apply now’ to forward an up-to-date copy of your CV, or call Shabnam at Hays on +65 60271964 or email View email address on mycareersfuture.gov.sg for a confidential discussion.

Referrals are welcome.

Registration ID No. R1873584 | EA License number: 07C3924 | Company Registration No. 200609504D