Data engineer (Azure)
Full-time
Flintex Consulting Pte Ltd
Data engineer (Azure) – Synapse and Pyspark, Python, Datawarehouse and Azure Data Explorer, Azure Devops
Job Scope
• Design, review and development of Pyspark scripts. Testing, troubleshooting of data pipelines, orchestration.
• Designing and developing reports and dashboards in PowerBI,setting up access control with row level security DAX query experience.
• Establishing connections to source data systems such as on-prem databases, IOT devices, APIs.
• Managing the collected data in appropriate storage/data-base solutions e.g. file systems, SQL servers, Big Data platforms such as Hadoop, HANA, etc. as required by the specific project requirements.
• Design, development of relevant data pipelines using pyspark, copy data activities for batch ingestion.
• Performing data integration e.g. using database table joins, or other mechanisms at an appropriate level as required by the analysis requirements of the project.
• Deployment of pipeline artifacts from one environment to the other using Azure Devops.
Skills & Experience
• Bachelor’s Degree in Computer Science or Engineering with 2 years of experience in Azure Data engineering, Python, Pyspark or Big Data development.
• Sound Knowledge of Azure Synapse analytics for pipelines, orchestration, set up.
• 1-2 experience in Visualization design and development with Power BI. Knowledge on row-level security, access control.
• Sound experience in SQL, Datawarehouse, data marts, data ingestion with Pyspark and Python.
• Expertise in developing and maintaining ETL processing pipelines in cloud-based platforms such as AWS, Azure, etc. (Azure Synapse or data factory preferred)
• Team player with good interpersonal, communication, and problem-solving skills.
• Preferred to have Devops expertise.
Working hours :
8:30am to 6pm (Monday to Friday) onsite, no hybrid option.
Vacancy posted 8 days ago
Similar jobs that could be interesting for youBased on the Data engineer (Azure) in Central, Singapore Country vacancy
- ...MUST SKILLS: Hands-on experience with Microsoft Azure Cloud architecture , including the design, implementation, and management of cloud-based solutions. Proven ability to manage both infrastructure and application delivery projects from planning through execution...
- ...systems. Your Role – Impactful, Meaningful And Future-Oriented We are seeking a highly motivated and detail-oriented CBI Data Development Engineer to join our team for CBTC GoA4 railway signalling projects. The engineer will be responsible for the design and development...
- ...About the Role We are seeking a Data Scientist (Machine Learning) to develop and deploy... ...for data collection, cleansing, feature engineering, and model training. • Build forecasting... ...with cloud platforms such as AWS, Azure, or GCP. • Experience with data visualization...
- .... • Prepare, clean, and transform structured and unstructured data for analysis and modelling. • Design statistical experiments and... ...in Data Science, Computer Science, Statistics, Mathematics, Engineering, or a related discipline. • At least 3 years of experience in...
- ...software. Join an all-star team. Take real ownership. Push boundaries. Build things that matter. We’re looking for our first Senior Data Analyst to turn data into insights and optimize revenue processes. You'll start with a deep focus on marketing performance and...
- ...compliance audits and assessments to ensure adherence to company policies and regulatory requirements. Analyze business processes and data to identify compliance risks and develop recommendations for improvement. Develop and implement compliance monitoring and...
- ...Junior DevOps Engineer Objectives of this position: Assist with the implementation of... ...cloud resources. • Monitor workloads on Azure (Log Analytics, Application Insights, Grafana... ...concepts required to support Data and AI workloads (e.g., GPU computing basics...
- ...Overview We are seeking a Applied AI Engineer to embed directly with our business units... ...• Retrain, evaluate, and tune models as data and operating conditions evolve Deploy... ...operate applications and models on Microsoft Azure/GCP behind production auth, logging, and...
- ...Key Job Accountabilities: As the CRM & Data Analytics Manager, you will be an integral part of the TRAPAC (Travel Retail Asia Pacific) Corporate Marketing team. You will be at the heart of our mission to build deeper traveler relationships by establishing our data infrastructure...
- ...our products Deploy and configure our solutions in customer data centers or cloud environments Develop and configure use... ...sales Serve as the interface between users and our product engineering teams. This is an excellent role to gain experience and insight...
- ...Good working knowledge and experience in managing Office365 and Azure • To demonstrate the ability to acquire and apply new technical... ...Security clearance Qualifications and Experience • IT / engineer related diploma from polytechnic or degree from university •...
- ...looking for a highly driven, self-motivated, and technically excellent engineer who is truly excited about building the "AI Operating System" of... ...solutions that bridge our AI tech stack with our enterprise data lake on AWS. We rely on our AI Engineers to possess "Deployment Autonomy...
- ...Reboot Monkey is a global leader in IT solutions, specializing in data center management that simplifies your IT operations. We provide hosting space, future-proof upgrades, and 24/7 support through our smart or remote hands, ensuring seamless, secure, and sustainable service...
- ...smooth production processes. Requirements Min Diploma in Engineering or related field. Prior experience in Wafer Fab/... ....: R1981808 By submitting your curriculum vitae or personal data to us in connection with your job application, you are deemed to...
- ...our products Deploy and configure our solutions in customer data centers or cloud environments Develop and configure use cases... ...during sales Serve as the interface between users and our product engineering teams. This is an excellent role to gain experience and insight...
- ...send to ****@*****.*** EA License No: 13C6305 Reg. No.: R1981808 By submitting your curriculum vitae or personal data to us in connection with your job application, you are deemed to have read and agreed to the terms of our Privacy Policy, and consented...
- ...Requirements Min Diploma in Mechanical / Electrical Engineering or its equivalent At least 1 year related working experience... ....: R1981808 By submitting your curriculum vitae or personal data to us in connection with your job application, you are deemed to...
- ...mentor, and grow a high-performing team of engineers, fostering a culture of innovation,... ...ML training, including but not limited to data pipeline optimization, large scale distributed... ...cloud computing platforms (e.g., AWS, GCP, Azure). Technical Skills: Deep expertise...
- ...We're looking for a Senior Software Engineer to lead the development and deployment of... ...establish a real-time Autonomous Vehicle-to-Cloud data pipeline, enabling critical services like... ...with cloud-native services (AWS IoT Core, Azure IoT) for telemetry, remote command and...
- ...About the Role We are seeking an experienced Data Center Technician for freelance/contract work in Singapore, SG . This is an on-site position supporting mission-critical infrastructure at premier data center facilities. Engagement Type Contract/Freelance basis...
- ...external auditing bodies, suppliers and customers) 4. Analyze quality data and metrics to identify trends and areas for improvement 5.... ...in a field of study such as chemistry, business, accounting, engineering, etc. · Certificate in Quality Engineering is preferred. · 4...
- ...and talented technologist to join us as a member of our Network Engineering team. The team is responsible for the architecture, design, administration... ...concepts - VXLAN and eVPN Experience with cloud - AWS and Azure Experience with network disaster recovery Financial...
- ...industry standards. Requirements Diploma or Degree in Engineering or related field. With 2 years of experience in the wafer... ....: R1981808 By submitting your curriculum vitae or personal data to us in connection with your job application, you are deemed to...
- ...customer relationships. Requirements Diploma or Degree in Engineering or related field. Prior experience in technical service or... ....: R1981808 By submitting your curriculum vitae or personal data to us in connection with your job application, you are deemed to...
- ...service. Requirements Bachelor's degree in Mechanical Engineering or related field. Prior experience in technical support or... ....: R1981808 By submitting your curriculum vitae or personal data to us in connection with your job application, you are deemed to...
- ...Review and analyze design specifications and standards. Perform engineering calculations to ensure switchboards meet safety and performance... ....: R1981808 By submitting your curriculum vitae or personal data to us in connection with your job application, you are deemed to...
- ...guidelines. Requirements Bachelor's degree in Civil Engineering or Geotechnical Engineering. Minimum 3 years of experience... ....: R1981808 By submitting your curriculum vitae or personal data to us in connection with your job application, you are deemed to...
- ...activities. Requirements Min Diploma / Degree in Civil Engineering or its’ equivalent Experience in underground civil works (... ....: R1981808 By submitting your curriculum vitae or personal data to us in connection with your job application, you are deemed to...
- ...compliance with quality standards. Monitor and analyze quality data to identify trends and areas for improvement. Collaborate... ...Bachelor's Degree in any Mechanical/Electronic/Mechatronics Engineering Good understanding in ISO standard for ISO 9000, 14000, and...
- ...service. Requirements Bachelor's degree in Mechanical Engineering or related field. Prior experience in technical support or... ....: R1981808 By submitting your curriculum vitae or personal data to us in connection with your job application, you are deemed to...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data engineer (Azure). Be the first to apply!
