Big Data Engineer, Data Ingestion
6500 - 13000 SGDBYTEDANCE PTE. LTD.
About Us
Founded in 2012, ByteDance's mission is to inspire creativity and enrich life. With a suite of more than a dozen products, including TikTok, Lemon8, CapCut and Pico as well as platforms specific to the China market, including Toutiao, Douyin, and Xigua, ByteDance has made it easier and more fun for people to connect with, consume, and create content.
Why Join ByteDance
Inspiring creativity is at the core of ByteDance's mission. Our innovative products are built to help people authentically express themselves, discover and connect – and our global, diverse teams make that possible. Together, we create value for our communities, inspire creativity and enrich life - a mission we work towards every day.
As ByteDancers, we strive to do great things with great people. We lead with curiosity, humility, and a desire to make impact in a rapidly growing tech company. By constantly iterating and fostering an "Always Day 1" mindset, we achieve meaningful breakthroughs for ourselves, our Company, and our users. When we create and grow together, the possibilities are limitless. Join us.
Diversity & Inclusion
ByteDance is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At ByteDance, our mission is to inspire creativity and enrich life. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach. We are passionate about this and hope you are too.
Job highlights
Positive team atmosphere, Career growth opportunity, Paid leave, Industry experts, Meals provided
Responsibilities
Team Introduction
The Data Management Suite team is building products that cover the whole lifecycle of data pipeline, including data ingestion and Integration, data development, data catalog, data security and data governance. These products support various businesses, so data engineers and data scientists could greatly boost their productivity.As a software engineer in the data management suite team, you will have the opportunity to build, optimize and grow one of the largest data platforms in the world. You'll have the opportunity to gain hands-on experience on core systems in the data platform ecosystem. Your work will have a direct and huge impact on the company's core products as well as hundreds of millions of users.
Responsibilities:
- Design, develop, and optimize the architecture of large-scale data ingestion systems to support real-time data pipelines, ensuring high throughput, low latency, and fault tolerance.
- Enhance the performance, scalability, and reliability of data ingestion pipelines.
- Develop and implement automated and intelligent operation and maintenance systems to monitor, diagnose, and ensure the stability and reliability of data ingestion pipelines.
- Collaborate with cross-functional teams to deliver event data ingestion, transformation, and storage solutions that meet diverse business requirements.
- Troubleshoot and resolve complex issues in production systems, ensuring minimal downtime and optimal performance.
Qualifications
Minimum Qualifications:
- Bachelor's degree in Computer Science or equivalent practical experience.
- 2+ years of experience in software development with proficiency in one or more programming languages, such as Java, Scala, Python, or Go.
- Strong understanding of data structures, algorithms, and distributed systems principles.
- Hands-on experience with big data technologies, such as Hadoop, Flink, Kafka, or similar frameworks.
Preferred Qualifications:
- 2+ years of experience in real-time data processing frameworks like Flink. Contributions to open-source projects are a plus.
- Experience in the event tracking domain, including event collection, real-time processing, governance, quality assurance and cost optimization.
- Deep understanding of data lakehouse architectures and the integration of stream and batch processing.
- Strong problem-solving skills and the ability to work in a collaborative environment.
- ...Responsibilities Design, develop, and optimize large-scale data ingestion systems for real-time pipelines with high throughput, low latency... ...and distributed systems principles. Hands-on experience with big data technologies such as Hadoop, Flink, Kafka, or similar frameworks...
10000 - 14000 SGD
...Job Description We are seeking an experienced Big Data Engineering Lead to design, develop and lead scalable enterprise data platforms, real... ..., Spark Structured Streaming and Kafka . Develop data ingestion, transformation and processing workflows using AWS, S3, Airflow...6000 - 9000 SGD
...Overview We are hiring a Data Engineer to join a high-performing enterprise data team responsible for delivering and supporting critical data ingestion capabilities across a large-scale data platform. This role is ideal for engineers who enjoy building production-grade data...7000 - 9000 SGD
...Location- Clementi Responsibilities ~ Big Data Ingestion Enhancement to switch from Canvas to D2L (Learning Management System) which will affect all 5 existing data sources. # Extract (taking the data from its current location). # Transform (cleansing and normalizing...7000 - 10000 SGD
...Job Summary We are looking for an experienced Big Data Administrator / Big Data Engineer – L3 to support and maintain enterprise Big Data infrastructure in a production environment. The successful candidate will be responsible for the availability, reliability, performance...- ...Responsibilities Design and implement offline data architectures for large-scale... ...degree or higher in computer science, software engineering, or a related field. At least 3 years of... ...relevant experience. Familiarity with big data frameworks including Hadoop, Hive, Flink...
- ...Responsibilities Design and implement offline data architectures for large-scale recommendation systems... ...'s degree or above in computer science, software engineering, or a related field. Familiarity with open-source big data frameworks such as Hadoop, Hive, Flink, FlinkSQL...
6000 - 9000 SGD
...We are looking for an experienced Big Data Administrator to join the IT infrastructure and production support team. The candidate will... ...Bachelor's degree in Computer Science, Information Technology, Engineering or a related discipline. Experience in financial services or...11250 - 22500 SGD
...than 10,000 employees globally. This is a data middle platform product built on user behavior... ...data analysis services.We are the global engineering team for Tea. We are passionate about... ...configuration, monitoring, and maintenance of big data platforms.- Implement and optimize...9000 - 10000 SGD
.../Scrum environment. Requirements Minimum 8 years of relevant experience in Java/J2EE development . Hands-on experience in Big Data technologies is preferred. Bachelor's or Technical Degree in Computer Science, Information Systems, Information Technology or a...- ...Opportunity We are building the foundational data infrastructure that powers one of the... ...'s leading crypto exchanges. As a Big Data Platform Engineer , you will design, build, and evolve... ...Build APIs-first tools (backfill, ingestion automation) designed for future MCP...
9000 - 11000 SGD
...serving clients across six continents. Together, we discover ideas and connect the dots to build a better and a bold new future. Big Data Administrator L3 Responsibilities & Requirement: Build and maintain Hadoop stack infrastructure Install Hadoop updates, patches...8000 - 11000 SGD
· Atleast 11-14 years of experience in Big Data / Data Platform Engineering inenterprise environments. Stronghands-on experience with: · Hadoopecosystem (HDFS, YARN, SPARK, MapReduce, HBase etc) · ApacheKafka (high-throughput environments) · OpenSearch/ Elasticsearch...- ...Architecture: Own the architecture, development, and optimization of our big data platform, supporting large-scale collection, processing, and... ...processing and pipelines for model training data and feature engineering; partner with Product, Risk, and Growth teams on data needs for...
9000 - 12000 SGD
...or IT is preferred · At least 8 years of relevant working experience in Java/J2EE development · At least 2 years of experience in big data technology development like the Hadoop platform. · Good understanding & working knowledge with Agile/Scrum · Hands-on experience...- ...Singapore. We use our products in a wide range of use-cases, including but not limited to Counter-terrorism and Computer Network Defence. We are looking for talented senior data engineers to lead and build mission-critical products that identify, analyse and investigate threats....
8000 - 10000 SGD
...Science, IS, or IT is preferred · At least 8 years of relevant working experience in Java/J2EE development · Hands-on experience in big data technology development is preferred · Good understanding & working knowledge with Agile/Scrum · Hands-on experience of end-to-end...4000 - 4500 SGD
...The Data Engineer plays a key role in designing, building and optimising data solutions. This role... ...in, the following: o Developing data ingestion, integration and extraction pipelines... ...Databricks). Preferred: knowledge of big data querying tools such as Hive and Hue....4000 - 4500 SGD
...Data engineer (Azure) – Synapse and Pyspark, Python, Datawarehouse and Azure Data Explorer, Azure... ...e.g. file systems, SQL servers, Big Data platforms such as Hadoop, HANA, etc.... ...pyspark, copy data activities for batch ingestion. • Performing data integration e.g. using...7000 - 10500 SGD
· 8-12 years of experience in Data Engineering, Big Data, Data Lake, or Lakehouse implementations. · Hands-on experience with Databricks, Snowflake... ...object storage platforms. · Experience implementing data ingestion, transformation, reconciliation and data quality frameworks....7000 - 10000 SGD
...We are looking for an experienced Senior Data Engineer with 8–12 years of experience in Data Engineering, Big Data, Data Lake, and Lakehouse platforms. The ideal candidate... ...batch, streaming, CDC, and API-based data ingestion pipelines. Build multimodal data ingestion pipelines...3000 - 4000 SGD
...program covering the latest technologies in Data Engineering, Cloud Platforms, and the Databricks... ...Assist in building and optimizing data ingestion, transformation, and orchestration workflows... ...in Data Engineering and the Databricks/Big Data ecosystem. Basic knowledge of...3500 - 4000 SGD
...Lead the transmission and ingest operations, ensuring seamless delivery and management of content workflows. The role involves supervising a team, coordinating closely with clients, and maintaining high operational standards. Key Responsibilities : 1. Responsible for...6500 - 8500 SGD
...Postion Overiew We are hiring a Data Engineer (2 years Direct Contract) for data migration projects... ...for routine tasks, such as data ingestion, transformation, and monitoring, using AWS... ...Preferred Skills: Experience with big data technologies like Apache Spark and Hadoop...15000 - 16500 SGD
...About the Role We are looking for a Senior Data Engineer to join our Data team and own data systems end to end, from the... ...responsible for Own data from source to delivery. Design ingestion from source systems into Big Query with Airflow (Cloud Composer), Dataflow and...7000 - 10000 SGD
...effectiveness as a team. As a Senior Data Engineer, you will: Contribute to the design... ...and maintain efficient data pipelines to ingest, transform, and store data from various... ...Python and Java or Scala ~ Experience with Big Data processing frameworks and data...11500 - 22500 SGD
...are looking for an ambitious, enthusiastic and highly motivated Data Engineer to join our Data team that spans across our international... ...and technical. The team builds and maintains thousands of data ingestion processes of varying complexity, feeding investment decisions...7000 - 11000 SGD
...Hugging Face Transformers, Lang Chain) Construct scalable data pipelines for ingesting and structuring archival content Interface with graph... ...Memgraph) for relationship mapping Collaborate with AI/ML engineers, designers, and other stakeholders to integrate LLM outputs...8000 - 13000 SGD
...monitoring, analytics and control capabilities. This full-time Data Engineer role is hands-on and delivery-focused: you will build secure,... ...scalable ETL/ELT pipelines for batch and near-real-time data ingestion, transformation and delivery. Apply strong SQL, Python, Spark...7000 - 8500 SGD
...Role Overview Support the design and development of data ingestion and ETL processes used within the Quantexa demo environment. Experience: 5-8 yrs Required Skills: Apache Spark ETL development Data integration Data transformation Data modelling Elasticsearch...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Big Data Engineer, Data Ingestion. Be the first to apply!
