Deskripsi Pekerjaan:

The Opportunity:

As a Data Engineer, you will be part of our growing Data Team. You will support the Data

Team in building a Data Warehouse that can cater to batch and streaming data, and provide

useful information to build efficient data metrics, which will be used to make smarter

business decisions and support business growth. You will be expected to be agile, hands-on,

and willing to learn with a growth mindset. You'll have ample room for growth and

opportunities to develop your skills as you contribute to creating world-class digital products.

What You Will Do:

• Assist in suggesting and implementing improvements to our existing data pipeline.
• Help understand stakeholder requirements and assist in integrating datasets into the data warehouse for stakeholder use in dashboard building.
• Support the building of data processing pipelines for optimal extraction, transformation, and loading from a wide variety of data sources.
• Perform data exploration on unstructured and structured data with guidance of senior team members.
• Communicate with various functions about their data needs and help ensure our data warehouse is ready to support them.
• Monitor data platform performance and assist in performance tuning when anomalies are detected.
• Collaborate with data analysts to address data quality and consistency issues.
• Help develop and maintain documentation for all data engineering processes.
• Ensure compliance with data governance and security policies.

Kualifikasi:

What We Need:

• A degree or higher in Computer Science, Electronics or Electrical Engineering, Software Engineering, Information Technology or other related technical disciplines.
• 1-2 years of experience in data engineering, software engineering, or a similar role.
• Fluency in SQL, Python/PySpark, or equivalent.
• Solid understanding of database concepts, including the differences between structured and unstructured data, and know how to handle and store each type effectively.
• Experience in writing clear, well-structured and concise documentation that promotes data discoverability.
• Capable of efficiently ingesting data into both data lakes and data warehouses, using appropriate tools and processes to ensure seamless data integration.
• Experience with different batching processes, including snapshot batching, full-load batching, and incremental batching, and be able to implement these methods as needed.
• Capable of performing data modeling tasks, particularly for processing logs data and handling intermediate-level SQL queries.
• Skilled in validating data to ensure its quality before it reaches the User Acceptance Testing (UAT) stage.
• Experience with processing Change Data Capture (CDC) is a plus.
• Experience within the Databricks and Google Cloud Platform environment is a plus.