Kompetensi

Data Engineering

Modern Data Engineering In The Cloud

Data engineering is the crucial part to enable and operationalize big data analytics and cloud applications in the big data ecosystem Modern data engineering ensures fast, secure, and high quality implementation of new systems that streamline operations and reduce costs with minimal workforce interruption It provides an extensible, highly scalable set of tools to access, transform, and integrate data from any business system This course is designed for anyone who wants to perform data integration and management tasks Participants work on projects to monitor the process and database changes In addition, participants would also learn about how cloud technology is able to help IT to reduce hardware dependencies, software management etc

Course Objective

1. Memahami dan melaksanakan konsep Extract, Transform and Load (ETL).
2. Melakukan proses integrasi data dan menyiapkan tugasan yang diberi.
3. Memantau proses dan perubahan pangkalan data.
4. Memahami kepentingan teknologi Cloud.

Course Outcomes

1. Understand and execute the concept of Extract, Transform and Load (ETL)
2. Perform data integration process and manage the tasks given
3. Monitor the process and database changes
4. Understand the importance of cloud technology 

Course Outline

1. What is Data Engineering?
   • Overview
   • Data Engineering Skillset
   • Data Engineering Roles
2. What is Cloud Data Engineering?
   • Different Types of Cloud Offering
   • Traditional ETL vs Modern Data Engineering in Cloud
   • Why on Cloud?
3. Data Engineering Execution
   • Working with files
   • iBasictransformation (Join, Filter, Expression Editor)
   • Using context variables
   • Error Handling
   • Working with Databases
   • Working with web services
   • Master Job
   • Documenting a Job
   • Parallel Execution
   • Joblets
   • Change Data Capture (CDC)
4. Introduction to Cloud
   • Overview architecture
   • Connecting to Talend Cloud
   • Publishing job
   • Cloud Engine and Remote Engine
   • Job Execution
   • Job Scheduling