Apache Hadoop: Manipulation and Transformation of Data Performance培訓

課程代碼

ApHadm1

課程時長

21 時間：同常來說包括休息是 3天

最低要求

Attendees are not required to have any specific skill as the training is focused on end users skills for both the administration and the manipulation of data under Apache Hadoop

概觀

本課程面向開發人員，架構師，數據科學家或任何需要密集或定期訪問數據的配置文件。

該課程的主要重點是數據處理和轉換。

在Hadoop生態系統的工具中，本課程包括Pig和Hive的使用，這兩者都大量用於數據轉換和操作。

此培訓還涉及性能指標和性能優化。

該課程完全是實踐性的，並通過理論方面的介紹來打斷。

Machine Translated

課程簡介

1.1Hadoop Concepts

1.1.1HDFS

The Design of HDFS
Command line interface
Hadoop File System

1.1.2Clusters

Anatomy of a cluster
Mater Node / Slave node
Name Node / Data Node

1.2Data Manipulation

1.2.1MapReduce detailed

Map phase
Reduce phase
Shuffle

1.2.2Analytics with Map Reduce

Group-By with MapReduce
Frequency distributions and sorting with MapReduce
Plotting results (GNU Plot)
Histograms with MapReduce
Scatter plots with MapReduce
Parsing complex datasets
Counting with MapReduce and Combiners
Build reports

1.2.3Data Cleansing

Document Cleaning
Fuzzy string search
Record linkage / data deduplication
Transform and sort event dates
Validate source reliability
Trim Outliers

1.2.4Extracting and Transforming Data

Transforming logs
Using Apache Pig to filter
Using Apache Pig to sort
Using Apache Pig to sessionize

1.2.5Advanced Joins

Joining data in the Mapper using MapReduce
Joining data using Apache Pig replicated join
Joining sorted data using Apache Pig merge join
Joining skewed data using Apache Pig skewed join
Using a map-side join in Apache Hive
Using optimized full outer joins in Apache Hive
Joining data using an external key value store

1.3Performance Diagnosis and Optimization Techniques

Map
- Investigating spikes in input data
- Identifying map-side data skew problems
- Map task throughput
- Small files
- Unsplittable files
Reduce
- Too few or too many reducers
- Reduce-side data skew problems
- Reduce tasks throughput
- Slow shuffle and sort
Competing jobs and scheduler throttling
Stack dumps & unoptimized code
Hardware failures
CPU contention
Tasks
- Extracting and visualizing task execution times
- Profiling your map and reduce tasks
Avoid the reducer
Filter and project
Using the combiner
Fast sorting with comparators
Collecting skewed data
Reduce skew mitigation

客戶評論

★★★★★

★★★★★

你最喜歡什麼培訓？：詳細介紹每個工具。

Amber Mehrotra - NIIT Limited

課程:Hadoop Administration

Machine Translated

事實上，所有數據和軟件都已準備好在已準備好的VM上使用，由外部磁盤中的培訓師提供。

vyzVoice

課程:Hadoop for Developers and Administrators

Machine Translated

我最喜歡教練提供真實的實例。

Simon Hahn

課程:Administrator Training for Apache Hadoop

Machine Translated

我真的很享受培訓師的巨大能力。

Grzegorz Gorski

課程:Administrator Training for Apache Hadoop

Machine Translated

我真的很喜歡這些實踐課程。

Jacek Pieczątka

課程:Administrator Training for Apache Hadoop

Machine Translated

這是非常親力親為的，我們花了一半的時間實際在Clouded / Hardtop做事，運行不同的命令，檢查系統，等等。額外的材料（書籍，網站等）真的很感激，我們將不得不繼續學習。安裝非常有趣，而且非常方便，從頭開始的集群設置非常好。

Ericsson

課程:Administrator Training for Apache Hadoop

Machine Translated

很多動手練習。

Ericsson

課程:Administrator Training for Apache Hadoop

Machine Translated

Ambari管理工具。能夠討論除電信之外的其他業務案例的實際Hadoop體驗。

Ericsson

課程:Administrator Training for Apache Hadoop

Machine Translated

我非常喜歡的虛擬機器老師對這個話題以及其他話題都很瞭解, 他很好, 也很友好。我喜歡迪拜的設施。

Safar Alqahtani - Elm Information Security

課程:Big Data Analytics in Health

Machine Translated

與參加培訓的人溝通。

Andrzej Szewczuk - Izba Administracji Skarbowej w Lublinie

課程:Apache NiFi for Administrators

Machine Translated

實際做的事情，理論也得到了Ajay的好評

Dominik Mazur - Capgemini Polska Sp. z o.o.

課程:Hadoop Administration on MapR

Machine Translated

演習

Capgemini Polska Sp. z o.o.

課程:Hadoop Administration on MapR

Machine Translated

練習的有用性

Algomine sp.z.o.o sp.k.

課程:Apache NiFi for Administrators

Machine Translated

我發現培訓很好，信息量很大......但是可以分發4到5天，讓我們可以了解不同方面的更多細節。

Veterans Affairs Canada

課程:Hadoop Administration

Machine Translated

我非常喜歡這種訓練。安東擁有豐富的知識，並以非常方便的方式提出了必要的理論。很棒的是，培訓是很多有趣的練習，所以我們從一開始就接觸到了我們所知道的技術。

Szymon Dybczak - Algomine sp.z.o.o sp.k.

課程:Apache NiFi for Administrators

Machine Translated

我發現這個課程給了一個很好的概述，並迅速觸及了一些我甚至沒有考慮的領域。

Veterans Affairs Canada

課程:Hadoop Administration

Machine Translated

課程分類

相關課程

促銷課程

ELK: Elasticsearch, Logstash and Kibana for Administrators

10/22/2019 - 09:30

The Hsinchu Wishun centre

更多

Artificial Intelligence Overview

10/29/2019 - 09:30

The Hsinchu Wishun centre

更多

Blockchain Development with Coinbase

10/30/2019 - 09:30

The Hsinchu Wishun centre

更多

Zeppelin for Interactive Data Analytics

01/09/2020 - 09:30

The Hsinchu Wishun centre

更多

Signavio Process Manager

01/30/2020 - 09:30

台北, Concord

更多

我們的客戶

ibm business consulting services

nexsan technologies canada

avid technology

toyota bank polska

santander consumer bank

european space agency (esa)

thomson reuters corporation

universal music publishing group

zf friedrichshafen

european organization for nuclear research (cern)

avon products, inc

royal dutch shell

procter & gamble (p&g)

the emirates group

environmental protection agency (epa) ireland

nokia siemens networks

starnet telecom

aviva commercial union

société générale

schneider electric

phoenix geophysics

ibm business consulting services

nexsan technologies canada

avid technology

toyota bank polska

santander consumer bank

european space agency (esa)

thomson reuters corporation

universal music publishing group

zf friedrichshafen

european organization for nuclear research (cern)

avon products, inc

royal dutch shell

procter & gamble (p&g)

the emirates group

environmental protection agency (epa) ireland

nokia siemens networks

starnet telecom

aviva commercial union

société générale

schneider electric

phoenix geophysics

is growing fast!

We are looking to expand our presence in Taiwan!

As a Business Development Manager you will:

expand business in Taiwan
recruit local talent (sales, agents, trainers, consultants)
recruit local trainers and consultants

We offer:

Artificial Intelligence and Big Data systems to support your local operation
high-tech automation
continuously upgraded course catalogue and content
good fun in international team

If you are interested in running a high-tech, high-quality training and consulting business.