HDP Developer Apache Pig and Hive (HW HDP PH)
About Individual Course:
About this course:
This course is designed for developers who need to create applications to analyze Big Data stored in Apache Hadoop using Pig and Hive. Topics include: Hadoop, YARN, HDFS, MapReduce, data ingestion, workflow definition, using Pig and Hive to perform data analytics on Big Data and an introduction to Spark Core and Spark SQL. This course also prepares the students for the HDP Certified Developer (HDPCD) Exam A Big Data Hadoop Certification
The average salary for a Hadoop Developer is $110,000 per year.
After completing this course, students will be able to:
- Describe Hadoop, YARN and use cases for Hadoop
- Describe Hadoop ecosystem tools and frameworks
- Describe the HDFS architecture
- Use the Hadoop client to input data into HDFS
- Transfer data between Hadoop and a relational database
- Explain YARN and MaoReduce architectures
- Run a MapReduce job on YARN
- Use Pig to explore and transform data in HDFS
- Understand how Hive tables are defined and implemented
- Use Hive to explore and analyze data sets
- Use the new Hive windowing functions
- Explain and use the various Hive file formats
- Create and populate a Hive table that uses ORC file formats
- Use Hive to run SQL-like queries to perform data analysis
- Use Hive to join datasets using a variety of techniques
- Write efficient Hive queries
- Create ngrams and context ngrams using Hive
- Perform data analytics using the DataFu Pig library
- Explain the uses and purpose of HCatalog
- Use HCatalog with Pig and Hive
- Define and schedule an Oozie workflow
- Present the Spark ecosystem and high-level architecture
- Perform data analysis with Spark's Resilient Distributed Dataset API
- Explore Spark SQL and the DataFrame API
Software developers who need to understand and develop applications for Hadoop.
Students should be familiar with programming principles and have experience in software development. SQL knowledge is also helpful. No prior Hadoop knowledge is required.
Suggested prerequisites courses:
Virtual Instructed-Led Outline
Module 1 - IN INTRODUCTION TO THE HADOOP DISTRIBUTED FILE SYSTEM
- Understanding Hadoop
- The Hadoop Distributed File System
- Ingesting Data into HDFS
- The MapReduce Framework
- Starting an HDP Cluster
- Demonstration: Understanding Block Storage
- Using HDFS Commands
- Importing RDBMS Data into HDFS
- Exporting HDFS Data to an RDBMS
- Importing Log Data into HDFS Using Flume
- Demonstration: Understanding MapReduce
- Running a MapReduce Job
Module 2 - AN INTRODUCTION TO APACHE PIG
- Introduction to Apache Pig
- Advanced Apache Pig Programming
- Demonstration: Understanding Apache Pig
- Getting Starting with Apache Pig
- Exploring Data with Apache Pig
- Splitting a Dataset
- Joining Datasets with Apache Pig
- Preparing Data for Apache Hive
- Demonstration: Computing Page Rank
- Analyzing Clickstream Data
- Analyzing Stock Market Data Using Quantiles
Module 3 - AN INTRODUCTION TO APACHE HIVE
- Apache Hive Programming
- Using HCatalog
- Advanced Apache Hive Programming
- Understanding Hive Tables
- Understanding Partition and Skew
- Analyzing Big Data with Apache Hive
- Demonstration: Computing NGrams
- Joining Datasets in Apache Hive
- Computing NGrams of Emails in Avro Format
- Using HCatalog withApachePig
Module 4 - WORKING WITH SPARK CORE, SPARK SQL AND OOZIE
- Advanced Apache Hive Programming (Continued)
- Hadoop 2 and YARN
- Introduction to Spark Core and Spark SQL
- Defining Workflow with Oozie
- Advanced Apache Hive Programming
- Running a YARN Application
- Getting Started with Apache Spark
- Exploring Apache Spark SQL
- Defining an Apache Oozie Workflow
|Learning Style||Virtual Classroom|
|Course Duration||4 Days|
Frequently Asked Questions About Virtual Instructor-Led Courses
I can't connect to my class, what are my options?
The link to the class is available upon logging in to your dashboard. If you are unable to see it, please contact our support team at 1-855-800-8240 and they will be happy to provide you the direct link via email or the dial in number.
I can't make it to attend to class. Can I reschedule?
Yes, you can reschedule your class. Please contact your Sales representative and they will arrange this for you. If you forgot his/her name, feel free to contact our support team at email@example.com or 1-855-800-8240.
Will I get my certificate upon completion?
Yes. Upon completion of the course, it will be available on your course as a Trophy Icon for you to download. If you do not see this, you will need to contact firstname.lastname@example.org with the following details so they can email you the certificate: Class Name, Class Date, Account Rep, and Your Email.
I cannot connect to my lab. Help!
Your Lab is accessible on the bottom part of your course. You will see a button that says "LAB". Just click it to launch the lab. Please note that some classes don’t need/require a LAB. You can verify with our support team by calling them at 1-855-800-8240 or by email at email@example.com. You can also check with your Instructor or the Associate Instructor if your class includes one.
What is my access code for Skillpipe?
A. Not all of the classes have or require Skillpipe. If your class includes one, please check your email as you should have received one from firstname.lastname@example.org. In case you do not find it in your inbox, please check the Spam / Junk folder. For any further assistance, you can call the support at 1-855-800-8240 or contact them via email at email@example.com.
I don't have audio. I can't hear the instructor.
Make sure you are using a compatible headset for your laptop or computer. If you don’t have a headset, you can use the built-in speaker of your laptop. Otherwise, you can use the dial in option by calling the dial in number provided in the class joining email. You may also contact support team for the dial in numbers associated for your training at 1-855-800-8240 or contact them via email at firstname.lastname@example.org.
How can I reach student support?
Support can be reach via phone at 1855-800-8240; via email at email@example.com or via chat support through the chat button on our website. Please note that support office hours will be from 8am-5pm CST Monday to Friday. Any concerns after office hours will be attended the following business day.
Get A Team Quote or Got Questions?
- Personalize learning based on competencies, goals & tools
- Expert Mentoring
- Hands on Labs & Assignments
- AI Curated Digital Book Content
- Adaptive Learning Paths
- Analytics & Benchmarking
- High certification Pass Rates – Over 200,000 people certified and more than 95% of our learners pass their certification on the first attempt