Flume and Sqoop for Ingesting Big Data

These hold/produce data to data stores like HDFS, HBase and Hive. Both the tools have default functionality and have the ability of abstracting away the users from the complication of transferring data among these systems.

Self-Paced

Learning Style

Course

Learning Style

Intermediate

Difficulty

2 Hours

Course Duration

Course Info

Download PDF

Certificate

See Sample

tab
About Individual Course:
  • Individual course plan gives you access to this course
On Sale!
Now Only $10.00 Regular Price $49.00
Now Only $10.00 Regular Price $49.00
/ Each
5 Learners Have Enrolled For This Course
When you subscribe, you get:
Learn Subscription plan gives you access to this course and over 840 other popular courses
On Sale!
Now Only $39.99 Regular Price $44.99
Now Only $39.99 Regular Price $44.99
/ Month
Team
Pricing
  • Buy 5-9 Enrollments And Save 68% ($12.74 monthly.)
  • Buy 10-19 Enrollments And Save 72% ($11.24 monthly.)
  • Buy 20-above Enrollments And Save 78% ($8.99 monthly.)
5 Learners Have Enrolled For This Course

You have already taken demo for this course.

If you want to get access to demo again, feel free to contact our support at (855) 800-8240

These hold/produce data to data stores like HDFS, HBase and Hive. Both the tools have default functionality and have the ability of abstracting away the users from the complication of transferring data among these systems.

Course Information

About this course:

Import data: Flume and Sqoop have a crucial part to play in the Hadoop ecosystem. They have the responsibility of transferring the data from sources like local file systems, HTTP, MySQL and Twitter. These hold/produce data to data stores like HDFS, HBase and Hive. Both the tools have default functionality and have the ability of abstracting away the users from the complication of transferring data among these systems.

Flume: Flume Agents have the ability to transfer data created by a streaming application to data stores like HDFS and HBase.

Sqoop: Sqoop can be used to bulk import data from typical RDBMS to Hadoop storage structures like HDFS or Hive.

Learning Objectives:

Practical application for the various sources and data stores:

  • Sources: Twitter, MySQL, Spooling Directory, HTTP
  • Data stores: HDFS, HBase, Hive

Flume components:

  • Flume Agents
  • Flume Events
  • Event bucketing
  • Channel selectors
  • Interceptors

Sqoop components:

  • Sqoop import from MySQL
  • Incremental imports using Sqoop Jobs

Audience:

This course will be highly useful for those engineers who have the responsibility designing an application with HDFS/HBase/Hive as the data store. This will also be suitable for those engineers who intend to port data from legacy data stores to HDFS.

Requirements:

The course has a mandatory requirement of having knowledge of HDFS. You should also be having fundamental understanding of HBase and Hive shells, as HBase and Hive examples require that. Additionally, you should also be having a working installation of HDFS, because it is required to run majority of the examples. 

Outline

More Information

More Information
SubjectsBig Data
Lab AccessNo
Learning StyleSelf-Paced Learning
Learning TypeCourse
DifficultyIntermediate
Course Duration2 Hours
LanguageEnglish
VPA DiscountVPA Discount

Reviews

Write Your Own Review
Only registered users can write reviews. Please Sign in or create an account

Course Expert:

Author

Tom Robertson
(Data Science Enthusiast)

Tom Robertson is a Python-based data scientist with domain knowledge in behavioral neuroscience research and non-profit development. He is currently employeed as Data Science Instructor at QuickStart Technologies. Tom is inspired by the belief that data science can improve quality of life by providing our researchers, doctors and policy-makers with new, data-driven insights.

Tom has expertise on Python, SQL, and Spark. He has worked on several libraries including but not limited to Scikit-learn, Pandas, NumPy, Matplotlib, Seaborn, SciPy, NLTK, Keras, and Tensorflow.

This Subscription Includes:

Information
Virtual instructor Led

Virtual Classroom Courses

Our virtual instructor-led courses give you access to live instructors training you with other live students in a virtual classroom environment.

200+ Virtual Classroom Courses
Information
Self-Paced

Online Self-Paced Courses

Take self-paced online courses at your convenience and own pace, with unlimited access to courses in various emerging technologies.

700+ Self-Paced Courses
Information
College

E-books blogs, case studies, ariticles

As part of informal learning, our platform will recommend E-books, whitepapers, case studies, articles, and videos. This is AI curated content closely aligned with your learning objectives.

E-books blogs, case studies,...
Information
College

College Accredited Courses

QuickStart courses are accredited by several top schools and universities, including Texas A&M and University of Phoenix. You can print out certificates and also apply them towards your degree plan with them.

College Accredited
Information
Dashboard

Full Learning Dashboard & Analytics

Access all your enrolled, completed, course statistics, and community discussions from one centralized and intuitive learning dashboard with built in analytics, course tracking, time spent, and more.

Analytics/Reporting
Information
Social

QuickStart Discussions

Engage with other learners where you can directly chat, ask questions, and socialize with other learners experts and instructors on a course subject.

Community Access Community Access
Information
Labs

Virtual Labs

Videos and lectures only go so far. Get real world, hands-on practice with virtual labs (not available for all courses).

Virtual Labs Virtual Labs
Information
Live Instructor Support

Live Instructor Mentoring & Support

Get your IT problems solved through a community of mentors, experts and peers. Get live help from experts to answer questions on course material or guidance on a project.

Mentoring & Discussions Mentoring & Discussions
Information
Dashboard

Career Paths

Start a learning pathway towards understanding and mastering your career. With QuickStart career paths, you can fully understanding and being the best in your field.

Learning Paths Learning Paths
Information
Dashboard

Informal Learning

Access to AI curated content from various content publishers which can help in self-directed learning.

Informal Learning Informal Learning
click here