Data ingestion, the process of collecting or streaming information from various sources like log files, social media files, and SQL databases. It encounters three important challenges while ingestion Large table ingestion, Schema changes in the source and Change data capture
To Load or store the extracted data in HDFS or NoSQL by the HBase. It can be easily accessed and processed by applications.
This is an actual process of getting a bigdata solution. To analyze big data sets at terabyte or even petabyte-scale by MapReduce or Spark framework.
Step 1: Data SourcesStep 2: Integration and Data StorageStep 3: Data Models and AnalyticsStep 4: Visualization and Reporting.
Step 1: Data Sources
Step 2: Integration and Data Storage
Step 3: Data Models and Analytics
Step 4: Visualization and Reporting.