- Basic data analysis concepts and terminology
- Kinesis Data Streams
- Kinesis Data Firehose
- Kinesis Video Streams
- Kinesis Data Analytics
- Visualize data for machine learning
Sources
Sources for streaming data
- Kaggle.com
- University of California Irvine
- Open Data on AWS
- Google big query also gives access to data sets
Kinesis Data Streams
- Gets data from data producers such as IoT, social media
- Uses shards to stream data to consumers such as EC2, Lambda, Kinesis Data Analytics, EMR clusters
- Consumers then send data to a data repository such as S3, DynamoDB, Redshift, or Business Intelligence Tools
Kinesis Data Firehose
- Receives data from producers such as IoT, social media
- Uses Lambda functioning instead of shards to transmit producer data
- Lambda function puts data to data stores such as S3, Redshift, ElasticSearch, or Splunk
- Can transmit directly from producers through Firehose to the data store (don’t have to use Lambda intermediary)
- S3 events to store to DynamoDB
Kinesis Video Streams
- Build video processing applications such as machine learning models
- Producers such as web cams, security cameras, audio feeds, images
- Data Consumers - Kinesis Video Stream applications
- Stores to S3
Kinesis Data Analytics
- Use SQL to process streaming data
- Sources: Kinesis Data Streams and Kinesis Data Firehose
- SQL queries put to S3, Redshift, or visualization and Business Intelligence Tools
Visualizing Data for Machine Learning
- Technique using static and interactive visuals to represent large amounts of data
- Visualizes patterns, trends and correlations that may be difficult to discern
- Data visualization helps monetize data as a product
本文由
Oscaner
创作, 采用
知识共享署名4.0
国际许可协议进行许可
本站文章除注明转载/出处外, 均为本站原创或翻译, 转载前请务必署名
-
Previous
[MLS-C01] [Data Engineering] AWS Migration services -
Next
[MLS-C01] [Exploratory Data Analysis] Kinesis Data Streams