VGU RESEARCH REPOSITORY
Please use this identifier to cite or link to this item:
https://epub.vgu.edu.vn/handle/dlibvgu/1861| Title: | Optimizing data pipeline for data analytics | Authors: | Nguyen Duy Phu Quang | Keywords: | Data warehousing;Centralized warehouses;Data pipeline architecture.;Data analytics | Issue Date: | 2024 | Abstract: | The thesis examines the optimization of data pipeline architecture by implementing a lakehouse approach, which improves data management and analytical processes. The research uses AWS S3, dbt Cloud, and Snowflake to clean and preprocess data in the AdventureWorks dataset efficiently. The paper investigates how orchestration, automated ingestion with Snowpipe and AWS SNS, and micro-partitioning can enhance pipeline performance. It focuses on integrating raw, historical, and transformed data to create dimensions and facts tables optimized for advanced visualization in PowerBI. The design of this pipeline allows for scalability and adaptability so that it can be used in contemporary data ecosystems that require fast processing speeds alongside insightful analytics capabilities. This study contributes new knowledge on faster decision making through better pipelines in information processing |
URI(1): | https://epub.vgu.edu.vn/handle/dlibvgu/1861 | Rights: | Attribution-NonCommercial 4.0 International |
| Appears in Collections: | Computer Science (CS) |
Files in This Item:
| File | Description | Size | Format | Existing users please Login |
|---|---|---|---|---|
| Optimizing data pipeline for data analytics.pdf | 22.34 MB | Adobe PDF |
Page view(s)
66
checked on Nov 15, 2025
Download(s)
29
checked on Nov 15, 2025
Google ScholarTM
Check
This item is licensed under a Creative Commons License