Sentiment Analysis and Data Visualization

Application of Sentiment Analysis of Italian tweet with Python and Spark

A Spark Streaming implementation for Online Twitter Sentiment Analysis.

Efficiently tackle large datasets and perform big data analysis with Spark and Python

Graduation project categorizes popular search phrases using Python and Spark and presents them on a website to inspire creators.

In this project I stream data and do crime classification using Spark. This dataset contains incidents derived from the SFPD Crime Incident Reporting system. The data ranges from 1/1/2003 to 5/13/2015. I do some data analysis of crime scenes in different areas and with respect to other parameters.

Apache Spark is one of the most widely used and supported open-source tools for machine learning and big data. In this repo, discover how to work with this powerful platform for machine learning. This repo discusses MLlib—the Spark machine learning library—which provides tools for data scientists and analysts who would rather find solutions to b…

对Spark ML进行二次封装，提供api调用

A movie recommender system using user-based collaborative filtering algorithm.

Real-time product recommendation system built using Apache Spark, Kafka, and Python.

Forecasting e-commerce product demand using PySpark MLlib. Includes data preprocessing, feature engineering, Random Forest modeling, and evaluation via Mean Absolute Error.

Streaming ETL data pipeline for health event monitoring and predictive analytics using Kafka, Airflow, Docker, Hadoop and Spark ML.

Apache Spark is a fast and general-purpose cluster computing system. It provides high-level APIs in Java, Scala, Python and R, and an optimized engine that supports general execution graphs. It also supports a rich set of higher-level tools including Spark SQL for SQL and structured data processing, MLlib for machine learning, GraphX for graph p…

In this project, I've created an end-to-end ETL pipeline and subsequently developed a machine learning model to predict the price of Amazon products based on several product-related features.

Perform sentiment analysis on Yelp dataset with Apache Spark

Using the Thunder Library for Image Processing with Spark ML Lib

This project demonstrates a complete ETL pipeline for Formula 1 racing data using Azure Databricks, Delta Lake, and Azure Data Factory. It covers data ingestion, transformation with PySpark and Spark SQL, data governance with Unity Catalog, and visualization through Power BI. Designed to showcase real-world data engineering workflows in Azure.

Implemented an auto-clustering tool with seed and number of clusters finder. Optimizing algorithms: Silhouette, Elbow. Clustering algorithms: k-Means, Bisecting k-Means, Gaussian Mixture. Module includes micro-macro pivoting, and dashboards displaying radius, centroids, and inertia of clusters. Used: Python, Pyspark, Matplotlib, Spark MLlib.

This repo contains code for restuarant recommendation system for users based upon business rating value.