Projects — Data Merging and Report Generation Script

This script merges user and order datasets with Pandas, generates shopping category reports with pie charts, and saves outputs to S3.

Datasets integrate with Redshift for analysis. The full interactive report tool is best on a larger screen.

Project video coming soon

For the complete page, please use a laptop or desktop.

Back to projects

This script streamlines the process of merging multiple datasets using the Pandas library and generating comprehensive reports with ease.

Whether data is being consolidated from various sources or in-depth analysis is being conducted, this script empowers users to efficiently merge, manipulate, and visualize the data.

This script will generate the pie chart after merging the data, providing a pictorial representation that offers insightful visualizations.

The report and graph will be saved in the S3 bucket.

This project exemplifies the process of merging multiple datasets to create comprehensive reports with insightful visualizations, such as pie charts.

The resulting reports and graphs are stored in an S3 bucket, while datasets are uploaded to Redshift.

Specifically, The script merges two datasets; User Data and Order Data.

User Data containing columns such as User ID,User Name,User DOB, User Email and Registration Date.

Order Data containing columns like Order ID, User ID, Number of Products, Product Name, Order Date and Total Amount.

Subsequently, the merged dataset is used to generate shopping category reports, revealing the percentage of user shopping activity for specific categories.

To access further details about the project, kindly click on the link provided below.

Connecting to Django for a security token…

Generate the default report

Please click on "Generate File" button

Generate the report for your files

Upload the user file and order file as per "User and Order Table" details

Then Click on "Upload File"

Select User file:

Select Order file:

Generate the report for customized fields

Records Range

Please Enter the number of records to be generated:

Dafult value is 1000; Min value be 100 Max value can be 10000

Table Fields

Please select the checkbox for optional fields(columns) which you don't want to be include in report.

By default, reports will be generated with all (mandatory + optional) fields.

For the user file and order file column reference, Please refer below "User and Order Table"

Select the checkbox and click on "GET FILE".

User Email: Registration Date: Order Date:

User and Order Table

Column details of user file and order file for the reference of Upload file and Get file

User File :

Mandatory Columns: User Id, User Name, User DOB

Column Heading: user_id, user_name, user_dob

Optional Columns: User Email, Registration Date

Column Heading: user_email, registration_date

Order File :

Mandatory Columns:Order Id, User Id, Number of Products, Products Name, Total Amount

Column Heading: order_id, user_id, number_of_product, product_name, total_amount

Optional Columns: Order Date

Column Heading: order_date

Data Merging and Report Generation ScriptProject 2

Data Merging and Report Generation ScriptProject 2