Hammad Yasir
Lahore
Summary
Analytical and process-oriented data engineer with strong knowledge of modern data engineering techniques. Proven expertise in data warehousing, databases, data quality, and data cleaning techniques. Proficient in utilizing cutting-edge cloud technologies and Big Data platforms.
Overview
5
5
years of professional experience
Work History
Senior Data Engineer
KAVTECH SOLUTIONS (PRIVATE) LTD.
06.2022 - Current
- Successfully implemented changes in ETL Jobs and Transformations to load real-state's data from source file to staging table and then to target table using Pentaho Data Integration
- Prepare build for complete ETL solution and deployed it on Linux Server
- Developed complete ETL pipeline which included extraction of data from tabular and non-tabular data sources and performed Merging of Data Streams, Data Cleaning, Data Validation
- Identify Existing issues of running ETL jobs on Pentaho Data Integration and make plan to shift existing ETL Jobs to Cloud Based Environment like AWS
- Work on following services AWS Lambda, S3, Athena, Redshift - Cloud Watch - SQS services - AWS Glue, AWS Glue DataBrew, EC2, EC2 Auto Scaling
- Implemented ETL Jobs and Transformations to load data from different sources(API based and non API sources) to pre_staging table, Cleansing data process, moving to the stage area using AWS Glue and AWS DataBrew, then store data to the target table using AWS lambda
- Developed a complete ETL pipeline, which included data extraction, transformation, Data Cleansing, and Data Validation
- Extract Data using python API, Write Parser to identify Meta data and data store to s3 bucket, cleaning data, manipulating data, running glue jobs for mapping data with targets tables.
Data Engineer
TECHNOGENICS SMC PVT LTD
08.2021 - 04.2022
- I was involved in the design, development, and management of large-scale data stores for turning data into information and insights
- From collecting and identifying requirements and translating them into scalable and stable solutions
- Facilitate strategic design and decisions related to core and new functions and features of the product
- Identify design problems and work with developers to implement state-of-the-art design solutions
- Build data warehouse pipeline using python, SQL, Kafka, AWS, Docker, Elastic Stack (ELK)
- Use different Data Ingesting and storing tool like Kinesis
- Cyber security product name Strikready having components, ingesting data, compressing data, forwarding data to the cloud
- Reverse steps like decompress, decrypt data, unpacking each dataset.
Data Engineer
BINARYTECH (PRIVATE) LIMITED
07.2020 - 06.2021
- Develop AWS ETL pipeline using AWS S3, AWS Glue
- Created AWS Lambda function that downloaded files from a website and uploaded it incrementally on S3 bucket and scheduled using AWS Event Bridge
- Developed Data pipeline using python which pulls JSON data from RDMS, extracts information from JSON, clean and manipulate data and then re-generate JSON and push data into target tables
- Developed python package using Boto3 which can perform CRUD operation on AWS RDS
- Developed Data pipeline using python which pulls JSON data from RDMS, extracts information from JSON, clean and manipulate data and then re-generate JSON and push data into target tables
- Making connecter using API with different clouds providers like IBM, AZURE, and AWS and ingest data
- Worked with spark jobs and Hadoop jobs
- Integrate with spark jobs using pylivy to run job on Spark cluster.
Data Engineer
JUMP SOLUTIONS
05.2019 - 06.2020
- Develop AWS ETL pipeline using AWS S3, AWS
- Created AWS Lambda function that downloaded files from a website and uploaded it incrementally on S3 bucket and scheduled using AWS Event Bridge
- Worked on AWS redshift to build End to End client solution.
Education
BS (Computer Science) -
UNIVERSITY OF AGRICULTURE FAISALABAD (UAF), FAISALABAD
01.2019
Skills
- Python3
- Pandas
- SQL
- ETL & ELT
- Data warehousing
- Hadoop & Apache Spark
- Pyspark
- Databricks
- Apache Airflow & Apache Nifi
- Apache Kafka
- Web Scrapping
- Data Manipulation
- Data Importing
Data Engineering
AWS
- IAM
- Boto3 & Aws wrangler
- Lamdba & Athena
- S3 & EC2
- RDS & DynamoDB
- Kinesis & AWS Glue
- Build Data Pipeline
- AWS Data Lake
- Amazon Redshift
- Amazon EMR
- Azure Blobs
- Azure Data Factory
- Azure Synapse Analytics
AZURE
CERTIFICATIONS & COURSES
- Python for Everybody Specialization (07/2020)
- IBM Data Science Professional Certificate (07/2020)
- Streamlined Data Ingestion with pandas (02/2023)
- SQL Fundamentals (02/2023)
- Intermediate SQL (02/2023)
- Joining Data in SQL (02/2023)
- Big Data Fundamentals with PySpark (12/2022)
- Introduction to SQL (12/2022)
- SQL Masterclass: SQL For Data Analytics (12/2022)
- Introduction to AWS Identity and Access Management (08/2022)
- IBM Data Engineering Professional Certificate | Coursera (06/2023)
Languages
Urdu
Native language
English
Proficient
C2
Timeline
Senior Data Engineer
KAVTECH SOLUTIONS (PRIVATE) LTD.
06.2022 - Current
Data Engineer
TECHNOGENICS SMC PVT LTD
08.2021 - 04.2022
Data Engineer
BINARYTECH (PRIVATE) LIMITED
07.2020 - 06.2021
Data Engineer
JUMP SOLUTIONS
05.2019 - 06.2020
BS (Computer Science) -
UNIVERSITY OF AGRICULTURE FAISALABAD (UAF), FAISALABAD
Similar Profiles
Haider AliHaider Ali
Software Engineer at Kavtech Solutions (Private) Ltd.Software Engineer at Kavtech Solutions (Private) Ltd.
Hamda AneesHamda Anees
Software Engineer at Kavtech SolutionsSoftware Engineer at Kavtech Solutions
Billy WijatmikoBilly Wijatmiko
Private Piano Teacher at PrivatePrivate Piano Teacher at Private