Muhammad Shoaib UR Rehman
Data Architect
Lahore
Summary
Senior Data Engineer with 4+ years of experience in data engineering, big data systems, and AI/ML-driven analytics. Skilled in designing scalable data architectures and deploying cost-effective solutions on AWS. Proficient in Python, SQL, and Spark, with a strong track record in building robust data pipelines, optimizing workflows, and leading high-performing teams.
Known for a strategic, solution-focused mindset and the ability to deliver impactful data solutions in fast-paced environments. Strong collaborator with a passion for innovation and data-driven decision-making.
Overview
5
5
years of professional experience
2021
2021
years of post-secondary education
2
2
Certificates
Work History
Senior Data Engineer
Arbisoft
04.2024 - Current
- Design and maintain Alternative Data pipelines from different vendors and perform ETL to generate Time Series data for financial insight and prediction models.
- Reengineered existing ETL workflows to improve performance up to 30% and cost by 50% by identifying bottlenecks and optimizing code accordingly.
- Mentored junior team members in best practices for software development, code optimization, and troubleshooting techniques.
- Ran statistical analyses and ingestion to process large datasets with TBs of data volume.
- Applied loss functions and variance explanation techniques to compare ticker performance metrics.
Python Expert - Data Engineer
Upwork
07.2020 - Current
- Upwork Top Rated Freelancer for Python and Big Data Domain with 100% Job Success Rate.
- Help Large Scale Businesses and Startups build Robust and Scalable IOT applications, Web Scrappers and Integration applications.
- Provide Support in Big Data Tools and AWS Cloud Infrastructure to optimize cost, scalability, security and fault-tolerance. Help them to reduce Production Errors and Monitor Application Performance to reduce Time-to-Market (TTM) significantly (up to 6 months).
- Employed NLP, Deep Learning, Machine Learning and Sci-Kit Learn libraries to extract insights from massive data sets and provided Predictive Analytics to ensure effective Decision Making for the clients.
Data Engineer - Team Lead
CureMD
07.2021 - 04.2024
- Led the development of AWS-based DataLake infrastructure for multi-tenant products, ingesting data from EHR, Patient Portal, and Claim Processing. Achieved 45% cost reduction through cloud optimization. Ensured historical and real-time data analysis capabilities.
- Designed data transformation pipelines converting TBs of patient data into HL7 FHIR standard, enabling rapid analysis and AI/ML workflows. Transformed 3+ terabytes of patient data. Reduced data processing time from months to days. Implemented healthcare data quality improvements, including data normalization, de-duplication, and standardization.
- Strengthened healthcare data security by implementing CIS benchmarks and achieving HIPAA compliance, resulting in a flawless security record with zero data breaches.
- Supported ML/AI applications with clean, aggregated data.
- Managed and trained a team of data engineers to deliver data analysis and critical business KPIs.
Certification
Certified Solutions Architect - Associate, Amazon Web Service, 2022, The focus of this certification is on the design of cost and performance optimized solutions, demonstrating a strong understanding of the AWS Well-Architected Framework.
Skills
Apache Spark
Apache Kafka
AWS Glue
Apache Nifi
AWS EMR
AWS Sagemaker
AWS Redshift
Snowflake/Snowpark
HDFS/Hive/Presto
Postgres/SQL/MongoDB
Redis/DuckDB
Python/NET/Java
Amazon Web Service (SAA certified)
Scikit-learn/TensorFlow/Matplotlib
Apache Superset
AWS QuickSight
Education
Bachelor of Science -
Timeline
Senior Data Engineer - Arbisoft
04.2024 - Current
Data Engineer - Team Lead - CureMD
07.2021 - 04.2024
Python Expert - Data Engineer - Upwork
07.2020 - Current
National University of Science and Technology - Bachelor of Science,