Summary
Overview
Work History
Education
Skills
Certification
Timeline
Generic

Syed Muhammad Hasan

Data Engineering Manager
Islamabad,IS

Summary

Dynamic Data Engineering Manager with a proven track record at QLU.AI, leading teams to optimize data architecture and enhance performance. Expert in Python and PostgreSQL, I successfully migrated 54 million records, improving system stability and search efficiency. Strong problem-solver, driving innovative solutions across data infrastructure.

Overview

1
1
year of professional experience
2
2
Certifications

Work History

Data Engineering Manager

QLU.ai
Islamabad, Pakistan, Federal
10.2024 - Current
  • Led a team of data engineers, overseeing architecture design and technical execution.
  • Proposed and implemented a serverless Elasticsearch deployment with auto-scaling to replace the static instance setup, optimizing performance for over 700 GB of data. The previous setup experienced CPU spikes due to I/O-bound operations like disk reads and query-time document scoring. The new architecture auto-scales resources to handle heavy search loads without performance degradation.
  • Evaluated alternatives like Manticore Search for cost and performance trade-offs; integrated into select features where suitable.
  • Drove solution-oriented decision-making across the data infrastructure stack.

Senior Data Engineer

QLU.ai
Islamabad, Federal
06.2024 - 10.2024
  • Fixed a long-standing issue with the location filter by building a clean mapping of cities, towns, and counties to their metro areas using internet and other external sources.
  • Migrated 54 million user records from an unstable Elasticsearch setup to a hybrid architecture using PostgreSQL and Elasticsearch, which helped stop site crashes and made searches faster.
  • Set up a basic data health check to make sure PostgreSQL and Elasticsearch stay in sync, using periodic checks and simple validations.
  • Added a lazy update system to keep people and company data in sync when one is updated, reducing manual fixes and keeping things consistent.
  • Cleaned and organized messy user data, fixing broken fields and making it easier to query and work with..

Data Engineer

QLU.ai
Islamabad, Federal
01.2024 - 06.2024
  • Set up a framework for running ETL pipelines using Airflow in a detached Docker container environment.
  • Ingested financial data for 60K public companies identified 40K+ PE-backed companies through web data analysis.
  • Synced financial records with our 8M company dataset to accurately link financial data to the correct entities.
  • Collected and labeled web images to support the AI team in building a demographics-based filtering feature.

Education

BEE - Electrical Engineering

NUST
Islamabad, Pakistan
04.2001 -

Skills

Python

Postgresql

Elastic search

Airflow

Sqlalchemy

Alembic

GCP cloud functions, cloud run

AWS lambda

Selenium

Certification

Azure Data Engineering Associate

Timeline

Data Engineering Manager

QLU.ai
10.2024 - Current

Azure Data Engineering Associate

07-2024

Senior Data Engineer

QLU.ai
06.2024 - 10.2024

Data Engineer

QLU.ai
01.2024 - 06.2024

Azure Data Fundamentals

01-2024

BEE - Electrical Engineering

NUST
04.2001 -

Similar Profiles

CREATE PROFILE
Syed Muhammad HasanData Engineering Manager