Senior Data Engineer - HFT

My client is seeking a Senior Data Engineer to join their dynamic team. This role is crucial for enhancing their data infrastructure and analytics capabilities in a fast-paced and rapidly evolving environment. The ideal candidate will have 4+ years of hands-on experience and a strong command of data engineering principles and technologies. Key Responsibilities Design, develop, and maintain robust data pipelines for real-time analytics and machine learning applications. Implement and optimize stream processing solutions using technologies such as Apache Flink or Spark. Work collaboratively with quants, data scientists and analysts to transform and validate datasets. Ensure efficient data storage and retrieval using AWS services (S3, ClickHouse) and SQL solutions. Core Technical Skills Stream Processing: Expertise in Flink, Spark Streaming, PyFlink, Kafka, and AWS MSK. Programming: Proficient in Python (including libraries like Pandas, Polars, boto3, and pyarrow) and SQL (Flink/Spark SQL). Data Storage: Experience with S3, ClickHouse, Parquet, and JSON. Monitoring/Logging: Familiarity with Grafana for data visualization and Fluentd for logging. Cloud & Infrastructure Proficient in AWS services such as EMR and Kinesis Data Analytics. Strong knowledge of containerization technologies (Docker, Kubernetes) and orchestration frameworks (Airflow, YARN). Familiarity with Infrastructure as Code (IaC) tools such as Terraform and Ansible is a plus. Understanding CI/CD practices for DevOps integration. Qualifications Bachelors degree in Computer Science, Engineering, or related field (Master's preferred). 4+ years of experience as a data engineer or in a similar role. Strong problem-solving skills and ability to work under pressure in a fast-paced environment.