The Senior Data Engineer will design and implement batch and streaming data pipelines, evolve AWS data lake architecture, and mentor peers while ensuring optimal performance and cost efficiency.
We are building a greenfield analytics platform supporting both batch and real-time data processing. We are looking for a Senior Data Engineer who can design, implement, and evolve scalable data systems in AWS.
This role combines hands-on development, architectural decision-making, and platform ownership.
Core Responsibilities:
- Design and implement batch and streaming data pipelines using Apache Spark.
- Build and evolve a scalable AWS-based data lake architecture.
- Develop and maintain real-time data processing systems (event-driven pipelines).
- Own performance tuning and cost optimization of Spark workloads.
- Define best practices for data modeling, partitioning, and schema evolution.
- Implement monitoring, observability, and data quality controls.
- Contribute to infrastructure automation and CI/CD for data workflows.
- Participate in architectural decisions and mentor other engineers.
Required Qualifications:
- 5+ years of experience in Data Engineering.
- Strong hands-on experience with Apache Spark (including Structured Streaming).
- Experience building both batch and streaming pipelines in production environments.
- Proven experience designing AWS-based data lake architectures: S3, EMR, Glue, Athena.
- Experience with event streaming platforms such as Apache Kafka or Amazon Kinesis.
- Experience implementing lakehouse formats such as Delta Lake.
- Strong understanding of partitioning strategies and schema evolution.
- Experience using SparkUI and AWS CloudWatch for profiling and optimization.
- Strong understanding of Spark performance tuning (shuffle, skew, memory, partitioning).
- Proven track record of cost optimization in AWS environments.
- Experience with Docker and CI/CD pipelines.
- Experience with Infrastructure as Code: Terraform, AWS CDK.
- Familiarity with monitoring and observability practices.
- Experience in the Financial domain.
- Experience running Spark workloads on Kubernetes.
- Experience implementing data quality frameworks or metadata/lineage systems.
- English - B2, Ukrainian- Native
Top Skills
Amazon Kinesis
Apache Kafka
Spark
AWS
Aws Cdk
Aws Cloudwatch
Ci/Cd
Delta Lake
Docker
Sparkui
Terraform
Similar Jobs
Artificial Intelligence • Information Technology • Consulting
Design and maintain data pipelines and ML systems, implement MLOps practices, and collaborate with teams to address client challenges in AI and data projects.
Top Skills:
Apache AirflowSparkAWSBedrockClickhouseEcrGlueKafkaLambdaPythonRedshiftS3SagemakerSnowflakeSQL
Information Technology • Software • Consulting
The Senior Data Engineer will optimize data integrations for Retail, design ETL workflows, enhance BigQuery performance, and maintain data quality in a collaborative, growth-driven environment.
Top Skills:
AirflowAPIsBigQueryCloud ComposerCloud Data WarehousingDatabasesDataflowDbtGitlabGCPPythonSnowflakeTerraform
Software
Develop and deliver data pipelines, manage deployments, ensure data quality, and support troubleshooting within the data delivery team.
Top Skills:
Azure Data FactoryCi/CdDbtGitSnowflakeSQL
What you need to know about the Toronto Tech Scene
Although home to some of the biggest names in tech, including Google, Microsoft and Amazon, Toronto has established itself as one of the largest startup ecosystems in the world. And with over 2,000 startups — more than 30 percent of the country's total startups — Toronto continues to attract new businesses. Be it helping entrepreneurs manage their finances, simplifying business operations by automating payroll or assisting pharmaceutical companies in launching new drugs, the city's tech scene is just getting started.
