Role Overview:
Are you an experienced Data Engineer ready to join our Enterprise Data Platform team as a Senior Data Ingestion Engineer? Reporting to the Senior Manager of Data Engineering, you'll be a key member of McAfee’s enterprise data platform team and responsible for data engineering activities on the Databricks platform.
This role requires a deep understanding of
- Data Lake ingestion processes and best practices
- ETL/ELT implementation
- CI/CD
- System integration tools
- Data pipeline management
This is a remote position based in Canada. We will only consider candidates in Canada and are not offering relocation assistance at this time.
About the role:
- Lead the Enterprise Data Platform Ingestion team, providing strategic guidance, mentorship, and leadership.
- Ingest data from a variety of source systems and tailor ingestion approaches on a per-system basis.
- Manage, maintain, and oversee ETL/ELT pipelines on the Databricks platform
- Optimize data pipelines for scalability and speed
- Document ingestion and integration flows and pipelines
- Use Airflow to schedule and automate ingestion jobs
- Manage metadata and master data in technical data catalog
- Ensure ELT/ETL design meets required security and compliance guidelines, and ensure PII management, flagging and risk assessment during ingestion
- Maintain ETL/ELT pipeline infrastructure and implement automated monitoring strategies
- Ensure adherence to SDLC best practices
- Strong deeper knowledge on SQL and Advance SQL using windows functions
- Ingestion of data from the product data to AWS real-time. Candidate should demonstrate the end-to-end data pipeline development.
- Product subscription model domain is a plus and good to have.
- Knowledge on Kong-API and Kinesis is a plus and good to have.
About you:
- Bring 10+ years of experience in Data Engineering, ingestion pipelining, and ETL/ELT
- Hold a bachelor’s degree in computer science, engineering, statistics, or related field
- Have hands-on experience with and understanding of the following:
- Spark/Scala
- SQL/SparkSQL
- Python/PySpark or similar programming language
- Kong-API and Kinesis
- Databricks / AWS
- Unity Catalog
- ETL/ELT development, monitoring and pipelining using tools such as Apache Airflow
- Ingestion tools such as Dell Boomi
- Data quality guidelines
- CI/CD pipelines
- Agile
- Git and version control
#LI-Remote
Company Overview
McAfee is a leader in personal security for consumers. Focused on protecting people, not just devices, McAfee consumer solutions adapt to users’ needs in an always online world, empowering them to live securely through integrated, intuitive solutions that protects their families and communities with the right security at the right moment.
Company Benefits and Perks:
We work hard to embrace diversity and inclusion and encourage everyone at McAfee to bring their authentic selves to work every day. We offer a variety of social programs, flexible work hours and family-friendly benefits to all of our employees.
- Bonus Program
- Pension and Retirement Plans
- Medical, Dental and Vision Coverage
- Paid Time Off
- Paid Parental Leave
- Support for Community Involvement
We're serious about our commitment to diversity which is why McAfee prohibits discrimination based on race, color, religion, gender, national origin, age, disability, veteran status, marital status, pregnancy, gender expression or identity, sexual orientation or any other legally protected status.
