Design, build, and maintain adversarial AI evaluation platforms and AI pipelines. Implement software engineering best practices for ML, integrate UI/UX, deploy on major cloud providers, and collaborate cross-functionally to improve model robustness and product infrastructure.
About Armilla AI
Armilla AI is a cutting-edge startup based in Toronto, Ontario, Canada, operating at the intersection of artificial intelligence and insurance. We're not just talking about AI risk; we're actively building the solutions to manage, underwrite, and insure it. Our dynamic team is passionate about pioneering the future of AI risk management through the development of sophisticated AI-powered tools and programmatic evaluation platforms.
The Role: Building & Evaluating AI SolutionsWe're looking for a talented and experienced AI Engineer to join our growing team. This is a pivotal role where you'll be instrumental in both building our core AI-powered tools and developing the programmatic platforms to rigorously evaluate AI systems. If you're passionate about applying robust engineering principles to AI, creating end-to-end solutions, and building systems for comprehensive AI assessment, we want to talk to you.
In this exciting role, you'll:
- Collaborate with senior AI/ML Scientists and Product Managers to design, build and maintain an adversarial AI evaluation platform.
- This role requires a combination of strong engineering skills, product development experience, and AI/ML process-flow knowledge.
- Architect and implement programmatic platforms for evaluating AI models, assessing their performance, robustness, and potential risks.
- Implement and optimize AI pipelines, ensuring efficient data flow, model training, inference, and evaluation processes.
- Apply software engineering best practices to machine learning (ML) projects, including version control, automated testing, code reviews, and continuous integration/continuous deployment (CI/CD).
- Develop and integrate UI/UX components to ensure seamless and intuitive user experiences for both our internal tools and evaluation platforms.
- Deploy and manage AI applications and evaluation infrastructures on various Cloud Service Providers (CSPs) like AWS, GCP, or Azure, ensuring scalability, reliability, and security.
- Collaborate closely with data scientists, actuaries, and product managers to translate complex requirements into robust technical solutions for both building and evaluating AI.
- Troubleshoot, debug, and optimize existing AI systems to improve performance and stability.
We're seeking a candidate who brings:
- Strong engineering skills with a proven track record in software development.
- Solid understanding of AI model lifecycle and especially concepts around AI model evaluations
- In-depth knowledge of AI pipelines and experience in building, managing, and optimizing them for both product development and evaluation.
- Demonstrable experience applying software engineering best practices to ML projects.
- Proficiency in creating end-to-end software stacks, including front-end (UI/UX) and back-end development.
- Hands-on experience with deployment and management on major Cloud Service Providers (CSPs) such as AWS, GCP, or Azure.
- Strong programming skills in languages like Python.
- Experience with relevant AI/ML frameworks and libraries (e.g., TensorFlow, PyTorch, scikit-learn).
- Exceptional problem-solving abilities and a commitment to producing high-quality, maintainable code.
- Strong communication and collaboration skills, with the ability to work effectively in a cross-functional team.
Joining Armilla AI means:
- Pioneering a New Frontier: You'll have a significant impact on our core products and infrastructure, shaping how AI risks are understood and managed globally.
- Diverse Technical Challenges: You'll work on both developing cutting-edge AI tools and building sophisticated systems for AI evaluation.
- Collaborative Environment: Be part of a highly skilled and supportive team that values innovation and continuous learning in a fast-paced startup.
- Dynamic Work Environment: Enjoy a hybrid work environment based in Toronto's trendy Liberty Village, fostering a collaborative and innovative atmosphere.
- Professional Growth: This role offers immense opportunities for personal and professional development in the burgeoning field of AI risk.
- Access to Cutting-Edge Tooling: You'll have access to your choice of leading AI models (e.g., GPT), operating systems, and cloud compute services to empower your work.
Ready to help us build and evaluate the future of AI risk solutions? If you're an experienced AI Engineer passionate about robust software and cutting-edge technology, we encourage you to apply! Please submit your resume and a cover letter outlining your relevant experience and why you're a great fit for Armilla AI
Armilla AI Toronto, Ontario, CAN Office
Toronto, Ontario, Canada
Similar Jobs
Cloud • Information Technology • Productivity • Security • Software • App development • Automation
Lead technical strategy and development of Confluence AI experiences across frontend, backend, and AI integrations. Design scalable systems, apply generative AI, retrieval, agents, evaluation, and safeguards, and improve product quality, adoption, trust, and customer value. Drive complex initiatives from architecture through launch, partner cross-functionally, mentor engineers, establish quality practices, and guide experimentation and continuous improvement across Atlassian.
Top Skills:
Ai AgentsAutomated EvaluationExperimentationFull-Stack DevelopmentGenerative AiInformation RetrievalMachine LearningProduction Monitoring
Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
Design and develop AI capabilities for CrowdStrike’s cross-platform endpoint sensor across macOS, Linux, and Windows. Responsibilities include operationalizing machine learning models, building telemetry pipelines for training and inference, integrating features with core OS subsystems, debugging performance and memory issues, and leading architecture, delivery, mentoring, and cross-functional collaboration.
Top Skills:
CC++Data PipelinesEbpfLinuxMachine LearningmacOSOn-Device InferenceSwiftWindowsWindows Driver DevelopmentWindows Os Kernel
Artificial Intelligence • Cloud • Computer Vision • Hardware • Internet of Things • Software
Architect and own marketing data infrastructure, including Databricks pipelines, Gold Layer tables, semantic models, and trusted analytics foundations. Lead cross-functional data initiatives, translate ambiguous business needs into scalable solutions, improve reliability and quality, and mentor engineers. Build and monitor production LLM and agent systems, automate workflows with AI, and deliver high-quality Python and SQL while validating AI-generated code. Partner with Marketing, Sales, BI, Data Science, and R&D teams.
Top Skills:
Ai AgentsBigQueryClaude CodeCursorData WarehousesDatabricksDbtEtl/EltLakehousesLlmsMarketing AutomationMartechPythonSalesforceSemantic LayersSnowflakeSQLStreamlitWeb Analytics
What you need to know about the Toronto Tech Scene
Although home to some of the biggest names in tech, including Google, Microsoft and Amazon, Toronto has established itself as one of the largest startup ecosystems in the world. And with over 2,000 startups — more than 30 percent of the country's total startups — Toronto continues to attract new businesses. Be it helping entrepreneurs manage their finances, simplifying business operations by automating payroll or assisting pharmaceutical companies in launching new drugs, the city's tech scene is just getting started.
