About Laivly
Seeking curious and creative types! We are an ambitious company of innovators building and shaping the future of customer service technology. Our solutions help the world’s biggest brands leverage artificial intelligence, machine learning, and digital automation in their contact centers to deliver better customer experiences. Led by a team of established contact center experts, Laivly addresses the unique needs and challenges of customer service programs, with an emphasis on ethics in AI and the customer service agent experience.
About the role
The team at Laivly is looking for a Sr. Machine Learning Scientist & Engineer to lead our prompt optimization and validation efforts for LLM-powered features.
As a Sr. Machine Learning Scientist & Engineer, you’ll work closely with engineering and product teams to design robust, accurate, and scalable experiments and monitoring systems that support safe and reliable deployment of AI features. You’ll own the end-to-end design and evaluation of prompts, build synthetic datasets, and develop pre- and post-release validation frameworks that measure performance, safety, and cost-effectiveness.
This remote role is open to all Canadian Residents.
As Sr. ML Scientist & Engineer, you will…
- Lead prompt design, optimization, versioning, and documentation
- Generate and maintain synthetic datasets for pre-release validation and testing
- Design and run experiments to measure accuracy, robustness, and safety of LLM behavior
- Define and monitor key metrics such as hallucination rates, latency, token cost, and regression risk
- Build evaluation pipelines, dashboards, and drift detection tools
- Implement rigorous statistical methods and produce reproducible reports with effect sizes and confidence intervals
- Investigate regressions or prompt failures in production and coordinate mitigation with stakeholders
- Collaborate with cross-functional teams on model integration, rate limiting, and cost controls
- Maintain prompt libraries, test suites, and operational runbooks
As Sr. ML Scientist & Engineer, you have…
- Bachelor’s degree in Computer Science, Computational Linguistics, Statistics, or related field; Master’s preferred
- 2+ years working directly with LLMs, prompt design, or applied NLP in production
- Experience creating synthetic datasets using programmatic generation or controlled sampling
- Deep knowledge of experimental design, statistical inference, and power analysis
- Strong Python programming skills and familiarity with statistical/ML frameworks
- Experience defining observability for model outputs and instrumenting alerting systems
- Familiarity with cloud environments (AWS, Azure), Kubernetes, and production deployment practices
- Proficiency in SQL and experience with reproducible analysis pipelines
- Excellent communication skills and strong documentation practices
Life at Laivly
Laivly gives you the opportunity to collaborate and grow your career with a creative, diverse, and passionate team. We work hard and play often, with a flexible environment that works with you. A career at Laivly means being part of a fun-loving, dedicated team of creatives, risk takers and game changers. It’s about sharing your talent and imagination to develop innovative tech that’s revolutionizing the way top brands interact with the world.
We’ve got a shared mission—and a Laivly future. Join us today!
Laivly provides Equal Employment Opportunities in accordance with all provincial and federal laws. Laivly is committed to ensuring equality of opportunity in all aspects of employment and does not discriminate based on protected characteristics.
Laivly is committed to accommodating persons with disabilities. If you need accommodation at any stage of the application process or want more information on our accommodation policies, please let us know.
.png)
