• Attention variants

• RoPE, ALiBi

• Grouped Query Attention (GQA)

• Mixture‑of‑Experts (MoE)

• Build models from first principles, not just adapt pre‑existing codebases

OptimisationDynamics & Training Stability

• Own optimizer and scheduler choices, including:

• AdamW

• Lion

• Adafactor

• Learning‑rate and warmup schedulers

• Understand and debug:

• Optimizer instability

• Gradient pathologies

• Divergence at large scale

Scaling Laws & Compute Tradeoffs

• Apply and validate scaling laws

• Navigate Chinchilla‑style compute vs data tradeoffs

• Make informed decisions about model size, dataset size, and training duration

Loss Functions & Alignment

• Design and experiment with loss functions including:

• Next‑token prediction

• Contrastive objectives

• RLHF, DPO, GRPO

• Understand how loss design impacts convergence, generalization, and alignment

Distributed Foundation Model Training

• Design and execute large‑scale training using:

• FSDP

• ZeRO‑3

• Tensor parallelism

• Pipeline parallelism

• Apply

• Mixed precision (bf16, fp8)

• Gradient checkpointing

• Partner closely with ML systems teams while retaining architectural ownership

Architecture Innovation

• Explore and implement novel model designs, including:

• MoE routing strategies

• Multimodal fusion architectures

• SSM / hybrid architectures

• Design architectures with KV cache efficiency and inference implications in mind

What Success Looks Like

• Training remains stable as models scale in size and complexity

• Architectural decisions are principled and defensible

• Models converge faster and generalize better due to architecture and optimisation choices

• Failure modes are understood, not mysterious

• The organization develops true in‑house foundation model expertise

Required Experience & Skills

Strongly Required

• Deep theoretical and practical understanding of modern deep learning

• Hands‑on experience training large models from scratch

• Ability to reason about optimization, not just tune hyperparameters

• Comfort operating in ambiguous, research‑driven environments

Critical Technical Skills

• Transformer internals and attention mechanisms

• Optimisationalgorithms and training dynamics

• Scaling laws and compute/data tradeoffs

• Distributed training strategies and mixed precision

• Architecture innovation for large, real‑world models

Common Problems You’ll Be Solving

• Why training diverges at scale

• How optimizer dynamics interact with architecture

• When scaling laws break down

• The real tradeoffs between data, compute, and model design

What we offer

We offer a generous compensation and benefits package (in addition to the base salary), including:

• Salary range$185,000.00 - $280,000.00 It is not typical for offers to be made at or near the top of the range. The actual salary will be determined based on experience and other job-related factors.

• Annual bonus opportunity

• Insurance coverage (medical, dental, vision, life, and disability)

• Paid time off

• Paid holidays

• Company contribution to the RRSP (Registered Retirement Savings Plan)

• Equity awards for certain positions and levels

• Remote and/or hybrid work available depending on the position

All compensation and benefits are subject to the terms and conditions of the underlying plans or programs, as applicable, and may be amended, terminated, or replaced from time to time.