Medical LLM & SLM Development
Built GPT-2–style Small Language Model from scratch with cross-hardware multi-GPU training on AMD MI300X systems using DDP and gradient accumulation.
- Custom Causal Self-Attention, LayerNorm, Transformer blocks
- 8-GPU AMD MI300X distributed training with DDP
- Fine-tuned 7B medical LLM using LoRA for parameter-efficient adaptation
- Mixed-precision strategies for memory efficiency
- Verifiable medical reasoning tasks with SFTTrainer