ML & GenAI Research Engineer.
I rebuild frontier AI systems from first principles in PyTorch — LLMs (DeepSeek-V3, LLaMA-3, GPT-OSS, Mamba-3, MoE, MLA, MTP), latent diffusion (Stable Diffusion 1.x), multimodal (PaliGemma-style VLMs), and agentic research platforms. Paper-faithful, single-GPU-first, fully from scratch.
I'm an ML & GenAI research engineer who builds frontier AI systems from first principles — no high-level wrappers, no black boxes. Every architecture in my portfolio was implemented bottom-up, from attention math to distributed training pipelines.
My recent work spans a 422M DeepSeek-V3 reproduction with MLA, aux-loss-free MoE, and multi-token prediction; a Stable Diffusion 1.x trained from scratch on 2× RTX 5090 (best loss 0.0947, epoch-42 checkpoint released); a hybrid 415M / 868M FusionLLM combining MLA, Gated Delta Net, MoE, and MTP; a GPT-OSS-style long-context MoE (502M / 247M active) with sliding-window/full attention alternation and learned attention sinks; a Mamba-3 complex64 SSD implementation with MIMO head mixing and zero causal convolution; a PaliGemma-style vision-language model trained end-to-end on a P100; and a 15-phase autonomous ML research platform that reads papers, plans experiments, writes patches, runs training, and iterates.
I care about the full stack — paper comprehension, architectural design, numerical stability, memory engineering, distributed training, and now agentic orchestration. See the GitHub profile README for the canonical list, or jump straight to Projects.
Faithful from-scratch reimplementation of DeepSeek-V3 — Multi-Head Latent Attention (MLA) with decoupled RoPE, aux-loss-free MoE routing, multi-token prediction (MTP), true MLA absorption at inference, and MTP-based speculative decoding. Architecture, training pipeline, microbench, and step-time benchmarks are complete and pass; the Chinchilla-optimal 422M / 8.4B-token run is designed for a single A100 80GB (training pending).
From-scratch reimplementation of OpenAI's GPT-OSS long-context MoE architecture. Twelve layers alternate sliding-window (window=128) and full attention; per-head learned attention-sink bias (clamped to [−10, 15] for BF16 SDPA stability); YaRN RoPE for 128K-context extrapolation; top-2-of-8 routed experts plus 1 shared; standard auxiliary load-balancing loss. Verifies a 2× KV-cache reduction at 128K (1.13 GB vs 2.25 GB pure GQA, BF16). 130 tests across 10 files cover SWA/full parity, sink bias, YaRN extrapolation, MoE routing, and gradient flow.
From-scratch reproduction of Mamba-3 at 404M params, written in pure PyTorch (no mamba-ssm, no custom CUDA, no causal conv). Promotes the state-space recurrence into the complex plane (N=64 complex64) — half the state dimension of Mamba-2 at parity capacity. Adds a fully-connected MIMO mixer across SSM heads and drops the causal conv block. Targeted at a Chinchilla-optimal 8.0B-token run on a single A100 80GB (12–15 h). The companion SSD.md walks through the chunkwise algorithm and its equivalence to the naive O(T) recurrence.
PaliGemma-inspired vision-language model built from scratch. SigLIP-style vision encoder with sinusoidal patch embeddings, GQA-based language decoder with RoPE and GeGLU, connected via a trainable linear projector. Zero pretrained weights — every component trained end-to-end on COCO 2014 captions on a single P100 GPU.
A 15-phase multi-agent platform that turns an arXiv paper into evaluated experiments — paper analysis, repo analysis, experiment planning, code patches, training runs, statistical evaluation, autonomous looping, and end-to-end research reports. Provider-agnostic LLM layer with per-agent model routing across coding / reasoning / orchestration models.
From-scratch LLaMA-3-style transformer engineered for single-GPU pretraining. Headline result: a 78% peak-memory reduction (92 GB → 20 GB) from gradient checkpointing, chunked cross-entropy (logits 50 GB → 0.3 GB), and disk-backed token caching (RAM 112 GB → 1 MB) — enabling 2× batch-size headroom on a single A100 80GB. Architecture complete; the 8.25B-token run is designed (training pending).
A Stable Diffusion 1.x-class latent diffusion model trained from scratch on 2× RTX 5090 (Blackwell). 860M-param U-Net, DDPM + DDIM schedulers, custom 7-phase curriculum on 1.3M+ images (LAION-Aesthetic → high-aesthetic LAION → DiffusionDB/JourneyDB → VGGFace2 → COCO → consolidation), DDP + BF16 + gradient checkpointing + EMA + Min-SNR. The 42-epoch checkpoint is publicly released on Hugging Face (atandra2000/sd-from-scratch-v1). Best loss 0.0947 (epoch 16); the 48-epoch curriculum continues.
A hybrid LLM pre-training framework fusing MLA, Gated Delta Net (linear-attention sequence mixer), DeepSeek-style MoE, and multi-token prediction into one 24-layer stack. Uses a dual optimizer (NorMuon + CautiousAdamW), WSD + μP LR schedule, and logit softcap.
Bidirectional face age transformation (young ↔ old) using a conditional CycleGAN with AdaIN style normalization, multi-scale discriminator, and VGG perceptual loss. Per-layer age conditioning enables smooth interpolation between age groups.
β-VAE for 128×128 face generation on the CelebA dataset. Uses bilinear upsampling in the decoder, KL annealing schedule, and a 512-dimensional latent space. Reconstruction MSE reached 0.0152. Tracked with Comet ML.
Deep Convolutional GAN trained on CelebA to generate 64×64 face images from scratch using PyTorch. Standard DCGAN architecture with spectral normalization; discriminator loss converges to the ln 2 ≈ 0.693 Nash equilibrium.
Full encoder-decoder Transformer for English → Italian neural machine translation. BPE tokenizer trained from scratch on the opus_books corpus. Training loss reduced from 6.17 → 2.28 over 20 epochs on a P100 GPU.
Decoder-only transformer language model built entirely from scratch with PyTorch. Implements multi-head self-attention, causal masking, layer normalization, and positional embeddings. Trained on Tiny Shakespeare as a character-level LM — the cleanest possible foundation study of the GPT architecture.
Real-time skeleton-based action recognition with from-scratch HRNet-like pose estimation and two-stream ST-GCN. Targeting the NTU RGB+D 120 benchmark. Pose + ST-GCN pipeline ready; full benchmark training not yet started.
KV-cache math, low-rank compression algebra, the absorption-trick derivation, decoupled RoPE mechanics, and SDPA vs manual attention path trade-offs in DeepSeek-V2 / V3. The companion reference to DeepSeek-V3-Lite — written for reviewers who want to see the math worked end-to-end, not summarized.
The learned per-head sink bias, its BF16 numerical-stability story (clamped to [−10, 15] to prevent SDPA mask-add overflow), and how it interacts with the sliding-window / full-attention alternation to hit 2× KV-cache reduction at 128K context.
Derives the chunkwise SSD recurrence used in Mamba-3-Lite, proves its equivalence to the naive O(T) scan, and shows how the complex64 promotion (N=128 → N=64) doubles the per-state expressive capacity without paying for an explicit rotation matrix.
Heritage Institute of Technology, Kolkata
Self-taught ML during a B.Tech in Civil Engineering (2020–2024). In parallel with coursework I read the seminal papers (Attention Is All You Need, DDPM, MoE, DeepSeek-V3, GPT-OSS, Mamba-3) and implemented 14 architectures from scratch in PyTorch — culminating in a 422M DeepSeek-V3 reproduction with MLA + MoE + MTP, a 502M GPT-OSS-style long-context MoE with sliding/full attention alternation, a 404M Mamba-3 complex64 SSD, an 860M Stable Diffusion trained from-scratch on 2× RTX 5090 (best loss 0.0947), and a 15-phase autonomous ML research platform with 878 tests. Civil Engineering gave me a foundation in numerical methods, optimization, and linear algebra that maps cleanly onto modern ML.