ML & GenAI Research Engineer.
I rebuild frontier AI systems from first principles in PyTorch — LLMs (HyMo, DeepSeek-V3, LLaMA-3, GPT-OSS, Mamba-3, MoE, MLA, MTP), latent diffusion (Stable Diffusion 1.x), multimodal (PaliGemma-style VLMs), and agentic research platforms. Paper-faithful, single-GPU-first, fully from scratch.
I'm an ML & GenAI research engineer who builds frontier AI systems from first principles — no high-level wrappers, no black boxes. Every architecture in my portfolio was implemented bottom-up, from attention math to distributed training pipelines.
My recent work spans a 434M-active / 1.13B-stored HyMo — the flagship hybrid LLM fusing Gated Delta Net (linear attention) with MLA (full attention) and Asymmetric MoE, pre-trained from scratch on 30B tokens with a custom Triton GDN kernel and FSDP-2; a 422M DeepSeek-V3 reproduction with MLA, aux-loss-free MoE, and multi-token prediction; a Stable Diffusion 1.x trained from scratch on 2× RTX 5090 (best loss 0.0947, epoch-42 checkpoint released); a GPT-OSS-style long-context MoE (502M / 247M active) with sliding-window/full attention alternation and learned attention sinks; a Mamba-3 complex64 SSD implementation with MIMO head mixing and zero causal convolution; a PaliGemma-style vision-language model trained end-to-end on a P100; and a 15-phase autonomous ML research platform that reads papers, plans experiments, writes patches, runs training, and iterates.
I care about the full stack — paper comprehension, architectural design, numerical stability, memory engineering, distributed training, and now agentic orchestration. See the GitHub profile README for the canonical list, or jump straight to Projects.
The flagship hybrid LLM — 434M active / 1.13B stored params fusing Gated Delta Net (linear attention) with Multi-Head Latent Attention (full attention) in a 3:1 ratio, plus Asymmetric MoE and Multi-Token Prediction. Pre-trained from scratch on 30B tokens with a hand-written Triton GDN kernel (chunked 1D selective scan) and FSDP-2. The only sanctioned kernel path is the custom Triton GDN — no fla dependency.
Faithful from-scratch reimplementation of DeepSeek-V3 — Multi-Head Latent Attention (MLA) with decoupled RoPE, aux-loss-free MoE routing, multi-token prediction (MTP), true MLA absorption at inference, and MTP-based speculative decoding. Architecture, training pipeline, microbench, and step-time benchmarks are complete and pass; the Chinchilla-optimal 422M / 8.4B-token run is designed for a single A100 80GB (training pending).
From-scratch reimplementation of OpenAI's GPT-OSS long-context MoE architecture. Twelve layers alternate sliding-window (window=128) and full attention; per-head learned attention-sink bias (clamped to [−10, 15] for BF16 SDPA stability); YaRN RoPE for 128K-context extrapolation; top-2-of-8 routed experts plus 1 shared; standard auxiliary load-balancing loss. Verifies a 2× KV-cache reduction at 128K (1.13 GB vs 2.25 GB pure GQA, BF16). 130 tests across 10 files cover SWA/full parity, sink bias, YaRN extrapolation, MoE routing, and gradient flow.
From-scratch reproduction of Mamba-3 at 404M params, written in pure PyTorch (no mamba-ssm, no custom CUDA, no causal conv). Promotes the state-space recurrence into the complex plane (N=64 complex64) — half the state dimension of Mamba-2 at parity capacity. Adds a fully-connected MIMO mixer across SSM heads and drops the causal conv block. Targeted at a Chinchilla-optimal 8.0B-token run on a single A100 80GB (12–15 h). The companion SSD.md walks through the chunkwise algorithm and its equivalence to the naive O(T) recurrence.
PaliGemma-inspired vision-language model built from scratch. SigLIP-style vision encoder with sinusoidal patch embeddings, GQA-based language decoder with RoPE and GeGLU, connected via a trainable linear projector. Zero pretrained weights — every component trained end-to-end on COCO 2014 captions on a single P100 GPU.
From-scratch real-time object detection: ResNet-50 + FPN backbone with a deformable set-prediction decoder (RT-DETR / DINO-style) — no anchors, no NMS. Targets ~42 mAP on COCO 2017, trained on 2× RTX 5090, with Gradio + ONNX serving.
A 15-phase multi-agent platform that turns an arXiv paper into evaluated experiments — paper analysis, repo analysis, experiment planning, code patches, training runs, statistical evaluation, autonomous looping, and end-to-end research reports. Provider-agnostic LLM layer with per-agent model routing across coding / reasoning / orchestration models.
Autonomous AI research-intelligence agent: daily multi-source web research across 15+ sources, LLM synthesis with self-critique, and provenance-tracked Markdown reports. 291 offline tests.
From-scratch LLaMA-3-style transformer engineered for single-GPU pretraining. Headline result: a 78% peak-memory reduction (92 GB → 20 GB) from gradient checkpointing, chunked cross-entropy (logits 50 GB → 0.3 GB), and disk-backed token caching (RAM 112 GB → 1 MB) — enabling 2× batch-size headroom on a single A100 80GB. Architecture complete; the 8.25B-token run is designed (training pending).
A Stable Diffusion 1.x-class latent diffusion model trained from scratch on 2× RTX 5090 (Blackwell). 860M-param U-Net, DDPM + DDIM schedulers, custom 7-phase curriculum on 1.3M+ images (LAION-Aesthetic → high-aesthetic LAION → DiffusionDB/JourneyDB → VGGFace2 → COCO → consolidation), DDP + BF16 + gradient checkpointing + EMA + Min-SNR. The 42-epoch checkpoint is publicly released on Hugging Face (atandra2000/sd-from-scratch-v1). Best loss 0.0947 (epoch 16); the 48-epoch curriculum continues.
Bidirectional face age transformation (young ↔ old) using a conditional CycleGAN with AdaIN style normalization, multi-scale discriminator, and VGG perceptual loss. Per-layer age conditioning enables smooth interpolation between age groups.
β-VAE for 128×128 face generation on the CelebA dataset. Uses bilinear upsampling in the decoder, KL annealing schedule, and a 512-dimensional latent space. Reconstruction MSE reached 0.0152. Tracked with Comet ML.
Deep Convolutional GAN trained on CelebA to generate 64×64 face images from scratch using PyTorch. Standard DCGAN architecture with spectral normalization; discriminator loss converges to the ln 2 ≈ 0.693 Nash equilibrium.
4× real-world photo super-resolution: a StableSR-style latent-diffusion U-Net with a mamba-ssm SSM refiner. Trained on DIV2K + Flickr2K + DF2K with Real-ESRGAN degradation; ~1.5s per image on an RTX 5090.
Full encoder-decoder Transformer for English → Italian neural machine translation. BPE tokenizer trained from scratch on the opus_books corpus. Training loss reduced from 6.17 → 2.28 over 20 epochs on a P100 GPU.
Decoder-only transformer language model built entirely from scratch with PyTorch. Implements multi-head self-attention, causal masking, layer normalization, and positional embeddings. Trained on Tiny Shakespeare as a character-level LM — the cleanest possible foundation study of the GPT architecture.
Real-time skeleton-based action recognition with from-scratch HRNet-like pose estimation and two-stream ST-GCN with CTR-GCN blocks. Architecture, training pipeline, inference stack (ONNX / TensorRT export, FastAPI serving), and 24 unit tests are implemented and passing; large-scale training on NTU RGB+D 120 has not yet started.
KV-cache math, low-rank compression algebra, the absorption-trick derivation, decoupled RoPE mechanics, and SDPA vs manual attention path trade-offs in DeepSeek-V2 / V3. The companion reference to DeepSeek-V3-Lite — written for reviewers who want to see the math worked end-to-end, not summarized.
The learned per-head sink bias, its BF16 numerical-stability story (clamped to [−10, 15] to prevent SDPA mask-add overflow), and how it interacts with the sliding-window / full-attention alternation to hit 2× KV-cache reduction at 128K context.
Derives the chunkwise SSD recurrence used in Mamba-3-Lite, proves its equivalence to the naive O(T) scan, and shows how the complex64 promotion (N=128 → N=64) doubles the per-state expressive capacity without paying for an explicit rotation matrix.
Heritage Institute of Technology, Kolkata
Self-taught ML during a B.Tech in Civil Engineering (2020–2024). In parallel with coursework I read the seminal papers (Attention Is All You Need, DDPM, MoE, DeepSeek-V3, GPT-OSS, Mamba-3) and implemented 17 architectures from scratch in PyTorch — culminating in a 434M-active / 1.13B-stored HyMo hybrid LLM (GDN + MLA + MoE + MTP), a 422M DeepSeek-V3 reproduction with MLA + MoE + MTP, a 502M GPT-OSS-style long-context MoE with sliding/full attention alternation, a 434M Mamba-3 complex64 SSD, an 860M Stable Diffusion trained from-scratch on 2× RTX 5090 (best loss 0.0947), and a 15-phase autonomous ML research platform with 878 tests. Civil Engineering gave me a foundation in numerical methods, optimization, and linear algebra that maps cleanly onto modern ML.