ML & GenAI Research Engineer.
I rebuild frontier AI systems from first principles in PyTorch — LLMs (HyMo, DeepSeek-V3, LLaMA-3, GPT-OSS, Mamba-3, MoE, MLA, MTP), latent diffusion (Stable Diffusion 1.x), multimodal (PaliGemma-style VLMs), and agentic research platforms. Paper-faithful, single-GPU-first, fully from scratch.
I'm an ML & GenAI research engineer who builds frontier AI systems from first principles — no high-level wrappers, no black boxes. Every architecture in my portfolio was implemented bottom-up, from attention math to distributed training pipelines.
My recent work spans a 434M-active / 1.13B-stored HyMo — the flagship hybrid LLM fusing Gated Delta Net (linear attention) with MLA (full attention) and Asymmetric MoE, pre-trained from scratch on 30B tokens with a custom Triton GDN kernel and FSDP-2; a ~412M DeepSeek-V3 reproduction with MLA, aux-loss-free MoE, and multi-token prediction; a Stable Diffusion 1.x trained from scratch on 2× RTX 5090 (best loss 0.0947, epoch-42 checkpoint released); a GPT-OSS-style long-context MoE (502M / 247M active) with sliding-window/full attention alternation and learned attention sinks; a Mamba-3 complex64 SSD implementation with MIMO head mixing and zero causal convolution; a PaliGemma-style vision-language model trained end-to-end on a P100; and a 15-phase autonomous ML research platform that reads papers, plans experiments, writes patches, runs training, and iterates.
I care about the full stack — paper comprehension, architectural design, numerical stability, memory engineering, distributed training, and now agentic orchestration. See the GitHub profile README for the canonical list, or jump straight to Projects.
The flagship hybrid LLM — 434M active / 1.13B stored params fusing Gated Delta Net (linear attention) with Multi-Head Latent Attention (full attention) in a 3:1 ratio, plus Asymmetric MoE and Multi-Token Prediction. Pre-trained from scratch on 30B tokens with a hand-written Triton GDN kernel (chunked 1D selective scan) and FSDP-2. The only sanctioned kernel path is the custom Triton GDN — no fla dependency.
Faithful from-scratch reimplementation of DeepSeek-V3 — Multi-Head Latent Attention (MLA) with decoupled RoPE, aux-loss-free MoE routing, multi-token prediction (MTP), true MLA absorption at inference, and MTP-based speculative decoding. Architecture, training pipeline, microbench, and step-time benchmarks are complete and pass; the Chinchilla-optimal ~412M / 8.4B-token run is designed for a single A100 80GB (training pending).
From-scratch reimplementation of OpenAI's GPT-OSS long-context MoE architecture. Twelve layers alternate sliding-window (window=128) and full attention; per-head learned attention-sink bias (clamped to [−10, 15] for BF16 SDPA stability); YaRN RoPE for 128K-context extrapolation; top-2-of-8 routed experts plus 1 shared; standard auxiliary load-balancing loss. Verifies a 2× KV-cache reduction at 128K (1.13 GB vs 2.25 GB pure GQA, BF16). The CPU suite runs 205 passed / 2 skipped across 12 files, covering SWA/full parity, sink bias, YaRN extrapolation, MoE routing, and gradient flow.
From-scratch reproduction of Mamba-3 at 404M params, written in pure PyTorch (no mamba-ssm, no custom CUDA, no causal conv). Promotes the state-space recurrence into the complex plane (N=64 complex64) — half the state dimension of Mamba-2 at parity capacity. Adds a fully-connected MIMO mixer across SSM heads and drops the causal conv block. Targeted at a Chinchilla-optimal 8.0B-token run on a single A100 80GB (12–15 h). The companion SSD.md walks through the chunkwise algorithm and its equivalence to the naive O(T) recurrence.
Learned sparse chunk attention (Tencent HiLS) reproduced from scratch: a ~341M-param GQA transformer where every block decides which chunks to read. Landmark-guided routing picks chunks under a causal mask, and a score-fused kernel combines the selected chunks so the whole path trains by the LM loss alone. 68 CPU tests gate every change; the A100 pretraining run has not started.
Block-autoregressive discrete diffusion language model in a Gemma-class stack: a 256-token canvas is denoised in parallel with block-causal attention, and entropy-bounded adaptive stopping commits partial answers while the rest re-noises. 343,516,160 params exactly, 8.0B-token Chinchilla budget on one A100. Trainer and eval scripts are CPU-tested; the 8B run is scripted but not started.
Replica of DeepSeek-V4.1-Flash — CED, the CSA2 hierarchical sparse indexer, sliding-window attention, mHC, Engram memory, MoE, DSpark decode, and a ViT front-end — sized at ~1.16B total / ~205M active for 4× A100 80GB under $500. Phases 0–2 are green and the trainer, recovery path and variant controls are CPU-verified at toy dimensions; the 200-step bitwise repeat waits on a pod.
The Gemma E2B text stack rebuilt from scratch and reduced to a single-A100 size: per-layer embeddings (PLE), local/global attention alternation, proportional partial RoPE, and cross-layer KV reuse across 20 layers at 768 hidden. A Lite adaptation rather than a checkpoint-compatible reproduction. 348,965,184 params exactly; 111 CPU tests including parity against tiny upstream-oracle models. No training run yet.
Hybrid pretraining framework fusing MLA, Gated Delta Net, DeepSeek MoE, and multi-token prediction at 415M active params over an 8.31B-token run. The direct predecessor of HyMo's architecture family, archived once HyMo superseded it.
PaliGemma-inspired vision-language model built from scratch. SigLIP-style vision encoder with sinusoidal patch embeddings, GQA-based language decoder with RoPE and GeGLU, connected via a trainable linear projector. Zero pretrained weights — every component trained end-to-end on COCO 2014 captions on a single P100 GPU.
From-scratch real-time object detection: ResNet-50 + FPN backbone with a deformable set-prediction decoder (RT-DETR / DINO-style) — no anchors, no NMS. Targets ~42 mAP on COCO 2017, trained on 2× RTX 5090, with Gradio + ONNX serving.
A 15-phase multi-agent platform that turns an arXiv paper into evaluated experiments — paper analysis, repo analysis, experiment planning, code patches, training runs, statistical evaluation, autonomous looping, and end-to-end research reports. Provider-agnostic LLM layer with per-agent model routing across coding / reasoning / orchestration models.
Autonomous AI research-intelligence agent: daily multi-source web research across 15+ sources, LLM synthesis with self-critique, and provenance-tracked Markdown reports. 291 offline tests.
Sovereign, air-gapped cybersecurity MoE at 4.55B total / 1.34B active: an auditable multi-turn tool harness (web probe, rate-capped network scan, binary analysis, DFIR triage, attack-graph state), hybrid sparse-dense RAG with a canary shield, and SFT / DPO / GRPO post-training over a 52-task RL gym. 227 CPU tests pass; datasets, checkpoints and the production embedder remain pod-side.
From-scratch LLaMA-3-style transformer engineered for single-GPU pretraining. Headline result: a 78% peak-memory reduction (92 GB → 20 GB) from gradient checkpointing, chunked cross-entropy (logits 50 GB → 0.3 GB), and disk-backed token caching (RAM 112 GB → 1 MB) — enabling 2× batch-size headroom on a single A100 80GB. Architecture complete; the 8.25B-token run is designed (training pending).
A Stable Diffusion 1.x-class latent diffusion model trained from scratch on 2× RTX 5090 (Blackwell). 860M-param U-Net, DDPM + DDIM schedulers, custom 7-phase curriculum on 1.3M+ images (LAION-Aesthetic → high-aesthetic LAION → DiffusionDB/JourneyDB → VGGFace2 → COCO → consolidation), DDP + BF16 + gradient checkpointing + EMA + Min-SNR. The 42-epoch checkpoint is publicly released on Hugging Face (atandra2000/sd-from-scratch-v1). Best loss 0.0947 (epoch 16); the 48-epoch curriculum continues.
Bidirectional face age transformation (young ↔ old) using a conditional CycleGAN with AdaIN style normalization, multi-scale discriminator, and VGG perceptual loss. Per-layer age conditioning enables smooth interpolation between age groups.
β-VAE for 128×128 face generation on the CelebA dataset. Uses bilinear upsampling in the decoder, KL annealing schedule, and a 512-dimensional latent space. Reconstruction MSE reached 0.0152. Tracked with Comet ML.
Deep Convolutional GAN trained on CelebA to generate 64×64 face images from scratch using PyTorch. Standard DCGAN architecture with spectral normalization; discriminator loss converges to the ln 2 ≈ 0.693 Nash equilibrium.
4× real-world photo super-resolution: a StableSR-style latent-diffusion U-Net with a mamba-ssm SSM refiner. Trained on DIV2K + Flickr2K + DF2K with Real-ESRGAN degradation; ~1.5s per image on an RTX 5090.
Sketch plus text in, five facial-composite hypotheses out: a from-scratch latent diffusion model that generates candidates, ranks them, refines the chosen one, and keeps an auditable decision ledger. Stage A is code-complete on CPU for $0 across four CPU-dry-runnable trainers, with a $100 RunPod budget reserved for training alone. Untrained by design so far — the deliverable is the working loop plus scaling curves.
Full encoder-decoder Transformer for English → Italian neural machine translation. BPE tokenizer trained from scratch on the opus_books corpus. Training loss reduced from 6.17 → 2.28 over 20 epochs on a P100 GPU.
Decoder-only transformer language model built entirely from scratch with PyTorch. Implements multi-head self-attention, causal masking, layer normalization, and positional embeddings. Trained on Tiny Shakespeare as a character-level LM — the cleanest possible foundation study of the GPT architecture.
Real-time skeleton-based action recognition with from-scratch HRNet-like pose estimation and two-stream ST-GCN with CTR-GCN blocks. Architecture, training pipeline, inference stack (ONNX / TensorRT export, FastAPI serving), and 24 unit tests are implemented and passing; large-scale training on NTU RGB+D 120 has not yet started.
KV-cache math, low-rank compression algebra, the absorption-trick derivation, decoupled RoPE mechanics, and SDPA vs manual attention path trade-offs in DeepSeek-V2 / V3. The companion reference to DeepSeek-V3-Lite — written for reviewers who want to see the math worked end-to-end, not summarized.
The learned per-head sink bias, its BF16 numerical-stability story (clamped to [−10, 15] to prevent SDPA mask-add overflow), and how it interacts with the sliding-window / full-attention alternation to hit 2× KV-cache reduction at 128K context.
Derives the chunkwise SSD recurrence used in Mamba-3-Lite, proves its equivalence to the naive O(T) scan, and shows how the complex64 promotion (N=128 → N=64) doubles the per-state expressive capacity without paying for an explicit rotation matrix.
Heritage Institute of Technology, Kolkata
Self-taught ML during a B.Tech in Civil Engineering (2020–2024). In parallel with coursework I read the seminal papers (Attention Is All You Need, DDPM, MoE, DeepSeek-V3, GPT-OSS, Mamba-3) and implemented 24 architectures from scratch in PyTorch — culminating in a 434M-active / 1.13B-stored HyMo hybrid LLM (GDN + MLA + MoE + MTP), a ~412M DeepSeek-V3 reproduction with MLA + MoE + MTP, a 502M GPT-OSS-style long-context MoE with sliding/full attention alternation, a 434M Mamba-3 complex64 SSD, an 860M Stable Diffusion trained from-scratch on 2× RTX 5090 (best loss 0.0947), and a 15-phase autonomous ML research platform with 20 agents and 61 typed tools. Civil Engineering gave me a foundation in numerical methods, optimization, and linear algebra that maps cleanly onto modern ML.