AB

Hi, I'm

Atandra Bharati

Machine Learning Engineer

Building LLMs, diffusion models & generative AI systems from scratch — end-to-end, from architecture design to multi-GPU training.

Kolkata, India atandrabharati@gmail.com
View Projects GitHub Profile

About Me

I'm a Machine Learning Engineer with a deep focus on building AI systems from the ground up. While pursuing a B.Tech in Civil Engineering at Heritage Institute of Technology Kolkata (2020–2024), I independently transitioned into ML through rigorous self-directed study and project-based learning — going far beyond coursework to implement production-grade architectures.

My work spans the full spectrum of modern deep learning: transformer-based LLMs with MoE routing and FP8 Triton kernels, latent diffusion models with custom DDPM/DDIM schedulers, vision-language models, and generative adversarial networks. I have hands-on experience with multi-GPU distributed training (FSDP, DDP) on cloud hardware.

I thrive on understanding systems at the lowest level — reading papers, then building clean, documented implementations from scratch. Every project in my portfolio was written without relying on high-level framework abstractions.

8+ ML Projects
27L Param LLM Built
~860M Param Diffusion UNet
8× GPU Training Infra

Technical Skills

Languages

Python CUDA C++ Triton Bash

Frameworks & Libraries

PyTorch HuggingFace Transformers HuggingFace Datasets Safetensors Diffusers PIL / Pillow NumPy

ML Architectures

Transformers (GPT-style) Diffusion Models (DDPM/DDIM) GANs (DCGAN, CycleGAN) VAEs (β-VAE) Vision-Language Models Mixture-of-Experts (MoE) Multi-Head Latent Attention RoPE / YaRN GQA / MLA SwiGLU / GeGLU Multi-Token Prediction FP8 Quantization

Training & Infrastructure

FSDP DDP / DataParallel Multi-GPU (8× RTX 5090) RunPod bfloat16 / FP8 AMP torch.compile NCCL Gradient Checkpointing

Tools & Experiment Tracking

Weights & Biases Comet ML Git / GitHub GitHub Actions img2dataset

Post-Training Techniques

SFT (Supervised Fine-Tuning) GRPO (RL) R1-Style Distillation KL Divergence Penalty Speculative Decoding EMA

Projects

All architectures designed and implemented from scratch — no high-level wrappers.

🧠
Featured

DeepSeek-V3-Lite

Full reimplementation of the DeepSeek-V3 architecture from scratch — a 27-layer transformer with 1 dense block and 26 Mixture-of-Experts blocks (2B effective parameters, 64 routed + 2 shared experts, top-6 activated). Implements Multi-Head Latent Attention with KV-cache compression (10–20×), custom FP8 Triton kernels, Multi-Token Prediction, and a complete post-training pipeline.

  • MLA with decoupled RoPE (YaRN), absorption trick, KV cache compression 10–20×
  • FP8 E4M3FN training — custom Triton kernels for block-wise quant, GEMM, dequant
  • Full post-training: SFT, GRPO (group_size=8), R1 distillation, speculative decoding
  • FSDP distributed training across 8× RTX 5090 GPUs with safetensors checkpointing
PyTorch Transformers MoE MLA Triton FP8 GRPO FSDP
View on GitHub →
🎨
Featured

Stable Diffusion

Stable Diffusion built entirely from scratch — custom UNet (~860M parameters), DDPM and DDIM schedulers, frozen VAE (stabilityai/sd-vae-ft-mse) and CLIP text encoder, trained on the LAION-2B-en-aesthetic dataset with a full 5-stage data pipeline. Pre-encodes latents into RAM (~42 GB) for fast training.

  • Custom UNet with cross-attention, spatial transformers, ch_mults=(1,2,4,4), 8 attention heads
  • DDPM (scaled_linear, β=0.00085–0.012, T=1000) + DDIM (25–50 steps, CFG=7.5)
  • Dual-GPU VAE encoding with process isolation (CUDA_VISIBLE_DEVICES per worker)
  • bfloat16 + torch.compile, fused AdamW, EMA (decay=0.9999), DataParallel (2× RTX PRO 4500)
PyTorch Diffusion UNet DDPM/DDIM Multi-GPU W&B
View on GitHub →
👁️

Vision Language Model

PaliGemma-inspired VLM built from scratch — SigLIP vision encoder, GQA-based language decoder with RoPE positional encodings and GeGLU activations, connected via a trainable linear projector. Trained on COCO 2014 captioning on a P100 GPU.

  • SigLIP vision encoder with patch embedding and learned positional encodings
  • GQA language decoder with GeGLU FFN and RoPE, fully from scratch
  • Linear projector bridging vision and language spaces
PyTorch VLM GQA RoPE SigLIP
View on GitHub →
🔄

Face Aging CycleGAN

Bidirectional face age transformation (young ↔ old) using a conditional CycleGAN with AdaIN style normalization, multi-scale discriminator, and VGG perceptual loss. Trained on RTX 6000 Ada with Weights & Biases experiment tracking.

  • AdaIN (Adaptive Instance Normalization) for age-conditioned style transfer
  • Multi-scale discriminator for improved texture fidelity
  • VGG perceptual loss + cycle-consistency + identity loss
PyTorch CycleGAN AdaIN W&B
View on GitHub →
💬

GPT From Scratch

Decoder-only Transformer language model built entirely from scratch with PyTorch. Implements multi-head self-attention, causal masking, layer normalization, and positional embeddings. Trained on Tiny Shakespeare as a character-level LM.

  • Full decoder-only transformer with causal multi-head attention from scratch
  • Character-level tokenization on Tiny Shakespeare
  • Foundational study of the GPT architecture and training dynamics
PyTorch Transformer LLM Python
View on GitHub →
🌍

Neural Machine Translation

Full encoder-decoder Transformer for English → Italian neural machine translation. BPE tokenizer trained from scratch on the opus_books corpus. Training loss reduced from 6.17 → 2.28 over 20 epochs on a P100 GPU.

  • Complete encoder-decoder transformer with cross-attention, from scratch
  • Custom BPE tokenizer trained on opus_books corpus
  • Loss: 6.17 → 2.28 over 20 epochs (P100 GPU)
PyTorch Seq2Seq BPE NMT
View on GitHub →
🎭

Face Generation VAE

β-VAE for 128×128 face generation on the CelebA dataset. Uses bilinear upsampling in the decoder, KL annealing schedule, and a 512-dimensional latent space. Tracked with Comet ML.

  • β-VAE with 512-dim latent space and KL annealing for stable training
  • Bilinear upsampling decoder for sharp 128×128 reconstructions
  • Comet ML experiment tracking with reconstruction + latent visualizations
PyTorch VAE Generative AI Comet ML
View on GitHub →
🤖

DCGAN Face Generation

Deep Convolutional GAN trained on CelebA to generate 64×64 face images from scratch using PyTorch. Implements the standard DCGAN architecture with spectral normalization and Comet ML experiment tracking.

  • DCGAN with transposed convolution generator and strided conv discriminator
  • Trained on CelebA (200K+ images) for 64×64 face synthesis
  • Comet ML tracking with generated sample grids at each epoch
PyTorch GAN Generative AI Comet ML
View on GitHub →

Education

🎓

Bachelor of Technology (B.Tech) — Civil Engineering

Heritage Institute of Technology, Kolkata

2020 – 2024

Alongside formal coursework, I independently pursued machine learning through rigorous self-study — reading seminal papers (Attention Is All You Need, DDPM, MoE, DeepSeek-V3) and implementing full architectures from scratch. This project-driven approach reflects my commitment to deep technical understanding rather than surface-level API usage.