cv
PhD Student, Computer Vision & Multimodal Learning — Michigan State University
General Information
| Full Name | Nitish Shukla |
| Position | PhD Student, Computer Science (Computer Vision & Multimodal Learning) |
| Affiliation | iPRoBe Lab, Michigan State University |
| shuklan3@msu.edu | |
| Languages | English, Hindi |
Research Overview
- My research lies at the intersection of representation learning and model reliability in deep learning, with a focus on how complex neural systems encode, reason over, and sometimes fail to disentangle fine-grained visual information in security-critical settings. I study how multimodal large language models (MLLMs) encode and reason over facial information, analyze failure modes in multi-image reasoning, and leverage internal representations for robust visual inference. A central theme of my work is generative face demorphing and biometric verification, where I develop diffusion-based inpainting models, benchmarks, and evaluation protocols that improve both face recognition accuracy and downstream morph attack detection.
Education
-
2024 - Present PhD in Computer Science
Michigan State University, MI, USA - Advised by Prof. Arun Ross in the iPRoBe Lab.
- Face demorphing, biometric security, and multimodal large language models.
-
2020 - 2022 MSc in Computer Science
Chennai Mathematical Institute (CMI), Chennai, India - Studied robustness properties of DNNs induced by prototypes with Prof. K. V. Subramanyam and Prof. Madhavan Mukund.
-
2017 - 2019 MSc in Mathematics
Indian Institute of Technology (IIT) Guwahati, India -
2014 - 2017 BSc in Physics, Mathematics and Computer Applications
Kanpur University, India
Experience
-
May 2025 - Aug 2025 PhD Research Intern
Adobe Research - Developed an attribute-aware style validation framework using zero-shot MLLM-induced embeddings for fine-grained artistic style analysis and retrieval.
- Proposed an attribute-based image decomposition strategy with FAISS-backed vector search that improved Recall@K by 2–3% over baseline representations. (Manuscript under review.)
-
Aug 2022 - Aug 2023 Data Scientist II
Micron Technology - Designed a synthetic data generation pipeline in Python and PyTorch to fine-tune a proprietary defect segmentation system for semiconductor die inspection.
- Synthetic augmentation improved defect shape mIoU by 9%, enabling robust detection of rare failure modes.
-
2019 - 2020 Research and Development Engineer
Next Education India Pvt. Ltd.
Honors and Awards
-
2025 - IAPR Best Biometrics Student Paper Award (BBSPA), IEEE International Joint Conference on Biometrics (IJCB) 2025.
Ongoing Research Projects
-
2026 Enhancing Single-Image Facial Demorphing using MLLMs
- Leveraged MLLM-derived semantic embeddings of morphs to condition a custom diffusion-based inpainting model, achieving 2–3% improvement in face recognition and biometric verification accuracy over baseline demorphing. (Under review, T-PAMI.)
-
2025 Style-Aware Open-Set Image Retrieval via MLLMs
- Introduced an open-set image retrieval framework leveraging zero-shot MLLM reasoning and attention to extract attribute-disentangled embeddings (brushstrokes, color palette, composition) via contrastive learning, achieving 2–3% mAP gains without supervised fine-tuning. (Under review.)
-
2026 Recursive Visual Pruning for Efficient Multi-Image Reasoning
- Designed an attention-guided recursive visual token pruning framework for vision-language models that reduces visual context via per-image budget allocation, enabling efficient multi-image inference without sacrificing reasoning accuracy. (In preparation.)
Selected Publications
-
2026 S2H-DPO: Hardness-Aware Preference Optimization for Vision–Language Models
- Nitish Shukla, Surgan Jandial, Arun Ross. Findings of the Association for Computational Linguistics (ACL).
-
2025 Facial Demorphing from a Single Morph Using a Latent Conditional GAN
- Nitish Shukla, Arun Ross. IEEE IJCB. IAPR Best Biometrics Student Paper Award.
-
2025 diffDeMorph: Extending Reference-Free Demorphing to Unseen Faces
- Nitish Shukla, Arun Ross. IEEE ICIP.
-
2025 dc-GAN: Dual-Conditioned GAN for Face Demorphing From a Single Morph
- Nitish Shukla, Arun Ross. IEEE FG.
-
2025 Metric for Evaluating Performance of Reference-Free Demorphing Methods
- Nitish Shukla, Arun Ross. IEEE/CVF WACV Workshops.
-
2024 Facial Demorphing via Identity Preserving Image Decomposition
- Nitish Shukla, Arun Ross. IEEE IJCB.
-
2023 Generating Adversarial Attacks in the Latent Space
- Nitish Shukla, Sudipta Banerjee. IEEE/CVF CVPR Workshops.
-
2023 SDeMorph: Towards Better Facial De-morphing from Single Morph
- Nitish Shukla. IEEE IJCB.
Technical Skills
-
Languages & Frameworks
- Python, PyTorch, OpenCV, HuggingFace Transformers, Scikit-learn
-
Computer Vision
- Face recognition, liveness detection, biometric verification, face demorphing, morph attack detection, image retrieval, inpainting
-
Deep Learning
- GANs, diffusion models, vision transformers (ViT), attention mechanisms, contrastive learning, few-shot learning, fine-tuning, RLHF, preference optimization
-
Multimodal Models
- CLIP, MLLMs, vision-language representation learning, zero-shot reasoning
-
Search & Retrieval
- FAISS, vector search, embedding-based retrieval
-
Experimentation
- MLflow, Pandas, large-scale benchmarking
-
Mathematics
- Bayesian methods, graph theory, linear algebra
References
- Arun Ross (Michigan State University) — rossarun@msu.edu
- K. V. Subramanyam (Chennai Mathematical Institute) — kv@cmi.ac.in
- Sudipta Banerjee (University of Wyoming) — sbanerj3@uwyo.edu