Entropy-Guided Attention for Private LLMs
Abstract
Discussion of entropy-guided attention, attention-entropy dynamics, and the role of architectural regularization in making private LLM inference more practical.
Date
Jan 1, 2025
Location
Virtual Event
Fireside chat on entropy-guided attention mechanisms and efficient private LLM inference.
Nandan Kumar Jha
Ph.D., Electrical and Computer Engineering
I study nonlinear representation dynamics in large language models, focusing on how nonlinearities, architecture, and optimization jointly shape representational geometry, scaling behavior, and usable computational capacity.