.. modify the LSTM memory structure, creating sLSTM with scalar memory and mLSTM with matrix memory, which integrate into xLSTM architectures.
Yesterday, I attended an xLSTM meetup in Munich, where Professor Sepp Hochreiter shared his latest research on “xLSTM: Much Faster Than Transformers”. Prof. Hochreiter’s talk was engaging and informative, providing a comprehensive overview of xLSTM’s architecture and its advantages over traditional transformer-based models. He demonstrated how xLSTM’s unique combination of scalar and matrix LSTMs, exponential gating mechanisms, and memory structures enables it to outperform state-of-the-art architectures on various tasks.
I also had the chance to talk with Sepp Hochreiter about Knowledge Graphs, a topic that has been gaining significant attention in the AI community.
Summary
Attending the xLSTM meetup was an enriching experience that deepened my understanding of this innovative architecture. I’m excited to see how xLSTM continues to evolve.
Example
I’ve tried the xLSTM-7b example on Huggingface “Hello xLSTM, how are you doing?” and got the response “Hello!”, and the question “Can you cook spaghetti with gasoline?” was answered with “Gasoline is flammable.”
Having a running example made it easy to run a Gradio instance with the xLSTM-7B model:
Any questions, comments, or ideas? Ping me on LinkedIn.
