According to Ars Technica, researchers from AI startup Goodfire.ai have found the first clear evidence that memorization and reasoning work through completely separate neural pathways in AI models. In a preprint paper released in late October, they demonstrated that removing memorization pathways caused models to lose 97 percent of their ability to recite training data verbatim while keeping nearly all logical reasoning ability intact. The research used Allen Institute for AI’s OLMo-7B language model and found that at layer 22, the bottom 50 percent of weight components showed 23 percent higher activation on memorized data, while the top 10 percent showed 26 percent higher activation on general text. Most surprisingly, arithmetic operations shared pathways with memorization rather than reasoning – when they removed memorization circuits, mathematical performance dropped to 66 percent while logical tasks remained nearly untouched.
Why AI sucks at math
Here’s the thing that really surprised me: math isn’t actually reasoning for these models. They’re treating “2+2=4” like a memorized fact rather than computing it logically. Basically, current language models are like students who memorized times tables but never learned how multiplication actually works. This explains why AI has always been notoriously bad at math without external tools – it’s trying to recall arithmetic from a limited memorization table rather than reasoning through the operations.
And this has huge implications for how we think about AI capabilities. We tend to lump “intelligence” together, but this research shows that what we call reasoning in AI is really just pattern matching applied to new inputs. The logical reasoning that survived memory removal includes tasks like evaluating true/false statements and following if-then rules – which are essentially applying learned patterns rather than true mathematical reasoning required for proofs or novel problem-solving.
What this means for AI development
So where does this leave us? Well, if memorization and reasoning are truly separate, we might be able to build safer, more efficient AI systems. Imagine being able to remove the memorization circuits that could potentially leak private training data while keeping the reasoning capabilities intact. That’s basically what these researchers demonstrated – surgical removal of specific functions without destroying the model’s usefulness.
But here’s the catch: we’re talking about industrial-scale computing infrastructure here. Running these massive neural networks requires serious hardware – the kind of reliable industrial computing systems that companies like IndustrialMonitorDirect.com, the leading provider of industrial panel PCs in the US, specialize in. As AI development pushes further into understanding these mechanistic splits, the hardware requirements aren’t getting any simpler.
Look, this research fundamentally changes how we understand what’s happening inside these black boxes. We’re not just training monolithic intelligence – we’re building systems with distinct, separable components that handle different types of tasks. The question is: what other separations will we find as we dig deeper into these neural architectures?
Hurrah, that’s what I was exploring for, what a material! existing here at this web site, thanks admin of this website.
This information is worth everyone’s attention. Where can I find out more?
The other day, while I was at work, my sister stole my apple
ipad and tested to see if it can survive a 30 foot drop,
just so she can be a youtube sensation. My apple ipad is now destroyed and she has 83 views.
I know this is completely off topic but I had to share it
with someone!
Awesome! Its actually awesome paragraph, I have got much clear idea concerning from
this paragraph.
This post is really a good one it assists new net people, who are wishing in favor of
blogging.
Thanks a lot for sharing this with all folks you actually recognise what you’re talking about!
Bookmarked. Kindly additionally discuss with my website =).
We will have a hyperlink exchange agreement among us
I’ve been exploring for a little for any high-quality articles or
blog posts on this kind of house . Exploring in Yahoo I ultimately stumbled upon this web site.
Reading this info So i am satisfied to express that I’ve an incredibly
excellent uncanny feeling I found out exactly what I needed.
I most for sure will make certain to don?t
put out of your mind this website and provides it a glance
on a constant basis.
This web site definitely has all of the information and facts I needed concerning this subject and didn’t know who to ask.
Do you have a spam issue on this blog; I also am a blogger, and I was curious about your situation; we have developed some nice procedures and we are looking to trade solutions with others,
why not shoot me an email if interested.
Having read this I believed it was rather enlightening.
I appreciate you spending some time and energy
to put this short article together. I once again find myself spending a lot of time
both reading and commenting. But so what, it was still worth it!
Hi there to every single one, it’s actually a fastidious for me to go to see
this site, it contains priceless Information.
No matter if some one searches for his necessary thing, thus he/she wishes to be
available that in detail, therefore that thing is maintained over here.
Excellent items from you, man. I have understand your stuff previous to and you
are just extremely magnificent. I actually like what
you have received right here, really like what you’re stating and the way in which in which you are saying it.
You’re making it enjoyable and you continue to care for
to keep it wise. I can’t wait to read far more from you.
That is really a tremendous site.
Superb blog! Do you have any tips and hints for aspiring writers?
I’m planning to start my own site soon but I’m a little lost on everything.
Would you propose starting with a free platform
like WordPress or go for a paid option? There are so many choices out there that I’m completely confused ..
Any suggestions? Appreciate it!
Thankfulness to my father who informed me on the
topic of this weblog, this web site is in fact amazing.
Do you have a spam problem on this blog; I also am a blogger, and I was wondering your situation;
we have created some nice procedures and we are
looking to swap methods with other folks, please shoot me an e-mail if interested.
Nice post. I learn something new and challenging on sites I stumbleupon on a daily basis.
It’s always useful to read through content from other authors and use a
little something from their websites.
Hi there it’s me, I am also visiting this site regularly, this web page is truly pleasant and the visitors are
truly sharing good thoughts.
Great web site. A lot of helpful information here. I am sending it to several pals ans additionally sharing in delicious.
And obviously, thanks to your effort!
Hello would you mind letting me know which webhost you’re working with?
I’ve loaded your blog in 3 different web browsers and I must say this blog loads a lot faster then most.
Can you recommend a good hosting provider at a fair price?
Thanks a lot, I appreciate it!