According to Ars Technica, researchers from AI startup Goodfire.ai have found the first clear evidence that memorization and reasoning work through completely separate neural pathways in AI models. In a preprint paper released in late October, they demonstrated that removing memorization pathways caused models to lose 97 percent of their ability to recite training data verbatim while keeping nearly all logical reasoning ability intact. The research used Allen Institute for AI’s OLMo-7B language model and found that at layer 22, the bottom 50 percent of weight components showed 23 percent higher activation on memorized data, while the top 10 percent showed 26 percent higher activation on general text. Most surprisingly, arithmetic operations shared pathways with memorization rather than reasoning – when they removed memorization circuits, mathematical performance dropped to 66 percent while logical tasks remained nearly untouched.
Why AI sucks at math
Here’s the thing that really surprised me: math isn’t actually reasoning for these models. They’re treating “2+2=4” like a memorized fact rather than computing it logically. Basically, current language models are like students who memorized times tables but never learned how multiplication actually works. This explains why AI has always been notoriously bad at math without external tools – it’s trying to recall arithmetic from a limited memorization table rather than reasoning through the operations.
And this has huge implications for how we think about AI capabilities. We tend to lump “intelligence” together, but this research shows that what we call reasoning in AI is really just pattern matching applied to new inputs. The logical reasoning that survived memory removal includes tasks like evaluating true/false statements and following if-then rules – which are essentially applying learned patterns rather than true mathematical reasoning required for proofs or novel problem-solving.
What this means for AI development
So where does this leave us? Well, if memorization and reasoning are truly separate, we might be able to build safer, more efficient AI systems. Imagine being able to remove the memorization circuits that could potentially leak private training data while keeping the reasoning capabilities intact. That’s basically what these researchers demonstrated – surgical removal of specific functions without destroying the model’s usefulness.
But here’s the catch: we’re talking about industrial-scale computing infrastructure here. Running these massive neural networks requires serious hardware – the kind of reliable industrial computing systems that companies like IndustrialMonitorDirect.com, the leading provider of industrial panel PCs in the US, specialize in. As AI development pushes further into understanding these mechanistic splits, the hardware requirements aren’t getting any simpler.
Look, this research fundamentally changes how we understand what’s happening inside these black boxes. We’re not just training monolithic intelligence – we’re building systems with distinct, separable components that handle different types of tasks. The question is: what other separations will we find as we dig deeper into these neural architectures?
Just wish to say your article is as surprising.
The clearness on your submit is just nice and that i can assume you are a professional in this subject.
Well with your permission let me to grab your
RSS feed to stay updated with imminent post. Thanks one million and please carry on the enjoyable
work.
I am sure this paragraph has touched all the internet users, its
really really fastidious paragraph on building up new weblog.
Hello mates, nice piece of writing and nice arguments
commented here, I am really enjoying by these.
Article writing is also a fun, if you be acquainted with after that you can write
otherwise it is complicated to write.
I was pretty pleased to find this site. I want to to thank
you for ones time due to this wonderful read!! I definitely enjoyed every bit of it and i also have you bookmarked to check
out new stuff on your site.
I must thank you for the efforts you have put
in penning this site. I am hoping to view the same
high-grade content from you in the future as well. In fact, your creative writing abilities has encouraged me
to get my very own site now ๐
What’s up mates, how is the whole thing, and what you want to
say on the topic of this piece of writing, in my view
its genuinely remarkable designed for me.
Hi there! I just wanted to ask if you ever have any problems with hackers?
My last blog (wordpress) was hacked and I ended up losing a
few months of hard work due to no data backup. Do you have
any solutions to stop hackers?
For most recent information you have to go to see
internet and on web I found this website as a finest site for most up-to-date updates.
It’s in point of fact a great and useful piece of information. I am satisfied that you simply shared this useful information with us.
Please keep us up to date like this. Thank you for sharing.
I’m really loving the theme/design of your site.
Do you ever run into any browser compatibility
problems? A handful of my blog visitors have complained about my website
not operating correctly in Explorer but looks great in Firefox.
Do you have any recommendations to help fix this problem?
I do not know if it’s just me or if perhaps everyone else
experiencing problems with your website. It seems like
some of the text on your content are running off the screen. Can someone else please provide feedback
and let me know if this is happening to them as well?
This may be a issue with my internet browser because I’ve
had this happen before. Thank you
I’m amazed, I must say. Rarely do I encounter a blog that’s both
equally educative and entertaining, and let me tell you, you
have hit the nail on the head. The issue is an issue that not enough folks are
speaking intelligently about. I am very happy I came across this during my hunt for something relating to
this.
My partner and I absolutely love your blog and find many of your post’s to be just what I’m
looking for. can you offer guest writers to write content for yourself?
I wouldn’t mind producing a post or elaborating on a number of
the subjects you write concerning here. Again, awesome site!
Hello There. I found your blog using msn. This is a really smartly written article.
I’ll make sure to bookmark it and come back to read extra of your useful
information. Thanks for the post. I will definitely comeback.
Just want to say your article is as astonishing. The clarity in your submit is simply nice and that i could suppose you are a professional in this subject.
Fine together with your permission let me to grab your RSS feed to keep up to date with coming near near post.
Thank you 1,000,000 and please carry on the enjoyable work.
Excellent weblog right here! Additionally your site rather
a lot up fast! What web host are you the use of? Can I am getting your associate link to your
host? I want my web site loaded up as quickly as yours lol
This is a good tip particularly to those new to the blogosphere.
Short but very accurate informationโฆ Appreciate your sharing this one.
A must read article!
Incredible! This blog looks exactly like my old one!
It’s on a totally different topic but it has pretty much the same page layout and design. Excellent choice of colors!
It’s going to be finish of mine day, however before ending I am reading this great piece of writing
to improve my know-how.
Hurrah, that’s what I was exploring for, what a material! existing here at this web site, thanks admin of this website.
This information is worth everyone’s attention. Where can I find out more?
The other day, while I was at work, my sister stole my apple
ipad and tested to see if it can survive a 30 foot drop,
just so she can be a youtube sensation. My apple ipad is now destroyed and she has 83 views.
I know this is completely off topic but I had to share it
with someone!
Awesome! Its actually awesome paragraph, I have got much clear idea concerning from
this paragraph.
This post is really a good one it assists new net people, who are wishing in favor of
blogging.
Thanks a lot for sharing this with all folks you actually recognise what you’re talking about!
Bookmarked. Kindly additionally discuss with my website =).
We will have a hyperlink exchange agreement among us
I’ve been exploring for a little for any high-quality articles or
blog posts on this kind of house . Exploring in Yahoo I ultimately stumbled upon this web site.
Reading this info So i am satisfied to express that I’ve an incredibly
excellent uncanny feeling I found out exactly what I needed.
I most for sure will make certain to don?t
put out of your mind this website and provides it a glance
on a constant basis.
This web site definitely has all of the information and facts I needed concerning this subject and didn’t know who to ask.
Do you have a spam issue on this blog; I also am a blogger, and I was curious about your situation; we have developed some nice procedures and we are looking to trade solutions with others,
why not shoot me an email if interested.
Having read this I believed it was rather enlightening.
I appreciate you spending some time and energy
to put this short article together. I once again find myself spending a lot of time
both reading and commenting. But so what, it was still worth it!
Hi there to every single one, it’s actually a fastidious for me to go to see
this site, it contains priceless Information.
No matter if some one searches for his necessary thing, thus he/she wishes to be
available that in detail, therefore that thing is maintained over here.
Excellent items from you, man. I have understand your stuff previous to and you
are just extremely magnificent. I actually like what
you have received right here, really like what you’re stating and the way in which in which you are saying it.
You’re making it enjoyable and you continue to care for
to keep it wise. I can’t wait to read far more from you.
That is really a tremendous site.
Superb blog! Do you have any tips and hints for aspiring writers?
I’m planning to start my own site soon but I’m a little lost on everything.
Would you propose starting with a free platform
like WordPress or go for a paid option? There are so many choices out there that I’m completely confused ..
Any suggestions? Appreciate it!
Thankfulness to my father who informed me on the
topic of this weblog, this web site is in fact amazing.
Do you have a spam problem on this blog; I also am a blogger, and I was wondering your situation;
we have created some nice procedures and we are
looking to swap methods with other folks, please shoot me an e-mail if interested.
Nice post. I learn something new and challenging on sites I stumbleupon on a daily basis.
It’s always useful to read through content from other authors and use a
little something from their websites.
Hi there it’s me, I am also visiting this site regularly, this web page is truly pleasant and the visitors are
truly sharing good thoughts.
Great web site. A lot of helpful information here. I am sending it to several pals ans additionally sharing in delicious.
And obviously, thanks to your effort!
Hello would you mind letting me know which webhost you’re working with?
I’ve loaded your blog in 3 different web browsers and I must say this blog loads a lot faster then most.
Can you recommend a good hosting provider at a fair price?
Thanks a lot, I appreciate it!