Memcached is often called a simple key value cache but internally, cache invalidation is actually one of the most carefully engineered parts of the system.
It does NOT actively track every key and delete it when TTL expires.
Instead, it uses a lazy invalidation model where expiration is only checked when a key is accessed or when memory pressure forces cleanup.
When you set a key with TTL, Memcached does not start a timer.it simply stores an absolute expiration timestamp.so the data still physically stays in memory even after it is logically expired.no background timers. No per key deletion jobs because that would be too expensive at scale.
cache invalidation happens in a reactive way.when a GET request comes in, Memcached checks the item.If it is expired, it is treated as a cache miss and removed at that moment.So cleanup happens inside normal traffic flow, not separately.but but but question comes in mind what if they are not accessed again haha they just sit in memory and waste space and that would be a great problem
to solve this, Memcached also uses LRU eviction and background cleanup.LRU means Least Recently Used.So old and unused data gets removed first when memory is needed.
Memcached does not use normal memory allocation.It uses something called slab allocator.memory is split into size-based groups.small objects in one group, medium in another, large in another.Each group manages its own memory and eviction.
Inside each slab, there is an LRU list.Recently used items stay at the front.Old unused items move to the back.When memory is full, Memcached removes items from the back.
In early versions, this LRU system used a single global lock.So every read and write had to fight for the same structure.On multi-core systems, this created serious slowdowns. daamn man this is soooo soo expensive every read comes with tons of update that LRU queue man thats gets fired
To fix this, Memcached evolved.Instead of one LRU, it now uses multiple groups:
HOT, WARM, and COLD.
HOT = frequently used data
WARM = sometimes used data
COLD = rarely used data
This reduces contention and improves performance.
Memcached also added background LRU crawler threads.These threads slowly scan memory and clean expired or unused items.So cleanup happens in the background without blocking real requests.
Cache invalidation in Memcached is not one system.It is multiple systems working together: TTL, LRU, slab allocation, and background cleanup.Each one handles a different job.
TTL decides if data is still valid.LRU decides what to remove when memory is full.Slab decides where data is stored.Crawler handles slow cleanup in background.
Because of this design, a key can be expired but still exist in memory.Or still valid but already evicted.This feels weird at first… but this is what keeps the system fast.
Modern Memcached avoids heavy work on the request path.No strict deletion. No expensive tracking. No immediate cleanup everywhere.Everything is delayed, batched, and handled in background.
At the end of the day, Memcached is not about perfect cache invalidation.It is about speed, simplicity, and scaling under heavy load.It chooses “fast and good enough” over “perfect and slow”.
boom boom Moral: hey you yes you fellow you don't design perfect system from day one . you build something that works, then you observe how it fails under scale, and then you improve it step by step. The more i learn about these internals the more i love the journey of the things and all how they are built and improve over the time.
Note: To Build the simple systems in the software industry is really hard and takes a lot of time and effort to get it right. but the good thing is that once you get it right, it's really easy to maintain and scale and the people will love the simple things and the things that are easy to understand and maintain.
I love how Memcached is built, and how it has been maintained and scaled over time. I love how systems are built and how they are continuously improved over time.