Question about the Operation Ratio in cachebench

[Mi5sssss]

Hello, I am a bit puzzled about the operation ratio outlined in the configuration file. Does this suggest that cachebench would execute both 'get' and 'set' operations concurrently at the stated ratio? Is there a possibility to load the key-value pairs first in cachelib and then run the 'get/set' (update) tests? It seems to me that if we run the operations simultaneously, the Hit Ratio may not be accurate. Have I overlooked any potential configurations that might be affecting this?

`===JSON Config===
{
"cache_config": {
"cacheSizeMB": 2048,
"poolRebalanceIntervalSec": 1,
"moveOnSlabRelease": false,
"numPools": 1,
"poolSizes": [
1
]
},
"test_config": {
"numOps": 5000000,
"numThreads": 32,
"numKeys": 100000000,
"keySizeRange": [
8,
9
],
"keySizeRangeProbability": [
1
],
"valSizeRange": [
256,
257
],
"valSizeRangeProbability": [
1,
1
],
"getRatio": 0.9,
"setRatio": 0.1,
"delRatio": 0,
"loneGetRatio": 0,
"keyPoolDistribution": [
1
],
"opPoolDistribution": [
1
]
}
}

Welcome to OSS version of cachebench
Created 99,976,016 keys in 1.30 mins
Generating 160.00M sampled accesses
Generated access patterns in 0.03 mins
Total 160.00M ops to be run
14:52:55 0.00M ops completed. Hit Ratio 0.00% (RAM 0.00%, NVM 0.00%)
== Test Results ==
== Allocator Stats ==
Items in RAM : 6,355,136
Items in NVM : 0
Alloc Attempts: 7,459,750 Success: 100.00%
Evict Attempts: 439,372 Success: -4198434145486952.50%
RAM Evictions : 618,369
Fraction of pool 0 used : 0.97
Cache Gets : 67,170,125
Hit Ratio : 6.51%
NVM Gets : 0, Coalesced : 0.00%
NVM Puts : 0, Success : 100.00%, Clean : 0.00%, AbortsFromDel : 0, AbortsFromGet : 0
NVM Evicts : 0, Clean : 0.00%, Unclean : 0, Double : 0
NVM Deletes : 0 Skipped Deletes: 0.00%
Released 15 slabs
Moves : attempts: 0, success: 0.00%
Evictions : attempts: 178,997, success: 100.00%
Total eviction executed 618369

== Throughput for ==
Total Ops : 74.63 million
Total sets: 7,459,750
get : 1,491,843/s, success : 6.51%
couldExist: 0/s, success : 0.00%
set : 165,680/s, success : 100.00%
del : 0/s, found : 0.00%`

[Mi5sssss]

Also, here is an uncommon number here:

Evict Attempts: 439,372 Success: -4198434145486952.50%

[jaesoo-fb]

Hi @Mi5sssss

We don't support the feature for warming up the cache. However, there are two relevant features that I can think of

• (1) use the cache persist feature to restore the cache from the cache state stored at the latest shutdown https://github.com/facebook/CacheLib/blob/main/cachelib/cachebench/util/CacheConfig.h#L70
• (2) enable the lookaside feature https://github.com/facebook/CacheLib/blob/main/cachelib/cachebench/util/Config.h#L184 : this will make cachebench emulate the lookaside cache, where cache sets the value for every missing key encountered

For weird success rate number, this is caused by the fact that we are not taking the stat numbers used for the computation here atomically. Let me fix this soon, but I think you can just interpret it as 100%.

[jaesoo-fb]

Could you clarify what you mean by the look-aside buffer?

I meant the lookaside cache (ref. https://tanzu.vmware.com/content/blog/an-introduction-to-look-aside-vs-inline-caching-patterns) and the option allows the cachebench to set the value on get miss so that later get would hit (See https://github.com/OpenMPDK/CacheLib/blob/fdp/blockcache_waf_reduction/cachelib/cachebench/runner/CacheStressor.h#L363)

[Mi5sssss]

Got it. Yes, that is what I mean for the lookaside cache. Can we customize the size of the lookaside cache? Thanks.

[Mi5sssss]

Here in this example:
== Allocator Stats ==
Items in RAM : 10,551,296
Items in NVM : 0
Alloc Attempts: 126,211,181 Success: 100.00%
Evict Attempts: 97,178,655 Success: -18982300252657.22%
RAM Evictions : 115,659,807
Fraction of pool 0 used : 0.63
Cache Gets : 160,000,000
Hit Ratio : 21.12%
NVM Gets : 0, Coalesced : 0.00%
NVM Puts : 0, Success : 100.00%, Clean : 0.00%, AbortsFromDel : 0, AbortsFromGet : 0
NVM Evicts : 0, Clean : 0.00%, Unclean : 0, Double : 0
NVM Deletes : 0 Skipped Deletes: 0.00%
Released 4,101 slabs
Moves : attempts: 0, success: 0.00%
Evictions : attempts: 18,481,152, success: 100.00%
Total eviction executed 115659807

== Throughput for ==
Total Ops : 160.00 million
Total sets: 126,211,181
get : 32,359/s, success : 21.12%
couldExist: 0/s, success : 0.00%
set : 25,526/s, success : 100.00%
del : 0/s, found : 0.00%

I only set the getRatio to 1, and other OPs to be 0. How could cachebench get the number of Alloc Attempts (126M) in this case? Thank you very much.

[jaesoo-fb]

As I mentioned, if you enable the lookaside, every miss will generate the sets; <# of lookaside sets> = <# of gets> x <miss ratio> = 160M x (1 - 0.2112) = 126M

[Mi5sssss]

Thank you for your assistance so far, but I'm still a bit puzzled about one aspect. Considering that we only have a getRatio of 1 and there are no other operating ratio involved, wouldn't it be logical to expect the miss ratio to be 100%? Could you please help me understand why this isn't the case?