drop collection or database does not free up disk space

[ysannezzz]

We have large MongoDB database (about 400GB data file each day), MongoDB 3.2.6/3.2.7, engine rocksDB, operating system redhat 6.3.
After dropping the collection or database, the disk space used on the machine was not reclaimed.
After rming sst files which i think should by dropped by rocksdb,mongodb
can still work fine.
By the way,show dbs is not corresponding to du in linux.
Best Regards

[igorcanadi]

Hey @ysannezzz, that's a pretty big database, nice. :)

The fix for this is to return true in CompactionFilter::IgnoreSnapshots() function here: https://github.com/facebook/rocksdb/blob/master/include/rocksdb/compaction_filter.h#L107-L115. The comments do a good job of describing this.

I was waiting until couple of RocksDB release prove this function works in production. I think enough time has passed that we can implement this. I'll try to get to this soon.

[ysannezzz]

Hey @igorcanadi ,thank you for this answer.
I have changed this function to return true and recompiled mongorocks.
But drop command still doesn't free up disk space.

[igorcanadi]

It's not immediate. You have to wait for compactions to happen as you keep writing more data into the database.

[ysannezzz]

How long should i wait for compaction to happen?
I find some sst files' mtime never change even if i keep writing more data into the database.

[wolfkdy]

After dropping a database. I found the actual physical size is not reduced.
I digged into the code and found that rocksdb::experimental::SuggestCompactRange is called in RocksEngine::dropIdent.
as @igorcanadi said,

It's not immediate. You have to wait for compactions to happen as you keep writing more data into the database.

Perhaps we should put each ident(index/datarecordstore) in different column family.
After dropping a table, the related column family can be removed, which is a procedure of file-deleting.