Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

colblk: implement HideObsoletePoints in DataBlockIter #3982

Merged
merged 1 commit into from
Oct 1, 2024

Conversation

RaduBerinde
Copy link
Member

colblk: minor benchmark refactoring

Adding a "short" version for the iterator benchmarks. We will add
combinations of various transforms with these selected configs.

We also slightly improve the benchmark naming.

colblk: implement HideObsoletePoints in DataBlockIter

Benchmark for the no-transform path before/after:

name                                                                                old time/op    new time/op    delta
CockroachDataBlockIterShort/AlphaLen=8,Prefix=8,Shared=4,ValueLen=8/Next-10           12.2ns ± 0%    12.5ns ± 3%  +2.38%  (p=0.008 n=5+5)
CockroachDataBlockIterShort/AlphaLen=8,Prefix=8,Shared=4,ValueLen=8/SeekGE-10          113ns ± 0%     114ns ± 0%  +0.80%  (p=0.008 n=5+5)
CockroachDataBlockIterShort/AlphaLen=8,Prefix=128,Shared=64,ValueLen=128/Next-10      10.1ns ± 0%    10.3ns ± 0%  +2.30%  (p=0.016 n=5+4)
CockroachDataBlockIterShort/AlphaLen=8,Prefix=128,Shared=64,ValueLen=128/SeekGE-10    89.4ns ± 0%    90.3ns ± 0%  +1.08%  (p=0.008 n=5+5)

Transforms benchmarks:

name                                                                                                  time/op
CockroachDataBlockIterTransforms/AlphaLen=8,Prefix=8,Shared=4,ValueLen=8/Next-10                      12.3ns ± 1%
CockroachDataBlockIterTransforms/AlphaLen=8,Prefix=8,Shared=4,ValueLen=8/SeekGE-10                     113ns ± 0%
CockroachDataBlockIterTransforms/AlphaLen=8,Prefix=8,Shared=4,ValueLen=8,SynthSeqNum/Next-10          12.5ns ± 0%
CockroachDataBlockIterTransforms/AlphaLen=8,Prefix=8,Shared=4,ValueLen=8,SynthSeqNum/SeekGE-10         114ns ± 1%
CockroachDataBlockIterTransforms/AlphaLen=8,Prefix=8,Shared=4,ValueLen=8,HideObsolete/Next-10         13.0ns ± 1%
CockroachDataBlockIterTransforms/AlphaLen=8,Prefix=8,Shared=4,ValueLen=8,HideObsolete/SeekGE-10        117ns ± 1%
CockroachDataBlockIterTransforms/AlphaLen=8,Prefix=128,Shared=64,ValueLen=128/Next-10                 10.3ns ± 0%
CockroachDataBlockIterTransforms/AlphaLen=8,Prefix=128,Shared=64,ValueLen=128/SeekGE-10               90.5ns ± 1%
CockroachDataBlockIterTransforms/AlphaLen=8,Prefix=128,Shared=64,ValueLen=128,SynthSeqNum/Next-10     10.5ns ± 0%
CockroachDataBlockIterTransforms/AlphaLen=8,Prefix=128,Shared=64,ValueLen=128,SynthSeqNum/SeekGE-10   90.9ns ± 1%
CockroachDataBlockIterTransforms/AlphaLen=8,Prefix=128,Shared=64,ValueLen=128,HideObsolete/Next-10    10.8ns ± 0%
CockroachDataBlockIterTransforms/AlphaLen=8,Prefix=128,Shared=64,ValueLen=128,HideObsolete/SeekGE-10  93.9ns ± 0%

@RaduBerinde RaduBerinde requested a review from a team as a code owner September 28, 2024 22:43
@cockroach-teamcity
Copy link
Member

This change is Reviewable

Copy link
Collaborator

@jbowens jbowens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 2 of 5 files at r1, all commit messages.
Reviewable status: 2 of 5 files reviewed, 3 unresolved discussions (waiting on @RaduBerinde)


sstable/colblk/data_block.go line 805 at r1 (raw file):

			i.nextObsoletePoint = i.r.isObsolete.SeekSetBitGE(i.row)
		} else {
			i.nextObsoletePoint = i.maxRow + 1

if i.row == i.maxRow, isn't it possible for i.r.isObsolete.At(i.row) == true, in which case we want to set i.nextObsoletePoint = i.row?


sstable/colblk/data_block.go line 806 at r1 (raw file):

		} else {
			i.nextObsoletePoint = i.maxRow + 1
		}

would it make things cleaner if Seek[Unset|Set]BitGE(n) where n >= BitmapBuilder.bitCount was defined as BitmapBuilder.bitCount? so that even if i.row == i.maxRow+1, we can blindly call Seek[Unset|Set]BitGE?


sstable/colblk/data_block.go line 963 at r1 (raw file):

//gcassert:inline
func (i *DataBlockIter) atObsoletePointForward() bool {
	return i.row == i.nextObsoletePoint && i.row <= i.maxRow

is there an invariant that if i.row != i.nextObsoletePoint, i.row < i.nextObsoletePoint? should we assert as much?

Copy link
Member Author

@RaduBerinde RaduBerinde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TFTR!

Reviewable status: 2 of 5 files reviewed, 3 unresolved discussions (waiting on @jbowens)


sstable/colblk/data_block.go line 805 at r1 (raw file):

Previously, jbowens (Jackson Owens) wrote…

if i.row == i.maxRow, isn't it possible for i.r.isObsolete.At(i.row) == true, in which case we want to set i.nextObsoletePoint = i.row?

Nice catch, added a test for this case.


sstable/colblk/data_block.go line 806 at r1 (raw file):

Previously, jbowens (Jackson Owens) wrote…

would it make things cleaner if Seek[Unset|Set]BitGE(n) where n >= BitmapBuilder.bitCount was defined as BitmapBuilder.bitCount? so that even if i.row == i.maxRow+1, we can blindly call Seek[Unset|Set]BitGE?

I was afraid it might cause other paths (like prefixChanged.SeekSetBitGE() to regress. I did it in a local method though, much better.


sstable/colblk/data_block.go line 963 at r1 (raw file):

Previously, jbowens (Jackson Owens) wrote…

is there an invariant that if i.row != i.nextObsoletePoint, i.row < i.nextObsoletePoint? should we assert as much?

Yes, added an assert here (I was afraid it would mess with the gcassert:inline check, but then I realized that the build for the check doesn't use the invariants tag).

Copy link
Collaborator

@jbowens jbowens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 2 of 2 files at r2, all commit messages.
Reviewable status: 4 of 5 files reviewed, all discussions resolved

@jbowens
Copy link
Collaborator

jbowens commented Sep 30, 2024

--- FAIL: TestLint (42.48s)
    --- FAIL: TestLint/TestGCAssert (10.61s)
        lint_test.go:129: See /tmp/gcassert-1156708372.log for full output.
        lint_test.go:132: 
            sstable/colblk/data_block.go:802:	i.setNextObsoletePoint(): call was not inlined
        lint_test.go:132: 
            sstable/colblk/data_block.go:831:	i.setNextObsoletePoint(): call was not inlined
        lint_test.go:132: 
            sstable/colblk/data_block.go:846:	i.setNextObsoletePoint(): call was not inlined
        lint_test.go:132: 
            sstable/colblk/data_block.go:931:	i.setNextObsoletePoint(): call was not inlined
        lint_test.go:132: 
            sstable/colblk/data_block.go:979:	i.setNextObsoletePoint(): call was not inlined

:'(

@RaduBerinde
Copy link
Member Author

So strange, SeekBitGE is not inlined, so how can just an if and a function call be too much to inline? Full logs show setNextObsoletePoint: function too complex: cost 81 exceeds budget 80

@RaduBerinde
Copy link
Member Author

Ok, moved the check to SeekBitGE and verified that it didn't affect the normal benchmarks:

name                                                                                old time/op    new time/op    delta
CockroachDataBlockIterShort/AlphaLen=8,Prefix=8,Shared=4,ValueLen=8/Next-10           12.1ns ± 0%    11.9ns ± 1%  -1.55%  (p=0.000 n=8+8)
CockroachDataBlockIterShort/AlphaLen=8,Prefix=8,Shared=4,ValueLen=8/SeekGE-10          113ns ± 0%     113ns ± 0%    ~     (p=0.631 n=8+8)
CockroachDataBlockIterShort/AlphaLen=8,Prefix=128,Shared=64,ValueLen=128/Next-10      10.1ns ± 0%    10.2ns ± 0%  +0.75%  (p=0.000 n=8+7)
CockroachDataBlockIterShort/AlphaLen=8,Prefix=128,Shared=64,ValueLen=128/SeekGE-10    89.6ns ± 0%    91.0ns ± 1%  +1.58%  (p=0.000 n=7+8)

Benchmark for the no-transform path before/after:
```
name                                                                                old time/op    new time/op    delta
CockroachDataBlockIterShort/AlphaLen=8,Prefix=8,Shared=4,ValueLen=8/Next-10           12.2ns ± 0%    12.5ns ± 3%  +2.38%  (p=0.008 n=5+5)
CockroachDataBlockIterShort/AlphaLen=8,Prefix=8,Shared=4,ValueLen=8/SeekGE-10          113ns ± 0%     114ns ± 0%  +0.80%  (p=0.008 n=5+5)
CockroachDataBlockIterShort/AlphaLen=8,Prefix=128,Shared=64,ValueLen=128/Next-10      10.1ns ± 0%    10.3ns ± 0%  +2.30%  (p=0.016 n=5+4)
CockroachDataBlockIterShort/AlphaLen=8,Prefix=128,Shared=64,ValueLen=128/SeekGE-10    89.4ns ± 0%    90.3ns ± 0%  +1.08%  (p=0.008 n=5+5)
```

Transforms benchmarks:
```
name                                                                                                  time/op
CockroachDataBlockIterTransforms/AlphaLen=8,Prefix=8,Shared=4,ValueLen=8/Next-10                      12.3ns ± 1%
CockroachDataBlockIterTransforms/AlphaLen=8,Prefix=8,Shared=4,ValueLen=8/SeekGE-10                     113ns ± 0%
CockroachDataBlockIterTransforms/AlphaLen=8,Prefix=8,Shared=4,ValueLen=8,SynthSeqNum/Next-10          12.5ns ± 0%
CockroachDataBlockIterTransforms/AlphaLen=8,Prefix=8,Shared=4,ValueLen=8,SynthSeqNum/SeekGE-10         114ns ± 1%
CockroachDataBlockIterTransforms/AlphaLen=8,Prefix=8,Shared=4,ValueLen=8,HideObsolete/Next-10         13.0ns ± 1%
CockroachDataBlockIterTransforms/AlphaLen=8,Prefix=8,Shared=4,ValueLen=8,HideObsolete/SeekGE-10        117ns ± 1%
CockroachDataBlockIterTransforms/AlphaLen=8,Prefix=128,Shared=64,ValueLen=128/Next-10                 10.3ns ± 0%
CockroachDataBlockIterTransforms/AlphaLen=8,Prefix=128,Shared=64,ValueLen=128/SeekGE-10               90.5ns ± 1%
CockroachDataBlockIterTransforms/AlphaLen=8,Prefix=128,Shared=64,ValueLen=128,SynthSeqNum/Next-10     10.5ns ± 0%
CockroachDataBlockIterTransforms/AlphaLen=8,Prefix=128,Shared=64,ValueLen=128,SynthSeqNum/SeekGE-10   90.9ns ± 1%
CockroachDataBlockIterTransforms/AlphaLen=8,Prefix=128,Shared=64,ValueLen=128,HideObsolete/Next-10    10.8ns ± 0%
CockroachDataBlockIterTransforms/AlphaLen=8,Prefix=128,Shared=64,ValueLen=128,HideObsolete/SeekGE-10  93.9ns ± 0%
```
Copy link
Collaborator

@jbowens jbowens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewable status: 3 of 6 files reviewed, all discussions resolved

@RaduBerinde
Copy link
Member Author

TFTR!

@RaduBerinde RaduBerinde merged commit b3453b7 into cockroachdb:master Oct 1, 2024
12 checks passed
@RaduBerinde RaduBerinde deleted the colblk-hide-obsolete branch October 1, 2024 16:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants