Skip to content
This repository has been archived by the owner on Oct 30, 2023. It is now read-only.

Improve performance of LongByteMappingStore #91

Open
wants to merge 4 commits into
base: trunk
Choose a base branch
from

Conversation

dlogothetis
Copy link
Contributor

@dlogothetis dlogothetis commented Oct 24, 2018

Part of the overhead comes from using an atomicinteger that's written to for every entry addition. I don't see any particular value in this keeping this counter, which incurs high overhead is it is accessed on every entry. There are better ways to update it as values are added, if we think this is useful, we can add it then.

Modified also the getByteTarget method to do one less hashmap access as per the comment by @spupyrev .

Tests

  • mvn -Phadoop_facebook clean install
  • mvn -Phadoop_2 clean install
  • Ran jobs that read a mapping, this reduces the time to load mapping by up to 50%.

https://issues.apache.org/jira/browse/GIRAPH-1210

Copy link

@spupyrev spupyrev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we also modify getByteTarget method to the following:
public byte getByteTarget(LongWritable vertexId) { long key = vertexId.get() >>> lowerOrder; byte[] bs = idToBytes.get(key); if (bs == null) { return -1; } int suffix = (int) (vertexId.get() & lowerBitMask); return bs[suffix]; }

@dlogothetis dlogothetis changed the title Remove the use of atomic counter in LongByteMappingStore Improve performance of LongByteMappingStore Oct 25, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants