pymoo hypervolume computation is very slow compared to moocore

[MLopez-Ibanez]

Hi,

I did a few tests comparing the time required to calculate the hypervolume using pymoo and using moocore: https://multi-objective.github.io/moocore/python/

With 3 objectives, pymoo is 200-1000 times slower than moocore:

With 4 objectives, pymoo is 300-600 times slower:

This is the script that I have used:

import numpy as np
import moocore

from pymoo.indicators.hv import Hypervolume as pymoo_HV

import timeit

def read_data(name):
    files = {"DTLZLinearShape.3d": "~/work/perfassess/hyperv/uc/test/inp/DTLZLinearShape.3d.front.1000pts.10",
             "DTLZLinearShape.4d" : "~/work/perfassess/hyperv/uc/test/inp/DTLZLinearShape.4d.front.1000pts.10"}
    x = moocore.read_datasets(files[name])[:, :-1]
    x = moocore.filter_dominated(x)
    return x

name = "DTLZLinearShape.4d"
x = read_data(name)

ref = np.ones(x.shape[1])
pymoo_hv = pymoo_HV(ref_point=ref)
moocore_hv = moocore.Hypervolume(ref = ref)

n = np.arange(500, 3000 + 1, 500)

pymoo_values = []
moocore_values =  []
pymoo_times = []
moocore_times =  []
for maxrow in n:
    z = x[:maxrow, :]
    pymoo_values += [pymoo_hv.do(z)]
    moocore_values += [moocore_hv(z)]
    duration = timeit.timeit('pymoo_hv.do(z)', globals=globals(), number = 2)
    pymoo_times += [duration]
    duration = timeit.timeit('moocore_hv(z)', globals=globals(), number = 2)
    moocore_times += [duration]
    
pymoo_values = np.array(pymoo_values)
moocore_values = np.array(moocore_values)
assert np.allclose(pymoo_values, moocore_values)

pymoo_times = np.array(pymoo_times)
moocore_times = np.array(moocore_times)

import pandas as pd
import matplotlib.pyplot as plt
cpu = "Intel i5-6200U 2.30GHz"
df = pd.DataFrame(dict(n = n, Ratio = pymoo_times/moocore_times)).set_index('n')
df.plot(grid=True, ylabel='Pymoo/moocore (seconds)', title = f'HV computation for {name} ({cpu})')
plt.show()

[blankjul]

Nice that you benchmarked both and let me know!

Do you know what code the other framework is using? Is it the method or the way it is implemented that causes the speed-up?
pymoo is borrowing its Hypervolume implementation as well from Simon Wessing (you might have already seen that).

[MLopez-Ibanez]

Do you know what code the other framework is using? Is it the method or the way it is implemented that causes the speed-up? pymoo is borrowing its Hypervolume implementation as well from Simon Wessing (you might have already seen that).

It is documented here: https://multi-objective.github.io/moocore/python/reference/generated/moocore.hypervolume.html#moocore.hypervolume

It is our C implementation of https://multi-objective.github.io/moocore/python/reference/generated/moocore.hypervolume.html#footcite-fonpaqlop06-hypervolume with a few more improvements contributed by Andreia P. Guerreiro that are not present in Simon's implementation.

[blankjul]

Are you the founder/main contributer of the framework (moocore)?

(saw your name popping up for the hypervolume code as well)

[MLopez-Ibanez]

For now I am the main maintainer but contributions are welcome. The idea is to implement all critical pieces in C/C++ but if some code is only available in Python or R, it can be added in the hopes that somebody will move it to C/C++ in the future. All the infrastructure to build and test C/C++ code for Python/R is already in place, so it is quite easy to add new stuff.