[CELEBORN-1802] Fail the celeborn master/worker start if CELEBORN_CONF_DIR is not directory

[turboFei]

What changes were proposed in this pull request?

Fail the celeborn master/worker start if CELEBORN_CONF_DIR is not directory. Otherwise, the process would run into unexpected status.

Why are the changes needed?

In the celeborn-daemon.sh , if we specify the --config <conf-dir> option. It would fail the master/worker start if the conf-dir is not a directory, likes the systemctl ConditionPathExists=$CELEBORN_CONF_DIR requirement check.

celeborn/sbin/celeborn-daemon.sh

Line 35 in fde6365

echo " <conf-dir> Override CELEBORN_CONF_DIR"

celeborn/sbin/celeborn-daemon.sh

Lines 53 to 62 in fde6365

	if [ "$1" == "--config" ]
	then
	shift
	conf_dir="$1"
	if [ ! -d "$conf_dir" ]
	then
	echo "ERROR : $conf_dir is not a directory"
	usage
	exit 1
	else

But before this PR, for the start master/worker scripts, it did not check if the CELEBORN_CONF_DIR is dirctory because the scripts did not leverage --config <conf-dir> option.

In this PR, we check the final CELEBORN_CONF_DIR before start celeborn, so that all the scripts would check if the CELEBORN_CONF_DIR is a directory before start.

Does this PR introduce any user-facing change?

Yes, it would fail the start if CELEBORN_CONF_DIR is not a directory.

How was this patch tested?

[turboFei]

Maybe I can use the below commands in systemctl.d/supervisor.d directly.

$CELEBORN_HOME/sbin/celeborn-daemon.sh --config $CELEBORN_CONF_DIR start org.apache.celeborn.service.deploy.master.Master

$CELEBORN_HOME/sbin/celeborn-daemon.sh --config $CELEBORN_CONF_DIR start org.apache.celeborn.service.deploy.master.Worker 1

[turboFei]

reopen it incase someone think it is necessary

[turboFei]

The celeborn-daemon.sh would load the celenorn-env.sh after checking the --config option.

celeborn/sbin/celeborn-daemon.sh

Line 79 in fde6365

. "${CELEBORN_HOME}/sbin/load-celeborn-env.sh"

So we just check it before run the start command.

[pan3793]

The change makes sense to me, also cc @FMX @RexXiong

[RexXiong]

LGTM, Thanks @pan3793

[FMX]

LGTM. Thanks. Merged into main(v0.6.0).