Not very good at detecting nearby vehicles.

[Leozyc-waseda]

I am trying to use the PV-RCNN model trained on pandaset to test it on my private dataset,

1.I converted my dataset's coordinate system to Normative coordinates are:
# - x pointing forward
# - y pointings to the left
# - z pointing to the top
2. pv_rcnn.yaml from kitti pv_rcnn.yaml，modified
_BASE_CONFIG_: cfgs/dataset_configs/pandaset_dataset.yaml
3. In pandaset_dataset.yaml I modified

POINT_CLOUD_RANGE: [-70, -50, -2, 50, 70, 0]# xmin, ymin, zmin, xmax, ymax, zmax
 VOXEL_SIZE: [0.05, 0.05, 0.05]

followed
#253 (comment)

You can see that at close range, there are one car that are not detected, and some strange results are output (yellow and cyan)

As MartinHahner mentioned in this issue, LiDAR models do not transfer well from one dataset to another one.
#599 (comment)

But here also mentioned the paper Train in Germany, Test in The USA: Making 3D Object Detectors Generalize, I found that the accuracy should be quite good in the case of close range.
#599 (comment)

Do you have any suggestions to improve the problem of not being able to detect cars at close range?

1.Do I need to retrain the model with my dataset height?
The lidar of pandaset is close to 2m off the ground, and my data is almost around 2.052m
2.Or are there any parameters I need to adjust carefully?
Such as POINT_CLOUD_RANGE, VOXEL_SIZE

[jihanyang]

I think our work ST3D can partly help this issue, especially for the 1st problem, which can be solved by just shifting the point cloud on z-axis.

[Leozyc-waseda]

@jihanyang
Thank you for your reply.
I will test my data with ST3D

BTW,

which can be solved by just shifting the point cloud on z-axis.

Is there anything I need to pay attention to about adjusting the point cloud z-axis?
If the lidar point is based on the ground 0m, or based on the actual height of 2.052m, how to adjust it when adjusting?

[jihanyang]

You can do similar shifting as
https://github.com/CVMI-Lab/ST3D/blob/812b64491c3a67a1b578fe46c375fcd0ed2c4305/tools/cfgs/da-waymo-kitti_models/secondiou/secondiou_old_anchor.yaml#L13

[Leozyc-waseda]

Thank you jihanyang!

[Leozyc-waseda]

@jihanyang Thanks a lot .
I remember you saying that it may be a problem with domain adaptation that it cannot detect a car at a nearby distance.
But today I tested it, using the PVRCNN model trained on pandaset, tested on pandaset, it feels very unsatisfactory..

score_thresh = 0.1
nms_thresh = 0.1

score_thresh = 0.7
nms_thresh = 0.1

Because I train on pandaset and test on pandaset, if I can't detect nearby cars, this shouldn't be a domain adaptation problem..

[jihanyang]

Can you highlight the car that you cannot detect?

[Leozyc-waseda]

Image

If the score value is 0.1, there will be a lot of FP.

If the score value is 0.7, there will be a lot of car cannot detect.

[jihanyang]

Could you provide how does adaptition model perform on this frame, such as pre-trained on waymo?

[Leozyc-waseda]

Do you mean to train the model of waymo->pandaset with ST3D?