动机
To exploit the radar information in this setting, radar-based features need to be mapped to the center of their corresponding object on the image, which requires an accurate association between the radar detections and objects in the scene
- 比较naive的融合思想
To accomplish this, a naive approach would be mapping each radar detection point to the image plane and associating it to an object if the point is mapped inside the 2D bounding box of that object
网络结构
个人理解:通过前面CenterNet输出的属性构建一个3D的透视模型,将该模型变换到BEV视图,让后将Radar的数据与BEV的视图做融合
视锥透视
If there are multiple radar detections inside this RoI, we take the closest point as the radar detection corresponding to
在驾驶领域这个处理是合理的
This significantly narrows down the radar detections that need to be checked for association, as any point outside this frustum can be ignored.
This is to account for inaccuracy in the estimated depth values, as the depth of the object at this stage is solely determined using the image-based features. Enlarging the frustum using this parameter increases the chance of including the corresponding radar detections inside the frustum even if the estimated depth is slightly off
如果CenterNet检测不出来,或者出现错误,明显会漏检;虽然他在后面做了一个3D BBox的扩大
Pillar Expansion
each radar point is expanded to a fixed-size pillar, as illustrated in Fig. 4. Pillars create a better representation for the physical objects detected by the radar, as these detections are now associated with a dimension in the 3D space.
将Radar的点直接3D柱体化,然后再通过Frustum(视锥)方式进行叠加,中图是直接叠加,下图是视锥的方式
for every radar detection associated to an object, we generate three heat map channels centered at and inside the object’s 2D bounding box, as shown in Fig. 4. The width and height of the heatmaps are proportional to the object’s 2D bounding box, and are controlled by a parameter α.
对radar关联好的每个OBJ,以中心点为远点,长宽以预测的2D BBOX为范围,抽取3张热力图,其特征为:正则化后的(d, vx or vy); 3通道的热力图是为了跟RGB做连接
CenterPoint
论文名字:Objects as points
跟一般BBOX的方式不一样,直接预测一个中心点,然后以中心点为基础去回归各项属性,比如朝向,深度等
这是论文粗读知乎地址: https://blog.csdn.net/c20081052/article/details/89358658