Problem on train dataset construction

[jarvishou829]

Thank you for the open-sourced code which is clear and easy to follow. I'm currently trying to train on my own dataset. I notice that the dataset loading relies on the pkl file. But the current pipeline does not include how to access. Is there any possible advice on how to form such a pkl on my own data, what information does the pkl include?
Further, it seems that a complete meta structure of the dataset is unavailable. Could you help provide a more detailed folder structre on any one dataset to help organize the custom dataset.

[jjxjiaxue]

Hi, you can temporarily refer to this:
https://github.com/OpenDriveLab/DetAny3D/blob/main/detect_anything/datasets/data_creator/custom_dataset.md

Regarding the construction of the 3-DOF rotation matrix, we will test and update it as soon as possible.

Previously, we constructed the 3-DOF rotation matrix as follows:

pose = obj['R_cam']
R_90 = np.array([[0, 0, 1],
                 [0, 1, 0],
                 [-1, 0, 0]])
pose = np.dot(pose, R_90)

Here, obj['R_cam'] is the rotation matrix from the Omni3D dataset.

[jarvishou829]

Thank you for the helpful instruction. I have realized the custom dataset construction process. During processing the dataset, I found that the bbox_2d_tensor is scaled here but the bbox_3d_tensor has not been processed in the returned dict. This cause the visualization here looks not correct. Is this a bug or it just affect the commented visulization code. I wonder will this affect computing the bbox_3d loss.

[jjxjiaxue]

During preprocessing, the image is resized, so the 2D bounding boxes need to be scaled accordingly.
However, the GT 3D bounding boxes are defined in the camera coordinate system, and their (x, y, z) are in metric units relative to the camera, not in pixel coordinates, so there is no need to resize them.