3D Shape Generation through Point Transformer Diffusion

Abstract

Diffusion models are a novel class of generative models, which have demonstrated promising results in 3D point cloud generation. Meanwhile, Transformer-based models have achieved impressive outcomes across various benchmarks in the field of 3D point cloud understanding. However, the integration of a Transformer based on the local self-attention mechanism with a diffusion model for 3D point cloud generation remains unexplored. In this work, we propose Point Transformer Diffusion (PTD), a probabilistic and flexible generative model. PTD integrates the standard Denoising Diffusion Probabilistic Model, adapted for 3D data, with Point Transformer, a local self-attention network specifically designed for 3D point clouds. To enhance PTD’s performance, several adjustments and techniques are implemented to align the Point Transformer model more effectively with the diffusion model and with the specific generation task. Experiments demonstrate PTD’s ability to generate realistic and diverse shapes. Furthermore, the evaluation results of PTD are comparable to, and in some cases even marginally superior to, those achieved by Point-Voxel Diffusion, which is a state-of-the-art approach. We hope that our work will inspire future investigations into architectures that combine diffusion models and Transformers, along with their application to a wide range of 3D generation tasks.