Diffusion probabilistic models have been successfully adapted to generate 3D point clouds and achieved impressive fidelity. However, existing methods can only generate a single point cloud from noise, leaving joint generation and conditional generation elusive to grasp. These two demands originate in various real-world problems and we take dressed avatar generation as an example. Specifically speaking, directly generating a dressed avatar as a single point cloud cannot meet the demand of changing garment, and generating garment and avatar as two point clouds inevitably leads to mismatch. Meanwhile, generating matched garment for an undressed avatar is not possible for current diffusion techniques. To this end, we present Hoodie which is the first method that successfully resolves aforementioned issues. Technically, Hoodie first trains two separate point cloud diffusion models with global latent codes, then trains a latent code diffusion model for the concatenation of human and garment latents. This hierarchical architecture not only supports the joint generation of human and matched garment, but also supports conditional inference that generates matched garment given a 3D human input. Besides, we integrate a point cloud upsampling GAN to improve the uniformity of generated point clouds. Large-scale quantitative and qualitative evaluations show that Hoodie achieves strong performance on aforementioned new tasks it enables. Code, data and models will be publicly available.