Project

  1. Pytorch Code for "Variational Hetero-Encoder Randomized GANs for Joint Image-Text Modeling" can be found HERE. This project is about to build a multi-layer probabilistic multi-modality model to explore interpretable relationships between images and texts. The model structure can be seen as follows, where (a) is image encoder and text decoder, (b) is the simple structure of image generator (StackGAN++), and (c) is our proposed raster-scan-GAN for image generarion.

    We give an example of generative images from texts.

    We show the learned hierarhcial relationships between images and texts.

  2. Python (Theano) code for "WHAI: Weibull hybrid autoencoding inference for deep topic modeling" can be found HERE. This project is about to build a multi-layer probabilistic autoencoding topic models. The learned topics and topic proportions from corpus are interpretable, which can be used for the downstream tasks. The graphical models with encoding network shown below illustrate the hierarchical model and inference.

    We also give an example of hierarchical topics learned from Wiki. More examples can be found in our paper.

    3. © Hao Zhang