Top-Down VAE Training

• train config

  {'dataset': 'textures',
   'dataset_params': {'batch_size': 128, 'test_batch_size': 128,
    'train_every': 2, 'test_every': 1, 'crop_dim': 40,
    'path': 'fakelabeled_natural_commonfiltered_640000_40px.pkl', 'offset': 0.0},
   'n_steps': 22500000, 'random_seed': None,
   'lr_init': 2.5e-05, 'lr_factor': 1.0, 'lr_schedule': [1],
   'output_type': 'normal', 'output_sd': 0.4,
   'n_y': 250, 'n_y_samples': 1, 'n_y_samples_reconstr': 1, 'n_z': 1800,
   'beta_y_evo': 1.0, 'beta_z_evo': 1.0,
   'encoder_kwargs': {'encoder_type': 'mlp', 'n_enc': [2000], 'enc_strides': [1]},
   'cluster_encoder_kwargs': {'encoder_type': 'mlp', 'n_enc': [1000, 500, 250]},
   'latent_y_to_concat_encoder_kwargs': {'y_to_concat_encoder_type': 'mlp',
    'y_to_concat_n_enc': [250, 500, 1000, 2000]},
   'latent_concat_to_z_encoder_kwargs': {'concat_to_z_encoder_type': 'mlp',
    'concat_to_z_n_enc': [2000]},
   'l2_lambda_w': 0.0, 'l2_lambda_b': 0.0,
   'gradskip_threshold': 10000000000.0, 'gradclip_threshold': 1000000000.0,
   'decoder_kwargs': {'decoder_type': 'mlp', 'n_dec': [], 'dec_up_strides': None},
   'latent_decoder_kwargs': {'decoder_type': 'mlp', 'n_dec': [2000]},
   'z1_distr_kwargs': {'distr': 'laplace', 'sigma_nonlin': 'exp',
    'sigma_param': 'var'},
   'z2_distr_kwargs': {'distr': 'normal', 'sigma_nonlin': 'exp',
    'sigma_param': 'var'},
   'report_interval': 750000,
   'save_dir': 'hiercont_logs/3_1_3_1_5_0_hiercont_40px_natural_nz1800_ny250_2layerconcattoz1_1layerz1generator_softplus',
   'restore_from': 'mycurl-0',
   'tb_dir': 'hiercont_tb_logs/3_1_3_1_5_0_hiercont_40px_natural_nz1800_ny250_2layerconcattoz1_1layerz1generator_softplus',
   'activation': tf.math.softplus}
  

• run time: 57 hours (l1)
• train components

![[/assets/elbo_components.png]]

• active tf coding z2 dims: [91, 86, 110, 11, 225]
• active non-tf coding z2 dims: [214, 163, 223, 10, 219, 136, 213, 236, 13, 202, 147, 161, 123, 192, 68, 245, 185, 170, 118, 177, 94, 69, 164, 128, 162, 84, 45, 131, 22, 36, 165, 155, 168, 187, 61, 188, 133, 25, 37, 195, 82, 87, 148, 77, 208, 104, 66, 142, 167, 59, 79, 38, 75, 172, 190, 240, 89, 96, 29, 247, 174, 6, 15, 145, 156, 137, 221, 85, 8, 135, 233, 26, 234, 197, 51, 124, 173, 19, 134, 5, 76, 178, 235, 140, 106, 143, 56, 50, 4, 237, 248, 83, 65, 52, 107, 175, 141, 122, 117, 207, 16, 157, 60, 12, 119, 158, 127, 186, 198, 132, 21, 54, 101, 220, 139, 194, 58, 23, 181, 105, 204, 57, 55, 154, 115, 239, 62, 160, 90, 196, 28, 182, 102, 201, 9, 73, 149, 64, 81, 238, 74, 3, 159, 231, 63, 42, 98, 97, 109, 205, 39, 14, 93, 17, 46, 217, 18, 92, 27, 130, 144, 44, 216, 100, 30, 244, 120, 215, 108, 0, 146, 116, 41, 199, 95, 112, 24, 226, 40, 203, 113, 99, 126, 72, 138, 191, 193, 7, 43, 121, 242, 152, 212, 211]
• examined:
  • [91, 86, 110, 11, 225]
  • [214, 163, 223, 10, 219]