largest gap between sum_i log2 p(x_i | x_<i) and log2 p(x), over 20000 images: 9.2e-14

largest |d logit_i / d x_j|    j >= i (future and self)   j < i (past)
masked network                 0.00e+00                   0.074
unmasked network               1.19e-01                   0.095

model                        bits per pixel on 20000 fresh images
uniform: every image equally likely  1.0000
independent pixels                   0.8371
learned autoregressive               0.3404
exact autoregressive (the truth)     0.3300

bits per pixel by image row       1     2     3     4     5     6     7     8
learned                        0.62  0.33  0.30  0.30  0.30  0.29  0.30  0.29
exact                          0.61  0.31  0.29  0.29  0.29  0.28  0.29  0.29
the flip noise alone costs 0.29 bits per pixel

network passes to draw 4000 images: 64 (one per pixel), against 1 pass to score them
share that look like a bar:  real data 0.988   autoregressive samples 0.842   independent-pixel samples 0.001
...##...   ...##...   ......##   ......##
....#...   ...##...   ...#..##   ......##
...##...   ...##...   ......##   ##....##
...#.#..   .#.##...   ......##   #.....##
...##...   ...##...   ......##   ......##
...###..   ...##...   ......##   ....#.##
...##...   .#.##...   ......##   ......##
...##...   ...##...   ......##   ......##

values in a 256x256 RGB image: 196,608; image tokens in DALL-E's 32x32 grid: 1,024; ratio 192
