• 1 Post
  • 1 Comment
Joined 9 months ago
cake
Cake day: September 30th, 2023

help-circle
  • This was not a convergence test; it was an “accuracy + aesthetics” test.

    There very much is a point to testing the accuracy and aesthetics of samplers, including ancestral ones. Indeed, that’s the entire point. By contrast, the whole point of doing a large number of tests is to compensate for the fact that you’re not going to get the same result with every sampler at every number of steps, and thus a large number of tests offsets the luck of the draw.

    I have no sampler named just “DPM++ 3M”. I have three DPM++ 3M samplers: SDE, SDE Karras, and SDE Exponential - the yellow samplers in the graphs.

    Anything can “feel off” to you, but this is what the data shows. I had some of my own biases coming into this that got busted in the process (while I wasn’t surprised by DPM++ SDE’s performance, I expected the new samplers to be standouts and old samplers like Euler a to be poor). If you feel the sample size is too small or is somehow biased, by all means contribute - I literally included the spreadsheet link so others could take part! :) Just a caveat: do your best to not mentally keep track of what sampler’s images you’re rating; we want it to be as “blind” of a test as can be reasonably done.