Ok, so what does closed under composition even mean here --- in this case, it means that we should end up only with colors that are a mix of the colors we started with. Specifically, if we take a bunch of "bright reds" and perform the "over" operation on them, we should only ever end up with more "bright reds." The non-premultiplied alpha fails in this case, because it takes a group of things (well, 2 things) with RGB value (1,0,0) and returns a new thing with RGB value (0.75,0,0), whereas the premultiplied alpha returns another (1,0,0).
I'm not sure how useful this observation is in the general case, because if you start with a set of different color values (e.g., (1,0,0) and (0,1,0)) you can get different mixes of those colors depending on the relative alpha values, and this is true for both premultiplied and non-premultiplied alpha (although the exact mixes of red and green will differ for the two methods). So, to my mind, I just treat this example as an interesting failure case that shows yet another reason why premultiplied alpha is probably a good idea.