Mathematical retrodiction and prediction

In this discussion over on Urs Schreiber's blog, The String Coffee Table, Greg Kuperberg writes:

Consider, for example, how category theory might guide you to define quantum groups. As a first try, you might consider group objects in the contravariant category of non-commutative algebras. This is a natural proposal, but it isn’t the most fruitful one. A much better idea is to use Hopf algebras. Hopf algebras are a non-cocommutative generalization of group algebras, but they are not group objects. They have their own category-theoretic motivation (in terms of monoidal categories), but it takes hindsight to see the way in which category theory is important here.

John Baez in some recent lectures shows how (involutory) Hopf algebras can be generated from groups (see p. 53), providing a category theoretic account of their nature. Kuperberg's point, however, is that this is reconstruction rather than construction. He also admits that category theory can play a suggestive role, providing targets for future research.

I was reminded during The String Coffee Table discussion of a debate which raged for a time in the philosophy of science as to the credit a theory deserves for retrospectively accounting for observations - retrodiction - as compared to the prediction of unseen phenomena. The consensus seems to admit that, to some extent, you should think your theory more likely to be accurate in the future if it has retrodicted an observation, so long as that observation wasn't used in the construction of the theory, by, say, fixing values of parameters. But there were sharp disagreements on what one could say about permissible uses of the observation. One of these days, I shall write up what statistical learning theory has to say about this topic. The key idea is that if you start with a class of hypotheses that isn't too rich, then relying only on training data you can say how probably accurate a selected hypothesis is. And richness isn't measured by the number of parameters.

All things considered, though, there's nothing like a bold new prediction. Now, higher-category theory performs something like this task too. Say you are interested in a certain construction, such as Hopf algebras. You then formulate the construction in a way that suggests features of a 'categorified' version, i.e., sets become categories, categories become bicategories, functions become functors, functors become natural transformations, product operations on elements become monoidal products on objects, and so on. Or perhaps you look to internalise constructions within a category, by, say, representing a group as a diagram, and realising that diagram within a category. You might end up then with a good idea of what your categorified entity, say, a Hopf 2-algebra, should look like. You predict that there should be an important concept which possesses specified properties. This is higher category theory in its guiding, predictive role, pushing you in promising directions. But there is no magic bullet. There's still very likely a huge amount of work to be done in getting the construction right, and in finding rich examples of it. Sometimes these examples already exist, but recognising them as a cases of concepts which fit into some hierarchical pattern can be extremely important.