There are some arguments in the quantum gravity community as to what constitutes the right notion of "background-free". But in general, we can say that a "background structure" is any sort of structure appearing in a mathematical model of a physical system that is fixed rather than dynamical - i.e., which does not depend on the state of the physical system in question.
Heuristically, we can think of a background structure as something which affects the dynamics of the system while remaining unaffected by it. From this viewpoint, background structures violate the commonly held dictum that "if A affects B, then B affects A" - the reciprocity of cause and effect.
This dictum has an interesting history. In 1277, when Westerners were getting really interested in the works of Aristotle, the Bishop of Paris declared this dictum a heresy. The reason is that the Catholic Church wanted God to be able to affect the universe while remaining unaffected by it.
However, this condemnation didn't stop the spread of the idea of reciprocity of cause and effect. Newton said that "for every action there is an equal and opposite reaction". And later, this idea was later built into the Lagrangian formalism. The notion of "background structure" is easiest to understand in the Lagrangian formalism (though one can also make sense of it in other approaches to physics).
If one variable in the Lagrangian of a physical theory exerts a force on another, then it receives a force in return. It's an immediate consequence of the equality of mixed partial derivatives. The only way out of this is to put a quantity in the Lagrangian but not allow oneself to vary it when working out the equations of motion. Thus we often find background structures appearing as quantities that appear in the Lagrangian of our theory but are not allowed to vary.
A simple example occurs when we consider a bead on a hoop - a standard problem in classical mechanics. If we demand that the hoop stay still no matter what the bead does, we cannot treat the hoop's position as a variable when working out the equations of motion from the Lagrangian. Instead, we must treat it as fixed. This means we pretend the hoop exerts a force on the bead while not being affected by the bead. For this reason, we say that the hoop's position is a "background structure". Of course, this is just an idealization of the real situation, where the hoop actually *is* affected by the motion of the bead.
For a deeper example, consider Maxwell's equations. The Lagrangian of this theory involves the metric on spacetime, but the metric is treated as fixed - it's not allowed to vary when derive the equations of motion. Thus the metric is a background structure in this theory. Concretely, what this means is in this theory, the geometry of spacetime affects the motion of light, but not conversely.
If we take a broader view of background structures, there are other background structures implicit in Maxwell's equations. For starters, the fact that spacetime is a manifold is a background structure - it holds regardless of the particular solution of Maxwell's equations that we're considering! The dimension of spacetime is also a background structure. So is specific topology of spacetime. So is the specific smooth structure.
However, let's focus on the metric right now.
General relativity differs from Maxwell's equations in that the metric, which was a background structure for Maxwell's equations, is now dynamical. In other words, the metric now depends on the actual solution of the equations of general relativity. We can see this clearly by noting that we allow the metric to vary in the Lagrangian formulation of general relativity. It not only affects the other fields - it's affected by them.
So when you combine electrodynamics with general relativity, the geometry of spacetime doesn't just affect the motion of light through spacetime - it's also affected by the motion of light through spacetime! More generally, the curvature of spacetime affects the motion of matter, while matter curves spacetime.
Getting more technical, one can see that making the metric dynamical instead of a background structure leads to the fact that all diffeomorphisms are gauge symmetries in general relativity. This in turn implies that quantities such as the energy density and momentum density as computed using Noether's theorem must vanish thanks to the equations of motion. And this in turn implies that when we try to quantize gravity following standard procedures, we have a Hamiltonian constraint rather than a Hamiltonian. So we get the Wheeler-DeWitt equation:
H psi = 0
which says that the Hamiltonian constraint must annihilate any physical state psi. This is the main shocking new thing about quantum gravity as opposed to, say, the quantization of Maxwell's equations.
Now, some people working on quantum gravity have been led by these considerations to argue that a good physical theory should be as background- free as possible.
The most articular exponent of this viewpoint is perhaps Lee Smolin. See:
He has argued that ANY sort of background structure is a bad thing in a fundamental theory of physics. Of course, taken to its ultimate limit, this philosophy can lead to problems. We can - and certainly should - treat the metric on spacetime as dynamical. We can try to treat the topology of spacetime as dynamical: people do this when they attempt to "sum over topologies" in quantum gravity. We can even try to treat the dimension of spacetime as dynamical - although we don't really know how to do this very well. But how can we do physics without any prior assumption on the nature of spacetime, e.g. that it is a manifold, or a spin foam, or some other kind of gadget?
Going to extremes, we can even argue that the equations describing any theory of physics are a background structure, because they hold regardless of the state of the system! To get rid of this background structure, we would need to do physics without any fundamental equations. Believe it or not, folks like Lee Smolin and Holger Nielsen have tried this! But I don't think they've been terribly successful so far. For more information on this line of thought, try:
Personally I think one can dig oneself into a hole by trying to do physics without any background structure - it's a bit like trying to paint a painting without any canvas. I prefer to stick more closely to the more limited lesson general relativity offers us. Namely, in general relativity there are no background structures of a GEOMETRICAL sort: i.e., no tensor fields (or bundle sections, or connections, etc.) that appear in the Lagrangian but are not varied over.
Of course, in a full-fledged theory of quantum gravity, we may have to interpret the term "geometrical structures" in a novel way. What counts as a "geometrical structure" depends on the kind of theory we've got.
It may, or may not, be useful to extend this lesson and assume that there are no background structures of a TOPOLOGICAL sort. For decades there has been a raging argument in the quantum gravity as to whether one should sum over topologies in the Euclidean path integral. This takes on new life in the spin foam approach to quantum gravity: people are arguing over whether one should sum over spin foams corresponding to different topologies or fix a topology at the outset. I would have to say the jury is still out on this issue.
For more on these issues, try this paper of mine:
For a more careful philosophical study of why background-free theories are good, I urge you to read this paper by Carlo Rovelli: