To really understand geometry, hence to understand GR, you gotta understand tangent vectors. Tangent to what, you ask? Tangent to a given point in spacetime! What does that mean? Well, this is easier to visualize if we consider not 4d curved spacetime, but a 2d curved space, like the surface of a pumpkin. (Yes, the pumpkin again.) Now the surface of a pumpkin is a "curved 2-dimensional Riemannian manifold", but it sits conveniently in (more or less) flat 3-dimensional Euclidean space, so we can think of a tangent vector to it as being an arrow whose base is at one point of the pumpkin, and which sticks out tangent to the pumpkin. We say it's a "tangent vector at a point" of the pumpkin.
Now we have to abstract things a bit! First, remove the 3d ambient Euclidean space and think only of the surface of the pumpkin! We can still define a "tangent vector"... the actual definition being rather mathematical... but one way to visualize it is as a teeny-weeny itsy-bitsy little arrow drawn on the surface of the pumpkin, with its base at the specified point. We make it small --- in fact, infinitesimal --- just in order to avoid worrying about the fact that the pumpkin is curved. After all, if we had an ambient 3d space as before, we could ignore the difference between a vector tangent to the pumpkin, and a vector actually drawn *on* the pumpkin, in the limit where the arrow became very small.
This is the sort of thing mathematicians can make precise; don't worry about it too much for now.
Oz asks: "Now of course the first thing (trying to keep 4D in mind) is what a tangent in this context would be. In particular a tangent to what? "
Tangent to spacetime itself! I hope my pumpkin metaphor made that clear... but let me repeat: if we think of a manifold, like the surface of a pumpkin, as embedded in some higher-dimensional Euclidean space: it's easy to visualize what we mean by a vector *tangent* to a point of that manifold. But in fact, even if we do not think of our spacetime as embedded in some higher-dimensional space, one may define the notion of a "tangent vector" at a point of spacetime. One could just as well drop the "tangent" business and call it a "vector", but hotshot physicists use so many different kinds of vectors they like to keep track of things this way, and besides, the "tangent vector" terminology encourages the useful kind of geometrical thinking that I was trying to cultivate in folks with that pumpkin metaphor.
Think of a tangent vector at a point of spacetime, if you like, as a wee arrow whose tail is pinned to that point.
Oz: "Now I never really needed to do anything with tensors in anger, or even at all. However I did read a little about them many years ago and I vaguely remember deciding they were basically vectors with position. "'
Hmm, that's not what tensors are... "vectors with position" is actually a pretty decent way of saying what *tangent vectors* are.
Oz reluctantly admits, "OK, perhaps, just perhaps, I get the idea. 'Normal' geometry is obsessed with orthogonality in some way. We abandon this and go for tangents. Now it's tempting to say that any small bit of pumpkin is flat, but that loses the curvaceousness of it. On the other hand we can't really define a big long tangent line on a 2D pumpkin since it cruises off into another non-existant dimension. We can however, always define an infinitesimally short one, possibly in a similarish way to Newton, at least in concept."
That's the basic idea. Mathematicians have had many years to make Newton's "infinitesimals" precise... in quite a variety of different ways which are pretty much equivalent for practical purposes. Thus we may unabashedly imagine a tangent vector to a pumpkin as an vector tangent to the pumpkin, but infinitesimal, so that it doesn't cruise off into the 3d space which is, after, quite nonexistent to the Pumpkin People, the 2-dimensional beings who inhabit the surface of the pumpkin.
Oz: "Incidentally I presume your warty old pumpkin is properly 2D, ie it's really completely smooth but has some strange spacial properties where different paths give different distances in "whatever co-ordinate system we define". Since we find it hard to visualise this we bend it up into a non-existent 3rd dimension."
Precisely. For us, of course, the 3rd dimension is real and the surface of the pumpkin is a mere "submanifold", but for the Pumpkin People the pumpkin is all of space and the 3rd dimension would simply be a mathematical fiction. Luckily, there is no need to argue. Mathematicians have mastered both "extrinsic geometry," in which a curved space is treated as a "submanifold" of some other, perhaps flat, space, and "intrinsic geometry", where you treat curved space in its own right and don't imagine it sitting in some higher-dimensional space.
The intrinsic viewpoint, developed by Gauss and Riemann in the late 1800s, is harder to get good at but it's often better. Why bother with extra dimensions you never really see or use anyway, if you don't need to?