Here is another example. Take a flat piece of paper, which we can think of as a 2-D space. Roll it into a cylinder. Is it curved? Of course it is... in the usual meaning of the word. But not for Gauss: technically the surface of a cylinder has zero Gaussian curvature. So what is going on here?
The trouble was, Gauss was a man with a secret. The biggest mathematical problem in Gauss' day was "the scandal of geometry". In ancient times Euclid had made geometry the archetype of a rigourous mathematical system. To deduce all the theorems of geometry, Euclid needed some definitions and some rules of reasoning. But he also needed to make some basic assumptions about geometrical entities (lines, circles etc), that he called postulates. There were five of them:
There is a formal analogy between the properties of "straight lines" in non-Euclidean geometry, and the properties of "geodesics" on curved surfaces. A geodesic is the shortest line between two points that is confined to the surface; the famous example is great circles on a sphere. Now if you look at a surface from "outside", as we look at the surface of a cylinder or a sphere, it is obvious whether or not it is curved. But suppose you were a 2-D being embedded in the surface: you can't see out of the surface, so curvature is not apparent in the same way. But you could do experiments in geometry, to see whether the straightest lines you could draw would agree with Euclidean plane geometry or not. This was the position Gauss was in, as a 3-D being trying to find out if his 3-D space was Euclidean or non-Euclidean. So when he wrote on the differential geometry of curved surfaces, he defined "curvature" in a way that could be measured entirely "from within". This had the unfortunate side effect that cylindrical surfaces etc. were no longer curved, because the geometry of geodesics on a cylinder is Euclidean. You can see this by drawing out some standard geometric proof on your flat piece of paper and then rolling it up. The formerly straight lines are still the shortest distances between their points; the angles are still the same; everything is unchanged.
But wait a minute! Suppose you go all the way round the cylinder: you travel along a "straight line", and get back to your starting point... that doesn't happen in Euclid! This is true, so we need to be a bit more careful. Gauss' curvature is defined locally, in the limit at each point on the surface. Now all smooth  surfaces are locally very close to Euclidean (if you look with a big enough magnifying glass, any smooth surface seems flat). But as you go to a bigger and bigger area, the properties become more and more distorted if the surface has a finite Gaussian curvature: for instance, the sum of angles of a triangle is no longer 180°, and the circumference of a circle is no longer 2 pi r. The smaller the curvature, the larger the region you need to get a given amount of distortion. But the surface of a cylinder is not like that. The properties stay precisely Euclidean until your region wraps all the way round, when you suddenly find that things are connected in an unexpected way. This is a feature of topology, not curvature. Topology describes the way things are connected up on a "global" scale, but is unconcerned with actual lengths; in contrast, lengths (like the circumference of small circles) are the essence of curvature, as we will see. It is important to keep the concepts of topology and curvature distinct, especially in cosmology where they are often muddled up.
So if a cylinder is not "curved" in the sense we want, how do we make a curved surface that would satisfy Gauss? Very simple. Spill some coffee on your paper and leave it out in the sun. The next day it will be all warped; you won't be able to lay it flat. Warp is a good word; it's the one that Gauss should have used. You can change the shape of a warped sheet, flip it inside-out by pushing down the bumps and pushing up the troughs, roll it up; but you can't make it flat. The warp is the thing that stays the same through all this. But what actually has happened? When the coffee is absorbed in the paper and later dries out, it changes the distance between neighbouring fibres in the paper. We get a region which is too big to fit inside its surroundings, if we stuck with the geometry of Euclid. For sheets of paper embedded in our 3-D world, this can be accomodated by the region bulging out, being curved in the everyday sense. But the crucial thing for the warp is not the bulging out, but the fact that the grid of distances between neighboring fibres is no longer consistent with the rules of planar Euclidean geometry.
Now imagine that our 2-D plane is purely abstract: not a slice of a 3-D world, but sufficient to itself. As Gauss realised, there is no reason why its grid of distances has to conform to Euclidean geometry. They can be distorted just as in the case of the fibres in the paper. But this time the plane has nowhere to "pop out" into, and no need to do so; it is the entirety of its own universe. If you think otherwise, this is your inbuilt geometric instinct talking; very useful for dealing with a real world which is Euclidean on a human scale, but no good at all for abstract mathematical arguments. In mathematics, to show that a concept is "wrong" you have to demonstrate a contradiction with the assumed axioms, and the whole point here is that we throw out the axiom which enforces the usual set of distances. Simply, the geometry of our warped plane is 2-D and non-Euclidean.
The grid of distances between neighboring points is such a useful concept that it has a special name: the metric. Of course, to call it a "grid" is an oversimplification; since our plane is a continuum, there are a continuous infinity of points in any interval. But this does not prevent us from specifying a rule that gives the distance between any pair of nearby points. To do that we need a way of labelling points, and for practical purposes we have to use coordinates, that is, a set of N real numbers which vary smoothly across the points in the continuum (N is the number of dimensions). The basic idea of the metric covers several closely-related concepts, but in cosmology it usually appears in the guise of a formula for calculating the infinitesimal distance between two infinitesimally-separated points. The metric therefore depends on the coordinates we choose, as well as on the actual geometry of the space. For example, on a Euclidean plane a few of the possible metrics are:
Even more bizarre is the "taxicab metric"
In cosmology, we have a 4-D space-time which, according to general relativity, is locally equivalent to the Minkowski metric of special relativity: