There was a period in my life that was clearly marked by my exuberant fascination with everything meta. Now I treat it as a remarkable but not indispensable concept. I remembered it after a post that contained some (really unjustified) hate for all things meta.
The concept is too simple to make an entire blog themed after it, I think, so the blog died but I'd like to cite one post I submitted there that explains my view on various ideas in the space and disambiguates them from meta proper:
One can think of meta as a function that takes an object and returns an object that describes the input object using the technique characteristic of the object itself. MP3 metadata, for example, is data about the mp3 file, but not the audio itself (for example, artist information, album cover picture, etc.). It's meta because, just like the mp3 file is described by data (sets of bits), the metadata is described by the same means.
In real-life meta is shorthand for a situation in which instead of the object, the focus is on the description of the object that shares the same characteristics as the object itself. For example, suppose we are documenting the goals of our projects. One project's goal could be to understand the relationships between customers' preferences and their purchasing power. Another project's goal could be to determine all players in the currency exchange market. If we step back, we might want to determine what's the goal of us documenting all those goals. This is a meta goal. It doesn't refer to the object itself (documenting the goals) but it describes it, and uses the same technique (determining the goal).
It is crucial to understand that sharing the characteristic of the underlying object is a necessary condition. In the example above, for example, we could do many things with the given collection of goals to describe it -- we could determine their number, for example, or figure out that they all take just one sentence. But it's the fact that we're determining the goal of writing up those goals that makes the situation meta.
Not everything can be metafied, but I claim that once it's been metafied, it can then be metafied ad infinitum.
This is most often confused with meta. If something refers to an instance of itself (either a different instance or the same), then it's self-referential. For example, if you're playing an RPG game and your character instantiates an RPG game, then you're using self-reference. Self-reference is often used to provide paradoxes: for example, take the following statement: "There are two misteaks in this sentence." There is only one mistake in this sentence, but the fact that the sentence failed to provide an accurate count of the number of mistakes is a mistake in itself -- so there are two mistakes in the sentence. Which means that the sentence only contains one mistake. But then... (ad inf.) The sentence is self-referential. Self-referential statements can cause infinite loops (but iterative, not recursive infinite loops), like in the example above.
Recursion is a special case of self-reference when the object invokes a new instance of itself rather than refer to itself. The sentence in the above case is not recursive because it does not invoke a new instance. Similarly, a piece of code that does recursion is itself not recursive: it's, however, still self-referential. An example of a recursive statement is "n factorial is n times larger than n-1 factorial". This is recursive: to determine what 5 factorial is, you have to instantiate this sentence multiple times. In the mistakes example above, you don't need to instantiate multiple copies of the sentence.
Reflection is the ability to inspect one's own definition. Reflection is a special case of meta.
An operation is reflexive if it operates on itself. This is not really self-referential (is the statement "my dog is cleaning himself" self-referential?).
It seems to be that self-relativity is the ability to relate to oneself. A simplest example of self-relativity is the word "I". If something is self-relative, it is capable of producing self-referential statements.
A lot of things in the world are just deep. They're not meta or self-referential. We like talking about them because they seem complex. That doesn't make them any less sophisticated or interesting, but a distinction must be made. An example of something that's not really meta or self-referential is the barber paradox:
"There's a barber in the town that shaves everyone who doesn't shave himself. Who shaves the barber?". This is paradoxical because if the barber shaves himself, then by the first statement, the barber cannot does not shave himself. If the barber doesn't shave himself, then by the first statement he shaves himself.
Actually, this is not really a paradox -- there is a flaw in how the first statement is constructed. Let S(i) be a function that assigns to every person i a person that shaves them. Then S(i)=b ^ S(i)=i encodes the first statement. It is meant to be a tautology (since the statement describes something about this hypothetical world that's supposed to be true for everyone in the town). But if we substitute b for i, we get S(b)=b ^ S(b)=b which is false. Hence the first statement cannot be a description of the world. In other words, it is impossible for the barber to shave everyone who doesn't shave himself. It's a little like saying, "The square root of every integer is an integer. What is the square root of 2?" and claiming it's a paradox. In actuality, the first statement is simply false.
In any case, this is not really self-referential; it's just deep. The fact that the barber shaves himself isn't self-referential, but "shaving" becomes reflexive when it's applied to the barber.