A Dystopia refers to a fictional or theoretical world in which the standard of human living has become so bad that it is often nightmarish - it is the exact opposite of a Utopia and has been used in numerous works of fiction with media such as "Big Brother" and "Animal Farm" being amongst the most well-known.
A Dystopia may resemble a Utopia at first (known as a "False Utopia") or may show little or no sign of hiding its true nature - in fantasy it is common for Dark Lords to proudly display their kingdom or empire as a dark and horrific realm, while other dictators try to disguise their cruelty with the use of grand statues or seemingly benevolent acts that hide the true horrors at hand.
Dystopias are most often seen as dictatorships but they can also be anarchies in which gangs and whatnot rule supreme (as popularised in Mad Max) - variants of Dystopia can even have non-human antagonists (such as Planet of The Apes) though this may overlap somewhat with the Alien Invasion scenario.