FOUR MAIN REASONS IMO.
1. The youth has lost touch with history. This is with all history. We all know the American greatness is dissolving in every way. Therefore our ability to be educated about all aspects of history is in peril. No one notices the transition into ignorance though. Money and materialism is more important than knowledge. The majority of the youth is distracted. It's like all kids now days have A.D.D. 2. The shit Menace was saying, the real shit is suppressed (due to reasons pertaining to the above).
3. There are no new major inspirations. That's why all the respectable artists are just spinning their wheels. They have no one to look to as an innovator (where the innovation actually worked). They're also simply frustrated.
My theory is give a guy a microphone and raise him without hip hop. Teach him the science of hip hop and then step away, see what he creates. Of course this isn't possible, but it's a for sure solution however impossible it is. That's really the only way we will get a mutant hip hop (beings it isn't evolving). Because current artists can't imagine out of their current realm. When we go to make a song, we think of other songs and many struggle with not borrowing from the songs they are thinking of. The film industry is suffering from the same problems. We suck the tittie of the past. That's why there is no progression.
I agree this shit needs to get more complex (but this isn't possible due to the reasons above^).
4. There are millions of topics to write about, yet it is always the same recycled shit. The artists now basically need to set their egos aside and find themselves, become in touch with themselves and just let the shit flow. Once you understand yourself you then truly grow. It has to do with self knowledge which is one of the biggest struggles that many face, gaining self knowledge. But after self knowledge, there is world knowledge. Most decent rappers are stuck on self knowledge and haven't yet attained world knowledge to the point where it is coherant in their music. I've always thought that Plato, Aristotle, Petrarch, Makiavelli, etc. would make good rappers. I'm rambling on here now, so basically what we need is artists deep in philosophy, psychology, history, and art (writing, poetry, storytelling), all supported by concrete logic, reason, and rational thinking and who set aside egos in order to focus on ^^^^^^^.
The fall of hip hop is basically one of many fallen practices. Just look at the world. Almost all forms of all activities (especially creative related) have fallen to cliches, platitudes, formulas, and simplicity (because complexity and unorthodox methods do not register in the common person's mind, so it is set aside for the generation of $$$$$$$$$$$ instead). It's not just hip hop, it's everything. There is no scale for this, because the entire scale is tipping. I'm telling you we're de-evolving!