There's truth to the thought Fisher can be a good head coach, as long as he stays away from the offense and hires a quality offensive coordinator. What has really made me dislike Fisher is his enormous ego. I'm sure basically every coach has one, but if you aren't Bill Belichick or Nick Saban, you need to at least hide it a little.
Just personality wise. Lanier was an ahole who never got another job. Fisher kept getting hired for some reason.