You must not look down on him just because he's poor.
Literal
Poor [because-から] [called-という] only [by-だけで] him [object-を] despise [must-not-てはいけない].
Uses 軽べつ, the partially-hiragana spelling of 軽蔑. からというだけで is the expanded form emphasizing 'merely for the reason that.' The specific target (彼) makes this more personal than the general versions.