That's not what I'm looking for.

Literal

I [of-の] searching [thing-の] [topic-は] such-thing is-not.

The head noun is elided — 僕の探している+の = 'the thing that I am searching for.' This nominalizing の lets you turn a relative clause into a noun-like unit meaning 'the thing that [clause].' そんなの is a colloquial そんなもの. じゃない is the casual negative of だ.