You say I should know him well, but actually I was only just introduced to him last week.
Literal
I [subject-が] him [object-を] well know-should [quotative-だと] you [topic-は] say [but-が], actually I [topic-は] last-week him [to-に] just introduced-passive [explanatory-のだ].
A complex contrastive sentence. はず expresses expectation or should-ness — 'should know.' 実際は ('actually, in reality') contrasts with the prior claim. 紹介されたばかり = '[was] just introduced' — ばかり after a past-tense verb means 'just (recently) [verbed].' Passive voice 紹介された places the speaker as the recipient of the introduction. A polished correction of a wrong assumption.