Father [with-と] together is [when-と] cannot-settle.
~と here is the conditional と meaning 'whenever X, then Y' — 'whenever I'm with father, I get restless.' 落ち着かない means 'can't settle down / feel restless.' A revealing sentence about a tense family relationship.