The post office is just down that street.

Literal

Post-office [topic-は] that street [object-を] descend-[past-った] place is-[polite-です].

The construction [verb-た]ところ ('a place where X has been done') is a useful structural pattern. 通りを下る ('to go down the street') uses を for the path and 下る ('to descend') in its horizontal sense — streets are conventionally described as 'going up' (toward the center) or 'going down' (away from center), not necessarily in altitude.