The former castle of the Dukes of Mazovia is a Brick Gothic building, whose origins reach back to the 13th century. In the 14th century the castle was extended by the Polish King Casimir the Great. Medieval remains of the castle that have survived until today are the Szlachecka tower, the clock tower and the western part of the former Benedictine abbey.
The Diocesan Museum is located here now, with precious treasures of Christian art, including Gothic sculptures. The treasure chamber includes the golden jewellery of the diocese and Gothic monstrances.