Looking at the reconstructed after World War II edifice of the palace of Brzostowski, it is difficult to see the fact that it was considered one of the most elegant residences in Vilnius around 1800. Its history is extremely interesting and dates back to the beginnings of German settlement in this part of Vilnius.

The Pociej (Brzostowski) Palace is located at the back of the Franciscan Church and the former cemetery at the Church of the Blessed Virgin Mary. It was adjacent to what was known in the 17th century as the old Woyn palace, or later the Tyzenhauz (Vittinghoff) palace, as well as to the church and buildings belonging to the Lutheran community, but also to the quarter occupied by the Jews.

The history of the seat dates back to the Middle Ages, when a brick building was erected on the site in the 14th century. At the end of the 16th century, the estate belonged to Jan Hlebowicz (d. 1590), governor of Trakai (1586-1590), treasurer and scribe of Lithuania (1580-1586), a Calvinist. He was succeeded by his eldest son Jan (d. c. 1604), and after his death by his younger son Mikolaj (d. 1632), future Smolensk Voivode (1611) and Vilnius Castellan (1621). From the descriptions of the time, it appears that the palace was not only spacious, but also richly furnished. Downstairs there were mainly utility rooms, including rooms for rent (e.g. a pharmacy), but also a treasury (from the courtyard, in the west wing). The representative rooms: the two dining rooms and the two "great rooms", were located in the west wing and in the body facing the street. There was a "brass" candlestick for 20 candles in the "great dining room" decorated entirely with fabrics (the so-called "stiles"), and a candlestick with a deer antler in the great hallway.

The master's suite was organised in the west wing, with an amphitheatre containing a 'chamber of rooms' preceding the bedroom, combined with a study and a library/archive. The suite of the lady of the house - Catherine of Krotoski - was situated on the first floor of the east wing, connecting with the kitchen through a porch. Tiled cookers stood in all rooms, including those decorated with the Hlebowicz coat of arms. At the edges of both flats, toilets were organised for the palace residents.

In the cobbled courtyard was an outbuilding with servants' flats, and further a kitchen, a bakery and brewery, as well as a bathhouse and two stables. A separate space in the south-western part of the plot was reserved for a garden of a utilitarian nature. In 1699 and 1701 a tax was collected from the palace to the municipal treasury for the provision of piped water. The 1609 inventory of the palace mentions, among other things, eight wooden pipes 'for shooting water', which were located in one of the utility rooms.

In 1636 the residence was described: "in all things prosperous" with a spacious courtyard extending to the Franciscan monastery. During the August procession with St Casimir's coffin through the streets of the city, one of the triumphal gates was organised next to the palace of the Hlebowicz family, by then already Catholics. This testifies to the importance of the owners and their residence in the landscape of Vilnius, as does the fact that kings visiting the Lithuanian capital have visited them.

During the Moscow occupation, the palace was severely damaged. Its owner at the time was Mikalojus Hlebovych's son, Jurgis Karol (1603-1669), after whose death the real estate fell into the hands of his widow Catherine (1614-1674), daughter of Kryštof II Radziwill, an ardent Reformed evangelical. She relinquished the palace to her daughters: Anna Marcybella (d. 1681/2), married to Marcjan Aleksander Ogiński (1632-1690), and Krystyna Barbara (d. 1695), wife of Kazimierz Jan Sapieha. Finally, in 1679, the palace fell into the hands of the Oginskis, after whom it was briefly held by Vitebsk voivode Leonard Gabriel Pociej (1635-1695), married to Regina Ludwika of Oginskis. Later their son, Lithuanian Hetman Ludvik Konstantin (1664-1730), became the owner of the palace, hence the residence was called "Hetman's Palace".

The next owner was Ludvik Konstanty's nephew, Antoni (d. 1749), a Lithuanian guardian, whom the uncle "bred at home from a young age and paid a considerable amount of money to other countries". However, the seat's glory days were already gone by then. By the early 1740s, the palace was derelict and neglected, but it was also rented out to craftsmen's lodgings, some of whose tenants were Jews.

It is possible that the seat, together with the indebted Brest estate of Pociej, was in the hands of Jerzy Detloff Flemming (1699-1771), Lithuanian treasurer from 1746. The palace burnt down in 1748 and in 1749, but the owner repaired it, as he regularly gave excellent balls there. At the latest in 1762 Detloff Flemming sold first the palace and in 1764 the office of the Lithuanian treasurer to Michal Brzostowski (1722-1784). He was succeeded by his daughter Barbara (d. 1792), married to another representative of the family and her father's namesake, Michał of Brzostów Brzostowski (d. 1800).

At the beginning of the 19th century, the two-storey palace became the property of the wealthy bourgeois Müller family, originally from Lviv. They raised the building one storey and gave a new look to its façade. In 1806, the palace had 53 large rooms and 20 small ones. The stables were for 12 horses and the coach house for 4 vehicles. At that time, the palace, and especially its great hall, was rented for various concerts, balls, but also for meetings of the noble resursa. In 1840-1858 one of the flats was occupied by Stanislaw Moniuszko (1819-1872), organist at the nearby St Catherine's Benedictine Church. The palace was heavily damaged in 1944, rebuilt and now houses the headquarters of Swedbank.