The palace was built on the initiative of Dominik Michał Słuszko (ca. 1655-1713), the last of the family, an adventurer famous throughout Lithuania and an ardent member of the Sapieha faction (until the defeat at Olkieniki), who married Konstancja of Podberesskė in 1686 (d. 1722).

The residence was built in Antokol, directly on the bank of the Neris River, organising a geometrical garden with green and water areas on the western side, i.e. from the city, and a courtyard with farm buildings on the eastern side. The residence was thus the only one in Vilnius to realise the scheme entre cour et jardin (between courtyard and garden), which was already well established in the Republic at that time.

Like the Sapieha residence in Antokol, the Słuszko palace was not built on raw roots. Previously, the Kishkas' estates granted to them by Stefan Batory had been located here, with a walled residence organised in the area then known as the New Town. On 17 November 1645, thanks to King Władysław IV, they passed into the hands of his favourite, Feliks Jan Pac (after 1615-1702). It was the Pacs who sold these estates to Słuszko.

The construction of the Słuszczański Palace seems to have been led by Giovanni Battista Frediani, who had built the Sapieha Palace and the Trinitarian temple in Antokol, and who had earlier supervised the construction of the Church of Sts Peter and Paul, planning its façade. He was certainly also the author of the design, which is a transformed scheme of the nearby Sapieha residence based on the so-called Poggio Reale variant, fashionable throughout Europe at the time. Doubtless, as with the Sapiehs, he incorporated the old Kiszczan-Pac walls into the new structure.

The basement palace of the Słuszko family was erected on an east-west axis as a two-storey (now four-storey) building on a rectangular plan with four tower annexes, higher than the main body by a mezzanine, next to which auxiliary staircases were organised. The western elevation, i.e. on the city side, was enriched with two-storey arcade-pillar loggias (walled up after 1833) overlooking the garden and the river, but also the castle with the cathedral and the city spread out on both banks of the Neris. The facades are framed by pilasters in the great order, with plain shafts and Ionic heads with festoons, supporting the beam, some with painted detailing.

The main representative hall was located on the first floor. It was preceded by a vestibule with a cupboard room, to which a parade staircase led from the ground floor. The tower annexes contained the bedrooms and offices of the master and mistress of the house.

In the façade, which opens onto the Antokolian hills, above the entrance was originally a plaque (lost) with an inscription in Latin, which translated into Polish read as follows: "I have removed the mountains, I have tamed the waters of the Neris, I have surpassed the hills without soaring, victor of the elements, I have become a house of rest, an Antokolian stately companion of heroes. Here under the Luna (Moon) [I am] a friend of rest, may the goddess of peace lay down her toga, may she lay down her armour for the peace of life under the Ostoja Bellona coat of arms." Above, in the pediment with stucco decoration made by an unknown workshop, the signs of the palace's owners are shown against a panoply: Ostoja Sluszka and Gozdawa Podbereski. They are embraced by figures of Chronos and a personification of the Ostoja coat of arms, which is a contamination of Bellona and Luna. Above them is the figure of Fortuna. The content of the inscription panel is the motto for the palace and the decoration of the pediment. It recalls Horatian ideals, but also shows Słuszko - the voivode of Vilnius and colonel-general of the Grand Duchy of Lithuania who took part in the Hungarian and Moldavian campaigns of John III - as a man who adhered to the principle of pax et bellum. This was one of the main political categories and appeared extremely frequently in 17th century literature and art, combining two seemingly mutually exclusive extremes. It was an ideal that promoted the service of the homeland in times of peace and war, which is precisely what the inscription of the Słuszek palace indicates. Importantly, the ideological sublimity of the residence, which Słuszka built by 'fighting' the elements of water and earth, was reinforced by the very scheme of the palace with its towers referring to the topos of chivalry, but also by the surrounding landscape - the river and the view of Vilnius, and the content was complemented by a chapel-eremitory standing in the garden.

It is important to emphasise the unusual location of the residence directly on the bank of the river with a garden laid out parallel to its course. It was composed as a geometrical, axial garden, but due to the narrow and elongated plot - with a single avenue. Behind the accommodation part, on the town side, there was a water parterre with a canal reaching the river, where the above-mentioned chapel was situated nearby. With this layout, the Neris became a component of the garden layout, flowing parallel to the delimited avenue and artificial parterre, which is unique in the Republic.

After the death of Dominik Michał Słuszko, the residence passed from hand to hand, increasingly falling into ruin. During the Kosciuszko Uprising, the palace housed headquarters, poured bullets and collected weapons. Following the collapse of the liberation uprising, the residence was taken over by the Russian government. In 1831, the building was incorporated into a fortress being organised in Vilnius. A hospital was placed in the former magnate's residence, then barracks, and in 1869 the palace was converted into a prison. In 1957-1959, renovations were undertaken to adapt the building for educational purposes. Today, the palace houses the Lietuvos muzikos ir teatro akademija.