The palace, located at Vilnius Market Square, in the immediate vicinity of St Casimir's Church, was built in 1782-1786 to a design by Augustyn Kossakowski for the Abramowicz family of the Jastrzębiec (var.) coat of arms. Before 1749, Andrzej Abramowicz, a Lithuanian scribe and castellan of Brzeskolitza, related to the Radziwills, purchased the estate in the central part of Vilnius, and was inherited by his childless brother Jerzy (d. 1767), starosta of Starodubov and stolnik of Vilnius, an active sejmik actor. The palace was built by the enterprising and enterprising wife of George, Marcjanna of Dernałowicz, who had "practised on more than one wall".

Before the Abramovichs' palace was built, it was decided to demolish the earlier 17th century buildings standing on the plot, so that the new one would have a regular and symmetrical plan. The foundations were laid in April 1783, and by the autumn of the following year, floor plans were already being drawn up. Augustyn Kossakowski designed a basement residence, situated on an east-west axis in a frontage building line, on a U-shaped plan, with a staircase in the southern part. Before 1797, the short sections of the side wings were enlarged and enclosed on the eastern side by an outbuilding on a roughly triangular plan and a walled enclosure. The body of the palace was originally two-storey with a mezzanine, double-arched, on the ground floor in a cross-corridor scheme, with the entrance gate on the axis, covered by a gabled, broken roof. The wings were single-arched. The facade of the palace, almost 32 m wide, had a Corinthian pseudo-portico. The storeys were separated by panels and the mezzanine was separated by a rich acanthus frieze.

In 1801-1806, the palace was modernised at the behest of Nikolai Abramovich (1788-1835), grandson of the first investors George and Martianna, Knight of the Officer's Cross of the Legion of Honour, Chamberlain of the Russian court and Marshal of the nobility of the Vilnius district, who ".In order not to idle, he embroidered slippers on canvas for his lovely wife [Emilia, née Bachmińska (1768-1844)], or sewed baskets with the art of saddlery,

He was well acquainted with artistic fashions and was renowned for his homemaking skills and excellent taste. The Vilnius palace, like Nikolai Abramovich's other residence, Vorniany, was furnished with splendour but tastefully. In both residences the Marshal ran an open house. The tax census for 1806, the year of the completion of the renovation, which mainly covered the interior, stated that the palace had 22 larger and 11 smaller rooms. The stables were for 15 horses and the coach house for four vehicles. However, after returning home from the Napoleonic wars, in 1814. Nicholas Abramovich decided to sell the palace. He found its buyer in the person of Antoni Wańkowicz.

In 1844, the building became the property of the Orthodox consistory. In 1853-1860, under the direction of Tomasz Tyszecki (listed 1824-1861), a renowned architect-composer who served as the eparchial architect, reconstruction of the edifice was undertaken, incorporating it into the structure of the St Casimir's Church - then an Orthodox church. A gable was introduced in the façade and the shape of the roof was changed. The Corinthian columns were replaced by Ionic columns and their plinths were significantly lowered. In addition, Ionic pilasters of great order appeared in the façade and in the façade facing St Casimir's Church. This is also where the window openings were pierced. The changes to the northern façade resulted from the demolition of the front part of the Jesuit building, which made it possible to open up the square in front of the church (orthodox church) directly onto the street, but also exposed the north wall of the palace, which had been hidden until then. Hence, it became necessary to develop it.

In addition, during the reconstruction carried out by Tyszecki, the mezzanine was framed by pilasters with a modified head with cannels. On the axis of the façade, the architect introduced floral garlands, and in the portico, rosettes and lion heads. The façade of the palace thus lost its austerity in favour of a decorativeness better suited to the tastes of the mid-19th century.

In 1914-1915, the Abramovichs' palace was the residence of the Lithuanian Orthodox Archbishop Tikhon (Vasily Belavin, 1865-1925), who was elected Patriarch of Moscow in 1917 and later became a saint of the Orthodox Church. Between the wars, the building belonged to the Jesuits, and was rented for flats, a school (mezzanine), a cinema (south wing), offices, and a slaughterhouse (south wing), among other uses. The palace suffered little damage during World War II, and after the end of the war it housed various institutions and services. It is currently abandoned, waiting for a new host.