Over the centuries, the small Idolta belonged to the Massalsky and Sapieha dukes and to the Miłosz family. A separate manor farm was established here in the 19th century and was owned by Józef Miłosz, then by his son Eugeniusz (d. 1885), married to Emilia née Targońskie. Subsequent owners were their children. Eugeniusz (d. 1908), Józef (d. 1914) and Jadwiga (née Iżycka). Both brothers lived in Drua, while their sister stayed in Idolt. She lived alone; she supposedly got rid of her husband shortly after her marriage. In 1909. Emilia (Józef's daughter) sold the Idolt estate to the Świderski family, who owned it until 1939.

The manor complex lies near the river of the same name. The manor house was built in the second half of the 19th century, it was erected on a rectangular plan as a brick, one-storey building with a wooden central part, topped with a triangular pediment with an oculus. It is fronted by a portico with three arcades, above which is a balcony. The façade is accentuated by a central risalit.

The interior of the mansion once housed the Miłosz family portraits and a library, but these collections did not survive the historical storms. After the end of World War II, the manor house became the seat of the kolkhoz administration and the estate belonged to the kolkhoz. In 2005-2008, the building was renovated to house a library and a village club. An outbuilding and farm buildings made of fieldstone have also been preserved in the vicinity of the Milosz residence. The whole is surrounded by the remains of the manor park with beautiful old trees.

A former chapel of the Miłosz family, built in 1862, has been preserved on a small peninsula on the lake, also called Idołta. The building was devastated after World War II. At the beginning of the 21st century, the chapel was repaired and returned to the faithful. Since then, it has been used by Catholics as a branch church of St John the Baptist in the Parish of Our Lady of the Scapular.

The church has a rectangular nave with a pentagonal chancel and two sacristies. The façade is crowned by a triangular pediment, above which rises a small wooden quadrilateral turret. Founder Eugeniusz Miłosz and his wife Emilia, née Targońskie, as well as their children Eugeniusz, Józef and Jadwiga, and Józef's wife Maria, née Kowalewska, are buried in the vaults. Some members of the Miłosz family were buried in the small cemetery surrounding the chapel, but this was destroyed after 1945. Only the tombstone of Konrad Miłosz, who died in 1852, has survived.