There are two historical references to Peter's wife. The first are church records recording the birth and baptism of his first three children and his wife's name is recorded as Gertrude. In 1795, there is a record of a land sale by Peter Flory and wife, Christina.

With respect to his children, there is a 14 year gap between the birth of Peter's third and fourth children. The first three children were born over a four year period from 1786 to 1790, while the last five (all sons) were born over an eight year period starting in 1804.

Taken together, this suggests that Peter probably had two wives. His first wife Gertrude probably died between 1790 (the year the third child was born) and 1795, since the land records record his wife's name as Christina. The census records suggests that Peter and Christina probably had at least two daughters, although their names are unknown.