I wonder whether we can remove him from "only two survived him" without affecting the meaning

Context:

Jefferson Finis Davis (June 3, 1808 – December 6, 1889) was an American statesman and leader of the http://en.wikipedia.org/wiki/Confederate_States_of_America during the http://en.wikipedia.org/wiki/American_Civil_War. Davis was born in Kentucky and grew up on plantations in http://en.wikipedia.org/wiki/Mississippi and http://en.wikipedia.org/wiki/Louisiana. He graduated from the U.S. Military Academy at http://en.wikipedia.org/wiki/West_Point and fought in the http://en.wikipedia.org/wiki/Mexican%E2%80%93American_War as the colonel of a volunteer regiment. He served as the http://en.wikipedia.org/wiki/United_States_Secretary_of_War under http://en.wikipedia.org/wiki/Historyof_the_Democratic_Party(United_States) President http://en.wikipedia.org/wiki/Franklin_Pierce, and as a Democratic U.S. senator from http://en.wikipedia.org/wiki/Mississippi. His plantation in Mississippi depended on slave labor, like most Southern plantations. As a senator, he argued against http://en.wikipedia.org/wiki/Secession_in_the_United_States, but did agree that each state was sovereign and had an unquestionable right to secede from the Union. Davis lost his first wife to malaria after three months of marriage, and the disease almost killed him as well. He had six children with his second wife, but only two survived him. He suffered from ill health for much of his life.

More:
http://en.wikipedia.org/wiki/Jefferson_Davis