The former warden will be temporarily placed at the bureau's Northeast Regional Office and the warden of the Otisville, New York, federal prison, James Petrucci, will serve as acting warden of the Metropolitan Correctional Center, where Epstein was being held.
Amid revelations about Jeffrey Epstein's death, authorities have intensified inquiries into what went wrong at the jail and who could still face charges in his case.
"Epstein's death must not deprive his victims of justice", the ministers said.
Epstein, accused of paying underage girls hundreds of dollars in cash for massages and then sexually abusing them at various locations, was shrouded in mystery.
In an interview with CBS West Palm Beach affiliate WPEC, a former Florida corrections worker said Epstein was "treated like a celebrity" during his 13-month stint in a county jail after he pleaded guilty in 2008 to charges of soliciting a minor for prostitution.
Before he took his own life, Epstein has been taken off suicide watch, according to a person familiar with the matter who spoke to The Associated Press. The New York City medical examiner wasn't able to determine his official cause of death and says that a determination is still pending. A private pathologist observed the examination, as well. But investigators have learned those checks weren't done for several hours before Epstein was found unresponsive, according to a person familiar with the episode.
Epstein served just 13 months in jail over the Florida charges a decade ago under a plea deal struck by the then federal prosecutor in the state, Alex Acosta, who was forced to resign as USA labour secretary last month over the issue.
He was arrested for a second time in July 2019 for alleged sex trafficking of underage girls at his mansion and faced up to 45 years in prison.
More details about exactly how Jeffrey Epstein committed suicide in jail are just raising more questions about how such an important prisoner was allowed the opportunity to kill himself.
With Epstein gone, potential co-conspirators involved in his alleged sex-trafficking network are shifting into focus.
Other politicians also took to social media to question the circumstances.
She declined to say which one or specify the person's regular role. The French government called for prosecutors to open an investigation into Epstein's links to France, with the secretaries of state for women's rights and protecting children releasing a statement Monday.