First off I think cultural relativism while a factors in the Star Trek Philospohy can be shelved in regards to these trials. The UFP won the war so it and its allies get to establish the rules, thus non-combatants are non-combatants despite the Jem'Hadars view on the situation. One thing in the series is the concept of a universal good in a way that can be applied here.
The Federation also seems to pursue a policy of rehabilitation as opposed to pure incarceration as we are familiar with today. So it is this that the Lady Founder will be subjected to and like Owen says if the UFP is successful to any degree that will definitely affect the Great Link when she returns.
As for the insanity defense I have always assumed from the series that the UFP considers all criminals insane from one degree to another so using this defense justs means that instead of a typical penal colony one gets a trip to one of the rehabilitative centers.
So I see special facilities for the Jem'Hadar though the UFP has to address the ketracel-white issue (this is a possible adventure as the PC's ship has to negotiate with the Dominion for ketracel-white for the prisoners being held on trial!).
The Vorta on the other hand will surely get the brunt of the punishments for a host of reasons. Such is the way of justice.