Home

Welcome to the AustinCEMBenchmarks wiki!

The benchmark suites in this repository aim to provide information about the state-of-the-art in electromagnetic simulation methods.

As a result of decades long research and development efforts in algorithms, software implementations, and advances in computer hardware/software infrastructure, today, a large (and expanding) set of computational methods are available for performing electromagnetic simulations. Indeed, various commercial, proprietary, and academic simulation tools currently rely on implementations of these computational methods. It is becoming more and more difficult to identify the “best” simulation method among alternatives to solve a problem of interest or to determine “how much better” one method is over others because

• a large and increasing number of competitive methods can be used
• underlying computer hardware/software infrastructure continues to evolve rapidly
• a high level of expertise is needed to apply specialized methods effectively
• methods are often evaluated primarily by their developers, who become judge, jury, and executioner of their own work; such assessments are prone to intentional or unintentional biases and over-optimistic performance estimates.

As a result, there is an increasing risk that simulation methods will be judged primarily on subjective factors (e.g., generality, simplicity, familiarity/popularity, or user friendliness) rather than objective scientific/engineering merits (e.g., accuracy, efficiency, scalability).

It is our contention that publicly available verification, validation, and performance benchmarks can help

• systematically combat the problem of the ubiquity of error
• inform researchers in the field and the public about the state of the art
• lower barriers to entry for new researchers/methods
• reduce importance of subjective factors when judging simulation methods
• increase the credibility of the results obtained and claims made by computational scientists and engineers.