User Tools
appendix:guidebook_authoring:benchmarks
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
appendix:guidebook_authoring:benchmarks [2018/08/15 16:47] jguerin Added notes about test isolation. |
appendix:guidebook_authoring:benchmarks [2018/08/15 17:01] (current) jguerin Added information about presentation of results. |
||
|---|---|---|---|
| Line 15: | Line 15: | ||
| ==== Present Results in Seconds ==== | ==== Present Results in Seconds ==== | ||
| + | * Consistency means little chance of misreading. | ||
| + | * Stick to the convention of displaying 3 places after the decimal. | ||
| + | * Avoid results that would be //time limit exceeded// wherever possible. | ||
| + | * 1s is a good cutoff for many purposes, 2s for some others. | ||
| + | * For a single column a '' | ||
| + | * Entire rows can be omitted when warranted. | ||
| + | * Ignore this rule if it will mislead the reader. | ||
| + | * E.g., If the last row is 10< | ||
| + | * Ignore this rule if insufficient data would otherwise be presented. | ||
| + | * A table with a single row is not typically useful. A table with a row at 0s, a row at //almost// 0s, and no other rows is also likely to be misleading. | ||
| ==== Make a Change -> Rerun All Tests ==== | ==== Make a Change -> Rerun All Tests ==== | ||
| + | * If you make //any// changes to a test file, please rerun //all// associated tests. | ||
| + | * In order to make this possible, put a new '' | ||
| + | * Ensure that all associated files are included on this '' | ||
| + | * This duplication is a violates terseness, but is important to guarantee that tests only rely on one page. | ||
| + | * Otherwise (in the event where many benchmarks refer to a single file location) it is impossible to know what to update and keep all results consistent. | ||
| + | * If you do not believe that you have changed results //prove it// rather than assuming that a change would not happen. | ||
| + | * In order for this guide to be usable, benchmarks must be implicitly worthy of trust. | ||
| ==== Make All Efforts to Only Test Your Target ==== | ==== Make All Efforts to Only Test Your Target ==== | ||
appendix/guidebook_authoring/benchmarks.1534369654.txt.gz ยท Last modified: 2018/08/15 16:47 by jguerin
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 4.0 International
