| Both sides previous revision
Previous revision
Next revision
|
Previous revision
Last revision
Both sides next revision
|
main:selectionmodelsearch [2018/02/01 22:20]
bshirley |
main:selectionmodelsearch [2018/02/09 15:56]
bshirley |
| "Report" functionality will be implemented in future version of ADCI. | "Report" functionality will be implemented in future version of ADCI. |
| |
| | <WRAP group> |
| | <WRAP column half> |
| {{ :file:wizard:optimalsearchdialog.png?nolink|}} | {{ :file:wizard:optimalsearchdialog.png?nolink|}} |
| | </WRAP> |
| | <WRAP column half> |
| | {{ :file:wizard:optimalsearchdialogplot.png?nolink|}} |
| | </WRAP> |
| | </WRAP> |
| |
| ===== Estimate time required to perform a search ===== | |
| The time needed to perform an image selection model optimization search depends on the number of models being searched. If the search space is relatively large, it may take hours to finish the optimization. The time required to perform a search can be estimated for certain evaluating samples. Doing so is encouraged in order to help plan a convenient time to start the search. | |
| |
| Before performing a potentially lengthy search, a small test search can be performed to get a sense of the time required to search each model. Start the wizard, check the first box 'Search image selection models applying image filtering' on the page 'Configure Image Selection Model Searching', and leave the other two choices unchecked. By doing this, we limit the number of models to be searched. On the page 'Select an Evaluation Method for Optimal Image Selection Model Search', choose 'p-value of Poisson' method or 'curve fit residual' method. Proceed to the 'Optimal Image Selection Model Search' dialog. | ===== Time required to perform a search ===== |
| | ==== Influences on search runtime ==== |
| | The time required to perform a search depends on the number of models being generated/examined. The number of models examined is determined by the selected image selection model categories to search (image filtering, combined z score, group bin) as well as settings specified in "Settings" -> "Image Selection Optimization Settings". If the search space is large, it may take hours to finish the optimization. |
| |
| Here, observe the current number of "generated models" and make sure it is no larger than 4000-5000. If the default settings for filter thresholds are used, the number will be 1152. Start the search and wait for it to finish. The dialog will show the time elapsed (in minutes) above the progress bar. This is the amount of time needed to finish the search for the listed number of generated models. When users are searching more image selection models for the same samples, the time required is proportional to the time spent in this small test search. | Search runtime is proportional to the number of evaluating samples. Evaluation methods also influence the search execution time. The 'p-value of Poisson' and 'curve fit residual' methods take approximately the same amount of time. The 'Leave-one-out' method takes longer, and is proportional to the number of evaluating samples. For example, to finish a search using 10 evaluating samples, 'leave-one-out' will take approximately 10-times longer than either 'curve fit residual' or 'p-value of Poisson'. |
| |
| The time required in a search is also proportional to the number of evaluating samples. If the number of evaluating samples in a real search is twice the number of samples in the test search, the real search is expected to take twice as long. | ==== Test to estimate required search time ==== |
| | After evaluating samples and parameters have been selected, click the button "Click to check" adjacent to the label "Time to finish: ". A quick test will be performed (approximately 10 seconds), after which the estimated time necessary to perform the full search is displayed. |
| |
| Finally, the 'p-value of Poisson' and 'curve fit residual' methods take approximately the same amount of time. The 'Leave-one-out' method takes longer, and is proportional to the number of evaluating samples. For example, to finish a search using 10 evaluating samples, 'leave-one-out' will take approximately 10-times longer than either 'curve fit residual' or 'p-value of Poisson'. | |
