| Both sides previous revision
Previous revision
Next revision
|
Previous revision
|
main:selectionmodelsearch [2018/02/01 21:57]
bshirley |
main:selectionmodelsearch [2018/02/09 15:57]
bshirley |
| </WRAP> | </WRAP> |
| </WRAP> | </WRAP> |
| | |
| ==== Image Selection Optimization Wizard ==== | ==== Image Selection Optimization Wizard ==== |
| Automated searches for optimal image selection model start from "Image Selection Optimization" wizard. It can be opened from the "Wizards" menu. A step by step guide to the wizard is provided below. | Automated searches for optimal image selection model start from "Image Selection Optimization" wizard. It can be opened from the "Wizards" menu. A step by step guide to the wizard is provided below. |
| | |
| === Introduction === | === Introduction === |
| Before proceeding to the next steps of the wizard, processed samples must be present within the main GUI. The "curve fit residual" evaluation method will require at least 3 samples, the "leave-1-out dose estimation" method at least 4 samples. | Before proceeding to the next steps of the wizard, processed samples must be present within the main GUI. The "curve fit residual" evaluation method will require at least 3 samples, the "leave-1-out dose estimation" method at least 4 samples. |
| | |
| === Select Samples === | === Select Samples === |
| Sample selecting in this wizard is the same as in the "Curve Calibration" wizard. Selected samples will be used to evaluate image selection models during optimization. Enter physical doses of these samples if they are not auto-filled or if they were auto-filled incorrectly. | Sample selecting in this wizard is the same as in the "Curve Calibration" wizard. Selected samples will be used to evaluate image selection models during optimization. Enter physical doses of these samples if they are not auto-filled or if they were auto-filled incorrectly. |
| | |
| === Select an SVM Sigma Value === | === Select an SVM Sigma Value === |
| Select an SVM sigma to use for optimization. It determines dicentric chromosome distributions, which are used to calculate p-values of Poisson fit, and dicentric chromosome frequencies, which are used in the "curve fit residual" method and the "leave-1-out dose estimation" method. | Select an SVM sigma to use for optimization. It determines dicentric chromosome distributions, which are used to calculate p-values of Poisson fit, and dicentric chromosome frequencies, which are used in the "curve fit residual" method and the "leave-1-out dose estimation" method. |
| <WRAP group> | <WRAP group> |
| <WRAP column half> | <WRAP column half> |
| | |
| === Configure Image Selection Model Search === | === Configure Image Selection Model Search === |
| As described in the "Method" section, possible image selection models are logically categorized into 3 groups. In ADCI, users have the option to include or exclude a group in the search of optimal models. Place a checkmark beside groups that are intended to be searched. Leave undesired groups unchecked. Image selection models in checked groups will be generated according to configurations in "Image Selection Optimization Settings". | As described in the "Method" section, possible image selection models are logically categorized into 3 groups. In ADCI, users have the option to include or exclude a group in the search of optimal models. Place a checkmark beside groups that are intended to be searched. Leave undesired groups unchecked. Image selection models in checked groups will be generated according to configurations in "Image Selection Optimization Settings". |
| </WRAP> | </WRAP> |
| </WRAP> | </WRAP> |
| | |
| <WRAP group> | <WRAP group> |
| <WRAP column half> | <WRAP column half> |
| </WRAP> | </WRAP> |
| </WRAP> | </WRAP> |
| | |
| === Summary === | === Summary === |
| Ensure your previous selections are correct on this summary screen. Note values entered on previous screens can be edited by clicking the blue button on the top left of the wizard dialog. Click “Finish” to complete the wizard and bring up the “Optimal Image Selection Model Search” dialog. | Ensure your previous selections are correct on this summary screen. Note values entered on previous screens can be edited by clicking the blue button on the top left of the wizard dialog. Click “Finish” to complete the wizard and bring up the “Optimal Image Selection Model Search” dialog. |
| | |
| ==== Optimal Image Selection Model Dialog ==== | ==== Optimal Image Selection Model Dialog ==== |
| <WRAP group> | |
| <WRAP> | |
| The automated search for optimal image selection models is performed in this dialog. A summary, including model generation configuration and evaluation method and samples, is shown in the top part. Users can check if everything is correct. | The automated search for optimal image selection models is performed in this dialog. A summary, including model generation configuration and evaluation method and samples, is shown in the top part. Users can check if everything is correct. |
| |
| |
| "Report" functionality will be implemented in future version of ADCI. | "Report" functionality will be implemented in future version of ADCI. |
| </WRAP> | |
| <WRAP> | <WRAP group> |
| | <WRAP column half> |
| {{ :file:wizard:optimalsearchdialog.png?nolink|}} | {{ :file:wizard:optimalsearchdialog.png?nolink|}} |
| | </WRAP> |
| | <WRAP column half> |
| | {{ :file:wizard:optimalsearchdialogplot.png?nolink|}} |
| </WRAP> | </WRAP> |
| </WRAP> | </WRAP> |
| |
| |
| ===== Searching Time ===== | ===== Time required to perform a search ===== |
| The time needed to perform an image selection model optimization search depends on the number of models being searched. If the search space is relatively large, it may take hours to finish the optimization. The time required to perform a search can be estimated for certain evaluating samples. Doing so is encouraged in order to help plan a convenient time to start the search. | ==== Influences on search runtime ==== |
| | The time required to perform a search depends on the number of models being generated/examined. The number of models examined is determined by the selected image selection model categories to search (image filtering, combined z score, group bin) as well as settings specified in "Settings" -> "Image Selection Optimization Settings". If the search space is large, it may take hours to finish the optimization. |
| Before performing a potentially lengthy search, a small test search can be performed to get a sense of the time required to search each model. Start the wizard, check the first box 'Search image selection models applying image filtering' on the page 'Configure Image Selection Model Searching', and leave the other two choices unchecked. By doing this, we limit the number of models to be searched. On the page 'Select an Evaluation Method for Optimal Image Selection Model Search', choose 'p-value of Poisson' method or 'curve fit residual' method. Proceed to the 'Optimal Image Selection Model Search' dialog. | |
| |
| Here, observe the current number of "generated models" and make sure it is no larger than 4000-5000. If the default settings for filter thresholds are used, the number will be 1152. Start the search and wait for it to finish. The dialog will show the time elapsed (in minutes) above the progress bar. This is the amount of time needed to finish the search for the listed number of generated models. When users are searching more image selection models for the same samples, the time required is proportional to the time spent in this small test search. | Search runtime is proportional to the number of evaluating samples. Evaluation methods also influence the search execution time. The 'p-value of Poisson' and 'curve fit residual' methods take approximately the same amount of time. The 'Leave-one-out' method takes longer, and is proportional to the number of evaluating samples. For example, to finish a search using 10 evaluating samples, 'leave-one-out' will take approximately 10-times longer than either 'curve fit residual' or 'p-value of Poisson'. |
| |
| The time required in a search is also proportional to the number of evaluating samples. If the number of evaluating samples in a real search is twice the number of samples in the test search, the real search is expected to take twice as long. | ==== Test to estimate required search time ==== |
| | **This feature is present in ADCI v1.9 or newer**\\ |
| | After evaluating samples and parameters have been selected, click the button "Click to check" adjacent to the label "Time to finish: ". A quick test will be performed (approximately 10 seconds), after which the estimated time necessary to perform the full search is displayed. |
| |
| Finally, the 'p-value of Poisson' and 'curve fit residual' methods take approximately the same amount of time. The 'Leave-one-out' method takes longer, and is proportional to the number of evaluating samples. For example, to finish a search of 10 evaluating samples, it will be 10-time longer for 'leave-one-out' compared to 'curve fit residual' or 'p-value of Poisson'. | |
