Home

Various approaches have used neural networks as probabilistic models for the design of protein sequences. These “inverse folding” models employ different objective functions, which come with trade-offs that have not been assessed in detail before. This study introduces probabilistic definitions of protein stability and conformational specificity and demonstrates the relationship between these chemical properties and the p(structure|seq) Boltzmann probability objective. This links the Boltzmann probability objective function to experimentally verifiable outcomes. We propose a novel sequence decoding algorithm, referred to as “BayesDesign”, that leverages Bayes’ Rule to maximize the p(structure|seq) objective instead of the p(seq|structure) objective common in inverse folding models. The efficacy of BayesDesign is evaluated in the context of two protein model systems, the NanoLuc enzyme and the WW structural motif. Both BayesDesign and the baseline ProteinMPNN algorithm increase the thermostability of NanoLuc and increase the conformational specificity of WW. The possible sources of error in the model are analyzed.

Objectives: Evaluate the gland-level annotations in the PANDA Dataset and provide specific recommendations for the development of an improved prostate adenocarcinoma dataset. Provide insight into why currently developed artificial intelligence (AI) algorithms designed for automatic prostate adenocarcinoma detection have failed to be clinically applicable.

Methods: A neural network model was trained on 5009 Whole Slide Images (WSIs) from PANDA. One expert pathologist repeatedly performed gland-level annotations on 50 PANDA WSIs to create a test set and estimate an intra-pathologist variability value. Dataset labels, expert annotations, and model predictions were compared and analyzed.

Results: We found an intra-pathologist accuracy of 0.83 and Prevalence-Adjusted Bias-Adjusted Kappa (kappa) value of 0.65. The model predictions and dataset labels comparison yielded 0.82 accuracy and 0.64 kappa. The model predictions and dataset labels showed low concordance with the expert pathologist.

Conclusions: Simple AI models trained on PANDA achieve accuracies comparable to intra-pathologist accuracies. Due to variability within or between pathologists these models will unlikely find clinically application. A shift in dataset curation must take place. We urge for the creation of a dataset with multiple annotations from a group of experts. This will enable AI models, trained on this dataset, to produce panel opinions which augment pathological decision making.

Context.—

Automated prostate cancer detection using machine learning technology has led to speculation that pathologists will soon be replaced by algorithms. This review covers the development of machine learning algorithms and their reported effectiveness specific to prostate cancer detection and Gleason grading.

Objective.—

To examine current algorithms regarding their accuracy and classification abilities. We provide a general explanation of the technology and how it is being used in clinical practice. The challenges to the application of machine learning algorithms in clinical practice are also discussed.

Data Sources.—

The literature for this review was identified and collected using a systematic search. Criteria were established prior to the sorting process to effectively direct the selection of studies. A 4-point system was implemented to rank the papers according to their relevancy. For papers accepted as relevant to our metrics, all cited and citing studies were also reviewed. Studies were then categorized based on whether they implemented binary or multi-class classification methods. Data were extracted from papers that contained accuracy, area under the curve (AUC), or κ values in the context of prostate cancer detection. The results were visually summarized to present accuracy trends between classification abilities.

Conclusions.—

It is more difficult to achieve high accuracy metrics for multiclassification tasks than for binary tasks. The clinical implementation of an algorithm that can assign a Gleason grade to clinical whole slide images (WSIs) remains elusive. Machine learning technology is currently not able to replace pathologists but can serve as an important safeguard against misdiagnosis.