bigbio / bigbio/quantms

Benchmarking ms2rscore (deeplc, ms2pip), quantms-rescoring in quantms.

Open
#410 13 comments 0 reactions 3 assignees Claimed by @ypriverol View on GitHub
enhancement high-priority research-projects
Dominant language
Nextflow
Stars
83
Forks
57
Avg merge
4h 14m
Merged PRs (30d)
1

Description

We recently reimplemented quantms-rescoring to generate the features for MS2PIP and DeepLC for quantms. This issue is to trace the benchmark performance within quantms and the gold-standard datasets we use. We should benchmark the following:

- [ ] ms2rescore and multiple search engine search SAGE, MSGF+ and COMET
- [ ] ms2rescore and multiple types of experiments and analytical methods (some of the tests could be combined):
- [ ] DDA-LFQ
- [ ] TMT
- [ ] inmunopeptidomics
- [ ] Low resolution, high resolution
- [ ] CID and HCD
- [ ] phospho-proteomics
- [ ] num_hits parameter, now that we have the ms2rescore, we can try to enable by default `num_hits == 3` (https://github.com/bigbio/quantms/issues/344) to see if we can recover rank 2 PSMs.

Here some of the previous efforts.

### PXD001819 Analysis

Currently, we have a workflow that can perform peptide identification using: `-> ms2rescore -> SNR + spectrum properties -> percolator`

Here the results can be found (https://ftp.pride.ebi.ac.uk/pub/databases/pride/resources/proteomes/quantms-benchmark/PXD001819-id-ms2rescore/).

## Total number of PMSs

Comet only + Percolator: 495306
Comet + MSGF + Percolator: 572496 (15.58% increase)
Comet + MSGF + ms2rescore: 589200 (18.95% increase)
Comet + MSGF + (SNR + ms2rescore): 587972 (18.71% increase)
Comet + MSGF + SAGE + (SNR + ms2rescore): 592918 (19.68% increase)

![psm_tools_plot](https://github.com/user-attachments/assets/0ccc875e-4b9b-4e12-89f2-f3aa3de64887)

## Total number of PSMs by RAW file and combination

![psms_by_file_and_tool](https://github.com/user-attachments/assets/3be887c5-00aa-4953-949d-7190e487bc5c)

> Currently, the combination of ms2rescore alone has more PSMs identifications, followed by ms2rescore + SNR.

The following questions would be interesting to understand:

- When the spectrum quality metrics are introduced, are the PSMs more high-quality meaning that while we have fewer PSMs for ms2rescore + SNR they have more quality than ms2rescore?
- Do we see the same results in other datasets?
- What is the impact at peptide level?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.