toctree breaks section hierarchy for latexpdf

Describe the bug

I have three documents from which I create my documentation (they should be its chapters): Introduction (my README from github), Manual (it is in Markdown, because it was easier for me to create a man page from that) and Details/Advanced. I join them from index.rst.

I want to have both html and pdf documentation. For html the documentation for my project looks good, and links hierarchy in the toctree is correct.

Unfortunately, for make latexpdf section numbering is broken: the manual is grabbed from toctree and inserted as a subsection 1.7.1 instead of chapter 2.0. To get a proper section numbering, I have to insert chapters in LaTeX manually with directives raw and only. Unfortunately, the sections from toctree still remain inserted, and there is no way to use only to prevent that.

It seems a bug of latexpdf, because for epub it works correctly. Is there a workaround?

How to Reproduce

**********************************************
Welcome to YARsync's documentation!
***********************************************

============
Introduction
============

.. include:: ../../README.rst
   :start-line: 4

.. toctree::
  :caption: Documentation:

  Manual <yarsync.1.md>
  Advanced <details>

.. raw:: latex

    \chapter{Manual}

.. only:: latex

    .. include:: yarsync.1.md
       :parser: myst_parser.sphinx_

.. only:: latex

    .. include:: details.rst

Environment Information

Platform:              linux; (Linux-6.1.1-arch1-1-x86_64-with-glibc2.36)
Python version:        3.10.9 (main, Dec 19 2022, 17:35:49) [GCC 12.2.0])
Python implementation: CPython
Sphinx version:        5.3.0
Docutils version:      0.19
Jinja2 version:        3.0.3


(same problem for readthedocs, which uses the same Sphinx version)

Sphinx extensions

["myst_parser"]

Additional context

It is strange that I have to use

   :parser: myst_parser.sphinx_

it was not documented in Sphinx, but without that it breaks.

It is sad that only works only for content. I found some other bugs which explain more why it is so (maybe they should be linked from the documentation?): https://github.com/sphinx-doc/sphinx/issues/9819, "toctree does not consider "only" directive" (with that I could had created two toctrees for pdf and all other formats), https://github.com/sphinx-doc/sphinx/issues/10642#issuecomment-1186146698 (more details about only directive).

Also it is sad that while my manual from Markdown is inserted as a separate section, its internal hierarchy is still broken (instead of being subsections, its parts become sub-subsections; I know no way to fix that). Maybe one time I will convert Markdown to rst, but first I will have to check how it would affect the manual page.

The reproduction is centered on index.rst, with README.rst, yarsync.1.md, and details.rst included through the toctree and latex-only blocks. First run make latexpdf and compare its hierarchy with the HTML and EPUB output; done means the generated PDF preserves the intended Introduction, Manual, and Advanced chapter structure without duplicate toctree content.