How to make the embedded language have corresponding language id

Hello everyone, I am struggling to find a correct way to support Textmate grammar with embedded language. For example, I want to embed sql into python and all the code inside sql( and ) should be considered as SQL code.

I add custom grammar rule to MagicPython grammer file and successfully make the token returned by tokenizeLine have SQL grammar scope name (something like *.sql). However, I don't know how to make the token returned by tokenizeLine2 to have language sql rather than python

e.g.
For SELECT in sql(SELECT):
tokenizeLine will return: keyword.other.DML.sql, which is defined in Sql.tmLanguage.json
tokenizeLine2 will return a Uint32Array, but the mask the array will show language is python rather than sql

Any idea about how to fix this? I appreciate any help and understand there are a lot of moving parts here to make this all work!

Start by tracing how tokenizeLine and tokenizeLine2 derive scopes and language masks for embedded grammars. Compare the MagicPython.tmLanguage grammar with Sql.tmLanguage.json, focusing on the sql(...) example. Done means the embedded SELECT span receives the sql language id in tokenizeLine2 instead of python.