SNOW-1413730: predicate for join doesn't seem to support null values in left side of == operator

Start by reproducing test_get_submitted_reviews_for_user in tests/manager_app_utils/manager_models/test_ma_sr.py, focusing on the get_submitted_reviews_for_user entry point and its chained joins. Trace how join predicates handle null values on the left side of ==, then verify that the join results can be shown without the NaN-to-bool error.

Please answer these questions before submitting your issue. Thanks!

1. What version of Python are you using?

Python 3.10.11

2. What are the Snowpark Python and pandas versions in the environment?

pandas==2.2.2
pandas-stubs==2.2.1.240316
snowflake-snowpark-python==1.15.0

3. What did you do?

def get_submitted_reviews_for_user(
    user_email: str,
    filter: dict,
    page_size: int = DEFAULT_PAGE_SIZE,
    page_number: int = 0,
) -> pd.DataFrame:
    session = db.init_connection("snowflake")
    offset = get_offset(
        page_size,
        page_number,
    )

    df_reviews = session.table(REVIEW_DATA_TABLE)
    df_lookback_answers = session.table(LOOKBACK_ANSWERS_TABLE)
    df_review_submissions = session.table(REVIEW_SUBMISSIONS_LATEST_VIEW).filter(
        (col("manager_email") == lit(user_email))
    )

    print("before the join")

    df_reviews_joined = (
        df_reviews.join(
            right=df_review_submissions,
            on="id",
            how="left",
            rsuffix="_submitted",
        )
        .join(
            df_lookback_answers,
            (
                (col("lookback_whygrantthen_id") == df_lookback_answers["id"])
                | (
                    col("lookback_whygrantthen_id_submitted")
                    == df_lookback_answers["id"]
                )
            ),
            join_type="left",
            lsuffix="_wt",
        )
        .join(
            df_lookback_answers,
            (col("lookback_whyrevokenow_id") == df_lookback_answers["id"])
            | (col("lookback_whyrevokenow_id_submitted") == df_lookback_answers["id"]),
            join_type="left",
            lsuffix="_wn",
        )
    )

    df_reviews_joined.show() # Fails on this statement

Fails with error:

FAILED tests/manager_app_utils/manager_models/test_ma_sr.py::test_get_submitted_reviews_for_user - ValueError: cannot convert float NaN to bool

4. What did you expect to see?

We expect to see the results of the JOIN.