open-telemetry / open-telemetry/opentelemetry-python-contrib

Santinize db.statement for static queries

Open
#2,003 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
1.1k
Forks
1.1k
Avg merge
4d 15h
Merged PRs (30d)
16

Description

Problem

When we use opentelemetry-instrument to automatically instrument a Flask app connected to a database, we notice that it doesn't sanitize queries with constant parameters, even though it collects db.statement by default. It goes against OTel semantic conventions .

Steps to reproduce
Sample code below can be run with command

 opentelemetry-instrument 
    --traces_exporter console \
    --metrics_exporter console \
    --service_name my-service \
    python app.py

Traces

When we hit the server curl http://localhost:9090/select we see a trace (Query is NOT sanitized)

{
    "name": "SELECT",
    "context": {
        "trace_id": "0xc759463ff38c3f9a40f08d4a02a72ec8",
        "span_id": "0xcc644e552f120684",
        "trace_state": "[]"
    },
    "kind": "SpanKind.CLIENT",
    "parent_id": null,
    "start_time": "2023-10-13T13:13:29.399042Z",
    "end_time": "2023-10-13T13:13:29.399442Z",
    "status": {
        "status_code": "UNSET"
    },
    "attributes": {
        "db.system": "sqlite",
        "db.name": "",
        "db.statement": "SELECT * FROM users where id = 1"
    },
    "events": [],
    "links": [],
    "resource": {
        "attributes": {
            "telemetry.sdk.language": "python",
            "telemetry.sdk.name": "opentelemetry",
            "telemetry.sdk.version": "1.20.0",
            "service.name": "my-service",
            "telemetry.auto.version": "0.41b0"
        },
        "schema_url": ""
    }
}

When we hit the server curl http://localhost:9090/user/1 we see a trace (Query is sanitized)

{
    "name": "SELECT",
    "context": {
        "trace_id": "0x1fe109bedcd50e879d22bc63da5b9bbb",
        "span_id": "0x304c499e2cc8383f",
        "trace_state": "[]"
    },
    "kind": "SpanKind.CLIENT",
    "parent_id": null,
    "start_time": "2023-10-14T11:09:15.530583Z",
    "end_time": "2023-10-14T11:09:15.530902Z",
    "status": {
        "status_code": "UNSET"
    },
    "attributes": {
        "db.system": "sqlite",
        "db.name": "",
        "db.statement": "SELECT * FROM users WHERE id=?"
    },
    "events": [],
    "links": [],
    "resource": {
        "attributes": {
            "telemetry.sdk.language": "python",
            "telemetry.sdk.name": "opentelemetry",
            "telemetry.sdk.version": "1.20.0",
            "service.name": "my-service",
            "telemetry.auto.version": "0.41b0"
        },
        "schema_url": ""
    }
}

Code To Reproduce
import os
import sqlite3

from flask import Flask, jsonify, request

DATABASE = "users.db"


def setup_database():
    with get_db() as conn:
        conn.execute(
            """
            CREATE TABLE IF NOT EXISTS users (
                id INTEGER PRIMARY KEY,
                name TEXT NOT NULL,
                email TEXT NOT NULL UNIQUE
            );
        """
        )


def get_db():
    return sqlite3.connect(DATABASE)


def create_app(test_config=None):
    app = Flask(__name__)
    setup_database()

    @app.route("/user/<int:user_id>", methods=["GET"])
    def get_user(user_id):
        with get_db() as conn:
            cur = conn.cursor()
            sql = f"select * from users whee id ={user_id}"
            cur.execute(sql)
            user = cur.fetchone()
        if user:
            return jsonify(user)
        return jsonify({"error": "User not found"}), 404


    @app.route("/select", methods=["GET"])
    def select_user():
        with get_db() as conn:
            cur = conn.cursor()
            cur.execute("SELECT * FROM users where id = 1")
            users = cur.fetchall()
        return jsonify(users)

    return app


if __name__ == "__main__":
    app = create_app()
    app.run(debug=False, host="0.0.0.0", port=os.environ.get("SERVER_PORT", 9090))

What is the expected behavior?
We should sanitize the queries with static parameters, so first trace should be something like

{
    "name": "SELECT",
    "context": {
        "trace_id": "0xc759463ff38c3f9a40f08d4a02a72ec8",
        "span_id": "0xcc644e552f120684",
        "trace_state": "[]"
    },
    "kind": "SpanKind.CLIENT",
    "parent_id": null,
    "start_time": "2023-10-13T13:13:29.399042Z",
    "end_time": "2023-10-13T13:13:29.399442Z",
    "status": {
        "status_code": "UNSET"
    },
    "attributes": {
        "db.system": "sqlite",
        "db.name": "",
        "db.statement": "SELECT * FROM users where id = ?"
    },
    "events": [],
    "links": [],
    "resource": {
        "attributes": {
            "telemetry.sdk.language": "python",
            "telemetry.sdk.name": "opentelemetry",
            "telemetry.sdk.version": "1.20.0",
            "service.name": "my-service",
            "telemetry.auto.version": "0.41b0"
        },
        "schema_url": ""
    }
}

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the supplied app.py reproduction, running it through the opentelemetry-instrument entry point and comparing the /select and /user/<user_id> traces. Trace the database instrumentation path that produces db.statement. Done means the static literal query is sanitized consistently with the parameterized query and the behavior is covered by a regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
flask, python, sqlite
Domain
databases, observability
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.