Skip to content

Skipping PostgreSQL dollar-quoted string constants extremely slow for queries with many query parameters #1851

@gkoller

Description

@gkoller

Describe the bug:

PostgreSQL has a something called dollar-quoted string constants. apm-agent-python's instrumentation.packages.dbapi2.scan function has logic to detect these dollar quoted string constants and skip them.

However, this logic/code path is also triggered when it comes across query parameters. After all, query parameters also start with a dollar character ($). For queries with a very large number of query parameters (eg, 3000-4000) this code path, as determined by running the app under PyCharm with yappi, is a severe bottle neck. Such a query can be handled in just tens of miliseconds (30-40 ms) by PostgreSQL, but it subsequently takes apm-agent-python more than 10 sec. (!!!). Almost all the time is spent in the skip_to function.

Leading to graphs like these:

screenshot_20230612_113424

What kind of a query takes up to 3000 query parameters? Queries with large VALUES lists.

SELECT * FROM (VALUES ($1::varchar, $2::varchar, $3::varchar), ($4::varchar, $5::varchar, $6::varchar), ($7::varchar, $8::varchar, $9::varchar), ... ($2992::varchar, $2993::varchar, $2994::varchar), ($2995::varchar, $2996::varchar, $2997::varchar), ($2998::varchar, $2999::varchar, $3000::varchar))

BTW the above query is a simplified example. In our use case that large VALUES list is part of a much more elaborate CTE (WITH statement)

To Reproduce

  1. Use SQL queries with a large VALUES list consisting of query parameters.

Environment (please complete the following information)

  • OS:
    • Windows 10
    • Linux (whatever is in the python:3.10.11 Docker image)
    • MacOS 12.6.6 on ARM
    • .... in other words it is not OS specific
  • Python version:
    • 3.10.11
    • 3.11.4
  • Framework and version [e.g. Django 2.1]:
    • FastAPI 0.96.0
  • APM Server version:
    • 8.6.2
  • Agent version:
    • (elastic-apm) 6.16.1

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions