Skip to content

No fast-track for bulk inserts in Cursor.executemany with INSERT/REPLACE syntax introduced in MySQL 8.0.19 #116

@kozchris

Description

@kozchris

asyncmy is not handling the new syntax for doing bulk inserts correctly. The same issue also exists in aiomysql with a good explanation of what is happening. That issue is: aio-libs/aiomysql#968

MySQL is deprecating this syntax:

INSERT INTO t1 (a,b,c) VALUES (1,2,3),(4,5,6)
  ON DUPLICATE KEY UPDATE c=VALUES(a)+VALUES(b);

and asyncmy emits a warning about it. However if you try to use the new syntax:

INSERT INTO t1 (a,b,c) VALUES (1,2,3),(4,5,6) AS new(m,n,p)
  ON DUPLICATE KEY UPDATE c = m+n;

the insertion speed becomes extremely slow.

I'm attaching the same example program as I did in the aiomysql bug:

I am seeing this issue in asyncmy = "0.2.10".

From my test app:

DEBUG:root:On deprecated named col duplicate insert of 8000 rows took 0.275493860244751 seconds
DEBUG:root:On named col duplicate insert of 8000 rows took 24.421008110046387 seconds
DEBUG:root:On placeholder duplicate insert of 8000 rows took 1.0064198970794678 seconds

I'll attach a sample program demonstrating the issue. For ease, add a .env file with the following information for your db and then run the dbtest2.py file:

#.env

DATABASE_USER=xxx
DATABASE_PASSWORD=yyy
DATABASE_HOST=aaa
DATABASE_NAME=zzzz

db_test2.py.zip

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions