- How to view the size of transmitted packets when using ORM
Please elaborate and be more specific with what you’re asking here. Size of what transmitted packets?
What question are you trying to answer? And why?
Are you trying to find an answer from inside Django or monitoring it from outside your system?
I want to know the packet size when the ORM is converted into SQL statement to request the database
Have you looked into creating your own database instrumentation? The example class QueryLogger is probably close to what you want. You would need to convert the SQL statement and params into a byte count.
Oh, thank you. It’s very helpful to me. I’m trying it
That’s part of where the ambiguity comes in, and why I was asking for more details about exactly what’s being looked for.
You might get a reasonable estimator of the SQL statement size, but that doesn’t neccessarily directly correspond to the packet size of the communications between psycopg2 (assuming PostgreSQL) and the database server. It might also have a completely different level of meaning if they’re using something like Sqlite.
Oh, yes, is there a better way to detect the packet size
From my earlier post -
I’m confused. I don’t know where to find the answer. I want to know the size of the data packet after converting my ORM into SQL statements
Let’s start with this one:
Or, to phrase it differently: What issue or problem has come up that makes this question about packets sizes relevent?
The question you’re asking is not trivial and has many aspects to it that may not be directly obvious to you.
Technically, your “packet size” (from a network perspective) has much less to do with the application you’re running than with the underlying environment. Django itself is not communicating with PostgreSQL - if that’s what you’re using. Django creates an SQL statement and passes it along to psycopg2 (if that is what you’re using).
It’s psycopg2 that creates the network traffic and manages the response from the database server - Django is not involved in that at all.
So for me to give you a valid answer to your question, I need to understand the root issue behind it. Otherwise, this all seems to me like an X-Y problem.
Good point. I made an assumption that a “good enough” calculation would suffice.
It might. But the OP has provided no guidance or clarification regarding what the actual requirement is or the degree of accuracy required - hence the additional questions.
1 Like
The thing is, I’m using filter (ID)__ In = []), I don’t know how many elements can be placed in the list. I consult the data, which explains that it is affected by Max in the database engine_ allowed_ The packet parameter is limited. The reason why I want to know the size of SQL statement packets is that I want to know that duo can be placed in the list
What database are you using?
What docs are you referencing with this “database engine_allowed_”?
Keep in mind that many things are going to affect the overall size of the SQL statement - you’re going to get different answers depending upon things including the name of the table and the size (number of digits) in the individual IDs. You’re not going to be able to come up with one number that is going to work in all cases.
I’m using mysql. I’ll use max_ allowed_ When packet is set to 1m__ In = [], when I put 200000 elements in the list, a connection error will be prompted. I don’t know if this is correct?
Databases tend to perform worse as you increase the size of the collection used in a field in (val1, val2, ..., val200000 ) filter. You may find creating a temporary table or adjusting the query to use a join statement for better performance. Changing your approach may also side-step the current error you’re attempting to address.
@KenWhitesell I believe the OP is referencing max_allowed_packet
1 Like
Well, yes, I’m trying to solve this problem
Do you want to explore reworking the approach to this query? If so, can you provide us with more context around what you’re trying to do?
1 Like