python - Log messages via TCP, significantly slower than similar code in Java -

in python 3 have queuehandler attached logger , queuelistener shipping logrecords sockethandler sends logs via tcp java app that's listening.

both programs run on localhost.

import logging import queue  log_q = queue.queue(-1) logger = logging.getlogger('testlogger') socket_handler = logging.handlers.sockethandler('localhost', 1337) q_handler = logging.handlers.queuehandler(log_q) q_listener = logging.handlers.queuelistener(log_q, socket_handler) logger.addhandler(q_handler) q_listener.start() 

i'm sending logrecords large lists attached them.

logger.info("prox_marker", extra={'vector': [some_list]}) 

where [some_list] list of ~100k double values.

i run following code test throughput:

for in range(1000):     logger.info("prox_marker", extra={'vector': [some_list]}) 

which takes 30-35 seconds complete.

if run similar test in java, java app twice fast.

in python queuehandler/-listener + sockethandler setup manages send 3 messages every 10 put in queue. time program finishes it'll have sent ~300 msgs 700 still in queue sent after main program finished.

the queuehandler/-listener use default, change default sockethandler use custom serializing method.

my goal (in case wasn't obvious) try , speed python code. unfortunately i'm still not 100% sure what's causing slow behavior in first place. might have socket (which know little about, i've tried playing around various timeout setting , tcp_nodelay - no avail).

i've tried ditch queuehandler/-listener , use sockethandler directly takes same amount of time before i'm assuming threading isn't issue.

any tips on issue might or how make faster appreciated.