php - Doctrine flush absolutely too long -
i have following situation: use symfony command list xml documents of given folder. parse each document , build document entity. want "import" new entities in mysql database using doctrine2.
php class looks this:
class importservice implements flushinterface { private $entitymanager; private $documentcache; private $flushcounter; public function __construct() { $this->documentcache = []; $this->flushcounter = 0; } public function savedocument(documententity $doc) { $this->persistdocumententity($doc); ... $this->checkbulk(); return $this; } protected function persistdocumententity(documententity $doc) { $cachekey = hash('sha256', $doc->getsource() . ':' . $doc->getsourceid() . ':' . $doc->getclient()); $this->entitymanager->persist($doc); $this->documentcache[$cachekey] = $doc; return $this; } protected function checkbulk() { $this->flushcounter++; if ($this->flushcounter > 20) { $this->flushcounter = 0; $this->flush(null, false); } return $this; } public function flush(flushinterface $flushinterface = null, $clearentitymanager = true) { // save database data $em = $this->entitymanager; $em->flush(); if ($clearentitymanager) { $em->clear(); } $this->documentcache = []; // call flush interface if ($flushinterface instanceof flushinterface) { $flushinterface->flush(); } return $this; } }
what happens? can execute app without problems on local system. folder contains 3,000 xml documents parse , import take 5 minutes. that's ok.
run same command (same application) on server , here import takes 5 days. that's absolutely long.
logged steps , made time measurements. found out problem line:
$em->flush();
it takes 12 seconds execute. looked log file , found out there 40 insert queries per flush execute. seems not much.
i've no idea why process executed fast locally , slow on server , i've no idea bottleneck comes from.
p.s. xdebug.profiling disabled in php.ini.
Comments
Post a Comment