FastNetMon

Thursday 28 August 2014

Как починить падения Puppet DB от ошибок OOM killer?

Проявляется бага как:
Warning: Error 400 on SERVER: Could not retrieve facts for ovz14.fastvps.ru: Failed to submit 'replace facts' command for ovzxxx.ru to PuppetDB at xxxxxx:8081: Connection refused - connect(2)
Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Failed to submit 'replace facts' command for oxxxxs.ru to PuppetDB at xxxxx:8081: Connection refused - connect(2)


В логе (/var/log/puppetdb/puppetdb-daemon.log ) при этом:
# java.lang.OutOfMemoryError: Java heap space
# -XX:OnOutOfMemoryError="kill -9 %p"
#   Executing /bin/sh -c "kill -9 15190"...
java.lang.OutOfMemoryError: Java heap space
Dumping heap to /var/log/puppetdb/puppetdb-oom.hprof ...
Heap dump file created [225222767 bytes in 1,713 secs]
Чиним, открываем конфиг: /etc/sysconfig/puppetdb и там заменяем Xmx192m на Xmx1024m.

Применяем:
/etc/init.d/puppetdb restart
Убеждаемся, что Java завелась с новым лимитом памяти:
 ps aux|grep Xmx -i --color
puppetdb 29429 11.7  1.8 1654504 593832 ?      Sl   11:35   0:26 /usr/lib/jvm/jre-1.7.0-openjdk.x86_64/bin/java -XX:OnOutOfMemoryError=kill -9 %p -Xmx1024m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/log/puppetdb/puppetdb-oom.hprof -Djava.security.egd=file:/dev/urandom -cp /usr/share/puppetdb/puppetdb.jar clojure.main -m com.puppetlabs.puppetdb.core services -c /etc/puppetdb/conf.d

No comments :

Post a Comment

Note: only a member of this blog may post a comment.