1
General Discussion / Re: OMS is slow, not working properly and disk is full ..
« Last post by NeySlim on May 23, 2013, 04:48:15 PM »Yep, it's working now, even the reserved ram meter.
Recent Posts
2
Installation and Configuration / Re: Cannot register openode nodes to OMS
« Last post by ilja_l on May 23, 2013, 01:41:33 PM »Hi, a couple of more things: at the moment latest OMS actually expects agent/tui package from the development repo. Not sure if it's available by default on the ISO, if not, you can add it manually by adding to /etc/yum.repos.d/opennode.repo (on HN) the following part:
[opennode-dev]
name=CentOS-$releasever - OpenNode OS dev packages for $basearch
baseurl=http://opennodecloud.com/CentOS/$releasever/opennode-dev
enabled=0
gpgcheck=1
protect=1
priority=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-ActiveSys
and executing 'yum update --enablerepo=opennode-dev opennode-tui'.
Also, although salt 0.12 should be fine, I suggest upgrading to version 0.14 for stability reasons. It can be done again via 'yum update salt' on both HN and OMS.
Also, could you please show the contents of the /etc/salt/minon of the HN?
Ilya
ps. certmaster is not used any more, we have migrated to a saltstack system.
[opennode-dev]
name=CentOS-$releasever - OpenNode OS dev packages for $basearch
baseurl=http://opennodecloud.com/CentOS/$releasever/opennode-dev
enabled=0
gpgcheck=1
protect=1
priority=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-ActiveSys
and executing 'yum update --enablerepo=opennode-dev opennode-tui'.
Also, although salt 0.12 should be fine, I suggest upgrading to version 0.14 for stability reasons. It can be done again via 'yum update salt' on both HN and OMS.
Also, could you please show the contents of the /etc/salt/minon of the HN?
Ilya
ps. certmaster is not used any more, we have migrated to a saltstack system.
3
General Discussion / Re: OMS is slow, not working properly and disk is full ..
« Last post by ilja_l on May 23, 2013, 01:28:11 PM »Well, this error can be caused by two things:
- not enough privileges
- incorrect objects in a db, which cause this.
The latter is easier to verify. Could you please do the following:
0. systemctl stop oms
1. mv /opt/oms/db /opt/oms/db.bak; mkdir /opt/oms/db
2. systemctl start oms
- not enough privileges
- incorrect objects in a db, which cause this.
The latter is easier to verify. Could you please do the following:
0. systemctl stop oms
1. mv /opt/oms/db /opt/oms/db.bak; mkdir /opt/oms/db
2. systemctl start oms
4
Installation and Configuration / Re: Cannot register openode nodes to OMS
« Last post by activesys on May 23, 2013, 01:03:58 PM »Check that after registration with TUI salt-minion service is actually running on hosts (hqkvm01 and hqkvm02): eg service salt-minion status
5
General Discussion / Re: OMS is slow, not working properly and disk is full ..
« Last post by NeySlim on May 23, 2013, 12:41:13 PM »Here is the nginx access log without common typo 
access-onc.log:192.168.1.93 - - [23/May/2013:13:39:03 +0400] "GET /computes?_dc=1369301943191&depth=1&attrs=hostname%2Curl%2Cnetwork%2Cnum_cores%2Cswap_size%2Cdiskspace%2Cmemory&exclude=openvz&sort=%5B%7B%22property%22%3A%22hostname%22%2C%22direction%22%3A%22ASC%22%7D%5D HTTP/1.1" 500 2137 "https://oms1.hq.eloquant.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31"
access-onc.log:192.168.1.93 - - [23/May/2013:13:39:03 +0400] "GET /computes?_dc=1369301943083&depth=1&attrs=hostname%2Cipv4_address%2Cstate%2Curl%2Ctags%2Cfeatures&exclude=openvz&q=tags%3Avirt%3Ano&filter=%5B%7B%22property%22%3Anull%2C%22value%22%3Anull%7D%5D HTTP/1.1" 500 2137 "https://oms1.hq.eloquant.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31"
access-onc.log:192.168.1.93 - - [23/May/2013:13:39:03 +0400] "GET /computes/?depth=2&attrs=features,diskspace,memory,__type__,tags&exclude=openvz&_dc=1369301943838 HTTP/1.1" 500 2137 "https://oms1.hq.eloquant.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31"
access-onc.log:192.168.1.93 - - [23/May/2013:13:39:07 +0400] "POST /stream?after=1369301922639 HTTP/1.1" 500 2326 "https://oms1.hq.eloquant.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31"
access-onc.log:192.168.1.93 - - [23/May/2013:13:39:03 +0400] "GET /computes?_dc=1369301943191&depth=1&attrs=hostname%2Curl%2Cnetwork%2Cnum_cores%2Cswap_size%2Cdiskspace%2Cmemory&exclude=openvz&sort=%5B%7B%22property%22%3A%22hostname%22%2C%22direction%22%3A%22ASC%22%7D%5D HTTP/1.1" 500 2137 "https://oms1.hq.eloquant.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31"
access-onc.log:192.168.1.93 - - [23/May/2013:13:39:03 +0400] "GET /computes?_dc=1369301943083&depth=1&attrs=hostname%2Cipv4_address%2Cstate%2Curl%2Ctags%2Cfeatures&exclude=openvz&q=tags%3Avirt%3Ano&filter=%5B%7B%22property%22%3Anull%2C%22value%22%3Anull%7D%5D HTTP/1.1" 500 2137 "https://oms1.hq.eloquant.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31"
access-onc.log:192.168.1.93 - - [23/May/2013:13:39:03 +0400] "GET /computes/?depth=2&attrs=features,diskspace,memory,__type__,tags&exclude=openvz&_dc=1369301943838 HTTP/1.1" 500 2137 "https://oms1.hq.eloquant.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31"
access-onc.log:192.168.1.93 - - [23/May/2013:13:39:07 +0400] "POST /stream?after=1369301922639 HTTP/1.1" 500 2326 "https://oms1.hq.eloquant.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31"
6
Installation and Configuration / Cannot register openode nodes to OMS
« Last post by rmaddodi on May 23, 2013, 12:07:01 AM »Hi All,
I am having issue registering my opennode nodes to OMS. I installed 2 opennode server from ISO file and I am running OMS VM on one of the node. I am able to login to web interface but when I click on Host Management buttom on right top plane and reload it, it doesn't seem to find it.
I tried to follow "http://opennodecloud.com/forum/index.php/topic,211.0.html" and https://docs.opennodecloud.com/wiki/doku.php?id=usrdoc:qsguide&#ensure_that_oms_service_is_running" but still no luck.
On my OMS server:
I am trying to register both nodes on port 51234 and 4056 but still I cannot register.
Ping test from OMS:
On my nodes: hqkvm01 and hqkvm02
Ping test from both nodes to OMS
Anything over here I am missing or I am doing some thing wrong over here?
I am having issue registering my opennode nodes to OMS. I installed 2 opennode server from ISO file and I am running OMS VM on one of the node. I am able to login to web interface but when I click on Host Management buttom on right top plane and reload it, it doesn't seem to find it.
I tried to follow "http://opennodecloud.com/forum/index.php/topic,211.0.html" and https://docs.opennodecloud.com/wiki/doku.php?id=usrdoc:qsguide&#ensure_that_oms_service_is_running" but still no luck.
On my OMS server:
| [root@opennode /]# systemctl status nginx nginx.service - The nginx HTTP and reverse proxy server Loaded: loaded (/usr/lib/systemd/system/nginx.service; enabled) Active: active (running) since Thu, 23 May 2013 00:34:50 +0400; 11min ago Process: 471 ExecStart=/usr/sbin/nginx (code=exited, status=0/SUCCESS) Process: 450 ExecStartPre=/usr/sbin/nginx -t (code=exited, status=0/SUCCESS) Main PID: 477 CGroup: name=systemd:/system/nginx.service â 477 nginx: master process /usr/sbin/nginx â 479 nginx: worker process May 23 00:34:50 opennode.sanfree.net nginx[450]: nginx: the configuration file /etc/nginx/nginx.conf syntax is ok May 23 00:34:50 opennode.sanfree.net nginx[450]: nginx: configuration file /etc/nginx/nginx.conf test is successful |
| [root@opennode /]# systemctl status oms oms.service - Opennode Management Service Loaded: loaded (/usr/lib/systemd/system/oms.service; enabled) Active: active (running) since Thu, 23 May 2013 00:34:48 +0400; 12min ago Main PID: 126 CGroup: name=systemd:/system/oms.service â 126 /usr/bin/python2.7 /opt/oms/bin/omsd --log /var/log/omsd â 583 /bin/sh -c bin/runzeo -f db/data.fs -a db/socket >db/zeo.log 2>&1 â 586 /usr/bin/python2.7 bin/runzeo -f db/data.fs -a db/socket |
| [root@opennode /]# netstat -ant Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:4505 0.0.0.0:* LISTEN tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:4506 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:6022 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN tcp 0 0 127.0.0.1:8080 127.0.0.1:57218 TIME_WAIT tcp 0 0 192.168.10.80:443 192.168.10.4:62652 TIME_WAIT tcp 0 0 127.0.0.1:8080 127.0.0.1:57206 TIME_WAIT tcp 0 0 127.0.0.1:8080 127.0.0.1:57235 TIME_WAIT tcp 0 0 127.0.0.1:8080 127.0.0.1:57216 TIME_WAIT tcp 0 0 127.0.0.1:8080 127.0.0.1:57210 TIME_WAIT tcp 0 0 127.0.0.1:8080 127.0.0.1:57219 TIME_WAIT tcp 0 0 127.0.0.1:8080 127.0.0.1:57197 TIME_WAIT tcp 0 0 127.0.0.1:8080 127.0.0.1:57236 TIME_WAIT tcp 0 0 127.0.0.1:8080 127.0.0.1:57205 TIME_WAIT tcp 0 0 127.0.0.1:8080 127.0.0.1:57241 TIME_WAIT tcp 0 0 127.0.0.1:8080 127.0.0.1:57196 TIME_WAIT tcp 0 0 127.0.0.1:8080 127.0.0.1:57240 TIME_WAIT tcp 0 0 127.0.0.1:8080 127.0.0.1:57228 TIME_WAIT tcp 0 0 127.0.0.1:8080 127.0.0.1:57229 TIME_WAIT tcp 0 0 127.0.0.1:8080 127.0.0.1:57238 TIME_WAIT tcp 0 0 127.0.0.1:8080 127.0.0.1:57232 TIME_WAIT tcp 0 0 127.0.0.1:8080 127.0.0.1:57200 TIME_WAIT tcp 0 0 127.0.0.1:8080 127.0.0.1:57227 TIME_WAIT tcp 0 0 127.0.0.1:8080 127.0.0.1:57207 TIME_WAIT tcp 0 0 127.0.0.1:8080 127.0.0.1:57203 TIME_WAIT tcp 0 0 127.0.0.1:8080 127.0.0.1:57208 TIME_WAIT tcp 0 0 127.0.0.1:8080 127.0.0.1:57204 TIME_WAIT tcp 0 0 127.0.0.1:8080 127.0.0.1:57224 TIME_WAIT tcp 0 0 127.0.0.1:8080 127.0.0.1:57225 TIME_WAIT tcp 0 0 127.0.0.1:8080 127.0.0.1:57199 TIME_WAIT tcp 0 0 127.0.0.1:8080 127.0.0.1:57233 TIME_WAIT tcp 0 0 127.0.0.1:8080 127.0.0.1:57237 TIME_WAIT tcp 0 0 127.0.0.1:8080 127.0.0.1:57230 TIME_WAIT tcp 0 0 192.168.10.80:443 192.168.10.4:62657 ESTABLISHED tcp 0 0 127.0.0.1:8080 127.0.0.1:57211 TIME_WAIT tcp 0 0 127.0.0.1:8080 127.0.0.1:57222 TIME_WAIT tcp 0 0 127.0.0.1:8080 127.0.0.1:57202 TIME_WAIT tcp 0 0 127.0.0.1:8080 127.0.0.1:57221 TIME_WAIT tcp6 0 0 :::22 :::* LISTEN tcp6 0 0 :::111 :::* LISTEN |
| [root@opennode /]# rpm -qa | grep salt salt-master-0.12.1-1.fc17.noarch salt-0.12.1-1.fc17.noarch salt-minion-0.12.1-1.fc17.noarch |
| [root@opennode /]# salt-key -L Accepted Keys: Unaccepted Keys: Rejected Keys: |
| [root@opennode /]# systemctl start certmaster Failed to issue method call: Unit certmaster.service failed to load: No such file or directory. See system logs and 'systemctl status certmaster.service' for details. |
| [root@opennode /]# certmaster-ca --list -bash: certmaster-ca: command not found |
I am trying to register both nodes on port 51234 and 4056 but still I cannot register.
Ping test from OMS:
| [root@opennode /]# ping hqkvm01.sanfree.net PING hqkvm01.sanfree.net (192.168.10.72) 56(84) bytes of data. 64 bytes from hqkvm01.sanfree.net (192.168.10.72): icmp_req=1 ttl=64 time=0.034 ms 64 bytes from hqkvm01.sanfree.net (192.168.10.72): icmp_req=2 ttl=64 time=0.019 ms 64 bytes from hqkvm01.sanfree.net (192.168.10.72): icmp_req=3 ttl=64 time=0.022 ms ^C --- hqkvm01.sanfree.net ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 1999ms rtt min/avg/max/mdev = 0.019/0.025/0.034/0.006 ms [root@opennode /]# ping hqkvm02.sanfree.net PING hqkvm02.sanfree.net (192.168.10.71) 56(84) bytes of data. 64 bytes from hqkvm02.sanfree.net (192.168.10.71): icmp_req=1 ttl=63 time=758 ms 64 bytes from hqkvm02.sanfree.net (192.168.10.71): icmp_req=2 ttl=63 time=0.127 ms 64 bytes from hqkvm02.sanfree.net (192.168.10.71): icmp_req=3 ttl=63 time=0.102 ms |
On my nodes: hqkvm01 and hqkvm02
| [root@hqkvm01 ~]# rpm -qa | grep salt salt-minion-0.12.1-1.el6.noarch salt-0.12.1-1.el6.noarch |
| [root@hqkvm02 ~]# rpm -qa | grep salt salt-minion-0.12.1-1.el6.noarch salt-0.12.1-1.el6.noarch |
Ping test from both nodes to OMS
| Node1 (hqkvm01): PING opennode.sanfree.net (192.168.10.80) 56(84) bytes of data. 64 bytes from opennode.sanfree.net (192.168.10.80): icmp_seq=1 ttl=64 time=0.030 ms 64 bytes from opennode.sanfree.net (192.168.10.80): icmp_seq=2 ttl=64 time=0.023 ms 64 bytes from opennode.sanfree.net (192.168.10.80): icmp_seq=3 ttl=64 time=0.016 ms ^C --- opennode.sanfree.net ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2087ms rtt min/avg/max/mdev = 0.016/0.023/0.030/0.005 ms Node2 (hqkvm02) PING opennode.sanfree.net (192.168.10.80) 56(84) bytes of data. 64 bytes from opennode.sanfree.net (192.168.10.80): icmp_seq=1 ttl=64 time=0.030 ms 64 bytes from opennode.sanfree.net (192.168.10.80): icmp_seq=2 ttl=64 time=0.023 ms 64 bytes from opennode.sanfree.net (192.168.10.80): icmp_seq=3 ttl=64 time=0.016 ms ^C --- opennode.sanfree.net ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2087ms rtt min/avg/max/mdev = 0.016/0.023/0.030/0.005 ms |
Anything over here I am missing or I am doing some thing wrong over here?
7
General Discussion / Re: OMS is slow, not working properly and disk is full ..
« Last post by ilja_l on May 22, 2013, 07:47:32 PM »"what HTTP request". The word was right next to the space key (c) http://bash.org/?5300
8
General Discussion / Re: OMS is slow, not working properly and disk is full ..
« Last post by NeySlim on May 22, 2013, 12:41:32 PM »What do you mean by "what http is causing it" ?
9
General Discussion / Re: OMS is slow, not working properly and disk is full ..
« Last post by ilja_l on May 22, 2013, 12:17:16 PM »Interesting, the version is more or less recent, but I cannot reproduce it. Could you please check what HTTP is causing it?
thanks,
Ilya
thanks,
Ilya
10
General Discussion / Re: OMS is slow, not working properly and disk is full ..
« Last post by NeySlim on May 22, 2013, 12:14:04 PM »Here you go:
/opt/oms/eggs/opennode.wsproxy-1.0-py2.7.egg
/opt/oms/eggs/opennode.oms.core-0.0_1431_g7747-py2.7.egg:
EGG-INFO logging.conf oms_acl oms_groups oms_permissions opennode opennode-oms.conf
/opt/oms/eggs/opennode.oms.knot-0.0_489_g4efe-py2.7.egg:
EGG-INFO opennode opennode-knot.conf
/opt/oms/eggs/opennode.oms.onc-0.0_911_g0c89-py2.7.egg:
app app.js beep.wav conf-default.js css EGG-INFO img index.html lib opennode portal.html support.js test-harness.js theme util.js
/opt/oms/eggs/opennode.wsproxy-1.0-py2.7.egg
/opt/oms/eggs/opennode.oms.core-0.0_1431_g7747-py2.7.egg:
EGG-INFO logging.conf oms_acl oms_groups oms_permissions opennode opennode-oms.conf
/opt/oms/eggs/opennode.oms.knot-0.0_489_g4efe-py2.7.egg:
EGG-INFO opennode opennode-knot.conf
/opt/oms/eggs/opennode.oms.onc-0.0_911_g0c89-py2.7.egg:
app app.js beep.wav conf-default.js css EGG-INFO img index.html lib opennode portal.html support.js test-harness.js theme util.js