Discussion:
[ovirt-users] New oVirt Node 4.0.6.1 network problem with bond and many vlans - Network.service timeout failed
Rogério Ceni Coelho
8 years ago
Permalink
Hi oVirt admins,

Yesterday, i install a new node 4.0.6.1 and after config network bond (1) +
vlans ( about 50 ), network.service failed with timeout as above. All my
old and working fine oVirt nodes run 4.0.5 and have the same 5 min timeout
on systemd.

When i try to run vms on this new server, some networks are ok and some not.

I install oVirt Node with 4.0.3 on same server and this problem do not
occur.

How can i workaround or change timeout to network.service ?

In time, there are any way to update oVirt Node to 4.0.5 from 4.0.3 install
iso ?

[***@prd-rbs-ovirt-kvm21-poa ~]# systemctl status network.service
â— network.service - LSB: Bring up/down networking
Loaded: loaded (/etc/rc.d/init.d/network; bad; vendor preset: disabled)
Active: failed (Result: timeout) since Thu 2017-05-04 11:29:43 BRT; 3h
12min ago
Docs: man:systemd-sysv-generator(8)
Process: 7574 ExecStart=/etc/rc.d/init.d/network start (code=killed,
signal=TERM)

May 04 11:29:12 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[7574]: Bringing
up interface o_p_s_a_lb_758: [ OK ]
May 04 11:29:18 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[7574]: Bringing
up interface o_p_sites_a_755: [ OK ]
May 04 11:29:23 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[7574]: Bringing
up interface o_prod_app_757: [ OK ]
May 04 11:29:28 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[7574]: Bringing
up interface o_prod_db_756: [ OK ]
May 04 11:29:38 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[7574]: Bringing
up interface ovirtmgmt: [ OK ]
*May 04 11:29:43 prd-rbs-ovirt-kvm21-poa.rbs.com.br
<http://prd-rbs-ovirt-kvm21-poa.rbs.com.br> systemd[1]: network.service
start operation timed out. Terminating.*
May 04 11:29:43 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]: Failed to
start LSB: Bring up/down networking.
May 04 11:29:43 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]: Unit
network.service entered failed state.
May 04 11:29:43 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]:
network.service failed.
May 04 11:29:43 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[7574]: Bringing
up interface paas_1305:
[***@prd-rbs-ovirt-kvm21-poa ~]#

[***@prd-rbs-ovirt-kvm21-poa ~]# systemctl restart network.service
Job for network.service failed because a timeout was exceeded. See
"systemctl status network.service" and "journalctl -xe" for details.
[***@prd-rbs-ovirt-kvm21-poa ~]# systemctl status network.service
â— network.service - LSB: Bring up/down networking
Loaded: loaded (/etc/rc.d/init.d/network; bad; vendor preset: disabled)
Active: failed (Result: timeout) since Thu 2017-05-04 14:55:34 BRT; 4min
46s ago
Docs: man:systemd-sysv-generator(8)
Process: 40810 ExecStart=/etc/rc.d/init.d/network start (code=killed,
signal=TERM)

May 04 14:55:21 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[40810]: [ OK ]
May 04 14:55:23 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[40810]: Bringing
up interface o_hlg_db_766: RTNETLINK answers: File exists
May 04 14:55:26 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[40810]: [ OK ]
May 04 14:55:28 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[40810]: Bringing
up interface o_p_s_a_lb_758: RTNETLINK answers: File exists
May 04 14:55:31 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[40810]: [ OK ]
May 04 14:55:33 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[40810]: Bringing
up interface o_p_sites_a_755: RTNETLINK answers: File exists
May 04 14:55:34 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]:
network.service start operation timed out. Terminating.
May 04 14:55:34 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]: Failed to
start LSB: Bring up/down networking.
May 04 14:55:34 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]: Unit
network.service entered failed state.
May 04 14:55:34 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]:
network.service failed.
[***@prd-rbs-ovirt-kvm21-poa ~]#
Rogério Ceni Coelho
8 years ago
Permalink
Hi again oVirt Admins,

After dig some time, i adjust network.service (/etc/rc.d/init.d/network)
timeout to zero creating a custom timeout.conf file for systemctl.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=818676
https://www.turnkeylinux.org/blog/debugging-systemd-sysv-init-compat
https://unix.stackexchange.com/questions/233468/how-does-systemd-use-etc-init-d-scripts

[***@prd-rbs-ovirt-kvm21-poa system]# systemctl show network.service |
grep -i timeout
TimeoutStartUSec=5min
TimeoutStopUSec=5min
Result=timeout
JobTimeoutUSec=0
JobTimeoutAction=none
[***@prd-rbs-ovirt-kvm21-poa system]# pwd
/usr/lib/systemd/system
[***@prd-rbs-ovirt-kvm21-poa system]# mkdir network.service.d
[***@prd-rbs-ovirt-kvm21-poa system]# cd network.service.d/
[***@prd-rbs-ovirt-kvm21-poa network.service.d]# vi timeout.conf
[Service]
TimeoutSec=0
TimeoutStartUSec=0
[***@prd-rbs-ovirt-kvm21-poa system]# systemctl daemon-reload
[***@prd-rbs-ovirt-kvm21-poa system]# systemctl show network.service |
grep -i timeout
TimeoutStartUSec=0
TimeoutStopUSec=0
Result=timeout
DropInPaths=/usr/lib/systemd/system/network.service.d/timeout.conf
JobTimeoutUSec=0
JobTimeoutAction=none
[***@prd-rbs-ovirt-kvm21-poa system]#
[***@prd-rbs-ovirt-kvm21-poa ~]# systemctl restart network.service
[***@prd-rbs-ovirt-kvm21-poa ~]# systemctl status network.service
â— network.service - LSB: Bring up/down networking
Loaded: loaded (/etc/rc.d/init.d/network; bad; vendor preset: disabled)
Drop-In: /usr/lib/systemd/system/network.service.d
└─timeout.conf
Active: active (exited) since Thu 2017-05-04 15:22:12 BRT; 5min ago
Docs: man:systemd-sysv-generator(8)
Process: 345 ExecStart=/etc/rc.d/init.d/network start (code=exited,
status=0/SUCCESS)

May 04 15:21:39 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[345]: Bringing
up interface rede_5: [ OK ]
May 04 15:21:44 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[345]: Bringing
up interface rede_6: [ OK ]
May 04 15:21:48 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[345]: Bringing
up interface server_lb_7: [ OK ]
May 04 15:21:50 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[345]: Bringing
up interface srv_1056: RTNETLINK answers: File exists
May 04 15:21:53 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[345]: [ OK ]
May 04 15:21:58 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[345]: Bringing
up interface streaming_106: [ OK ]
May 04 15:22:02 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[345]: Bringing
up interface tef_100: [ OK ]
May 04 15:22:07 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[345]: Bringing
up interface telecom_25: [ OK ]
May 04 15:22:12 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[345]: Bringing
up interface zh_32: [ OK ]
May 04 15:22:12 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]: Started LSB:
Bring up/down networking.
[***@prd-rbs-ovirt-kvm21-poa ~]#

Em qui, 4 de mai de 2017 às 15:01, Rogério Ceni Coelho <
...
After reboot :

[***@prd-rbs-ovirt-kvm21-poa ~]# systemctl --failed
0 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use 'systemctl list-unit-files'.
[***@prd-rbs-ovirt-kvm21-poa ~]# systemctl status network.service
â— network.service - LSB: Bring up/down networking
Loaded: loaded (/etc/rc.d/init.d/network; bad; vendor preset: disabled)
Drop-In: /usr/lib/systemd/system/network.service.d
└─timeout.conf
Active: active (exited) since Thu 2017-05-04 15:41:34 BRT; 1min 50s ago
Docs: man:systemd-sysv-generator(8)
Process: 6886 ExecStart=/etc/rc.d/init.d/network start (code=exited,
status=0/SUCCESS)

May 04 15:40:47 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[6886]: Bringing
up interface rbs_farm_200: [ OK ]
May 04 15:40:53 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[6886]: Bringing
up interface rede_5: [ OK ]
May 04 15:40:58 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[6886]: Bringing
up interface rede_6: [ OK ]
May 04 15:41:04 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[6886]: Bringing
up interface server_lb_7: [ OK ]
May 04 15:41:13 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[6886]: Bringing
up interface srv_1056: [ OK ]
May 04 15:41:18 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[6886]: Bringing
up interface streaming_106: [ OK ]
May 04 15:41:23 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[6886]: Bringing
up interface tef_100: [ OK ]
May 04 15:41:28 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[6886]: Bringing
up interface telecom_25: [ OK ]
May 04 15:41:34 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[6886]: Bringing
up interface zh_32: [ OK ]
May 04 15:41:34 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]: Started LSB:
Bring up/down networking.
[***@prd-rbs-ovirt-kvm21-poa ~]#
Sandro Bonazzola
8 years ago
Permalink
Il 04/Mag/2017 20:02, "Rogério Ceni Coelho" <***@gmail.com>
ha scritto:

Hi oVirt admins,

Yesterday, i install a new node 4.0.6.1


I would suggest to go with 4.1.1 nodes since 4.0 is not supported anymore.


and after config network bond (1) + vlans ( about 50 ), network.service
failed with timeout as above. All my old and working fine oVirt nodes run
4.0.5 and have the same 5 min timeout on systemd.

When i try to run vms on this new server, some networks are ok and some not.

I install oVirt Node with 4.0.3 on same server and this problem do not
occur.

How can i workaround or change timeout to network.service ?

In time, there are any way to update oVirt Node to 4.0.5 from 4.0.3 install
iso ?

[***@prd-rbs-ovirt-kvm21-poa ~]# systemctl status network.service
â— network.service - LSB: Bring up/down networking
Loaded: loaded (/etc/rc.d/init.d/network; bad; vendor preset: disabled)
Active: failed (Result: timeout) since Thu 2017-05-04 11:29:43 BRT; 3h
12min ago
Docs: man:systemd-sysv-generator(8)
Process: 7574 ExecStart=/etc/rc.d/init.d/network start (code=killed,
signal=TERM)

May 04 11:29:12 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[7574]: Bringing
up interface o_p_s_a_lb_758: [ OK ]
May 04 11:29:18 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[7574]: Bringing
up interface o_p_sites_a_755: [ OK ]
May 04 11:29:23 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[7574]: Bringing
up interface o_prod_app_757: [ OK ]
May 04 11:29:28 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[7574]: Bringing
up interface o_prod_db_756: [ OK ]
May 04 11:29:38 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[7574]: Bringing
up interface ovirtmgmt: [ OK ]
*May 04 11:29:43 prd-rbs-ovirt-kvm21-poa.rbs.com.br
<http://prd-rbs-ovirt-kvm21-poa.rbs.com.br> systemd[1]: network.service
start operation timed out. Terminating.*
May 04 11:29:43 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]: Failed to
start LSB: Bring up/down networking.
May 04 11:29:43 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]: Unit
network.service entered failed state.
May 04 11:29:43 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]:
network.service failed.
May 04 11:29:43 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[7574]: Bringing
up interface paas_1305:
[***@prd-rbs-ovirt-kvm21-poa ~]#

[***@prd-rbs-ovirt-kvm21-poa ~]# systemctl restart network.service
Job for network.service failed because a timeout was exceeded. See
"systemctl status network.service" and "journalctl -xe" for details.
[***@prd-rbs-ovirt-kvm21-poa ~]# systemctl status network.service
â— network.service - LSB: Bring up/down networking
Loaded: loaded (/etc/rc.d/init.d/network; bad; vendor preset: disabled)
Active: failed (Result: timeout) since Thu 2017-05-04 14:55:34 BRT; 4min
46s ago
Docs: man:systemd-sysv-generator(8)
Process: 40810 ExecStart=/etc/rc.d/init.d/network start (code=killed,
signal=TERM)

May 04 14:55:21 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[40810]: [ OK ]
May 04 14:55:23 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[40810]: Bringing
up interface o_hlg_db_766: RTNETLINK answers: File exists
May 04 14:55:26 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[40810]: [ OK ]
May 04 14:55:28 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[40810]: Bringing
up interface o_p_s_a_lb_758: RTNETLINK answers: File exists
May 04 14:55:31 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[40810]: [ OK ]
May 04 14:55:33 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[40810]: Bringing
up interface o_p_sites_a_755: RTNETLINK answers: File exists
May 04 14:55:34 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]:
network.service start operation timed out. Terminating.
May 04 14:55:34 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]: Failed to
start LSB: Bring up/down networking.
May 04 14:55:34 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]: Unit
network.service entered failed state.
May 04 14:55:34 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]:
network.service failed.
[***@prd-rbs-ovirt-kvm21-poa ~]#
Rogério Ceni Coelho
8 years ago
Permalink
To install 4.1.1 node i need to update ovirt engine ?
...
Sandro Bonazzola
8 years ago
Permalink
On Sat, May 6, 2017 at 11:27 PM, Rogério Ceni Coelho <
Post by Rogério Ceni Coelho
To install 4.1.1 node i need to update ovirt engine ?
Since 4.0 is unsupported I would recomment to update ovirt-engine to 4.1.1
too.
...
--
SANDRO BONAZZOLA

ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D

Red Hat EMEA <https://www.redhat.com/>
<https://red.ht/sig>
TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
Loading...