<div dir="ltr"><div> Hi all,</div><div><br></div><div>Running a 2 node proxmox cluster :<br></div><div><br></div><div>pveversion -v<br>proxmox-ve: 5.4-2 (running kernel: 4.15.18-24-pve)<br>pve-manager: 5.4-13 (running version: 5.4-13/aee6f0ec)<br>pve-kernel-4.15: 5.4-12</div><div><br></div><div>linstor-client 1.0.10-1<br>linstor-common 1.4.1-1<br>linstor-controller 1.4.1-1<br>linstor-proxmox 4.1.1-1<br>linstor-satellite 1.4.1-1<br>python-linstor 1.0.10-1</div><div><br></div><div>Last update - upgrade : 24/01/2020.</div><div><br></div><div>Storage : drbd on top of lvm.</div><div><br></div><div>After extending drbd storage size from 2Tb to 5Tb, we can not restore / create any VM.</div><div>A timeout occurs :</div><div>
TASK ERROR: unable to create VM 8001 - error with cfs lock
'storage-drbdstorage': Could not set allow-two-primaries on resource
definition vm-8001-disk-2, because: got lock timeout - aborting command
at /usr/share/perl5/PVE/Storage/Custom/LINSTORPlugin.pm line 285.
PVE::Storage::Custom::LINSTORPlugin::alloc_image("PVE::Storage::Custom::LINSTORPlugin",
"drbdstorage", HASH(0x56188be2bb70), 8001, "raw", undef, 1048576)
called at /usr/share/perl5/PVE/Storage.pm line 738 eval {...} called at
/usr/share/perl5/PVE/Storage.pm line 738 PVE::Storage::__ANON__()
called at /usr/share/perl5/PVE/Cluster.pm line 915 eval {...} called at
/usr/share/perl5/PVE/Cluster.pm line 883
PVE::Cluster::__ANON__("storage-drbdstorage", undef,
CODE(0x56188bc35a68)) called at /usr/share/perl5/PVE/Cluster.pm line 954
PVE::Cluster::cfs_lock_storage("drbdstorage", undef,
CODE(0x56188bc35a68)) called at /usr/share/perl5/PVE/Storage/Plugin.pm
line 390
PVE::Storage::Plugin::cluster_lock_storage("PVE::Storage::Custom::LINSTORPlugin",
"drbdstorage", 1, undef, CODE(0x56188bc35a68)) called at
/usr/share/perl5/PVE/Storage.pm line 743
PVE::Storage::vdisk_alloc(HASH(0x56188b409290), "drbdstorage", 8001,
"raw", undef, 1048576) called at /usr/share/perl5/PVE/API2/Qemu.pm line
181 PVE::API2::Qemu::__ANON__("scsi0", HASH(0x56188bc359d8)) called at
/usr/share/perl5/PVE/QemuServer.pm line 3288
PVE::QemuServer::foreach_drive(HASH(0x56188b405ab8),
CODE(0x56188bc35a50)) called at /usr/share/perl5/PVE/API2/Qemu.pm line
216 eval {...} called at /usr/share/perl5/PVE/API2/Qemu.pm line 216
PVE::API2::Qemu::__ANON__(PVE::RPCEnvironment=HASH(0x56188bc37128),
"root\@pam", HASH(0x56188b405ab8), "x86_64", HASH(0x56188b409290), 8001,
undef, HASH(0x56188b405ab8), ...) called at
/usr/share/perl5/PVE/API2/Qemu.pm line 604 eval {...} called at
/usr/share/perl5/PVE/API2/Qemu.pm line 602 PVE::API2::Qemu::__ANON__()
called at /usr/share/perl5/PVE/Tools.pm line 203 eval {...} called at
/usr/share/perl5/PVE/Tools.pm line 203
PVE::Tools::lock_file_full("/var/lock/qemu-server/lock-8001.conf", 1, 0,
CODE(0x56188bc365e8)) called at /usr/share/perl5/PVE/Tools.pm line 212
PVE::Tools::lock_file("/var/lock/qemu-server/lock-8001.conf", 1,
CODE(0x56188bc365e8)) called at /usr/share/perl5/PVE/AbstractConfig.pm
line 78 PVE::AbstractConfig::lock_config_full("PVE::QemuConfig", 8001,
1, CODE(0x56188bc365e8)) called at /usr/share/perl5/PVE/API2/Qemu.pm
line 636 PVE::API2::Qemu::__ANON__() called at
/usr/share/perl5/PVE/API2/Qemu.pm line 659 eval {...} called at
/usr/share/perl5/PVE/API2/Qemu.pm line 659
PVE::API2::Qemu::__ANON__("UPID:px1-administration:00005D93:000AA66C:5E2B1A0E:qmcreate:8"...)
called at /usr/share/perl5/PVE/RESTEnvironment.pm line 610 eval {...}
called at /usr/share/perl5/PVE/RESTEnvironment.pm line 601
PVE::RESTEnvironment::fork_worker(PVE::RPCEnvironment=HASH(0x56188bc37128),
"qmcreate", 8001, "root\@pam", CODE(0x56188be420e0)) called at
/usr/share/perl5/PVE/API2/Qemu.pm line 671
PVE::API2::Qemu::__ANON__(HASH(0x56188b405ab8)) called at
/usr/share/perl5/PVE/RESTHandler.pm line 446
PVE::RESTHandler::handle("PVE::API2::Qemu", HASH(0x5618890d5be8),
HASH(0x56188b405ab8)) called at /usr/share/perl5/PVE/HTTPServer.pm line
167 eval {...} called at /usr/share/perl5/PVE/HTTPServer.pm line 133
PVE::HTTPServer::rest_handler(PVE::HTTPServer=HASH(0x56188bc367e0),
"10.249.15.146", "POST", "/nodes/px1-administration/qemu",
HASH(0x56188be46af8), HASH(0x56188bc791f8), "extjs") called at
/usr/share/perl5/PVE/APIServer/AnyEvent.pm line 712 eval {...} called
at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 686
PVE::APIServer::AnyEvent::handle_api2_request(PVE::HTTPServer=HASH(0x56188bc367e0),
HASH(0x56188be25c50), HASH(0x56188be46af8), "POST",
"/api2/extjs/nodes/px1-administration/qemu") called at
/usr/share/perl5/PVE/APIServer/AnyEvent.pm line 939 eval {...} called
at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 931
PVE::APIServer::AnyEvent::handle_request(PVE::HTTPServer=HASH(0x56188bc367e0),
HASH(0x56188be25c50), HASH(0x56188be46af8), "POST",
"/api2/extjs/nodes/px1-administration/qemu") called at
/usr/share/perl5/PVE/APIServer/AnyEvent.pm line 1325
PVE::APIServer::AnyEvent::__ANON__(AnyEvent::Handle=HASH(0x56188be1a520),
"numa=0\x{26}vmid=8001\x{26}sockets=1\x{26}cores=1\x{26}net0=virtio%2Cbridge%3Dvmb"...)
called at /usr/lib/x86_64-linux-gnu/perl5/5.24/AnyEvent/Handle.pm line
1522 AnyEvent::Handle::__ANON__(AnyEvent::Handle=HASH(0x56188be1a520))
called at /usr/lib/x86_64-linux-gnu/perl5/5.24/AnyEvent/Handle.pm line
1332
AnyEvent::Handle::_drain_rbuf(AnyEvent::Handle=HASH(0x56188be1a520))
called at /usr/lib/x86_64-linux-gnu/perl5/5.24/AnyEvent/Handle.pm line
2032 AnyEvent::Handle::__ANON__(EV::IO=SCALAR(0x56188be19860), 1)
called at /usr/lib/x86_64-linux-gnu/perl5/5.24/AnyEvent/Impl/EV.pm line
88 eval {...} called at
/usr/lib/x86_64-linux-gnu/perl5/5.24/AnyEvent/Impl/EV.pm line 88
AnyEvent::CondVar::Base::_wait(AnyEvent::CondVar=HASH(0x56188b405278))
called at /usr/lib/x86_64-linux-gnu/perl5/5.24/AnyEvent.pm line 2026
AnyEvent::CondVar::Base::recv(AnyEvent::CondVar=HASH(0x56188b405278))
called at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 1612
PVE::APIServer::AnyEvent::run(PVE::HTTPServer=HASH(0x56188bc367e0))
called at /usr/share/perl5/PVE/Service/<a href="http://pvedaemon.pm">pvedaemon.pm</a> line 52
PVE::Service::pvedaemon::run(PVE::Service::pvedaemon=HASH(0x561885644508))
called at /usr/share/perl5/PVE/Daemon.pm line 171 eval {...} called at
/usr/share/perl5/PVE/Daemon.pm line 171
PVE::Daemon::__ANON__(PVE::Service::pvedaemon=HASH(0x561885644508))
called at /usr/share/perl5/PVE/Daemon.pm line 391 eval {...} called at
/usr/share/perl5/PVE/Daemon.pm line 380
PVE::Daemon::__ANON__(PVE::Service::pvedaemon=HASH(0x561885644508),
undef) called at /usr/share/perl5/PVE/Daemon.pm line 552 eval {...}
called at /usr/share/perl5/PVE/Daemon.pm line 550
PVE::Daemon::start(PVE::Service::pvedaemon=HASH(0x561885644508), undef)
called at /usr/share/perl5/PVE/Daemon.pm line 661
PVE::Daemon::__ANON__(HASH(0x56188562a410)) called at
/usr/share/perl5/PVE/RESTHandler.pm line 446
PVE::RESTHandler::handle("PVE::Service::pvedaemon",
HASH(0x561885644820), HASH(0x56188562a410)) called at
/usr/share/perl5/PVE/RESTHandler.pm line 855 eval {...} called at
/usr/share/perl5/PVE/RESTHandler.pm line 838
PVE::RESTHandler::cli_handler("PVE::Service::pvedaemon", "pvedaemon
start", "start", ARRAY(0x5618858e94d0), ARRAY(0x561885644e98), undef,
undef, undef) called at /usr/share/perl5/PVE/CLIHandler.pm line 580
PVE::CLIHandler::__ANON__(ARRAY(0x56188562a638), CODE(0x5618858aa2d8),
undef) called at /usr/share/perl5/PVE/CLIHandler.pm line 657
PVE::CLIHandler::run_cli_handler("PVE::Service::pvedaemon", "prepare",
CODE(0x5618858aa2d8)) called at /usr/bin/pvedaemon line 27
</div><div><br></div><div>DRBD resources are correctly created on both sides. VMID.conf appears briefly, then disappears. And VM is NOT created.<br></div><div><br></div><div>We noticed long response time when :</div><div>linstor sp pl</div><div><br></div><div>An strace give a long wait (20s) to :</div><div>sendto(3, "GET /v1/view/storage-pools HTTP/"..., 153, 0, NULL, 0) = 153<br>poll([{fd=3, events=POLLIN}], 1, 300000</div><div><br></div><div>Live migration of running VMs works.<br></div><div><br></div><div>Any idea?</div><div><br></div><div> Thank you,</div><div><br></div><div> Christophe.<br></div></div>