<div dir="ltr">Hi!<div><br></div><div>Yes it seems, the copy (ssh/dd) command fails while trying to dd the data into the drbd resource.</div><div>Unfortunately the log line is cut of:</div><div><span style="font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px">Oct 06 15:24:33 nts-cloud-100 cp[16558]: INFO running shell command: bash -c /var/lib/one/remotes//</span><span style="font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px">datastore/downloader.sh </span><a href="https://marketplace.opennebula.systems//appliance/e902e91f-4e83-4">https://marketplace.opennebula.systems//appliance/e902e91f-4e83-4</a><br></div><div><br></div><div>This line should fully contain the command executed, maybe if you try to run it manually it will give you a better hint to what fails,</div><div>maybe it is a simple ssh connection(wrong keys, handshake question) problem.</div><div><br></div><div>Cheers,</div><div>Rene</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Oct 7, 2019 at 8:47 AM Notespeak Fyyar <<a href="mailto:notespeak@ntshosts.com">notespeak@ntshosts.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><u></u><div><div style="font-family:Verdana,Arial,Helvetica,sans-serif;font-size:10pt"><div>Greetings DRBD users,<br></div><div>we have recently started using drbd9 as a primary mean of storing opennebula user data.<br></div><div>We have followed every instruction in the plugin guide, but unfortunately we are facing some sort of an execution/permission issues. We have tried to give oneadmin full tmp and datastore permissions and tweaked datastore settings, but the problem is always the same, and has been reproduced at least 3 times with a clean installation of opennebula, and drbd9. <br></div><div>It should also needs to be noted, that a similar case has already been posted recently <a href="https://lists.gt.net/drbd/users/30739" target="_blank">https://lists.gt.net/drbd/users/30739</a>. <br></div><div><br></div><div>ERROR (taken form /var/log/oned.log)<br></div><div><br></div><div>Command execution failed (exit code: 1): /var/lib/one/remotes/datastore/linstor/cp PERTX0RSSVZFUl9BQ1RJT05fREFUQT48SU1BR0U+PElEPjIyPC9JRD48VUlEPjA8L1VJRD48R0lEPjA8L0dJRD48VU5BTUU+b25lYWRtaW48L1VOQU1FPjxHTkFNRT5vbmVhZG1pbjwvR05BTUU+PE5BTUU+RGViaWFuIDggLSBLVk08L05BTUU+PFBFUk1JU1NJT05TPjxPV05FUl9VPjE8L09XTkVSX1U+PE9XTkVSX00+MTwvT1dORVJfTT48T1dORVJfQT4wPC9PV05FUl9BPjxHUk9VUF9VPjA8L0dST1VQX1U+PEdST1VQX00+MDwvR1JPVVBfTT48R1JPVVBfQT4wPC9HUk9VUF9BPjxPVEhFUl9VPjA8L09USEVSX1U+PE9USEVSX00+MDwvT1RIRVJfTT48T1RIRVJfQT4wPC9PVEhFUl9BPjwvUEVSTUlTU0lPTlM+PFRZUEU+MDwvVFlQRT48RElTS19UWVBFPjI8L0RJU0tfVFlQRT48UEVSU0lTVEVOVD4wPC9QRVJTSVNURU5UPjxSRUdUSU1FPjE1NzAzNzU0NzA8L1JFR1RJTUU+PFNPVVJDRT48IVtDREFUQVtdXT48L1NPVVJDRT48UEFUSD48IVtDREFUQVtodHRwczovL21hcmtldHBsYWNlLm9wZW5uZWJ1bGEuc3lzdGVtcy8vYXBwbGlhbmNlL2U5MDJlOTFmLTRlODMtNDBhZi1iNDFlLWI4OTVhYmYwNDYxZC9kb3dubG9hZC8wXV0+PC9QQVRIPjxGU1RZUEU+PCFbQ0RBVEFbXV0+PC9GU1RZUEU+PFNJWkU+MjA0ODwvU0laRT48U1RBVEU+NDwvU1RBVEU+PFJVTk5JTkdfVk1TPjA8L1JVTk5JTkdfVk1TPjxDTE9OSU5HX09QUz4wPC9DTE9OSU5HX09QUz48Q0xPTklOR19JRD4tMTwvQ0xPTklOR19JRD48VEFSR0VUX1NOQVBTSE9UPi0xPC9UQVJHRVRfU05BUFNIT1Q+PERBVEFTVE9SRV9JRD4xMDU8L0RBVEFTVE9SRV9JRD48REFUQVNUT1JFPmxpbnN0b3JfYXV0b19wbGFjZTwvREFUQVNUT1JFPjxWTVM+PC9WTVM+PENMT05FUz48L0NMT05FUz48QVBQX0NMT05FUz48L0FQUF9DTE9ORVM+PFRFTVBMQVRFPjxERVZfUFJFRklYPjwhW0NEQVRBW3ZkXV0+PC9ERVZfUFJFRklYPjxEUklWRVI+PCFbQ0RBVEFbcWNvdzJdXT48L0RSSVZFUj48Rk9STUFUPjwhW0NEQVRBW3Fjb3cyXV0+PC9GT1JNQVQ+PEZST01fQVBQPjwhW0NEQVRBWzE4XV0+PC9GUk9NX0FQUD48RlJPTV9BUFBfTUQ1PjwhW0NEQVRBWzcxOTI3ZmQwOTUxNjA1OWJjZGI5ZTgzZjM2ZDc3OWI3XV0+PC9GUk9NX0FQUF9NRDU+PEZST01fQVBQX05BTUU+PCFbQ0RBVEFbRGViaWFuIDggLSBLVk1dXT48L0ZST01fQVBQX05BTUU+PC9URU1QTEFURT48U05BUFNIT1RTPjxBTExPV19PUlBIQU5TPjwhW0NEQVRBW05PXV0+PC9BTExPV19PUlBIQU5TPjxDVVJSRU5UX0JBU0U+PCFbQ0RBVEFbLTFdXT48L0NVUlJFTlRfQkFTRT48TkVYVF9TTkFQU0hPVD48IVtDREFUQVswXV0+PC9ORVhUX1NOQVBTSE9UPjwvU05BUFNIT1RTPjwvSU1BR0U+PERBVEFTVE9SRT48SUQ+MTA1PC9JRD48VUlEPjA8L1VJRD48R0lEPjA8L0dJRD48VU5BTUU+b25lYWRtaW48L1VOQU1FPjxHTkFNRT5vbmVhZG1pbjwvR05BTUU+PE5BTUU+bGluc3Rvcl9hdXRvX3BsYWNlPC9OQU1FPjxQRVJNSVNTSU9OUz48T1dORVJfVT4xPC9PV05FUl9VPjxPV05FUl9NPjE8L09XTkVSX00+PE9XTkVSX0E+MDwvT1dORVJfQT48R1JPVVBfVT4xPC9HUk9VUF9VPjxHUk9VUF9NPjA8L0dST1VQX00+PEdST1VQX0E+MDwvR1JPVVBfQT48T1RIRVJfVT4wPC9PVEhFUl9VPjxPVEhFUl9NPjA8L09USEVSX00+PE9USEVSX0E+MDwvT1RIRVJfQT48L1BFUk1JU1NJT05TPjxEU19NQUQ+PCFbQ0RBVEFbbGluc3Rvcl1dPjwvRFNfTUFEPjxUTV9NQUQ+PCFbQ0RBVEFbbGluc3Rvcl1dPjwvVE1fTUFEPjxCQVNFX1BBVEg+PCFbQ0RBVEFbL3Zhci9saWIvb25lLy9kYXRhc3RvcmVzLzEwNV1dPjwvQkFTRV9QQVRIPjxUWVBFPjA8L1RZUEU+PERJU0tfVFlQRT4yPC9ESVNLX1RZUEU+PFNUQVRFPjA8L1NUQVRFPjxDTFVTVEVSUz48SUQ+MDwvSUQ+PC9DTFVTVEVSUz48VE9UQUxfTUI+MzY0MzgwNDwvVE9UQUxfTUI+PEZSRUVfTUI+MzY0MzgwNDwvRlJFRV9NQj48VVNFRF9NQj4wPC9VU0VEX01CPjxJTUFHRVM+PC9JTUFHRVM+PFRFTVBMQVRFPjxBTExPV19PUlBIQU5TPjwhW0NEQVRBW3llc11dPjwvQUxMT1dfT1JQSEFOUz48QlJJREdFX0xJU1Q+PCFbQ0RBVEFbbnRzLWNsb3VkLW4xMDEgbnRzLWNsb3VkLTEwMCBudHMtY2xvdWQtbjEwMl1dPjwvQlJJREdFX0xJU1Q+PENMT05FX1RBUkdFVD48IVtDREFUQVtTRUxGXV0+PC9DTE9ORV9UQVJHRVQ+PENMT05FX1RBUkdFVF9TSEFSRUQ+PCFbQ0RBVEFbU0VMRl1dPjwvQ0xPTkVfVEFSR0VUX1NIQVJFRD48Q0xPTkVfVEFSR0VUX1NTSD48IVtDREFUQVtTRUxGXV0+PC9DTE9ORV9UQVJHRVRfU1NIPjxESVNLX1RZUEU+PCFbQ0RBVEFbQkxPQ0tdXT48L0RJU0tfVFlQRT48RElTS19UWVBFX1NIQVJFRD48IVtDREFUQVtCTE9DS11dPjwvRElTS19UWVBFX1NIQVJFRD48RElTS19UWVBFX1NTSD48IVtDREFUQVtCTE9DS11dPjwvRElTS19UWVBFX1NTSD48RFNfTUFEPjwhW0NEQVRBW2xpbnN0b3JdXT48L0RTX01BRD48TElOU1RPUl9BVVRPX1BMQUNFPjwhW0NEQVRBWzJdXT48L0xJTlNUT1JfQVVUT19QTEFDRT48TElOU1RPUl9DTE9ORV9NT0RFPjwhW0NEQVRBW3NuYXBzaG90XV0+PC9MSU5TVE9SX0NMT05FX01PREU+PExJTlNUT1JfU1RPUkFHRV9QT09MPjwhW0NEQVRBW2RhdGFzdG9yZS1zc2RdXT48L0xJTlNUT1JfU1RPUkFHRV9QT09MPjxMTl9UQVJHRVQ+PCFbQ0RBVEFbTk9ORV1dPjwvTE5fVEFSR0VUPjxMTl9UQVJHRVRfU0hBUkVEPjwhW0NEQVRBW05PTkVdXT48L0xOX1RBUkdFVF9TSEFSRUQ+PExOX1RBUkdFVF9TU0g+PCFbQ0RBVEFbTk9ORV1dPjwvTE5fVEFSR0VUX1NTSD48UkVTVFJJQ1RFRF9ESVJTPjwhW0NEQVRBWy9dXT48L1JFU1RSSUNURURfRElSUz48U0FGRV9ESVJTPjwhW0NEQVRBWy92YXIvdG1wXV0+PC9TQUZFX0RJUlM+PFRNX01BRD48IVtDREFUQVtsaW5zdG9yXV0+PC9UTV9NQUQ+PFRNX01BRF9TWVNURU0+PCFbQ0RBVEFbc3NoLHNoYXJlZF1dPjwvVE1fTUFEX1NZU1RFTT48VFlQRT48IVtDREFUQVtJTUFHRV9EU11dPjwvVFlQRT48L1RFTVBMQVRFPjwvREFUQVNUT1JFPjxNQVJLRVRQTEFDRT48SUQ+MDwvSUQ+PFVJRD4wPC9VSUQ+PEdJRD4wPC9HSUQ+PFVOQU1FPm9uZWFkbWluPC9VTkFNRT48R05BTUU+b25lYWRtaW48L0dOQU1FPjxOQU1FPk9wZW5OZWJ1bGEgUHVibGljPC9OQU1FPjxNQVJLRVRfTUFEPjwhW0NEQVRBW29uZV1dPjwvTUFSS0VUX01BRD48Wk9ORV9JRD48IVtDREFUQVswXV0+PC9aT05FX0lEPjxUT1RBTF9NQj4wPC9UT1RBTF9NQj48RlJFRV9NQj4wPC9GUkVFX01CPjxVU0VEX01CPjA8L1VTRURfTUI+PE1BUktFVFBMQUNFQVBQUz48SUQ+MDwvSUQ+PElEPjE8L0lEPjxJRD4yPC9JRD48SUQ+MzwvSUQ+PElEPjQ8L0lEPjxJRD41PC9JRD48SUQ+NjwvSUQ+PElEPjc8L0lEPjxJRD44PC9JRD48SUQ+OTwvSUQ+PElEPjEwPC9JRD48SUQ+MTE8L0lEPjxJRD4xMjwvSUQ+PElEPjEzPC9JRD48SUQ+MTQ8L0lEPjxJRD4xNTwvSUQ+PElEPjE2PC9JRD48SUQ+MTc8L0lEPjxJRD4xODwvSUQ+PElEPjE5PC9JRD48SUQ+MjA8L0lEPjxJRD4yMTwvSUQ+PElEPjIyPC9JRD48SUQ+MjM8L0lEPjxJRD4yNDwvSUQ+PElEPjI1PC9JRD48SUQ+MjY8L0lEPjxJRD4yNzwvSUQ+PElEPjI4PC9JRD48SUQ+Mjk8L0lEPjxJRD4zMDwvSUQ+PElEPjMxPC9JRD48SUQ+MzI8L0lEPjxJRD4zMzwvSUQ+PElEPjM0PC9JRD48SUQ+MzU8L0lEPjxJRD4zNjwvSUQ+PElEPjM3PC9JRD48L01BUktFVFBMQUNFQVBQUz48UEVSTUlTU0lPTlM+PE9XTkVSX1U+MTwvT1dORVJfVT48T1dORVJfTT4xPC9PV05FUl9NPjxPV05FUl9BPjE8L09XTkVSX0E+PEdST1VQX1U+MTwvR1JPVVBfVT48R1JPVVBfTT4wPC9HUk9VUF9NPjxHUk9VUF9BPjA8L0dST1VQX0E+PE9USEVSX1U+MTwvT1RIRVJfVT48T1RIRVJfTT4wPC9PVEhFUl9NPjxPVEhFUl9BPjA8L09USEVSX0E+PC9QRVJNSVNTSU9OUz48VEVNUExBVEU+PERFU0NSSVBUSU9OPjwhW0NEQVRBW09wZW5OZWJ1bGEgU3lzdGVtcyBNYXJrZXRQbGFjZV1dPjwvREVTQ1JJUFRJT04+PE1BUktFVF9NQUQ+PCFbQ0RBVEFbb25lXV0+PC9NQVJLRVRfTUFEPjwvVEVNUExBVEU+PC9NQVJLRVRQTEFDRT48L0RTX0RSSVZFUl9BQ1RJT05fREFUQT4= 22<br></div><div>Sun Oct 6 15:25:34 2019 [Z0][ImM][E]: Error copying image in the datastore<br></div><div><br></div><div>journalctl opennebula entry:<br></div><div>INFO Entering datastore cp <br></div><div>Oct 06 15:24:30 nts-cloud-100 cp[16558]: INFO Deploying resource 'OpenNebula-Image-22'<br></div><div>Oct 06 15:24:32 nts-cloud-100 cp[16558]: INFO running shell command: bash -c source /var/lib/one/remotes//datastore/libfs.sh && set_up_datastore /var/lib/one//datastores/105 / /var/tmp<br></div><div>Oct 06 15:24:32 nts-cloud-100 cp[16558]: INFO running shell command: bash -c source /var/lib/one/remotes//datastore/libfs.sh && set_downloader_args "" "" "" "" <a href="https://marketplace.opennebula" target="_blank">https://marketplace.opennebula</a><br></div><div>Oct 06 15:24:32 nts-cloud-100 cp[16558]: INFO Downloading <a href="https://marketplace.opennebula.systems//appliance/e902e91f-4e83-40af-b41e-b895abf0461d/download/0" target="_blank">https://marketplace.opennebula.systems//appliance/e902e91f-4e83-40af-b41e-b895abf0461d/download/0</a> to the image repository<br></div><div>Oct 06 15:24:32 nts-cloud-100 cp[16558]: INFO running shell command: bash -c source /var/lib/one/remotes//datastore/libfs.sh && check_restricted <a href="https://marketplace.opennebula.systems//appli" target="_blank">https://marketplace.opennebula.systems//appli</a><br></div><div>Oct 06 15:24:32 nts-cloud-100 cp[16558]: INFO Copying local image <a href="https://marketplace.opennebula.systems//appliance/e902e91f-4e83-40af-b41e-b895abf0461d/download/0" target="_blank">https://marketplace.opennebula.systems//appliance/e902e91f-4e83-40af-b41e-b895abf0461d/download/0</a> to the image repository<br></div><div>Oct 06 15:24:33 nts-cloud-100 cp[16558]: INFO running shell command: bash -c /var/lib/one/remotes//datastore/downloader.sh <a href="https://marketplace.opennebula.systems//appliance/e902e91f-4e83-4" target="_blank">https://marketplace.opennebula.systems//appliance/e902e91f-4e83-4</a><br></div><div>Oct 06 15:24:50 nts-cloud-100 monitor[16860]: INFO Entering datastore monitor.<br></div><div>Oct 06 15:24:50 nts-cloud-100 monitor[16854]: INFO Entering datastore monitor.<br></div><div>Oct 06 15:24:50 nts-cloud-100 monitor[16877]: INFO Entering tm monitor.<br></div><div>Oct 06 15:24:50 nts-cloud-100 monitor[16860]: INFO Exiting datastore monitor[274/3643804/3643530].<br></div><div>Oct 06 15:24:50 nts-cloud-100 monitor[16854]: INFO Exiting datastore monitor[274/3643804/3643530].<br></div><div>Oct 06 15:24:51 nts-cloud-100 monitor[16877]: INFO Leaving tm monitor.<br></div><div>Oct 06 15:25:33 nts-cloud-100 cp[16558]: [334B blob data]<br></div><div>Oct 06 15:25:33 nts-cloud-100 cp[16558]: ERROR "Error copying data to OpenNebula-Image-22 on nts-cloud-100"<br></div><div><br></div><div>journalctl DRBD9 entry:<br></div><div>Oct 06 15:10:17 nts-cloud-100 Controller[1545]: 15:10:17.294 [grizzly-http-server-26] INFO LINSTOR/Controller - SYSTEM - New volume definition with number '0' of resource definition 'OpenNe<br></div><div>Oct 06 15:24:30 nts-cloud-100 Controller[1545]: 15:24:30.825 [grizzly-http-server-27] INFO LINSTOR/Controller - SYSTEM - New volume definition with number '0' of resource definition 'OpenNe<br></div><div>Oct 06 15:58:28 nts-cloud-100 Controller[1545]: 15:58:28.694 [grizzly-http-server-24] INFO LINSTOR/Controller - SYSTEM - New volume definition with number '0' of resource definition 'OpenNe<br></div><div>Oct 06 15:59:43 nts-cloud-100 Controller[1545]: 15:59:43.348 [grizzly-http-server-7] INFO LINSTOR/Controller - SYSTEM - New volume definition with number '0' of resource definition 'OpenNeb<br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div>The datastore configs in opennebula are following:<br></div><div><br></div><div>NAME="datastore-ssd-image"<br></div><div>TYPE="IMAGE_DS"<br></div><div>STORAGE_POOL="datastore-ssd"<br></div><div>AUTO_PLACE="2"<br></div><div>BRIDGE_LIST="n101 n100 n102"<br></div><div>DISK_TYPE="BLOCK"<br></div><div>DS_MAD="linstor_un"<br></div><div>TM_MAD="linstor_un"<br></div><div><br></div><div><br></div><div><br></div><div>NAME = linstor_auto_place233<br></div><div>TM_MAD = linstor<br></div><div>TYPE = SYSTEM_DS<br></div><div>LINSTOR_AUTO_PLACE = 2<br></div><div>LINSTOR_STORAGE_POOL = "datastore-ssd"<br></div><div>BRIDGE_LIST = "n101 n100 n102"<br></div><div><br></div><div><br></div><div>All the best,<br></div><div>Notespeak.</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div></div><br></div>_______________________________________________<br>
Star us on GITHUB: <a href="https://github.com/LINBIT" rel="noreferrer" target="_blank">https://github.com/LINBIT</a><br>
drbd-user mailing list<br>
<a href="mailto:drbd-user@lists.linbit.com" target="_blank">drbd-user@lists.linbit.com</a><br>
<a href="https://lists.linbit.com/mailman/listinfo/drbd-user" rel="noreferrer" target="_blank">https://lists.linbit.com/mailman/listinfo/drbd-user</a><br>
</blockquote></div>