[DRBD-user] Error when copying/downloading an image in opennebula datastore

Tue Oct 8 10:49:54 CEST 2019

Hi!

Yes it seems, the copy (ssh/dd) command fails while trying to dd the data
into the drbd resource.
Unfortunately the log line is cut of:
Oct 06 15:24:33 nts-cloud-100 cp[16558]: INFO running shell command: bash
-c /var/lib/one/remotes//datastore/downloader.sh
https://marketplace.opennebula.systems//appliance/e902e91f-4e83-4

This line should fully contain the command executed, maybe if you try to
run it manually it will give you a better hint to what fails,
maybe it is a simple ssh connection(wrong keys, handshake question) problem.

Cheers,
Rene

On Mon, Oct 7, 2019 at 8:47 AM Notespeak Fyyar <notespeak at ntshosts.com>
wrote:

> Greetings DRBD users,
> we have recently started using  drbd9 as a primary mean of storing
> opennebula user data.
> We have followed every instruction in the plugin guide, but unfortunately
> we are facing some sort of an execution/permission issues. We have tried to
> give oneadmin full tmp and datastore permissions and tweaked datastore
> settings, but the problem is always the same, and has been reproduced at
> least 3 times with a clean installation of opennebula, and drbd9.
> It should also needs to be noted, that a similar case has already been
> posted recently https://lists.gt.net/drbd/users/30739.
>
> ERROR (taken form /var/log/oned.log)
>
> Command execution failed (exit code: 1):
> /var/lib/one/remotes/datastore/linstor/cp
> PERTX0RSSVZFUl9BQ1RJT05fREFUQT48SU1BR0U+PElEPjIyPC9JRD48VUlEPjA8L1VJRD48R0lEPjA8L0dJRD48VU5BTUU+b25lYWRtaW48L1VOQU1FPjxHTkFNRT5vbmVhZG1pbjwvR05BTUU+PE5BTUU+RGViaWFuIDggLSBLVk08L05BTUU+PFBFUk1JU1NJT05TPjxPV05FUl9VPjE8L09XTkVSX1U+PE9XTkVSX00+MTwvT1dORVJfTT48T1dORVJfQT4wPC9PV05FUl9BPjxHUk9VUF9VPjA8L0dST1VQX1U+PEdST1VQX00+MDwvR1JPVVBfTT48R1JPVVBfQT4wPC9HUk9VUF9BPjxPVEhFUl9VPjA8L09USEVSX1U+PE9USEVSX00+MDwvT1RIRVJfTT48T1RIRVJfQT4wPC9PVEhFUl9BPjwvUEVSTUlTU0lPTlM+PFRZUEU+MDwvVFlQRT48RElTS19UWVBFPjI8L0RJU0tfVFlQRT48UEVSU0lTVEVOVD4wPC9QRVJTSVNURU5UPjxSRUdUSU1FPjE1NzAzNzU0NzA8L1JFR1RJTUU+PFNPVVJDRT48IVtDREFUQVtdXT48L1NPVVJDRT48UEFUSD48IVtDREFUQVtodHRwczovL21hcmtldHBsYWNlLm9wZW5uZWJ1bGEuc3lzdGVtcy8vYXBwbGlhbmNlL2U5MDJlOTFmLTRlODMtNDBhZi1iNDFlLWI4OTVhYmYwNDYxZC9kb3dubG9hZC8wXV0+PC9QQVRIPjxGU1RZUEU+PCFbQ0RBVEFbXV0+PC9GU1RZUEU+PFNJWkU+MjA0ODwvU0laRT48U1RBVEU+NDwvU1RBVEU+PFJVTk5JTkdfVk1TPjA8L1JVTk5JTkdfVk1TPjxDTE9OSU5HX09QUz4wPC9DTE9OSU5HX09QUz48Q0xPTklOR19JRD4tMTwvQ0xPTklOR19JRD48VEFSR0VUX1NOQVBTSE9UPi0xPC9UQVJHRVRfU05BUFNIT1Q+PERBVEFTVE9SRV9JRD4xMDU8L0RBVEFTVE9SRV9JRD48REFUQVNUT1JFPmxpbnN0b3JfYXV0b19wbGFjZTwvREFUQVNUT1JFPjxWTVM+PC9WTVM+PENMT05FUz48L0NMT05FUz48QVBQX0NMT05FUz48L0FQUF9DTE9ORVM+PFRFTVBMQVRFPjxERVZfUFJFRklYPjwhW0NEQVRBW3ZkXV0+PC9ERVZfUFJFRklYPjxEUklWRVI+PCFbQ0RBVEFbcWNvdzJdXT48L0RSSVZFUj48Rk9STUFUPjwhW0NEQVRBW3Fjb3cyXV0+PC9GT1JNQVQ+PEZST01fQVBQPjwhW0NEQVRBWzE4XV0+PC9GUk9NX0FQUD48RlJPTV9BUFBfTUQ1PjwhW0NEQVRBWzcxOTI3ZmQwOTUxNjA1OWJjZGI5ZTgzZjM2ZDc3OWI3XV0+PC9GUk9NX0FQUF9NRDU+PEZST01fQVBQX05BTUU+PCFbQ0RBVEFbRGViaWFuIDggLSBLVk1dXT48L0ZST01fQVBQX05BTUU+PC9URU1QTEFURT48U05BUFNIT1RTPjxBTExPV19PUlBIQU5TPjwhW0NEQVRBW05PXV0+PC9BTExPV19PUlBIQU5TPjxDVVJSRU5UX0JBU0U+PCFbQ0RBVEFbLTFdXT48L0NVUlJFTlRfQkFTRT48TkVYVF9TTkFQU0hPVD48IVtDREFUQVswXV0+PC9ORVhUX1NOQVBTSE9UPjwvU05BUFNIT1RTPjwvSU1BR0U+PERBVEFTVE9SRT48SUQ+MTA1PC9JRD48VUlEPjA8L1VJRD48R0lEPjA8L0dJRD48VU5BTUU+b25lYWRtaW48L1VOQU1FPjxHTkFNRT5vbmVhZG1pbjwvR05BTUU+PE5BTUU+bGluc3Rvcl9hdXRvX3BsYWNlPC9OQU1FPjxQRVJNSVNTSU9OUz48T1dORVJfVT4xPC9PV05FUl9VPjxPV05FUl9NPjE8L09XTkVSX00+PE9XTkVSX0E+MDwvT1dORVJfQT48R1JPVVBfVT4xPC9HUk9VUF9VPjxHUk9VUF9NPjA8L0dST1VQX00+PEdST1VQX0E+MDwvR1JPVVBfQT48T1RIRVJfVT4wPC9PVEhFUl9VPjxPVEhFUl9NPjA8L09USEVSX00+PE9USEVSX0E+MDwvT1RIRVJfQT48L1BFUk1JU1NJT05TPjxEU19NQUQ+PCFbQ0RBVEFbbGluc3Rvcl1dPjwvRFNfTUFEPjxUTV9NQUQ+PCFbQ0RBVEFbbGluc3Rvcl1dPjwvVE1fTUFEPjxCQVNFX1BBVEg+PCFbQ0RBVEFbL3Zhci9saWIvb25lLy9kYXRhc3RvcmVzLzEwNV1dPjwvQkFTRV9QQVRIPjxUWVBFPjA8L1RZUEU+PERJU0tfVFlQRT4yPC9ESVNLX1RZUEU+PFNUQVRFPjA8L1NUQVRFPjxDTFVTVEVSUz48SUQ+MDwvSUQ+PC9DTFVTVEVSUz48VE9UQUxfTUI+MzY0MzgwNDwvVE9UQUxfTUI+PEZSRUVfTUI+MzY0MzgwNDwvRlJFRV9NQj48VVNFRF9NQj4wPC9VU0VEX01CPjxJTUFHRVM+PC9JTUFHRVM+PFRFTVBMQVRFPjxBTExPV19PUlBIQU5TPjwhW0NEQVRBW3llc11dPjwvQUxMT1dfT1JQSEFOUz48QlJJREdFX0xJU1Q+PCFbQ0RBVEFbbnRzLWNsb3VkLW4xMDEgbnRzLWNsb3VkLTEwMCBudHMtY2xvdWQtbjEwMl1dPjwvQlJJREdFX0xJU1Q+PENMT05FX1RBUkdFVD48IVtDREFUQVtTRUxGXV0+PC9DTE9ORV9UQVJHRVQ+PENMT05FX1RBUkdFVF9TSEFSRUQ+PCFbQ0RBVEFbU0VMRl1dPjwvQ0xPTkVfVEFSR0VUX1NIQVJFRD48Q0xPTkVfVEFSR0VUX1NTSD48IVtDREFUQVtTRUxGXV0+PC9DTE9ORV9UQVJHRVRfU1NIPjxESVNLX1RZUEU+PCFbQ0RBVEFbQkxPQ0tdXT48L0RJU0tfVFlQRT48RElTS19UWVBFX1NIQVJFRD48IVtDREFUQVtCTE9DS11dPjwvRElTS19UWVBFX1NIQVJFRD48RElTS19UWVBFX1NTSD48IVtDREFUQVtCTE9DS11dPjwvRElTS19UWVBFX1NTSD48RFNfTUFEPjwhW0NEQVRBW2xpbnN0b3JdXT48L0RTX01BRD48TElOU1RPUl9BVVRPX1BMQUNFPjwhW0NEQVRBWzJdXT48L0xJTlNUT1JfQVVUT19QTEFDRT48TElOU1RPUl9DTE9ORV9NT0RFPjwhW0NEQVRBW3NuYXBzaG90XV0+PC9MSU5TVE9SX0NMT05FX01PREU+PExJTlNUT1JfU1RPUkFHRV9QT09MPjwhW0NEQVRBW2RhdGFzdG9yZS1zc2RdXT48L0xJTlNUT1JfU1RPUkFHRV9QT09MPjxMTl9UQVJHRVQ+PCFbQ0RBVEFbTk9ORV1dPjwvTE5fVEFSR0VUPjxMTl9UQVJHRVRfU0hBUkVEPjwhW0NEQVRBW05PTkVdXT48L0xOX1RBUkdFVF9TSEFSRUQ+PExOX1RBUkdFVF9TU0g+PCFbQ0RBVEFbTk9ORV1dPjwvTE5fVEFSR0VUX1NTSD48UkVTVFJJQ1RFRF9ESVJTPjwhW0NEQVRBWy9dXT48L1JFU1RSSUNURURfRElSUz48U0FGRV9ESVJTPjwhW0NEQVRBWy92YXIvdG1wXV0+PC9TQUZFX0RJUlM+PFRNX01BRD48IVtDREFUQVtsaW5zdG9yXV0+PC9UTV9NQUQ+PFRNX01BRF9TWVNURU0+PCFbQ0RBVEFbc3NoLHNoYXJlZF1dPjwvVE1fTUFEX1NZU1RFTT48VFlQRT48IVtDREFUQVtJTUFHRV9EU11dPjwvVFlQRT48L1RFTVBMQVRFPjwvREFUQVNUT1JFPjxNQVJLRVRQTEFDRT48SUQ+MDwvSUQ+PFVJRD4wPC9VSUQ+PEdJRD4wPC9HSUQ+PFVOQU1FPm9uZWFkbWluPC9VTkFNRT48R05BTUU+b25lYWRtaW48L0dOQU1FPjxOQU1FPk9wZW5OZWJ1bGEgUHVibGljPC9OQU1FPjxNQVJLRVRfTUFEPjwhW0NEQVRBW29uZV1dPjwvTUFSS0VUX01BRD48Wk9ORV9JRD48IVtDREFUQVswXV0+PC9aT05FX0lEPjxUT1RBTF9NQj4wPC9UT1RBTF9NQj48RlJFRV9NQj4wPC9GUkVFX01CPjxVU0VEX01CPjA8L1VTRURfTUI+PE1BUktFVFBMQUNFQVBQUz48SUQ+MDwvSUQ+PElEPjE8L0lEPjxJRD4yPC9JRD48SUQ+MzwvSUQ+PElEPjQ8L0lEPjxJRD41PC9JRD48SUQ+NjwvSUQ+PElEPjc8L0lEPjxJRD44PC9JRD48SUQ+OTwvSUQ+PElEPjEwPC9JRD48SUQ+MTE8L0lEPjxJRD4xMjwvSUQ+PElEPjEzPC9JRD48SUQ+MTQ8L0lEPjxJRD4xNTwvSUQ+PElEPjE2PC9JRD48SUQ+MTc8L0lEPjxJRD4xODwvSUQ+PElEPjE5PC9JRD48SUQ+MjA8L0lEPjxJRD4yMTwvSUQ+PElEPjIyPC9JRD48SUQ+MjM8L0lEPjxJRD4yNDwvSUQ+PElEPjI1PC9JRD48SUQ+MjY8L0lEPjxJRD4yNzwvSUQ+PElEPjI4PC9JRD48SUQ+Mjk8L0lEPjxJRD4zMDwvSUQ+PElEPjMxPC9JRD48SUQ+MzI8L0lEPjxJRD4zMzwvSUQ+PElEPjM0PC9JRD48SUQ+MzU8L0lEPjxJRD4zNjwvSUQ+PElEPjM3PC9JRD48L01BUktFVFBMQUNFQVBQUz48UEVSTUlTU0lPTlM+PE9XTkVSX1U+MTwvT1dORVJfVT48T1dORVJfTT4xPC9PV05FUl9NPjxPV05FUl9BPjE8L09XTkVSX0E+PEdST1VQX1U+MTwvR1JPVVBfVT48R1JPVVBfTT4wPC9HUk9VUF9NPjxHUk9VUF9BPjA8L0dST1VQX0E+PE9USEVSX1U+MTwvT1RIRVJfVT48T1RIRVJfTT4wPC9PVEhFUl9NPjxPVEhFUl9BPjA8L09USEVSX0E+PC9QRVJNSVNTSU9OUz48VEVNUExBVEU+PERFU0NSSVBUSU9OPjwhW0NEQVRBW09wZW5OZWJ1bGEgU3lzdGVtcyBNYXJrZXRQbGFjZV1dPjwvREVTQ1JJUFRJT04+PE1BUktFVF9NQUQ+PCFbQ0RBVEFbb25lXV0+PC9NQVJLRVRfTUFEPjwvVEVNUExBVEU+PC9NQVJLRVRQTEFDRT48L0RTX0RSSVZFUl9BQ1RJT05fREFUQT4=
> 22
> Sun Oct  6 15:25:34 2019 [Z0][ImM][E]: Error copying image in the datastore
>
> journalctl opennebula entry:
> INFO Entering datastore cp
> Oct 06 15:24:30 nts-cloud-100 cp[16558]: INFO Deploying resource
> 'OpenNebula-Image-22'
> Oct 06 15:24:32 nts-cloud-100 cp[16558]: INFO running shell command: bash
> -c source /var/lib/one/remotes//datastore/libfs.sh && set_up_datastore
> /var/lib/one//datastores/105 / /var/tmp
> Oct 06 15:24:32 nts-cloud-100 cp[16558]: INFO running shell command: bash
> -c source /var/lib/one/remotes//datastore/libfs.sh && set_downloader_args
> "" "" "" "" https://marketplace.opennebula
> Oct 06 15:24:32 nts-cloud-100 cp[16558]: INFO Downloading
> https://marketplace.opennebula.systems//appliance/e902e91f-4e83-40af-b41e-b895abf0461d/download/0
> to the image repository
> Oct 06 15:24:32 nts-cloud-100 cp[16558]: INFO running shell command: bash
> -c source /var/lib/one/remotes//datastore/libfs.sh && check_restricted
> https://marketplace.opennebula.systems//appli
> Oct 06 15:24:32 nts-cloud-100 cp[16558]: INFO Copying local image
> https://marketplace.opennebula.systems//appliance/e902e91f-4e83-40af-b41e-b895abf0461d/download/0
> to the image repository
> Oct 06 15:24:33 nts-cloud-100 cp[16558]: INFO running shell command: bash
> -c /var/lib/one/remotes//datastore/downloader.sh
> https://marketplace.opennebula.systems//appliance/e902e91f-4e83-4
> Oct 06 15:24:50 nts-cloud-100 monitor[16860]: INFO Entering datastore
> monitor.
> Oct 06 15:24:50 nts-cloud-100 monitor[16854]: INFO Entering datastore
> monitor.
> Oct 06 15:24:50 nts-cloud-100 monitor[16877]: INFO Entering tm monitor.
> Oct 06 15:24:50 nts-cloud-100 monitor[16860]: INFO Exiting datastore
> monitor[274/3643804/3643530].
> Oct 06 15:24:50 nts-cloud-100 monitor[16854]: INFO Exiting datastore
> monitor[274/3643804/3643530].
> Oct 06 15:24:51 nts-cloud-100 monitor[16877]: INFO Leaving tm monitor.
> Oct 06 15:25:33 nts-cloud-100 cp[16558]: [334B blob data]
> Oct 06 15:25:33 nts-cloud-100 cp[16558]: ERROR "Error copying data to
> OpenNebula-Image-22 on nts-cloud-100"
>
> journalctl DRBD9 entry:
> Oct 06 15:10:17 nts-cloud-100 Controller[1545]: 15:10:17.294
> [grizzly-http-server-26] INFO  LINSTOR/Controller - SYSTEM - New volume
> definition with number '0' of resource definition 'OpenNe
> Oct 06 15:24:30 nts-cloud-100 Controller[1545]: 15:24:30.825
> [grizzly-http-server-27] INFO  LINSTOR/Controller - SYSTEM - New volume
> definition with number '0' of resource definition 'OpenNe
> Oct 06 15:58:28 nts-cloud-100 Controller[1545]: 15:58:28.694
> [grizzly-http-server-24] INFO  LINSTOR/Controller - SYSTEM - New volume
> definition with number '0' of resource definition 'OpenNe
> Oct 06 15:59:43 nts-cloud-100 Controller[1545]: 15:59:43.348
> [grizzly-http-server-7] INFO  LINSTOR/Controller - SYSTEM - New volume
> definition with number '0' of resource definition 'OpenNeb
>
>
>
>
>
>
>
>
>
>
>
>
> The datastore configs in opennebula are following:
>
> NAME="datastore-ssd-image"
> TYPE="IMAGE_DS"
> STORAGE_POOL="datastore-ssd"
> AUTO_PLACE="2"
> BRIDGE_LIST="n101 n100 n102"
> DISK_TYPE="BLOCK"
> DS_MAD="linstor_un"
> TM_MAD="linstor_un"
>
>
>
> NAME = linstor_auto_place233
> TM_MAD = linstor
> TYPE = SYSTEM_DS
> LINSTOR_AUTO_PLACE = 2
> LINSTOR_STORAGE_POOL = "datastore-ssd"
> BRIDGE_LIST = "n101 n100 n102"
>
>
> All the best,
> Notespeak.
>
>
>
>
>
>
>
> _______________________________________________
> Star us on GITHUB: https://github.com/LINBIT
> drbd-user mailing list
> drbd-user at lists.linbit.com
> https://lists.linbit.com/mailman/listinfo/drbd-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20191008/620d2b1c/attachment-0001.htm>