Downloading fail loop


#1

I have a problem downloading an image.

Running on a Beaglebone Black Resin OS 1.24.0

This is what happened when i started:

19.01.17 16:58:53 [+0000] Starting application 'registry.resin.io/loggerbox/998985246fc4d39114484efe4115afafa0dfaa8c' 19.01.17 16:58:54 [+0000] Application is already running 'registry.resin.io/loggerbox/998985246fc4d39114484efe4115afafa0dfaa8c' 19.01.17 16:58:57 [+0000] Downloading application 'registry.resin.io/loggerbox/1d13e377e983b158f3d5bc7dd391ab5b55c41adc' 19.01.17 17:09:36 [+0000] Failed to download application 'registry.resin.io/loggerbox/1d13e377e983b158f3d5bc7dd391ab5b55c41adc' due to 'Error pulling image (latest) from registry.resin.io/loggerbox/1d13e377e983b158f3d5bc7dd391ab5b55c41adc, failed to register layer: ApplyLayer flate: corrupt input before offset 56661767 stdout: {"layerSize":147275709} stderr: ' 19.01.17 17:09:41 [+0000] Downloading application 'registry.resin.io/loggerbox/1d13e377e983b158f3d5bc7dd391ab5b55c41adc' 19.01.17 17:16:53 [+0000] Failed to download application 'registry.resin.io/loggerbox/1d13e377e983b158f3d5bc7dd391ab5b55c41adc' due to 'Error pulling image (latest) from registry.resin.io/loggerbox/1d13e377e983b158f3d5bc7dd391ab5b55c41adc, failed to register layer: ApplyLayer exit status 1 stdout: stderr: unexpected EOF' 19.01.17 17:16:54 [+0000] Downloading application 'registry.resin.io/loggerbox/1d13e377e983b158f3d5bc7dd391ab5b55c41adc' 19.01.17 17:09:36 [+0000] Failed to download application 'registry.resin.io/loggerbox/1d13e377e983b158f3d5bc7dd391ab5b55c41adc' due to 'Error pulling image (latest) from registry.resin.io/loggerbox/1d13e377e983b158f3d5bc7dd391ab5b55c41adc, failed to register layer: ApplyLayer flate: corrupt input before offset 56661767 stdout: {"layerSize":147275709} stderr: ' 19.01.17 17:09:41 [+0000] Downloading application 'registry.resin.io/loggerbox/1d13e377e983b158f3d5bc7dd391ab5b55c41adc' 19.01.17 17:16:53 [+0000] Failed to download application 'registry.resin.io/loggerbox/1d13e377e983b158f3d5bc7dd391ab5b55c41adc' due to 'Error pulling image (latest) from registry.resin.io/loggerbox/1d13e377e983b158f3d5bc7dd391ab5b55c41adc, failed to register layer: ApplyLayer exit status 1 stdout: stderr: unexpected EOF' 19.01.17 17:16:54 [+0000] Downloading application 'registry.resin.io/loggerbox/1d13e377e983b158f3d5bc7dd391ab5b55c41adc' 19.01.17 17:24:06 [+0000] Downloading application 'registry.resin.io/loggerbox/1d13e377e983b158f3d5bc7dd391ab5b55c41adc'

then by this morning it’s doing this

20.01.17 10:11:23 [+0000] Downloading application 'registry.resin.io/loggerbox/1d13e377e983b158f3d5bc7dd391ab5b55c41adc' 20.01.17 10:12:49 [+0000] Failed to download application 'registry.resin.io/loggerbox/1d13e377e983b158f3d5bc7dd391ab5b55c41adc' due to 'Error pulling image (latest) from registry.resin.io/loggerbox/1d13e377e983b158f3d5bc7dd391ab5b55c41adc, open /var/lib/docker/tmp/GetImageBlob491910678: read-only file system' 20.01.17 10:12:51 [+0000] Error applying device configuration: Error: update "deviceConfig" set "targetValues" = '{}' - SQLITE_READONLY: attempt to write a readonly database 20.01.17 10:12:51 [+0000] Downloading application 'registry.resin.io/loggerbox/1d13e377e983b158f3d5bc7dd391ab5b55c41adc' 20.01.17 10:14:18 [+0000] Failed to download application 'registry.resin.io/loggerbox/1d13e377e983b158f3d5bc7dd391ab5b55c41adc' due to 'Error pulling image (latest) from registry.resin.io/loggerbox/1d13e377e983b158f3d5bc7dd391ab5b55c41adc, open /var/lib/docker/tmp/GetImageBlob474350563: read-only file system' 20.01.17 10:14:19 [+0000] Error applying device configuration: Error: update "deviceConfig" set "targetValues" = '{}' - SQLITE_READONLY: attempt to write a readonly database


SOLVED: Downloading Loop Fail - "no space left on device"
#2

That seems like a file system corruption, taking a look!


#3

@keithejc it wasn’t a corruption (just the symptoms are similar), rather that the file system seem to have run out of space. It seemed that it has updated fine in the end, though,

It’s a BTRFS issue as Docker + BTRFS uses quite a bit more space for metadata for all data too. Looks like the other devices in your application are also near similar state as this was, and might require some attention in the near future.

By the way, are Binary Delta Updates enabled on the device? That might help with the download and space issues (basically needs adding RESIN_SUPERVISOR_DELTA equal to 1 to the Fleet or Device configuration):

UPDATE: no, not quite fixed on the next update, checking it a bit.


#4

So, have run a fix routine on the device, and it seems to be working again, but it’s on the edge, just enough space to maybe run the update correctly (but very little free space left). Could you let us know if deltas are enabled for this device or application?


#5

Thanks. Deltas were not enabled. I have enabled it now.

Yeah, the internal mmc is not very large on the BBB. Would give us more leeway if we could boot from the external sdcard, but looks like we’re going now.

Thanks for your help