[Cyberduck-trac] [Cyberduck] #10612: Failure to upload large files (200+ gb)

[Cyberduck]

#10612: Failure to upload large files (200+ gb)
---------------------------+-------------------------
    Reporter:  MarkBlaise  |      Owner:
        Type:  defect      |     Status:  new
    Priority:  normal      |  Milestone:
   Component:  core        |    Version:  6.9.0
    Severity:  normal      |   Keywords:
Architecture:  Intel       |   Platform:  Windows 8.1
---------------------------+-------------------------
 It may just be network hiccups, but Cyberduck v6.9.0 (29768) seems to have
 a lot of trouble uploading large files to my Wasabi cloud account.

 I have been using both Cyberduck and the Duck CLI, and have seen the issue
 with both.

 For example, this morning Duck failed to upload a 290 gb file, after 3
 hours of trying. I have a gigabit fiber optic connection, so speed is not
 the issue.

 I tried to enable debugging by following directions on
 https://trac.cyberduck.io/wiki/help/en/faq#Enabledebuglogging.
 - I created the file
 ''C:\Users\Mark\AppData\Roaming\Cyberduck\default.properties'' It has 1
 line of text ('''logging=debug''') - attached.
 - I also defined the system environment variable '''logging''' set equal
 to '''debug'''.

 When I run a Duck command I get:

 C:\Users\Mark>duck  -l  wasabisys://T5ZHIOV9VGMNBAKU1HQ4@Blaise-
 Archive/Magni/
  - log4j:WARN No appenders could be found for logger
 (ch.cyberduck.core.preferences.Preferences).
 - log4j:WARN Please initialize the log4j system properly.
 - log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
 more info.
 - Login successful…
 - Backup-Image-C-20190203.ndf

 So it would seem I can't get Duck to log.

 While the file is copying, there are 2 entries in the Cyberduck listing in
 gray text; 1 in the target folder, and 1 in the bucket root (see attached
 screen shot). These entries do not display a size. Once an upload
 completes the gray entry in the bucket root is gone, and the entry in the
 target folder is black, and displays a size.

 When an upload fails, the two gray entries remain. If I attempt the upload
 3 times and have 3 failures, I will have 6 gray entries (3 in the root, 3
 in the target folder), all with the same name.

 I guess the gray entries are temp files. Regrettably, when a failed upload
 is tried again, Duck does not "pickup where it left off" - it starts from
 the beginning again.

 So, I have to delete the grayed root entry, which will also remove the
 entry in the target folder. One cannot delete the entry in the target
 folder.

 Unfortunately, Wasabi cloud storage charges accrue for 3 months for any
 file uploaded, even if it is deleted prior to 3 months. This means that I
 will be paying for every failed file upload for 3 months of storage - and
 then also paying for the actual file once successfully uploaded.

 I would love to figure out why these uploads fail and correct the issue,
 so I can stop paying for 3 months of storage for each failed upload file
 fragment.

 I would also like to get logging working for Duck.

 Right now another copy attempt is being made for this morning's failure,
 After that, I'll try a large upload from the Cyberduck GUI to see if that
 is logging.

--
Ticket URL: <https://trac.cyberduck.io/ticket/10612>
Cyberduck <https://cyberduck.io>
Libre FTP, SFTP, WebDAV, S3 & OpenStack Swift browser for Mac and Windows