[Cyberduck-trac] [Cyberduck] #10612: Failure to upload large files (200+ gb)

[Cyberduck]

#10612: Failure to upload large files (200+ gb)
-------------------------+-------------------------
 Reporter:  MarkBlaise   |         Owner:  dkocher
     Type:  defect       |        Status:  assigned
 Priority:  normal       |     Milestone:  7.0
Component:  cli          |       Version:  6.9.0
 Severity:  normal       |    Resolution:
 Keywords:               |  Architecture:  Intel
 Platform:  Windows 8.1  |
-------------------------+-------------------------

Comment (by MarkBlaise):

 Two things ...

 (1) Last night a large file upload to the Wasabi cloud failed. I was
 working with Wasabi support, and I had turned on Wasabi account logging,
 Here's what they said:

 ''I and one of the engineering team member who is working on this case
 looked into the logs and found that when the upload is almost complete,
 Cyberduck sends a PUT request of the whole file which in turn is rejected
 by our API as it is above the threshold of the multi-part upload for a
 single part. The error generated is given below:
 ''

 {{{
 <Error>
  <Code>EntityTooLarge</Code>
  <Message>Your proposed upload exceeds the maximum allowed size</Message>
  <ProposedSize>297379400337</ProposedSize>
  <MaxSizeAllowed>5368709120</MaxSizeAllowed>
  <RequestId>03FF43F2B13795F8</RequestId>
 <HostId>l1BlqtLcUAY/sAC1U8mJhc7dsAlJf+qNqRz3MwqBwZh3DqbSuDaCP64ZkiCRtzkaGYdlLEva+/uH</HostId>
 </Error>

 }}}

 (2) Note that I am currently using a Wasabi trial account to test the
 feasibility of using their service and Cyberduck for our backup purposes.
 The trial has a 1 TB storage limit.

 This morning I attempted to upload a 122 GiB file via Duck with the -v
 switch. There was more than 300 GiB of space available. The upload failed.
 Here is the command line used:


 {{{
 Z:\Backup-Staging\Uploaded>duck  -v -P -e rename --upload
 wasabisys://HKIAYRYIVIYKGMHKPSN8@Bax-Backu
 p/Test/ \\odin\Backup-Staging\Uploaded\20190209-Guinan-Full-Bax-
 Bck_20190210015533.nbd > Copy-122gb-
 File.log
 log4j:WARN No appenders could be found for logger
 (ch.cyberduck.core.preferences.Preferences).
 log4j:WARN Please initialize the log4j system properly.
 log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
 more info.
 Z:\Backup-Staging\Uploaded>
 }}}

 I have attached a ZIP of the log to which output was redirected, '''Copy-
 122gb-File.zip'''

 If I understand the Wasabi log correctly, this is a problem Cyberduck has
 uploading large files (to Wasabi); it is not about available space.

 Please let me know what you determine.

 Also, why can I not get Duck logging working?[[BR]]
 '''log4j:WARN No appenders could be found for logger
 (ch.cyberduck.core.preferences.Preferences)'''[[BR]]
 If I understand the information at the link in the error message[[BR]]
 http://logging.apache.org/log4j/1.2/faq.html#noconfig [[BR]]
 it has to do with Duck/Cyberduck Java configuration.

--
Ticket URL: <https://trac.cyberduck.io/ticket/10612#comment:6>
Cyberduck <https://cyberduck.io>
Libre FTP, SFTP, WebDAV, S3 & OpenStack Swift browser for Mac and Windows