[Cyberduck-trac] [Cyberduck] #8919: Failed On Large Uploads: Mismatch between MD5 hash and ETag

Cyberduck trac at trac.cyberduck.io
Thu Jul 9 13:42:27 UTC 2015


#8919: Failed On Large Uploads:  Mismatch between MD5 hash and ETag
--------------------------------------------+------------------------
 Reporter:  collector                       |         Owner:  dkocher
     Type:  defect                          |        Status:  new
 Priority:  high                            |     Milestone:
Component:  s3                              |       Version:  4.7.1
 Severity:  blocker                         |    Resolution:
 Keywords:  Mismatch MD5 ETag large upload  |  Architecture:  Intel
 Platform:  Mac OS X 10.10                  |
--------------------------------------------+------------------------
Description changed by dkocher:

Old description:

> For several months since buying CyberDuck I've been unable to upload
> large files to Amazon S3 (greater than 4 or 5 GB) about 95% of the time.
> The error returned is "Mismatch between MD5 hash (long string) of
> uploaded data and ETAG (long string) returned by the server. I've
> uploaded with and without a VPN and from different locations and
> different computers (All Mac). Attached is the log file from yesterday
> (folder names obscured for security). The first upload of 10.45GB failed
> about 8GB into it. I retried it and it worked (VERY rare). The second
> file failed 17.7GB into a 26.1GB file.
>
> GET /?max-keys=1000&prefix=XXXX.XX.-converted.videotapes%2F&delimiter=%2F
> HTTP/1.1
> Date: Wed, 08 Jul 2015 18:58:28 GMT
> x-amz-request-payer: requester
> Authorization: AWS AKIAICYF5VD7M5F7J5JQ:Dcza6UzeiOyuSSUzCRFcuW2BGzI=
> Host: XXXX-XXXXXX-videos.s3.amazonaws.com:443
> Connection: Keep-Alive
> User-Agent: Cyberduck/4.7.1.17920 (Mac OS X/10.10.3) (x86_64)
> HTTP/1.1 200 OK
> x-amz-id-2:
> y4am2JuQYXUQnnPrdhgja1qpAixHb6jgDE/KGd6e1u/aM/RujV2n6pMk8pdMBWNo
> x-amz-request-id: 17127AE0D31F8BA9
> Date: Wed, 08 Jul 2015 18:58:29 GMT
> x-amz-bucket-region: us-east-1
> Content-Type: application/xml
> Transfer-Encoding: chunked
> Server: AmazonS3
> GET /?versioning HTTP/1.1
> Date: Wed, 08 Jul 2015 18:58:28 GMT
> x-amz-request-payer: requester
> Authorization: AWS AKIAICYF5VD7M5F7J5JQ:beq57viq5PVQL1DiDQj8fwCf0Sw=
> Host: XXX-XXXXXX-videos.s3.amazonaws.com:443
> Connection: Keep-Alive
> User-Agent: Cyberduck/4.7.1.17920 (Mac OS X/10.10.3) (x86_64)
> HTTP/1.1 200 OK
> x-amz-id-2:
> it75FkKMTGARsIevWq1VuUA4sStEWQoakCDe7TtyAfQoQaq6dVsj2GkksqB350Ne
> x-amz-request-id: C4DEC4D20395AF2A
> Date: Wed, 08 Jul 2015 18:58:29 GMT
> Transfer-Encoding: chunked
> Server: AmazonS3
> GET /?max-keys=1000&prefix=XXXX.XX.-converted.videotapes%2F&delimiter=%2F
> HTTP/1.1
> Date: Wed, 08 Jul 2015 18:58:29 GMT
> x-amz-request-payer: requester
> Authorization: AWS AKIAICYF5VD7M5F7J5JQ:fGhb848vvpg0OKN45HM+cZR1Km0=
> Host: XXXX-XXXXXX-videos.s3.amazonaws.com:443
> Connection: Keep-Alive
> User-Agent: Cyberduck/4.7.1.17920 (Mac OS X/10.10.3) (x86_64)
> HTTP/1.1 200 OK
> x-amz-id-2:
> Fwz+JjYrmOV9fsKlLJTNnI/I+ZbulOUdv0Fzdzoeb7bEJ4BdX6Y2SPuzV8n17k7o
> x-amz-request-id: 4CD54467246DE112
> Date: Wed, 08 Jul 2015 18:58:29 GMT
> x-amz-bucket-region: us-east-1
> Content-Type: application/xml
> Transfer-Encoding: chunked
> Server: AmazonS3
> GET /?max-keys=1000&prefix=XXXX.XX.-converted.videotapes%2F&delimiter=%2F
> HTTP/1.1
> Date: Wed, 08 Jul 2015 20:27:52 GMT
> x-amz-request-payer: requester
> Authorization: AWS AKIAICYF5VD7M5F7J5JQ:NU0ekJA7ykAcmt5QzPB16oS7OmM=
> Host: XXXX-XXXXXX-videos.s3.amazonaws.com:443
> Connection: Keep-Alive
> User-Agent: Cyberduck/4.7.1.17920 (Mac OS X/10.10.3) (x86_64)
> HTTP/1.1 200 OK
> x-amz-id-2:
> 1kLo6DGW4sLS2iaowUSql91seqA+m1fAymLUbj7xJqv6s24QrquuVo3A6WwEeMgc
> x-amz-request-id: 9C05ED42504A1C68
> Date: Wed, 08 Jul 2015 20:27:53 GMT
> x-amz-bucket-region: us-east-1
> Content-Type: application/xml
> Transfer-Encoding: chunked
> Server: AmazonS3

New description:

 For several months since buying CyberDuck I've been unable to upload large
 files to Amazon S3 (greater than 4 or 5 GB) about 95% of the time. The
 error returned is "Mismatch between MD5 hash (long string) of uploaded
 data and ETAG (long string) returned by the server. I've uploaded with and
 without a VPN and from different locations and different computers (All
 Mac). Attached is the log file from yesterday (folder names obscured for
 security). The first upload of 10.45GB failed about 8GB into it. I retried
 it and it worked (VERY rare). The second file failed 17.7GB into a 26.1GB
 file.


 {{{
 GET /?max-keys=1000&prefix=XXXX.XX.-converted.videotapes%2F&delimiter=%2F
 HTTP/1.1
 Date: Wed, 08 Jul 2015 18:58:28 GMT
 x-amz-request-payer: requester
 Authorization: AWS AKIAICYF5VD7M5F7J5JQ:Dcza6UzeiOyuSSUzCRFcuW2BGzI=
 Host: XXXX-XXXXXX-videos.s3.amazonaws.com:443
 Connection: Keep-Alive
 User-Agent: Cyberduck/4.7.1.17920 (Mac OS X/10.10.3) (x86_64)
 HTTP/1.1 200 OK
 x-amz-id-2:
 y4am2JuQYXUQnnPrdhgja1qpAixHb6jgDE/KGd6e1u/aM/RujV2n6pMk8pdMBWNo
 x-amz-request-id: 17127AE0D31F8BA9
 Date: Wed, 08 Jul 2015 18:58:29 GMT
 x-amz-bucket-region: us-east-1
 Content-Type: application/xml
 Transfer-Encoding: chunked
 Server: AmazonS3
 GET /?versioning HTTP/1.1
 Date: Wed, 08 Jul 2015 18:58:28 GMT
 x-amz-request-payer: requester
 Authorization: AWS AKIAICYF5VD7M5F7J5JQ:beq57viq5PVQL1DiDQj8fwCf0Sw=
 Host: XXX-XXXXXX-videos.s3.amazonaws.com:443
 Connection: Keep-Alive
 User-Agent: Cyberduck/4.7.1.17920 (Mac OS X/10.10.3) (x86_64)
 HTTP/1.1 200 OK
 x-amz-id-2:
 it75FkKMTGARsIevWq1VuUA4sStEWQoakCDe7TtyAfQoQaq6dVsj2GkksqB350Ne
 x-amz-request-id: C4DEC4D20395AF2A
 Date: Wed, 08 Jul 2015 18:58:29 GMT
 Transfer-Encoding: chunked
 Server: AmazonS3
 GET /?max-keys=1000&prefix=XXXX.XX.-converted.videotapes%2F&delimiter=%2F
 HTTP/1.1
 Date: Wed, 08 Jul 2015 18:58:29 GMT
 x-amz-request-payer: requester
 Authorization: AWS AKIAICYF5VD7M5F7J5JQ:fGhb848vvpg0OKN45HM+cZR1Km0=
 Host: XXXX-XXXXXX-videos.s3.amazonaws.com:443
 Connection: Keep-Alive
 User-Agent: Cyberduck/4.7.1.17920 (Mac OS X/10.10.3) (x86_64)
 HTTP/1.1 200 OK
 x-amz-id-2:
 Fwz+JjYrmOV9fsKlLJTNnI/I+ZbulOUdv0Fzdzoeb7bEJ4BdX6Y2SPuzV8n17k7o
 x-amz-request-id: 4CD54467246DE112
 Date: Wed, 08 Jul 2015 18:58:29 GMT
 x-amz-bucket-region: us-east-1
 Content-Type: application/xml
 Transfer-Encoding: chunked
 Server: AmazonS3
 GET /?max-keys=1000&prefix=XXXX.XX.-converted.videotapes%2F&delimiter=%2F
 HTTP/1.1
 Date: Wed, 08 Jul 2015 20:27:52 GMT
 x-amz-request-payer: requester
 Authorization: AWS AKIAICYF5VD7M5F7J5JQ:NU0ekJA7ykAcmt5QzPB16oS7OmM=
 Host: XXXX-XXXXXX-videos.s3.amazonaws.com:443
 Connection: Keep-Alive
 User-Agent: Cyberduck/4.7.1.17920 (Mac OS X/10.10.3) (x86_64)
 HTTP/1.1 200 OK
 x-amz-id-2:
 1kLo6DGW4sLS2iaowUSql91seqA+m1fAymLUbj7xJqv6s24QrquuVo3A6WwEeMgc
 x-amz-request-id: 9C05ED42504A1C68
 Date: Wed, 08 Jul 2015 20:27:53 GMT
 x-amz-bucket-region: us-east-1
 Content-Type: application/xml
 Transfer-Encoding: chunked
 Server: AmazonS3
 }}}

--

-- 
Ticket URL: <https://trac.cyberduck.io/ticket/8919#comment:2>
Cyberduck <https://cyberduck.io>
Libre FTP, SFTP, WebDAV, S3 & OpenStack Swift browser for Mac and Windows


More information about the Cyberduck-trac mailing list