[Cyberduck-trac] [Cyberduck] #8919: Mismatch between MD5 hash and ETag for multipart upload
Cyberduck
trac at trac.cyberduck.io
Fri Jul 24 12:29:15 UTC 2015
#8919: Mismatch between MD5 hash and ETag for multipart upload
--------------------------------------------+-------------------------
Reporter: collector | Owner: dkocher
Type: defect | Status: assigned
Priority: high | Milestone: 4.8
Component: s3 | Version: 4.7.1
Severity: blocker | Resolution:
Keywords: Mismatch MD5 ETag large upload | Architecture: Intel
Platform: Mac OS X 10.10 |
--------------------------------------------+-------------------------
Description changed by dkocher:
Old description:
> For several months since buying CyberDuck I've been unable to upload
> large files to Amazon S3 (greater than 4 or 5 GB) about 95% of the time.
> The error returned is "Mismatch between MD5 hash (long string) of
> uploaded data and ETAG (long string) returned by the server. I've
> uploaded with and without a VPN and from different locations and
> different computers (All Mac). Attached is the log file from yesterday
> (folder names obscured for security). The first upload of 10.45GB failed
> about 8GB into it. I retried it and it worked (VERY rare). The second
> file failed 17.7GB into a 26.1GB file.
>
> {{{
> GET /?max-keys=1000&prefix=XXXX.XX.-converted.videotapes%2F&delimiter=%2F
> HTTP/1.1
> Date: Wed, 08 Jul 2015 18:58:28 GMT
> x-amz-request-payer: requester
> Authorization: AWS AKIAICYF5VD7M5F7J5JQ:Dcza6UzeiOyuSSUzCRFcuW2BGzI=
> Host: XXXX-XXXXXX-videos.s3.amazonaws.com:443
> Connection: Keep-Alive
> User-Agent: Cyberduck/4.7.1.17920 (Mac OS X/10.10.3) (x86_64)
> HTTP/1.1 200 OK
> x-amz-id-2:
> y4am2JuQYXUQnnPrdhgja1qpAixHb6jgDE/KGd6e1u/aM/RujV2n6pMk8pdMBWNo
> x-amz-request-id: 17127AE0D31F8BA9
> Date: Wed, 08 Jul 2015 18:58:29 GMT
> x-amz-bucket-region: us-east-1
> Content-Type: application/xml
> Transfer-Encoding: chunked
> Server: AmazonS3
> GET /?versioning HTTP/1.1
> Date: Wed, 08 Jul 2015 18:58:28 GMT
> x-amz-request-payer: requester
> Authorization: AWS AKIAICYF5VD7M5F7J5JQ:beq57viq5PVQL1DiDQj8fwCf0Sw=
> Host: XXX-XXXXXX-videos.s3.amazonaws.com:443
> Connection: Keep-Alive
> User-Agent: Cyberduck/4.7.1.17920 (Mac OS X/10.10.3) (x86_64)
> HTTP/1.1 200 OK
> x-amz-id-2:
> it75FkKMTGARsIevWq1VuUA4sStEWQoakCDe7TtyAfQoQaq6dVsj2GkksqB350Ne
> x-amz-request-id: C4DEC4D20395AF2A
> Date: Wed, 08 Jul 2015 18:58:29 GMT
> Transfer-Encoding: chunked
> Server: AmazonS3
> GET /?max-keys=1000&prefix=XXXX.XX.-converted.videotapes%2F&delimiter=%2F
> HTTP/1.1
> Date: Wed, 08 Jul 2015 18:58:29 GMT
> x-amz-request-payer: requester
> Authorization: AWS AKIAICYF5VD7M5F7J5JQ:fGhb848vvpg0OKN45HM+cZR1Km0=
> Host: XXXX-XXXXXX-videos.s3.amazonaws.com:443
> Connection: Keep-Alive
> User-Agent: Cyberduck/4.7.1.17920 (Mac OS X/10.10.3) (x86_64)
> HTTP/1.1 200 OK
> x-amz-id-2:
> Fwz+JjYrmOV9fsKlLJTNnI/I+ZbulOUdv0Fzdzoeb7bEJ4BdX6Y2SPuzV8n17k7o
> x-amz-request-id: 4CD54467246DE112
> Date: Wed, 08 Jul 2015 18:58:29 GMT
> x-amz-bucket-region: us-east-1
> Content-Type: application/xml
> Transfer-Encoding: chunked
> Server: AmazonS3
> GET /?max-keys=1000&prefix=XXXX.XX.-converted.videotapes%2F&delimiter=%2F
> HTTP/1.1
> Date: Wed, 08 Jul 2015 20:27:52 GMT
> x-amz-request-payer: requester
> Authorization: AWS AKIAICYF5VD7M5F7J5JQ:NU0ekJA7ykAcmt5QzPB16oS7OmM=
> Host: XXXX-XXXXXX-videos.s3.amazonaws.com:443
> Connection: Keep-Alive
> User-Agent: Cyberduck/4.7.1.17920 (Mac OS X/10.10.3) (x86_64)
> HTTP/1.1 200 OK
> x-amz-id-2:
> 1kLo6DGW4sLS2iaowUSql91seqA+m1fAymLUbj7xJqv6s24QrquuVo3A6WwEeMgc
> x-amz-request-id: 9C05ED42504A1C68
> Date: Wed, 08 Jul 2015 20:27:53 GMT
> x-amz-bucket-region: us-east-1
> Content-Type: application/xml
> Transfer-Encoding: chunked
> Server: AmazonS3
> }}}
New description:
For several months since buying CyberDuck I've been unable to upload large
files to Amazon S3 (greater than 4 or 5 GB) about 95% of the time. The
error returned is
{{{
Mismatch between MD5 hash (long string) of uploaded data and ETAG (long
string) returned by the server.
}}}
I've uploaded with and without a VPN and from different locations and
different computers (All Mac). Attached is the log file from yesterday
(folder names obscured for security). The first upload of 10.45GB failed
about 8GB into it. I retried it and it worked (VERY rare). The second file
failed 17.7GB into a 26.1GB file.
{{{
GET /?max-keys=1000&prefix=XXXX.XX.-converted.videotapes%2F&delimiter=%2F
HTTP/1.1
Date: Wed, 08 Jul 2015 18:58:28 GMT
x-amz-request-payer: requester
Authorization: AWS AKIAICYF5VD7M5F7J5JQ:Dcza6UzeiOyuSSUzCRFcuW2BGzI=
Host: XXXX-XXXXXX-videos.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7.1.17920 (Mac OS X/10.10.3) (x86_64)
HTTP/1.1 200 OK
x-amz-id-2:
y4am2JuQYXUQnnPrdhgja1qpAixHb6jgDE/KGd6e1u/aM/RujV2n6pMk8pdMBWNo
x-amz-request-id: 17127AE0D31F8BA9
Date: Wed, 08 Jul 2015 18:58:29 GMT
x-amz-bucket-region: us-east-1
Content-Type: application/xml
Transfer-Encoding: chunked
Server: AmazonS3
GET /?versioning HTTP/1.1
Date: Wed, 08 Jul 2015 18:58:28 GMT
x-amz-request-payer: requester
Authorization: AWS AKIAICYF5VD7M5F7J5JQ:beq57viq5PVQL1DiDQj8fwCf0Sw=
Host: XXX-XXXXXX-videos.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7.1.17920 (Mac OS X/10.10.3) (x86_64)
HTTP/1.1 200 OK
x-amz-id-2:
it75FkKMTGARsIevWq1VuUA4sStEWQoakCDe7TtyAfQoQaq6dVsj2GkksqB350Ne
x-amz-request-id: C4DEC4D20395AF2A
Date: Wed, 08 Jul 2015 18:58:29 GMT
Transfer-Encoding: chunked
Server: AmazonS3
GET /?max-keys=1000&prefix=XXXX.XX.-converted.videotapes%2F&delimiter=%2F
HTTP/1.1
Date: Wed, 08 Jul 2015 18:58:29 GMT
x-amz-request-payer: requester
Authorization: AWS AKIAICYF5VD7M5F7J5JQ:fGhb848vvpg0OKN45HM+cZR1Km0=
Host: XXXX-XXXXXX-videos.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7.1.17920 (Mac OS X/10.10.3) (x86_64)
HTTP/1.1 200 OK
x-amz-id-2:
Fwz+JjYrmOV9fsKlLJTNnI/I+ZbulOUdv0Fzdzoeb7bEJ4BdX6Y2SPuzV8n17k7o
x-amz-request-id: 4CD54467246DE112
Date: Wed, 08 Jul 2015 18:58:29 GMT
x-amz-bucket-region: us-east-1
Content-Type: application/xml
Transfer-Encoding: chunked
Server: AmazonS3
GET /?max-keys=1000&prefix=XXXX.XX.-converted.videotapes%2F&delimiter=%2F
HTTP/1.1
Date: Wed, 08 Jul 2015 20:27:52 GMT
x-amz-request-payer: requester
Authorization: AWS AKIAICYF5VD7M5F7J5JQ:NU0ekJA7ykAcmt5QzPB16oS7OmM=
Host: XXXX-XXXXXX-videos.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7.1.17920 (Mac OS X/10.10.3) (x86_64)
HTTP/1.1 200 OK
x-amz-id-2:
1kLo6DGW4sLS2iaowUSql91seqA+m1fAymLUbj7xJqv6s24QrquuVo3A6WwEeMgc
x-amz-request-id: 9C05ED42504A1C68
Date: Wed, 08 Jul 2015 20:27:53 GMT
x-amz-bucket-region: us-east-1
Content-Type: application/xml
Transfer-Encoding: chunked
Server: AmazonS3
}}}
--
--
Ticket URL: <https://trac.cyberduck.io/ticket/8919#comment:4>
Cyberduck <https://cyberduck.io>
Libre FTP, SFTP, WebDAV, S3 & OpenStack Swift browser for Mac and Windows
More information about the Cyberduck-trac
mailing list