[Cyberduck-trac] [Cyberduck] #8919: Failed On Large Uploads: Mismatch between MD5 hash and ETag
Cyberduck
trac at trac.cyberduck.io
Thu Jul 9 13:42:27 UTC 2015
#8919: Failed On Large Uploads: Mismatch between MD5 hash and ETag
--------------------------------------------+------------------------
Reporter: collector | Owner: dkocher
Type: defect | Status: new
Priority: high | Milestone:
Component: s3 | Version: 4.7.1
Severity: blocker | Resolution:
Keywords: Mismatch MD5 ETag large upload | Architecture: Intel
Platform: Mac OS X 10.10 |
--------------------------------------------+------------------------
Description changed by dkocher:
Old description:
> For several months since buying CyberDuck I've been unable to upload
> large files to Amazon S3 (greater than 4 or 5 GB) about 95% of the time.
> The error returned is "Mismatch between MD5 hash (long string) of
> uploaded data and ETAG (long string) returned by the server. I've
> uploaded with and without a VPN and from different locations and
> different computers (All Mac). Attached is the log file from yesterday
> (folder names obscured for security). The first upload of 10.45GB failed
> about 8GB into it. I retried it and it worked (VERY rare). The second
> file failed 17.7GB into a 26.1GB file.
>
> GET /?max-keys=1000&prefix=XXXX.XX.-converted.videotapes%2F&delimiter=%2F
> HTTP/1.1
> Date: Wed, 08 Jul 2015 18:58:28 GMT
> x-amz-request-payer: requester
> Authorization: AWS AKIAICYF5VD7M5F7J5JQ:Dcza6UzeiOyuSSUzCRFcuW2BGzI=
> Host: XXXX-XXXXXX-videos.s3.amazonaws.com:443
> Connection: Keep-Alive
> User-Agent: Cyberduck/4.7.1.17920 (Mac OS X/10.10.3) (x86_64)
> HTTP/1.1 200 OK
> x-amz-id-2:
> y4am2JuQYXUQnnPrdhgja1qpAixHb6jgDE/KGd6e1u/aM/RujV2n6pMk8pdMBWNo
> x-amz-request-id: 17127AE0D31F8BA9
> Date: Wed, 08 Jul 2015 18:58:29 GMT
> x-amz-bucket-region: us-east-1
> Content-Type: application/xml
> Transfer-Encoding: chunked
> Server: AmazonS3
> GET /?versioning HTTP/1.1
> Date: Wed, 08 Jul 2015 18:58:28 GMT
> x-amz-request-payer: requester
> Authorization: AWS AKIAICYF5VD7M5F7J5JQ:beq57viq5PVQL1DiDQj8fwCf0Sw=
> Host: XXX-XXXXXX-videos.s3.amazonaws.com:443
> Connection: Keep-Alive
> User-Agent: Cyberduck/4.7.1.17920 (Mac OS X/10.10.3) (x86_64)
> HTTP/1.1 200 OK
> x-amz-id-2:
> it75FkKMTGARsIevWq1VuUA4sStEWQoakCDe7TtyAfQoQaq6dVsj2GkksqB350Ne
> x-amz-request-id: C4DEC4D20395AF2A
> Date: Wed, 08 Jul 2015 18:58:29 GMT
> Transfer-Encoding: chunked
> Server: AmazonS3
> GET /?max-keys=1000&prefix=XXXX.XX.-converted.videotapes%2F&delimiter=%2F
> HTTP/1.1
> Date: Wed, 08 Jul 2015 18:58:29 GMT
> x-amz-request-payer: requester
> Authorization: AWS AKIAICYF5VD7M5F7J5JQ:fGhb848vvpg0OKN45HM+cZR1Km0=
> Host: XXXX-XXXXXX-videos.s3.amazonaws.com:443
> Connection: Keep-Alive
> User-Agent: Cyberduck/4.7.1.17920 (Mac OS X/10.10.3) (x86_64)
> HTTP/1.1 200 OK
> x-amz-id-2:
> Fwz+JjYrmOV9fsKlLJTNnI/I+ZbulOUdv0Fzdzoeb7bEJ4BdX6Y2SPuzV8n17k7o
> x-amz-request-id: 4CD54467246DE112
> Date: Wed, 08 Jul 2015 18:58:29 GMT
> x-amz-bucket-region: us-east-1
> Content-Type: application/xml
> Transfer-Encoding: chunked
> Server: AmazonS3
> GET /?max-keys=1000&prefix=XXXX.XX.-converted.videotapes%2F&delimiter=%2F
> HTTP/1.1
> Date: Wed, 08 Jul 2015 20:27:52 GMT
> x-amz-request-payer: requester
> Authorization: AWS AKIAICYF5VD7M5F7J5JQ:NU0ekJA7ykAcmt5QzPB16oS7OmM=
> Host: XXXX-XXXXXX-videos.s3.amazonaws.com:443
> Connection: Keep-Alive
> User-Agent: Cyberduck/4.7.1.17920 (Mac OS X/10.10.3) (x86_64)
> HTTP/1.1 200 OK
> x-amz-id-2:
> 1kLo6DGW4sLS2iaowUSql91seqA+m1fAymLUbj7xJqv6s24QrquuVo3A6WwEeMgc
> x-amz-request-id: 9C05ED42504A1C68
> Date: Wed, 08 Jul 2015 20:27:53 GMT
> x-amz-bucket-region: us-east-1
> Content-Type: application/xml
> Transfer-Encoding: chunked
> Server: AmazonS3
New description:
For several months since buying CyberDuck I've been unable to upload large
files to Amazon S3 (greater than 4 or 5 GB) about 95% of the time. The
error returned is "Mismatch between MD5 hash (long string) of uploaded
data and ETAG (long string) returned by the server. I've uploaded with and
without a VPN and from different locations and different computers (All
Mac). Attached is the log file from yesterday (folder names obscured for
security). The first upload of 10.45GB failed about 8GB into it. I retried
it and it worked (VERY rare). The second file failed 17.7GB into a 26.1GB
file.
{{{
GET /?max-keys=1000&prefix=XXXX.XX.-converted.videotapes%2F&delimiter=%2F
HTTP/1.1
Date: Wed, 08 Jul 2015 18:58:28 GMT
x-amz-request-payer: requester
Authorization: AWS AKIAICYF5VD7M5F7J5JQ:Dcza6UzeiOyuSSUzCRFcuW2BGzI=
Host: XXXX-XXXXXX-videos.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7.1.17920 (Mac OS X/10.10.3) (x86_64)
HTTP/1.1 200 OK
x-amz-id-2:
y4am2JuQYXUQnnPrdhgja1qpAixHb6jgDE/KGd6e1u/aM/RujV2n6pMk8pdMBWNo
x-amz-request-id: 17127AE0D31F8BA9
Date: Wed, 08 Jul 2015 18:58:29 GMT
x-amz-bucket-region: us-east-1
Content-Type: application/xml
Transfer-Encoding: chunked
Server: AmazonS3
GET /?versioning HTTP/1.1
Date: Wed, 08 Jul 2015 18:58:28 GMT
x-amz-request-payer: requester
Authorization: AWS AKIAICYF5VD7M5F7J5JQ:beq57viq5PVQL1DiDQj8fwCf0Sw=
Host: XXX-XXXXXX-videos.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7.1.17920 (Mac OS X/10.10.3) (x86_64)
HTTP/1.1 200 OK
x-amz-id-2:
it75FkKMTGARsIevWq1VuUA4sStEWQoakCDe7TtyAfQoQaq6dVsj2GkksqB350Ne
x-amz-request-id: C4DEC4D20395AF2A
Date: Wed, 08 Jul 2015 18:58:29 GMT
Transfer-Encoding: chunked
Server: AmazonS3
GET /?max-keys=1000&prefix=XXXX.XX.-converted.videotapes%2F&delimiter=%2F
HTTP/1.1
Date: Wed, 08 Jul 2015 18:58:29 GMT
x-amz-request-payer: requester
Authorization: AWS AKIAICYF5VD7M5F7J5JQ:fGhb848vvpg0OKN45HM+cZR1Km0=
Host: XXXX-XXXXXX-videos.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7.1.17920 (Mac OS X/10.10.3) (x86_64)
HTTP/1.1 200 OK
x-amz-id-2:
Fwz+JjYrmOV9fsKlLJTNnI/I+ZbulOUdv0Fzdzoeb7bEJ4BdX6Y2SPuzV8n17k7o
x-amz-request-id: 4CD54467246DE112
Date: Wed, 08 Jul 2015 18:58:29 GMT
x-amz-bucket-region: us-east-1
Content-Type: application/xml
Transfer-Encoding: chunked
Server: AmazonS3
GET /?max-keys=1000&prefix=XXXX.XX.-converted.videotapes%2F&delimiter=%2F
HTTP/1.1
Date: Wed, 08 Jul 2015 20:27:52 GMT
x-amz-request-payer: requester
Authorization: AWS AKIAICYF5VD7M5F7J5JQ:NU0ekJA7ykAcmt5QzPB16oS7OmM=
Host: XXXX-XXXXXX-videos.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7.1.17920 (Mac OS X/10.10.3) (x86_64)
HTTP/1.1 200 OK
x-amz-id-2:
1kLo6DGW4sLS2iaowUSql91seqA+m1fAymLUbj7xJqv6s24QrquuVo3A6WwEeMgc
x-amz-request-id: 9C05ED42504A1C68
Date: Wed, 08 Jul 2015 20:27:53 GMT
x-amz-bucket-region: us-east-1
Content-Type: application/xml
Transfer-Encoding: chunked
Server: AmazonS3
}}}
--
--
Ticket URL: <https://trac.cyberduck.io/ticket/8919#comment:2>
Cyberduck <https://cyberduck.io>
Libre FTP, SFTP, WebDAV, S3 & OpenStack Swift browser for Mac and Windows
More information about the Cyberduck-trac
mailing list