[Cyberduck-trac] [Cyberduck] #8158: Uploads failing with CyberDuck via AXR/WebDev with AmpliData
Cyberduck
trac at trac.cyberduck.io
Thu Aug 7 14:08:56 UTC 2014
#8158: Uploads failing with CyberDuck via AXR/WebDev with AmpliData
---------------------+------------------------
Reporter: jfallon | Owner: dkocher
Type: defect | Status: new
Priority: highest | Milestone:
Component: webdav | Version: 4.5.1
Severity: blocker | Resolution:
Keywords: | Architecture:
Platform: |
---------------------+------------------------
Changes (by jfallon):
* severity: normal => blocker
Old description:
> CyberDuck fails to upload files
>
> I have tcpdump's of clients that work, like curl/bitkinex, versus the
> failures that occur with CyberDuck.
>
> Details below, please reach out if tcpdumps and or additional
> triage/debug data would be useful
>
> The cause of the bug in Cyberduck is (again) the combination of the use
> of HTTP digest authentication and the Expect: 100-continue header.
> The issue with 100-continu is that there is no guarantee that the server
> will actually send this. The HTTP RFC instructs the client not to wait
> indefinitely on the 100-continue reply. This means that the server has no
> way of knowing whether the body of the request will be sent by the client
> or not, regardless of whether it has sent the 100-continue reply or it
> has sent an error code back. To make this work in all cases, there are
> only 2 options available to the server if it replies with something else
> than 100-continue:
> 1. Always close the connection
> 2. Always read the body (and expect that the body is in fact sent by the
> client)
> The behaviour of DSS is configurable (there is a threshold content-length
> for which it will close the connection if the actual content-length is
> larger than this value, and it will read the body when it is smaller).
> This only works correctly if the client also is aware of this behaviour
> of 100-continue. E.g. for Curl, this is the case, but Cyberduck will wait
> indefinitely for the 100-continue response, and not send the body if it
> receives something else (e.g. 401). DSS expects the body to be sent in
> all cases (if option 2 is chosen), and thus start reading the body which
> Cyberduck never has sent. DSS will thus interpret the next Cyberduck
> requests as part of the body. Once it has read content-length bytes, DSS
> expects that the next bytes on the connection will be a new request. In
> the Cyberduck case, it is likely that we are now in the middle of the
> body of another request, which will most likely not be a valid HTTP
> request, and definitely not an intended HTTP request.
> If option 1 is active, this problem should not occur, but it appears that
> Cyberduck also does not handle server side disconnects gracefully: it
> should retry the request on a new connection, but it seems to simply
> throw a broken pipe exception to the user. There is a remote chance that
> this is because DSS does not send Connection: close in its response. This
> is still something in AXR that is not 100% correct according to the HTTP
> specs. Still, Cyberduck should handle this more gracefully.
> There is no way to make the behaviour of (2) in DSS work with Cyberduck
> without breaking other (valid) HTTP clients like Curl. This should be
> fixed in Cyberduck. We can try to make (1) work with Cyberduck by adding
> Connection: close, but most likely Cyberduck will still need to be fixed
> as well.
New description:
CyberDuck fails to upload files
The strange thing is, this was working for quite some time with this
System and then it suddenly started failing. I have tried various
revisions on the MAC, and Windows with no luck.
tcpdumps, of successful curl and failure of CyberDuck uploads included
Server side, log file also attached, which demonstrates the issue.
Details below from AmpliData Engineering.
The cause of the bug in Cyberduck is (again) the combination of the use of
HTTP digest authentication and the Expect: 100-continue header.
The issue with 100-continu is that there is no guarantee that the server
will actually send this. The HTTP RFC instructs the client not to wait
indefinitely on the 100-continue reply. This means that the server has no
way of knowing whether the body of the request will be sent by the client
or not, regardless of whether it has sent the 100-continue reply or it has
sent an error code back. To make this work in all cases, there are only 2
options available to the server if it replies with something else than
100-continue:
1. Always close the connection
2. Always read the body (and expect that the body is in fact sent by the
client)
The behaviour of DSS is configurable (there is a threshold content-length
for which it will close the connection if the actual content-length is
larger than this value, and it will read the body when it is smaller).
This only works correctly if the client also is aware of this behaviour of
100-continue. E.g. for Curl, this is the case, but Cyberduck will wait
indefinitely for the 100-continue response, and not send the body if it
receives something else (e.g. 401). DSS expects the body to be sent in all
cases (if option 2 is chosen), and thus start reading the body which
Cyberduck never has sent. DSS will thus interpret the next Cyberduck
requests as part of the body. Once it has read content-length bytes, DSS
expects that the next bytes on the connection will be a new request. In
the Cyberduck case, it is likely that we are now in the middle of the body
of another request, which will most likely not be a valid HTTP request,
and definitely not an intended HTTP request.
If option 1 is active, this problem should not occur, but it appears that
Cyberduck also does not handle server side disconnects gracefully: it
should retry the request on a new connection, but it seems to simply throw
a broken pipe exception to the user. There is a remote chance that this is
because DSS does not send Connection: close in its response. This is still
something in AXR that is not 100% correct according to the HTTP specs.
Still, Cyberduck should handle this more gracefully.
There is no way to make the behaviour of (2) in DSS work with Cyberduck
without breaking other (valid) HTTP clients like Curl. This should be
fixed in Cyberduck. We can try to make (1) work with Cyberduck by adding
Connection: close, but most likely Cyberduck will still need to be fixed
as well.
--
--
Ticket URL: <https://trac.cyberduck.io/ticket/8158#comment:1>
Cyberduck <http://cyberduck.io>
Libre FTP, SFTP, WebDAV, S3 & OpenStack Swift browser for Mac and Windows
More information about the Cyberduck-trac
mailing list