The Wayback Machine - https://web.archive.org/web/20211029225747/https://github.com/rclone/rclone/pull/5590
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ftp: fix deadlock after failed update when concurrency=1 #5590

Merged
merged 1 commit into from Oct 1, 2021

Conversation

Projects
ftp
Done
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants
@ivandeex
Copy link
Member

@ivandeex ivandeex commented Sep 9, 2021

What is the purpose of this change?

Our users complained many times that rclone FTP client manifests random problems with completing a file upload.

As my investigations have shown, the bug triggers because jlaffaye/ftp does not properly nudge the fshttp anti-hanging timer on command channel after a file upload. I have a working fix for jlaffaye/ftp and backend/ftp which I will submit later.

This pull request solves a second-stage bug triggered by the above bug which can be fixed solely in rclone. Frequently when users reported the above bug we recommended them to simplify their FTP client settings, in particular set --ftp-concurrency=1. When the FTP backend's upload routine encounters a error, it will remove the failed file. The removal needs another connection but due to low concurrency setting it will wait for connection token forever.

This patch modifies the upload routine so that after error it will first release the failed connection and then remove the failed file and thus avoid the deadlock.
Due to presence of another bug I will include a relevant unit test with a dedicated pull request #5596, not here. I just hope that the change looks self-evident. @ncw ??

Was the change discussed in an issue or in the forum before?

Checklist

  • I have read the contribution guidelines.
  • I have added tests for all changes in this PR if appropriate (see manual test below)
  • I have added documentation for the changes if appropriate.
  • All commit messages are in house style.
  • I'm done, this Pull Request is ready for review :-)
@ivandeex ivandeex added this to the v1.57 milestone Sep 9, 2021
@ivandeex ivandeex requested a review from ncw Sep 9, 2021
@ivandeex ivandeex self-assigned this Sep 9, 2021
@ivandeex ivandeex added this to In progress in ftp Sep 9, 2021
@ivandeex
Copy link
Member Author

@ivandeex ivandeex commented Sep 11, 2021

Manual unit test:
Trigger yet unfixed bug in rclone uploading a large file >100M and see it FAIL with i/o timeout, not HANG

rclone copy ./large_file :ftp,host=XXX,user=XXX,pass=XXX,concurrency=1:

@ivandeex
Copy link
Member Author

@ivandeex ivandeex commented Sep 11, 2021

@ncw Please look

@ivandeex
Copy link
Member Author

@ivandeex ivandeex commented Sep 26, 2021

@ncw
This request is ready for review
Can we have it in 1.57 please?

@ivandeex ivandeex force-pushed the pr-ftp-deadlock branch from c289eb3 to 4fc7bcb Sep 30, 2021
ncw
ncw approved these changes Sep 30, 2021
Copy link
Member

@ncw ncw left a comment

That looks sensible - let's merge :-)

@ivandeex ivandeex merged commit 69f4b48 into rclone:master Oct 1, 2021
10 checks passed
@ivandeex ivandeex deleted the pr-ftp-deadlock branch Oct 1, 2021
@ivandeex ivandeex moved this from In progress to Done in ftp Oct 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment