The Wayback Machine - https://web.archive.org/web/20220602112453/https://github.com/rclone/rclone/pull/5947
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

opendrive: resolve lag and truncate bugs fixes #5936 #5947

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

scott-grimes
Copy link

@scott-grimes scott-grimes commented Jan 24, 2022

What is the purpose of this change?

Resolves two issues with opendrive:

  1. fetches metadata directly from /file/info.json in readMetaData when possible (if the object id is already known)

Previously all calls in readMetaData relied on the /folder/itembyname.json api which is buggy: items written to a directory take an arbitrary amount of time to appear via this endpoint, and the file info returned does not include the file hash.

  1. resolves a bug caused by an undocumented feature of the move_copy endpoint which silently truncates new filenames to 255 char. rclone (correctly) throws an error when attempting to create a file or directory that surpasses the max length for opendrive, but there is no check performed in the move_copy command. Without such a check files written to the remote may have their names modified unknowingly.

Was the change discussed in an issue or in the forum before?

#5936

Checklist

  • I have read the contribution guidelines.
  • I have added tests for all changes in this PR if appropriate.
  • I have added documentation for the changes if appropriate.
  • All commit messages are in house style.
  • I'm done, this Pull Request is ready for review :-)

Copy link
Member

@ncw ncw left a comment

This looks really good - thank you.

Looking things up by ID is always better.

I put some very minor comments about the code inline.

@@ -429,6 +429,12 @@ func (f *Fs) Move(ctx context.Context, src fs.Object, remote string) (fs.Object,
return nil, err
}

// move_copy will silently truncate new filenames
if len(leaf) > 255 {
fs.Debugf("Can't move file: name (%s) exceeds 255 char", leaf)
Copy link
Member

@ncw ncw Jan 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The debug message needs an object as its first parameter - this will render as its path in the log.

So something like

fs.Debugf(src, "Can't move file: name %q exceeds 255 chars", leaf)

if "" != o.id {
fileInfo := File{}
err = o.fs.pacer.Call(func() (bool, error) {
opts := rest.Opts{
Copy link
Member

@ncw ncw Jan 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We usually define opts outside the Call function as it is essentially a loop.

return fmt.Errorf("failed to get fileinfo: %w", err)
}

o.id = fileInfo.FileID
Copy link
Member

@ncw ncw Jan 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels like this should be an if / else with the setting of the o.id etc variables done after the if / else otherwise there is a bit of duplicated code here.

Copy link
Author

@scott-grimes scott-grimes Jan 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done!

@scott-grimes scott-grimes force-pushed the issue-5936 branch 2 times, most recently from b0edc45 to 43e9eb9 Compare Jan 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants