You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
So far, rsync had not way to atomically rename directories into place, because one could not replace one non-empty directory with another.
Thus, rsync had to move the old dir "out of the way", or delete it recursively, and the new dir into its place.
This means rsync fails when 2 rsyncs write the same files to the same destination directory, because one would delete files that the other is in the middle of writing.
rsync: [generator] failed to set times on "/my-sourcedir/mydir": No such file or directory (2)
rsync: [generator] recv_generator: mkdir "/my-sourcedir/mydir/myfile" failed: No such file or directory (2)
** Skipping any contents from this failed directory ***
Edit: I think the above errors happen only in specific situations, e.g. when the target directory contains some subdirectories which are not writable or so. I haven't quite figured the situation out yet. I suspect that in such cases, rsync needs to delete the target directory first, causing the errors. If write permissions are available in the target directory, ``--temp-dirseems to already be atomic, because inside it only plain files are created, and not parallel directory hierarchies -- at least if--delay-updates` is not used.
I feature-request that rsync use this to make --temp-dir moves actually atomic.
Then 2 concurrent rsyncs to the same target directory should work without errors, even if subdirectories are involved.
The text was updated successfully, but these errors were encountered:
nh2
changed the title
Use renameat2(RENAME_EXCHANGE) for truly atomic renames with --partial-dir
Use renameat2(RENAME_EXCHANGE) for truly atomic renames with --temp-dirOct 25, 2024
So far, rsync had not way to atomically rename directories into place, because one could not replace one non-empty directory with another.
Thus, rsync had to move the old dir "out of the way", or delete it recursively, and the new dir into its place.
This means rsync fails when 2 rsyncs write the same files to the same destination directory, because one would delete files that the other is in the middle of writing.
This would manifest as errors such as:
or
Edit: I think the above errors happen only in specific situations, e.g. when the target directory contains some subdirectories which are not writable or so. I haven't quite figured the situation out yet. I suspect that in such cases,
rsync
needs to delete the target directory first, causing the errors. If write permissions are available in the target directory, ``--temp-dirseems to already be atomic, because inside it only plain files are created, and not parallel directory hierarchies -- at least if
--delay-updates` is not used.Now it's possible
I feature-request that rsync use this to make
--temp-dir
moves actually atomic.Then 2 concurrent rsyncs to the same target directory should work without errors, even if subdirectories are involved.
The text was updated successfully, but these errors were encountered: