add PutMultiFileJob to send many files at once
use it in BulkPropagatorJob to implement bulk upload feature
Signed-off-by: Matthieu Gallien <matthieu.gallien@nextcloud.com>
Previously the source was deleted (or attempted to be deleted), even if
the new location was not acceptable for upload. This could make data
unavilable on the server.
For #7410
As far as I'm aware local discovery can be skipped on folders that are
selective-sync blacklisted, so a local discovery is required when an
entry is removed from the blacklist.
Also rename
avoidReadFromDbOnNextSync() -> schedulePathForRemoteDiscovery()
since the old name might also imply it's not read from db in the local
discovery - which is not the case. Use Folder::
schedulePathForLocalDiscovery() for that.
It could happen that readyRead was emitted for incoming data while the
download was not yet finished. Then the network job could finish with
no more data arriving - so readyRead wasn't emitted again.
To fix this, the finished signal also gets connected to the readyRead
slot.
Paths can contain the wildcards % and _ and that would lead to odd
behavior.
This patch also clarifies the behavior of avoidReadFromDbOnNextSync()
which previously dependend on whether "foo/bar" or "foo/bar/" was
passed as input.
Possibly affects #6322
Previously we required matching mtimes but that's actually
unnecessary when the question is about whether to skip the
download. We will still update the file's metadata.
Also, adjust behavior when the checksum is weak (Adler32):
in these cases we still depend on equal mtimes.
... even if the file is not changed.
We get an UPDATE_METADATA in that case, so make sure we let the
SyncFileStatusTracker know about it.
That means we need to filter out UPDATE_METADATA in the other listeners
of this signal.
Issue #6098
When users share the same tree several times (say A/ and A/B/ are both
shared) the remote tree can have several entries that have the same
file id. This needs to be respected in rename detection.
Also adds several tests and fixes for issues noticed during testing.
See #6096
We mostly trust the file watchers meaning that we don't re-scan the
local tree if we have done that recently and no file watcher events
have arrived. If the file watchers invalidate a subtree, we rescan
only that subtree.
Since we're not entirely sure the file watchers are reliable, we still
do full local discoveries regularly (1h by default). There is a config
file setting as well as an environment variable to control the interval.
Comparison of file sizes for potential conflicts was added in
0eb9401c62, but did not extend to checking
the file size in case of potential local moves.
This commit adds this check and adds tests for various move+change
scenarios.
This gets rid of the csync_statedb sqlite layer and use
the same code and same connection as the rest of the SyncEngine.
Missing functions are added to SyncJournalDb and change a few minor
things (like changing SyncJournalFileRecord::_modtime to be an int64
instead of a QDateTime, like it was in csync).
The current implementation would return the same value whether the query failed
or if no row would be found. This is something that is currently checked by csync
and needs to be provided if we want to use SyncJournalDB there.
Adjusted all call sites to also check the return value even though they
could still just rely on rec.isValid(), but makes it more explicit as to what
happens for database errors in those cases, if we ever want to gracefully handle
them.
This remove the remaining "other" fields of the sync log to save a
bit of memory.
other_etag and other_fileId don't give much information to the users
and other_instruction will always be INST_NONE anyway.
other_modtime and other_size are kept since they are sometimes used.
They were renamed to have a bit more meaningful name.
SyncEngine::checkPermissions will now fetch its information from the
csync trees since they are now preserved until right after this point.
Fixes#3213
