Commit graph

497 commits

Author SHA1 Message Date
Olivier Goffart
75f66ddaa1
Local discovery: always recurse within touched directory
If the file system watcher tells us a directory was modified, we should
recurse into it because it means it is probably a new directory

Issue #6804
2020-12-15 10:58:11 +01:00
Christian Kamm
e10e953c66
PropagateUpload: Avoid crash due to cascading aborts
https://sentry.io/owncloud/desktop-win-and-mac/issues/698694072/activity/
2020-12-15 10:58:11 +01:00
Olivier Goffart
640cf0c71e
Fix leaks in tests
As discovered by AddressSanitizer
2020-12-15 10:58:11 +01:00
Olivier Goffart
f666511a4b
Discovery: Remove stale DB entries
And test the Remove/Remove case.

This means we need to always query the database for all the entries.
This showed another small bug in the test in which sync item for virtual
files at the root could have a slash in front of them.
2020-12-15 10:58:10 +01:00
Olivier Goffart
afed46afcc
New discovery algorithm: Parallel PROPFIND 2020-12-15 10:58:08 +01:00
Olivier Goffart
d25d87e92c
New Discovery Algorithm: Ge tthe size of new folders
Also add a test that this works properly
2020-12-15 10:58:07 +01:00
Christian Kamm
e58a25d7c5
Virtual files: Renames propagate #6718 2020-12-15 10:58:07 +01:00
Olivier Goffart
f0ddf179a5
Fix tx.pl
The conflict filename changed
2020-12-15 10:58:06 +01:00
Christian Kamm
6b571b609c
Conflicts: Rename BasePath to InitialBasePath #6709 2020-12-15 10:58:05 +01:00
Olivier Goffart
bf6e4174c8
Move the checksum related routine from FileSystem to checksum, where they belong 2020-12-15 10:58:05 +01:00
Olivier Goffart
9c99544e0b
Fix test link with older Qt 2020-12-15 10:58:04 +01:00
Olivier Goffart
c3f745fa76
Conflict handling: add the OC-ConflictBasePath header
Issue #6702
2020-12-15 10:58:04 +01:00
Olivier Goffart
73d933115a
Virtual Files: Set the mtime of the virtual file to that of the server
Note: When the server mtime is modified, the mtime of the file is not updated

Issue #6727
2020-12-15 10:58:04 +01:00
Christian Kamm
c10f103fb8
Virtual files: Renaming to virtual doesn't delete data #6718
Unfortunately to do this, the local update phase must write to the
database, creating a new side-effect and order dependency (local update
must run before remote update).
2020-12-15 10:58:03 +01:00
Olivier Goffart
a670431a48
SyncEngine: Fix renames making hierarchy inversion
Issue #6694
2020-12-15 10:58:03 +01:00
Christian Kamm
be0fa72fcb
RemotePermissions: Fix empty vs null
Fixes two bugs that appeared since the introduction of the struct:
- when reading permissions from the journal, null ("") was read as
  empty-not-null
- when reading permissinos from the server, empty ("") was read as null

Addresses #4608
2020-12-15 10:58:03 +01:00
Olivier Goffart
732069aaa7
SyncEngine: Fix the "direction" of the "all file delted" message when the server is reset
Using the direction of the "first" item is not enough as it might be a directory
which is a UPDATE_META_DATA,  or there can have been a few changes locally as well.

As reported on https://github.com/owncloud/client/issues/6317#issuecomment-412163113
2020-12-15 10:58:02 +01:00
Olivier Goffart
d6fc3b6e0a
Skip failing test
These tests are failling because now we consider that any files that ends
with .owncloud is a virtual file, regardless what the DB say.
2020-12-15 10:58:01 +01:00
Olivier Goffart
ca32eb1cf7
More cleanup of csync remains 2020-12-15 10:58:01 +01:00
Olivier Goffart
ac24cdbde6
New Discovery Algo: Permsission check 2020-12-15 10:58:01 +01:00
Olivier Goffart
0a6d2d0f46
Remove csync_update and csync_reconcile
I guess some other csync utilities can also be remove dnow but
that will be for later
2020-12-15 10:58:00 +01:00
Olivier Goffart
5994c65ecb
Remove the check_csync_update test
since csync_update is going away
2020-12-15 10:58:00 +01:00
Olivier Goffart
57068b0fd9
New Discovery Algorithm: handle server errors 2020-12-15 10:58:00 +01:00
Olivier Goffart
6794306dde
Added test that checks what happens when there is an error in the remote discovery
(Many of the expected error string are left empty because the current
error message is not insterresting
2020-12-15 10:58:00 +01:00
Olivier Goffart
75a1f2d089
Convert p7.pl to a C++ test
This is just a translation of test/scripts/txpl/t7.pl to C++ using the test
framework.
2020-12-15 10:58:00 +01:00
Olivier Goffart
1e8c37d3d6
New discovery algorithm: Virtual files
The commented tests lines were implementation details
2020-12-15 10:57:59 +01:00
Olivier Goffart
bdd1e72dda
New discovery algoritmh: more on Renames 2020-12-15 10:57:58 +01:00
Dominik Schmidt
07f331717f
Rename ocsync library to ${APPLICATION_EXECUTABLE}_csync 2020-12-15 10:57:56 +01:00
Dominik Schmidt
aad928a6be
Install libocsync to lib/ without subfolder.
Installing to lib/${APPLICATION_EXECUTABLE} has caused a bunch of
irritations in the past and subtle annoying to fix bugs. To avoid name
clashes with branded clients ${APPLICATION_EXECUTABLE} becomes now
part of the filename instead of the subfolder.

The concrete motivation to change this now is that on Windows there
is no RPATH and it's not possible to run owncloud directly from the
Craft Root folder, which is nice when you're developing on Windows.

It would have been possible to change this just for Windows but as
written earlier this has caused lots of issues and thus I think it's
a good idea to just stay consistent accross platforms when touching it.
2020-12-15 10:57:56 +01:00
Olivier Goffart
857afd32df
PropagateDownload: Don't discard the body of error message
We want to keep the body so we can get the message from it
(Issue #6459)

TestDownload::testErrorMessage did not fail because the FakeErrorReply
did not emit readyRead and did not implement bytesAvailable.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-12-15 10:57:55 +01:00
Olivier Goffart
4ee244190b
Virtual Files: Allow to download a folder recursively from the socket API
Issue: #6466
2020-12-15 10:57:54 +01:00
Christian Kamm
6dba2e8b06
Virtual files: Wipe virtual after download completes, not before
Otherwise a interrupted or unsuccessful download would mean that the
download-intend was forgotten. The next sync would reestablish the
virtual file instead.
2020-12-15 10:57:54 +01:00
Christian Kamm
a6c19572a2
Virtual files: Only remove virtual file once on download
With thanks to @ogoffart for spotting the problem.
2020-12-15 10:57:54 +01:00
Christian Kamm
aa6f5f59c4
Rename Placeholders to Virtual Files in code #6531 2020-12-15 10:57:53 +01:00
Olivier Goffart
09cacc4cd4
Blacklist: remember the X-Request-ID
Issue #6420
Store the X-Request-ID in the SyncFileItem and also in the blacklist.
Note that for consistency reason, the X-Request-ID is also in the
SyncFileItem if the request succeeds.

Currently there is no UI to access it, but it can be queried with sql
commands
2020-12-15 10:57:53 +01:00
Christian Kamm
4e3f2f755a
Placeholder: Stop adding ignore pattern
Because we can't make older clients preserve the version directive that
was attached to it.

See #6504 and #6498
2020-12-15 10:57:52 +01:00
Christian Kamm
8b6ac63ddb
Placeholder: Don't contain "stub" 2020-12-15 10:57:52 +01:00
Olivier Goffart
600b1a72c1
Fix test: QTemporaryDir::filePath was added in Qt 5.9 2020-12-15 10:57:51 +01:00
Christian Kamm
765c12dae1
LocalDiscoveryTracker: Separate from Folder and move to libsync
To allow relevant code to be closer together and for testing in
unittests without having to get a gui Folder.

See #6120
2020-12-15 10:57:51 +01:00
Olivier Goffart
718843ffce
Download: Use the <s:message> from the reply in the error message if any
Issue: #6459
2020-12-15 10:57:50 +01:00
Olivier Goffart
50223d889b
Add a test for resuming downloads 2020-12-15 10:57:50 +01:00
Christian Kamm
6a37a7a42c
Placeholders: Fix migration issues
Some edgecases weren't covered and didn't have tests yet.
2020-12-15 10:57:49 +01:00
Christian Kamm
12d6f680f2
Placeholders: Ignore placeholder files in older clients
To do this, we add the placeholder extension to the user exclude file
automatically. However, newer clients shouldn't use that exclude
pattern: so we also add version directives that allow making exclude
patterns dependent on the client version.
2020-12-15 10:57:49 +01:00
Christian Kamm
b1de184bc8
Placeholders: Safe migration to older client versions
Now the db entries for placeholders will have the full placeholder
paths. That way older clients will, on remote discovery, delete the
placeholders and download the real files.
2020-12-15 10:57:48 +01:00
Christian Kamm
0cd83a2c09
Placeholders: Deal with conflicts when a placeholder exists
So "foo.owncloud" exists but the user adds a new "foo".
2020-12-15 10:57:48 +01:00
Olivier Goffart
1049fb74d9
Placeholders: Move the placeholder option from the account to the folder 2020-12-15 10:57:48 +01:00
Christian Kamm
f5cf03b6a6
On-demand downloading: Placeholder-file based prototype
- Controled by an option.
- New remote files start out as ItemTypePlaceholder, are created with a
  .owncloud extension.
- When their db entry is set to ItemTypePlaceholderDownload the next
  sync run will download them.
- Files that aren't in the placeholder state sync as usual.
- See test cases in testsyncplaceholders.

Missing:
- User ui for triggering placeholder file download
- Maybe: Going back from file to placeholder?
2020-12-15 10:57:43 +01:00
Christian Kamm
74382ddcc0 FolderWatcher linux: Make automatically recursive #7068
Previously it depended on addFolder() / removeFolder() calls to adjust
watchers when new folders were added or removed. There also needed to be
complex move handling.

Now, any folder creation/move-in notifications automatically trigger
watcher additions and folder deletion/move-out triggers removal.
2020-11-24 16:56:49 +00:00
Christian Kamm
351bada552 Folderwatcher: On linux, fix paths after dir renames #6808
If a folder was renamed A -> B, the folder watcher for the inode
would be unaware and still report changes for A/foo. Now directory
renames in the watched folders are tracked and paths are updated
accordingly.
2020-11-24 16:56:49 +00:00
Kevin Ottens
4fdb203cdb
Move some of the SocketApi conflict handling in a ConflictSolver class
The socket api move and delete commands are not strictly about conflicts
since they also deal with files which couldn't be uploaded for some
other reason. Still the new ConflictSolver could be used in those cases.

This opens the door at reusing that logic in other places.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-10-22 16:40:46 +02:00