Commit graph

2244 commits

Author SHA1 Message Date
Kevin Ottens
90fbb7d322
Implement callback between wrapper and vfs object for hydration requests
This comes with a test simulating an open request coming from another
process (although in our case it's really just a thread). The actual
hydration works as expected by cfapi, handling of encrypted files is for
now missing.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-12-30 09:44:58 +01:00
Kevin Ottens
7668c521c1
Split the CfAPI lower level code in a wrapper
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-12-30 09:44:58 +01:00
Kevin Ottens
423780bf79
Add initial version of the CfApi backend
For now this implements only the logic necessary to get a test suite
equivalent to the TestSyncVirtualFiles one to pass. It doesn't (yet)
honor request to fetch files from the system.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-12-30 09:44:58 +01:00
Kevin Ottens
216b251b40
One byte placeholders assumption only holds for the suffix backend
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-12-30 09:44:57 +01:00
Kevin Ottens
4d4b87c9c3
Don't consider CfApi backend as experimental
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-12-30 09:44:53 +01:00
rakekniven
e1979fdb33 Removed translation for "etag" and changed spelling
Shouldn't be translated.

Just checked https://en.wikipedia.org/wiki/HTTP_ETag and all available localized pages.

Signed-off-by: rakekniven <mark.ziegler@rakekniven.de>
2020-12-28 08:51:56 +00:00
Camila
016410dbaa
Make monochrome icons the default on Linux and Mac OS.
Unless it is branded with other icons.

Signed-off-by: Camila <hello@camila.codes>
2020-12-16 14:42:24 +01:00
Kevin Ottens
af57a702ef
Workaround bug in older GCC used for the AppImage toolchain
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-12-15 11:01:53 +01:00
Kevin Ottens
c57eff6fd8
Please the clang-tidy overlord
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-12-15 11:01:53 +01:00
Hannah von Reth
c03a5da670
Don't warn if everything is fine 2020-12-15 11:01:53 +01:00
Hannah von Reth
ebd8047cb6
Replace custome date parser QDateTime::fromString(value, Qt::RFC2822Date) 2020-12-15 11:01:52 +01:00
Hannah von Reth
9d9eadba8e
Use time the request was send,..
not when it was processed by the client, to determine the quality of the connection.
2020-12-15 11:01:08 +01:00
Hannah von Reth
1b4ccea083
Owncloud - virtual files smaller <1KB - problems with syncing
The issue was caused by gziped responses not providing a
content lenght header.

Fixes: #8248
2020-12-15 10:59:27 +01:00
Hannah von Reth
440b31986a
Ensure the callback is triggered 2020-12-15 10:59:25 +01:00
Hannah von Reth
f6faba48e2
Don`t block main thread when displaying all files removed dialog
Fixes: #8170
2020-12-15 10:59:25 +01:00
Kevin Ottens
201dbd54db
Remove the plugin loader mechanism for VFS backends
We will have all the code in public anyway so it can just be compiled
in. Thus no need to go through the plugin loading dance. Replaced the
loading with factory functions. Kept mostly the same structure
otherwise.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-12-15 10:59:24 +01:00
Kevin Ottens
1acb2679dd
Get rid of now unused GetFolderEncryptStatusJob
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-12-15 10:59:23 +01:00
Kevin Ottens
36b8e7c2a4
Remove the encryption flag check from encrypted propagation code
If we use those encrypted propagation code paths, we already know from
the discovery phase (and thus the journal db) that the folders are
encrypted so no need to check again.

This will remove another expensive round trip with the server.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-12-15 10:59:23 +01:00
Kevin Ottens
b2533e6451
Remove encrypt status job fetching from ClientSideEncryption
This is now unused.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-12-15 10:59:23 +01:00
Kevin Ottens
d22046d679
No need to fetch encryption data separately before discovery
This step isn't necessary anymore, no one looks at ClientSideEncryption
for that information anyway.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-12-15 10:59:23 +01:00
Kevin Ottens
580b36fe20
Remove unused methods on ClientSideEncryption
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-12-15 10:59:23 +01:00
Kevin Ottens
7a4a359704
Stop using e2e()->isFolderEncrypted() in the jobs
Thanks to the new discovery algorithm, we got all the freshest E2EE
information straight from the database so reuse it instead of going
through an in memory copy.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-12-15 10:59:22 +01:00
Kevin Ottens
37a51872e1
Set the encrypted flag in the database when EncryptFolderJob succeeds
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-12-15 10:59:22 +01:00
Kevin Ottens
b667bdda14
Change EncryptFolderJob path convention
It had a different path convention than all the other jobs, most likely
for legacy reasons because of the tight coupling it had to the settings
dialog.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-12-15 10:59:22 +01:00
Kevin Ottens
ee8e0fa332
Have the folder token inside the EncryptFolderJob
No need to look for a token on the outside we can just work properly by
keeping all the state encapsulated in the job.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-12-15 10:59:22 +01:00
Kevin Ottens
af00ef1cca
Don't use getFileRecordByE2eMangledName in jobs anymore
Thanks to the new discovery algorithm we got both mangled and original
file names in the item so no need to go through the database for
nothing.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-12-15 10:59:21 +01:00
Kevin Ottens
66f24241d8
Repair E2EE deletions
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-12-15 10:59:21 +01:00
Kevin Ottens
567f678fd5
Make sure the encrypted flag gets all the way to the journal db
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-12-15 10:59:21 +01:00
Kevin Ottens
961794669e
Repair E2EE uploads
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-12-15 10:59:21 +01:00
Kevin Ottens
1d07af07a5
Start hooking up E2EE to the new discovery algorithm
Now we pull the encrypted metadata during the discovery which is a
better approach than before. This shall remove the need for some of the
deep propfinds we have been using so far. It already simplifies the code
a bit for the download scenario.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-12-15 10:59:21 +01:00
Hannah von Reth
44175aca94
Fix: Log chunked messages 2020-12-15 10:59:20 +01:00
Hannah von Reth
93152761a1
Use verbose function names instead of direct member access 2020-12-15 10:59:20 +01:00
Hannah von Reth
4f596dae8c
Log http status
Fixes: #8073
2020-12-15 10:59:19 +01:00
Hannah von Reth
0eabc75039
Improve log message 2020-12-15 10:59:18 +01:00
Hannah von Reth
fd6146d09e
Clazy: Fix some warnings 2020-12-15 10:59:18 +01:00
Hannah von Reth
c1a91e91e7
Use the servers checksum type by default 2020-12-15 10:59:16 +01:00
Hannah von Reth
8c3749cbe2
Log state before specific assertion 2020-12-15 10:59:16 +01:00
Hannah von Reth
4589772393
Remove more legacy C code 2020-12-15 10:59:15 +01:00
Hannah von Reth
47dc7e6c49
Remove custome string functions 2020-12-15 10:59:15 +01:00
Hannah von Reth
c9ee0a0343
Log: Improve log message [ info gui.folder ]: Folder sync result: 4...
by including the name of the folder and print the name of the result
2020-12-15 10:59:15 +01:00
Hannah von Reth
6c2c544713
Log HTTP requests and responses
Issue: #7873
2020-12-15 10:59:13 +01:00
Hannah von Reth
aaca3e7ce5
Use separate loggin category for upload v1 and ng 2020-12-15 10:59:12 +01:00
Hannah von Reth
4d615c31da
Rename csync_instructions_e -> SyncInsturctions 2020-12-15 10:59:10 +01:00
Hannah von Reth
8cbdb4451b
Remove now unneede csync_instruction_str 2020-12-15 10:59:10 +01:00
Hannah von Reth
38c4d5a406
Cookies: Do set cookies in DetermineAuthTypeJob too
As we don't support cookie based authentication anymore we can provide cookies here.
This fixes issues with loadbalancers access policy managers.
2020-12-15 10:59:10 +01:00
Hannah von Reth
f073997aee
Cookies: Don't override cookies with outdated values
This code was actually not breaking most cookie handling by accident.
As the raw cookies where not split properly we added cookies with values like
"key: val; key2 = val2; key3 = val3"
When the code was corrected we overwrote the newer values in the jar with
the old ones from a request.
2020-12-15 10:59:09 +01:00
Olivier Goffart
317d3735d7
SocketAPI: fix status after a failure to move
Issue #7759
2020-12-15 10:59:09 +01:00
Hannah von Reth
85aefa4232
Sync: Fix handling of virtual files in error state
Issue: #7799
2020-12-15 10:59:09 +01:00
Hannah von Reth
5927beb08d
Fix potential null pointer access
https://sentry.io/organizations/owncloud/issues/1529161263/events/02509984b5ca42ffb3960d9c9e161414/?project=79001&statsPeriod=14d
2020-12-15 10:59:07 +01:00
Olivier Goffart
a6a0e361c1
fixup test 2020-12-15 10:59:07 +01:00
Hannah von Reth
106a35d242
[SSL] Print warning only if something changed 2020-12-15 10:59:06 +01:00
Hannah von Reth
ba87fc9e78
[SSL] Properly restore user accepted certificats 2020-12-15 10:59:06 +01:00
Hannah von Reth
9564e5e92e
Fix saving of cookies
Fixes: #7700
2020-12-15 10:59:06 +01:00
Olivier Goffart
1c10fceacc
SyncEngine: no need to use QAtomicInt
This was done because the propagator jobs where running in a thread a long
time ago, but this is no longer the case.

(Also QAtomicInt::load is marked as deprecated now)
2020-12-15 10:59:05 +01:00
Hannah von Reth
b6e8d47644
Use Q_ENUM on AuthType to get the name printed in the log 2020-12-15 10:59:04 +01:00
Hannah von Reth
6c19b02888
Disable http2 support for now
Issue: #7610
2020-12-15 10:59:04 +01:00
Olivier Goffart
0e7c56e81c
Don't show the "All files deleted" popup when unselecting everything with selective sync
Issue #7337
2020-12-15 10:59:04 +01:00
Olivier Goffart
392d3c257c
Discovery: Allow more HTTP error code to be treated as ignored dir
The original code from csync was stopping at any error.
But we have been whitelisting soeme http error code one by one
to ignore the directory instead of aborting the sync.
However, as there are more requests to continue the sync in case
of error, just ignore most HTTP errors

Issue #7586
2020-12-15 10:59:03 +01:00
Hannah von Reth
43c7e32ee1
[libsync] Export verifyFileUnchanged
This fixes Windows builds that are broken since #7562 was merged
2020-12-15 10:59:03 +01:00
Olivier Goffart
66f7b27121
VFS: Do not overwrite existing files by placeholder
For issue #7557 and #7556

Note: this change the API of the VFS plugin, so the VFS plugin needs small
adaptations
2020-12-15 10:59:03 +01:00
Olivier Goffart
fc0aeb53b1
Discovery: fix TestSyncVirtualFiles::testExtraFilesLocalDehydrated on windows
On windows we do a test to know if we should change the case of the files,
but that conflict with the test that checks if the file was still there
when the filename is actually the same. Which can happen with virtual files
as they have two representation (the one with and without suffix).
2020-12-15 10:59:02 +01:00
Christian Kamm
4c4cbf0d97
Vfs: Lots of tests and corrections for suffix edge cases
Avoid or deal with problems that happen when suffixed files exist on the
server or suffix and non-suffixed files exist locally.

See #7350, #7261.
2020-12-15 10:59:02 +01:00
Olivier Goffart
88f86a56b1
Discovery: Attempt to fix issue with windows VFS and new files
(or moved files)
As seen in the log of #7558, a conflict may be issued by mistake.

See investigation in https://github.com/owncloud/client/issues/7558#issuecomment-547385362

This hopefully fix #7558
2020-12-15 10:59:02 +01:00
Olivier Goffart
115a53134c
SyncEngine: Save a bit of memory by not keeping a set of all filename
This is only used for conflict files, so only save conflict files.
(The _seenFile was used for other things in 2.5, but not anymore)
2020-12-15 10:59:01 +01:00
Olivier Goffart
83d743b66b
When moving is allowed but deleting is not, do not restore moved items
Issue #7293
2020-12-15 10:59:01 +01:00
Olivier Goffart
ee611e6004
Restoration items should appear in the sync protocol
When an item is downloaded because it is restored, it shall be shown in the
sync protocol.
(It is also going to be shown in the not synchronized for a short while, but
that's fine)
2020-12-15 10:59:01 +01:00
Christian Kamm
9d422284af
Discovery: Distinguish readdir and closedir errors 2020-12-15 10:59:01 +01:00
Olivier Goffart
adbd3d869b
Fix warning about serverJob not being used
And fix a FIXME in the same time
2020-12-15 10:59:00 +01:00
Christian Kamm
89216daee6
Propagate dir: Never write the etag on remote mkdir #7481
It must always only be written once all children are successfully
propagated.
2020-12-15 10:59:00 +01:00
Christian Kamm
e91e1ca78f
Test: Disable local discovery parallelism in permission test
Adding parallelism broke the test because it depended on the order of
discovery.
2020-12-15 10:59:00 +01:00
Christian Kamm
e596b55977
Discovery: Change local job signal signatures
To make it more explicit that data a copy of the data is transfered
between threads. (though Qt will make a copy of the arguments anyway)
2020-12-15 10:58:59 +01:00
Christian Kamm
71f71b38f1
Discovery: local job shouldn't be parented
Since it'll be deleted by the thread pool.
2020-12-15 10:58:59 +01:00
Christian Kamm
53a217d4e4
Don't store pointer to local job
There were crashes in the QPointer assignment. Possibly the thread pool
is done with the job and deletes it before the assignment is done.
2020-12-15 10:58:59 +01:00
Markus Goetz
26b5e36351
Discovery: List local directories from thread #7456 #7439 2020-12-15 10:58:59 +01:00
Markus Goetz
09a0dbbf82
Reconcile: Sort already during discovery #7445 2020-12-15 10:58:59 +01:00
Markus Goetz
41dc68f99c
Propagator: Delay job execution a bit #7439 2020-12-15 10:58:58 +01:00
Christian Kamm
e37e954720
Discovery: Remove level of indent around moves
No code changes.
2020-12-15 10:58:58 +01:00
Christian Kamm
28797baa39
Discovery: If a move is forbidden, restore the source
Previously the source was deleted (or attempted to be deleted), even if
the new location was not acceptable for upload. This could make data
unavilable on the server.

For #7410
2020-12-15 10:58:58 +01:00
Christian Kamm
c9d1037622
Propagation: Fix delete-before-rename bug #7441
By introducing a PropagateRootDirectory job that explicitly
separates the directory deletion jobs from all the other jobs.

Note that this means that if there are errors in subJobs the
dirDeletionJobs won't get executed.
2020-12-15 10:58:58 +01:00
Markus Goetz
475117dd60
Propagator: Make sure we schedule only one job #7439
To not starve the event loop.
(There is still ~= 3 jobs running at the same time)
2020-12-15 10:58:58 +01:00
Christian Kamm
3446412d92
PropagateDownload: Don't try to open readonly temporaries
This situation could arrise when receiving a read-only share and the
temporary rename failed for some reason.

See #7419
2020-12-15 10:58:58 +01:00
Christian Kamm
0ac8a3e6be
Don't fatal on "Storage temporarily unavailable"
This is an unreliable workaround. The real fix will need to be deferred
to another release.

For #5088
2020-12-15 10:58:57 +01:00
Hannah von Reth
e5a36c3bc6
CMake VFS: Enable us to provided vfs plugins from an external directory 2020-12-15 10:58:57 +01:00
Dominik Schmidt
67656fb136
Add Q_PROPERTYs for gui testing 2020-12-15 10:58:56 +01:00
Hannah von Reth
e97784bb9d
Don't use mutable lambdas as they hide where we modify work on a copy 2020-12-15 10:58:55 +01:00
Christian Kamm
18e1098e38
Remove maxLogLines config option
It's no longer used.

For owncloud/docs#1365
2020-12-15 10:58:55 +01:00
Christian Kamm
ea829f96ca
Download: Don't trigger too many concurrent hash computations
Previously the job would only become "active" when the downloads
started. That meant that arbitrarily many hash computations could be
queued at the same time.

Since the the file was opened during future creation this could lead to
a "too many open files" problem if there were lots of new-new conflicts.

To change this:
- Make PropagateDownload become active when computing a hash
  asynchronously.
- Make the computation future open the file only once it gets run. This
  will make it less likely for this problem to occur even if thousands
  of these futures are queued.

For #7372
2020-12-15 10:58:55 +01:00
Christian Kamm
8a5a185752
Fix 'unused parameter name' warnings 2020-12-15 10:58:55 +01:00
Christian Kamm
5761f4cd8a
SyncEngine: Don't duplicate fatal errors
Previously fatal error texts were duplicated: Once they entered the
SyncResult via the SyncFileItem and once via syncError().

syncError is intended for folder-wide sync issues that are not pinned
to particular files. Thus that duplicated path is removed.

For #5088
2020-12-15 10:58:54 +01:00
Christian Kamm
61972c35a8
Discovery: Parse etag to be consistent with RequestEtagJob
This avoids unnecessary sync runs.

For #7345
2020-12-15 10:58:54 +01:00
Christian Kamm
55ee3f440b
Vfs: Remove old db record when dehydrating via rename
For #7338
2020-12-15 10:58:54 +01:00
Christian Kamm
e6990bd04e
Vfs: Make move detection work with virtual files #7001
Previously a checksum computation could be done on a suffix-placeholder
file, making discovery believe that no move took place.
2020-12-15 10:58:54 +01:00
Christian Kamm
00dcf3ef59
Vfs: Ensure pins change with (de-)hydration
Previously an implicit hydration of a file in an online-only folder
would not change the pin state and cause a dehydration on the next
sync.
2020-12-15 10:58:53 +01:00
Christian Kamm
0df3b83bd2
Vfs: Add hook to allow update-metadata for unchanged files
Allows winvfs to convert files to placeholders when vfs is enabled. This
is required to mark files as in-sync #7329.
2020-12-15 10:58:53 +01:00
Christian Kamm
5bc2180478
Vfs: Preserve pin state on hydration
For #7322 and #7323
2020-12-15 10:58:53 +01:00
Dominik Schmidt
92f6d866e1
Add missing OWNCLOUDSYNC_EXPORT 2020-12-15 10:58:52 +01:00
Christian Kamm
5acb157a7e
VfsSuffix: Wipe stale pin states #7273
Previously the pin states of deleted files stayed in the 'flags'
database and could be inadvertently reused when a new file with the same
name appeared. Now they are deleted.

To make this work right, the meaning of the 'path' column in the 'flags'
table was changed: Previously it never had the .owncloud file suffix.
Now it's the same as in metadata.path.

This takes the safe parts from #7274 for inclusion in 2.6. The more
elaborate database schema changes (why use 'path' the join the two
tables in the first place?) shall go into master.
2020-12-15 10:58:52 +01:00
Christian Kamm
0e9f030b0f
RequestEtagJob: Consistently parse etags #7271
Previously RequestEtagJob did return the etag verbatim (including extra
quotes) while the db had the parsed form. That caused the etag
comparison during discovery move detection to always fail. The test
didn't catch it because the etags there didn't have quotes.

Now:
- RequestEtagJob will parse the etag, leading to a consistent format
- Tests have etags with quotes, detecting the problem
2020-12-15 10:58:52 +01:00
Christian Kamm
69fa1e4775
UploadDevice: Fix windows issues #7264
- Close the UploadDevice to close the QFile after the PUT job is done.
  This allows winvfs to get an oplock on the file later.

- Don't rely on QFile::fileName() to be valid after
  openAndSeekFileSharedRead() was called. The way it is openend on
  Windows makes it have an empty filename.
2020-12-15 10:58:52 +01:00
Christian Kamm
fb47419e84
Vfs: Move pin state if files move #7250
Previously renames of items didn't carry the pin state with them.
2020-12-15 10:58:52 +01:00
Christian Kamm
34dc5e4e88
Vfs: Don't let new local files start out unpinned #7250
If one adds a new file to an online-only folder the previous behavior
was to upload the file in one sync and dehydrate it in the next. Now
these new files get set to Unspecified pin state, making them retain
their data.
2020-12-15 10:58:52 +01:00
Markus Goetz
62d876b09a
OAuth2: Better error logging
This does not fix a bug, just was found while spotting a bug that was no bug.
For https://github.com/owncloud/enterprise/issues/2951
2020-12-15 10:58:51 +01:00
Christian Kamm
452ed56571
Upload: Read file chunks gradually #7226
Instead of all at once, to reduce peak memory use.

Changing UploadDevice in this way requires keeping the file open for the
duration of the upload. It also means changes to open(), seek(), close()
to ensure that uses of the device work right when a request needs to
be resent.
2020-12-15 10:58:51 +01:00
Christian Kamm
a7852e3aba
Http2: Resend requests on ContentReSend error #7174
Since Qt does not yet transparently resend HTTP2 requests in some cases
we do it manually.

The test showed a problem where the initial non-200 reply would close
the target temporary file and the follow-up request couldn't store any
data. Removing that close() call is safe because there also is a
_saveBodyToFile flag that guards writes to the target file.
2020-12-15 10:58:51 +01:00
Olivier Goffart
c335f69a65
Discovery: Do not abort the sync in case of error 404 (or 500)
Issue: #7199
2020-12-15 10:58:51 +01:00
Olivier Goffart
9f08636a4a
SyncEngine: Fix renaming a single file cause the "delete all file" popup
Possibly a regression, since the new discovery discovers rist the renamed
files as removed

Issue #7204
2020-12-15 10:58:51 +01:00
Christian Kamm
edad7ce7d7
Vfs: Mark sqlite temporaries excluded on db-open #7141
The previous patch ensured that the sqlite temporaries weren't deleted
and recreated for every sync run, but there was still time between
client startup and the first sync run where they would have the
"needs-sync" icon.
2020-12-15 10:58:50 +01:00
Christian Kamm
7774b8049e
Vfs: Distinguish availability error kinds #7143
Previously "no-availability" meant db-error and querying the
availability of a nonexistant path returned AllHydrated.

Now, the availability has a DbError and a NoSuchItem error case.
2020-12-15 10:58:49 +01:00
Christian Kamm
21cb93e3ce
SyncEngine: Don't close db when done #7141
The db-close operation is likely a leftover from when the SyncEngine
owned its own db connection and serves no purpose anymore.

Closing the db causes the removal of the temporary wal and shm files.
These files are recreated when the db is opened again, which happens
almost immediately.

This is a problem for winvfs because the delete-recreate step wipes the
exclusion state on these files just after the sync is done. That meant
that the db temporaries permanently had a "needs sync" icon marker shown
in the explorer.

Avoiding reopening the db also reduces the number of log messages per
sync.
2020-12-15 10:58:48 +01:00
Christian Kamm
cfbcdc01db
Discovery: Improvements to doc comments 2020-12-15 10:58:48 +01:00
Christian Kamm
fbe2dbf4ab
Discovery: Query data-fingerprint on root item
Previously the property wasn't queried, meaning the fingerprint logic
couldn't get triggered.
2020-12-15 10:58:48 +01:00
Christian Kamm
c50f041c5b
Discovery: 403 and 503 on root cause error
Previously these result codes during remote discovery of the sync root
would not cause an error and the discovery would get stuck.

Also extends RemoteDiscovery tests to check for errors on the root item.
2020-12-15 10:58:48 +01:00
Christian Kamm
7b96321df2
PropagateIgnore: Default to NormalError for INSTRUCTION_ERROR
Previously if one set the instruction to ERROR while forgetting to set
an error status, it'd propagate as FileIgnored. Now the default is
NormalError for INSTRUCTION_ERROR and FileIgnored for
INSTRUCTION_IGNORE.
2020-12-15 10:58:47 +01:00
Christian Kamm
da40e84aec
About: Add remark about vfs plugin that's in use #7137 2020-12-15 10:58:47 +01:00
Christian Kamm
2bffde2600
Fix windows build 2020-12-15 10:58:47 +01:00
Christian Kamm
22234e0e73
LogWindow: Remove output, add "go to log folder" button #6475 2020-12-15 10:58:47 +01:00
Christian Kamm
dcf34316fd
Vfs: Add 'availability', a simplified, user-facing pin state #7111
The idea is that the user's question is "is this folder's data available
offline?" and not "does this folder have AlwaysLocal pin state?".
The the answers to the two questions can differ: an always-local
folder can have subitems that are not always-local and are dehydrated.

The new availability enum intends to describe the answer to the user's
actual question and can be derived from pin states. If pin states aren't
stored in the database the way of calculating availability will depend
on the vfs plugin.
2020-12-15 10:58:47 +01:00
Christian Kamm
590db28541
Vfs: Clear up relationship between _type and pin state
The pin state is a per-item attribute that has an effect on _type:
AlwaysLocal dehydrated files will be marked for hydration and OnlineOnly
hydrated files will be marked for dehydration.

Where exactly this effect materializes depends on how the pin states are
stored. If they're stored in the db (suffix) the dbEntry._type is
changed during the discovery.

If the pin state is stored in the filesystem, the localEntry._type must
be adjusted by the plugin's stat callback.

This patch makes pin states behave more consistently between plugins.
Previously with suffix-vfs pin states only had an effect on new remote
files. Now the effect of pinning or unpinning files or directories is as
documented and similar to other plugins.
2020-12-15 10:58:46 +01:00
Christian Kamm
cd10e3d28c
PropagateDirectory: Set initial dir mtime to server mtime #7119
It's still not synced in any way later.
2020-12-15 10:58:46 +01:00
Christian Kamm
69887c531e
PropagateDirectory: Remove dead code
1. The _firstJob is usually deleted by the time the PropagateDirectory
   finishes. (deleteLater() is called early)
2. The PropagateDirectory::_item and PropagateRemoteMkdir::_item point
   to the same SyncFileItem anyway. This code is a leftover from when
   each job had its own instance.
2020-12-15 10:58:46 +01:00
Christian Kamm
fd9b01981b
Detect missing server data during discovery #7112
This has two positive effects:
- We can put the error on the particular file that has missing data
- We can sync all other files
2020-12-15 10:58:46 +01:00
Christian Kamm
4bab93b246
Vfs: Better handling and more tests for suffix file renames
Previously removing the vfs suffix of a file always triggered a
conflict. Now it may just cause a file download.

This was done because users expected symmetry in the rename actions and
renaming foo -> foo.owncloud already triggers the "make the file
virtual" action. Now foo.owncloud -> foo triggers the "download the
contents" action.
2020-12-15 10:58:46 +01:00
Christian Kamm
5e5b0b3f76
Vfs suffix: Require suffix when creating placeholder files 2020-12-15 10:58:45 +01:00
Christian Kamm
3f55f9302e
Vfs: Hydrating a virtual is SYNC not NEW #7101
Previously it'd be NEW(ItemTypeFile), but now it has changed to be
SYNC(ItemTypeVirtualFileDownload) which allows better classification.
2020-12-15 10:58:45 +01:00
Christian Kamm
0c0049c908
Local discovery tracking: On success, also wipe _renameTarget
It's possible that the rename target was in the local discovery list.
2020-12-15 10:58:45 +01:00
Christian Kamm
47f10fbf96
Fix and test _file and _renameTarget
There was a bunch of inconsistency around whether _file was set to
_renameTarget or not. This is now never done, passing on more
information.
2020-12-15 10:58:44 +01:00
Christian Kamm
0cf19123a7
VFS: Unbreak behavior for rename+hydrate #7001
Users can rename a file *and* add/remove the vfs suffix at the same time
leading to very complex sync actions. This patch doesn't add support for
them, but adds tests and makes sure these cases do not cause unintened
behavior.

The rename will be propagated, but the users's hydrate/dehydrate request
will be ignored.
2020-12-15 10:58:44 +01:00
Olivier Goffart
4346567a03
Async Poll: keep the size in the database
This was not required with 2.5 because a size of 0 was ignorted when comparing
size by the csync updater, to be compatible with very old version of the database.
But the we discovery will still think the file is changed if the database contains
a size of 0
2020-12-15 10:58:44 +01:00
Olivier Goffart
46bf3ed31a
Upload: asynchronious operations
Implements https://github.com/owncloud/core/pull/31851
2020-12-15 10:58:44 +01:00
Markus Goetz
32c60c2f5d
macOS: Fix vfs suffix plugin paths #7090 2020-12-15 10:58:44 +01:00
Christian Kamm
ee6a48b3dc
Client certs: Store pkcs12 in config, password in keychain
It still reads and writes the old format too, but all newly stored
client certs will be in the new form.

For #6776 because Windows limits credential data to 512 bytes in older
versions.
2020-12-15 10:58:43 +01:00
Christian Kamm
cc840534c0
Add PLUGINDIR cmake setting and define #7027
By default, plugins are only searched next to the binary or next to the
other Qt plugins. This optional build variable allows another path to be
configured.

The idea is that on linux the oC packaging probably wants the binary in
something like /opt/owncloud/bin and the plugins in
/opt/owncloud/lib/plugins.

Similarly, distribution packagers probably don't want the plugins next
to the binary or next to the other Qt plugins. This flag allows them to
configure another path that the executable will look in.
2020-12-15 10:58:43 +01:00
Christian Kamm
93afc2a04b
Discovery win: Fix detection of case-only renames
Previously they were detected as DELETE+NEW because if "a" is renamed to
"A" then QFile::exists("a") will still return true on Windows.
2020-12-15 10:58:42 +01:00
Christian Kamm
238ac53666
Ensure local discovery on selective sync changes
As far as I'm aware local discovery can be skipped on folders that are
selective-sync blacklisted, so a local discovery is required when an
entry is removed from the blacklist.

Also rename
avoidReadFromDbOnNextSync() -> schedulePathForRemoteDiscovery()
since the old name might also imply it's not read from db in the local
discovery - which is not the case. Use Folder::
schedulePathForLocalDiscovery() for that.
2020-12-15 10:58:41 +01:00
Christian Kamm
4c04351360
Fix warnings about signedness
Sizes are always qint64, not unsigned.
TransferIds are always uint.
2020-12-15 10:58:41 +01:00
Christian Kamm
910ccaf600
Vfs: Improve sync protocol entries for actions
Creating a new virtual file and replacing a file with a virtual one now
have their own text in the protocol, not just "Downloaded".

To do this, the SyncFileItem type is kept as
ItemTypeVirtualFileDehydration for these actions. Added new code to
ensure the type isn't written to the database.

While looking at this, I've also added documentation on SyncFileItem's
_file, _renameTarget, _originalFile and destination() because some of
the semantics weren't clear.
2020-12-15 10:58:41 +01:00
Christian Kamm
f24687ec93
Vfs: Make dehydration a SYNC and not a NEW action
That change will be useful for the notifications. Previously the
dehydrated files were reported as "newly downloaded", now they're
reported as "updated".
2020-12-15 10:58:40 +01:00
Christian Kamm
797734870f
PropagateDownload: Create conflict even if local file changed
Fixes a bug introduced while moving the attribute propagation before the
conflict-renaming.
2020-12-15 10:58:40 +01:00
Christian Kamm
8a8e93827f
Vfs suffix: Fix dehydration creating the wrong db entry 2020-12-15 10:58:40 +01:00
Christian Kamm
6a977edeee
Vfs: Remove VfsDefaults
That just complicated things. It's ok if Vfs is not a fully abstract
interface class.

The pinstate-in-db methods are instead provided directly on Vfs and
VfsSuffix and VfsOff use them to implement pin states.

The start() method is simply non-virtual and calls into startImpl() for
the plugin-specific startup code.
2020-12-15 10:58:40 +01:00
Christian Kamm
e9cbe13598
Vfs: Add option to hide socketapi pin actions
Because some plugins provide alternative ui.
2020-12-15 10:58:40 +01:00
Christian Kamm
e6ee5d0f8a
PropagateDownload: Conflict-rename later
The block of code that propagated attributes etc from the previously
existing file was placed *after* the block that renamed the previously
existing file to a conflict name. That meant the propagation didn't work
in the conflict case.
2020-12-15 10:58:39 +01:00
Christian Kamm
6c5fa1dadd
Vfs: dehydration is separate action
Allows for better attribute preservation.

Also add verifyFileUnchanged() call before dehydration to avoid data
loss when discovery takes a while.
2020-12-15 10:58:39 +01:00
Christian Kamm
83a818678f
PinStates cleanup
- SyncJournalDB functions now behind internalPinStates() to avoid
accidental usage, when nearly everyone should go through Vfs.
- Rename Vfs::getPinState() to Vfs::pinState()
2020-12-15 10:58:39 +01:00
Christian Kamm
597cc60f52
Vfs: Enable propagating attributes on download 2020-12-15 10:58:38 +01:00
Christian Kamm
01f7bc7b49
SyncFileStatusTracker: Distinguish Warning and Excluded
Any folder with a (potentially deeply) contained error will have
StatusWarning. StatusExcluded marks exclusions. The difference is useful
to know for VFS.
2020-12-15 10:58:38 +01:00
Christian Kamm
7f400e3226
Pin state updates
- unspecified and inherited are different
- move enum to header in common/
- access through Vfs instead of directly in Journal
2020-12-15 10:58:38 +01:00
Christian Kamm
f89450648f
Vfs: Clarify SyncEngine::wipeVirtualFiles()
Possibly the behavior should actually change and the function should
de-placeholder all items, not just dehydrated ones.
2020-12-15 10:58:38 +01:00
Christian Kamm
5820ac8b41
Discovery: Files can have dehydrate/download actions
This will be used in conjunction with vfs plugins that detect whether a
file has a pending hydration/dehydration through independent means and
communicate that to the discovery through local file type.
2020-12-15 10:58:38 +01:00
Christian Kamm
af1666788e
Discovery: Add signal for silentlyExcluded files
This allows SyncFileStatusTracker to also know about these. After all
its information is used to provide icons for them too.
2020-12-15 10:58:37 +01:00