Commit graph

15835 commits

Author SHA1 Message Date
Christian Kamm
eb58352286
Vfs: Make test pass with different suffix #7279 2020-12-15 10:58:52 +01:00
Christian Kamm
5acb157a7e
VfsSuffix: Wipe stale pin states #7273
Previously the pin states of deleted files stayed in the 'flags'
database and could be inadvertently reused when a new file with the same
name appeared. Now they are deleted.

To make this work right, the meaning of the 'path' column in the 'flags'
table was changed: Previously it never had the .owncloud file suffix.
Now it's the same as in metadata.path.

This takes the safe parts from #7274 for inclusion in 2.6. The more
elaborate database schema changes (why use 'path' the join the two
tables in the first place?) shall go into master.
2020-12-15 10:58:52 +01:00
Christian Kamm
0e9f030b0f
RequestEtagJob: Consistently parse etags #7271
Previously RequestEtagJob did return the etag verbatim (including extra
quotes) while the db had the parsed form. That caused the etag
comparison during discovery move detection to always fail. The test
didn't catch it because the etags there didn't have quotes.

Now:
- RequestEtagJob will parse the etag, leading to a consistent format
- Tests have etags with quotes, detecting the problem
2020-12-15 10:58:52 +01:00
Christian Kamm
69fa1e4775
UploadDevice: Fix windows issues #7264
- Close the UploadDevice to close the QFile after the PUT job is done.
  This allows winvfs to get an oplock on the file later.

- Don't rely on QFile::fileName() to be valid after
  openAndSeekFileSharedRead() was called. The way it is openend on
  Windows makes it have an empty filename.
2020-12-15 10:58:52 +01:00
Christian Kamm
fb47419e84
Vfs: Move pin state if files move #7250
Previously renames of items didn't carry the pin state with them.
2020-12-15 10:58:52 +01:00
Christian Kamm
34dc5e4e88
Vfs: Don't let new local files start out unpinned #7250
If one adds a new file to an online-only folder the previous behavior
was to upload the file in one sync and dehydrate it in the next. Now
these new files get set to Unspecified pin state, making them retain
their data.
2020-12-15 10:58:52 +01:00
Markus Goetz
62d876b09a
OAuth2: Better error logging
This does not fix a bug, just was found while spotting a bug that was no bug.
For https://github.com/owncloud/enterprise/issues/2951
2020-12-15 10:58:51 +01:00
Christian Kamm
452ed56571
Upload: Read file chunks gradually #7226
Instead of all at once, to reduce peak memory use.

Changing UploadDevice in this way requires keeping the file open for the
duration of the upload. It also means changes to open(), seek(), close()
to ensure that uses of the device work right when a request needs to
be resent.
2020-12-15 10:58:51 +01:00
Christian Kamm
a7852e3aba
Http2: Resend requests on ContentReSend error #7174
Since Qt does not yet transparently resend HTTP2 requests in some cases
we do it manually.

The test showed a problem where the initial non-200 reply would close
the target temporary file and the follow-up request couldn't store any
data. Removing that close() call is safe because there also is a
_saveBodyToFile flag that guards writes to the target file.
2020-12-15 10:58:51 +01:00
Olivier Goffart
c335f69a65
Discovery: Do not abort the sync in case of error 404 (or 500)
Issue: #7199
2020-12-15 10:58:51 +01:00
Olivier Goffart
9f08636a4a
SyncEngine: Fix renaming a single file cause the "delete all file" popup
Possibly a regression, since the new discovery discovers rist the renamed
files as removed

Issue #7204
2020-12-15 10:58:51 +01:00
Olivier Goffart
2c975b1e70
SocketAPI: Fix string claiming a folder is a file
Issue #7206
2020-12-15 10:58:51 +01:00
Markus Goetz
46c336503a
Log Window: Create and open folder properly #7166 2020-12-15 10:58:50 +01:00
Christian Kamm
ee1078b768
owncloudcmd: log sync errors
Previously it was hard to debug some errors since the messages would not
show up in the output.
2020-12-15 10:58:50 +01:00
Christian Kamm
b974f579ae
Chunked upload: Fix percent encoding in If header #7176 2020-12-15 10:58:50 +01:00
Christian Kamm
9f3578b276
Tests: Fix DB locking issue in permissions test 2020-12-15 10:58:50 +01:00
Christian Kamm
a0457d5e1d
Tests: fix vfs availability test case 2020-12-15 10:58:50 +01:00
Christian Kamm
edad7ce7d7
Vfs: Mark sqlite temporaries excluded on db-open #7141
The previous patch ensured that the sqlite temporaries weren't deleted
and recreated for every sync run, but there was still time between
client startup and the first sync run where they would have the
"needs-sync" icon.
2020-12-15 10:58:50 +01:00
Christian Kamm
7774b8049e
Vfs: Distinguish availability error kinds #7143
Previously "no-availability" meant db-error and querying the
availability of a nonexistant path returned AllHydrated.

Now, the availability has a DbError and a NoSuchItem error case.
2020-12-15 10:58:49 +01:00
Christian Kamm
1e5ae77994
Fix logic for duration that an etag reply indicates connectivity
This got inverted accidentally when std::chrono was introduced.

For #7160
2020-12-15 10:58:49 +01:00
Christian Kamm
7f3f13fd97
Vfs: "free space" only shows when it has an effect #7143
To do this, introduce AllDehydrated availability and rename
SomeDehydrated to Mixed - it now guarantees there are also hydrated
items.
2020-12-15 10:58:49 +01:00
Christian Kamm
40d9fc4f4b
Vfs: Adjust and centralise action text #7143
Saying "Currently available locally" sounds more like an indicator than
"Availably locally" does. Centralizing translations avoids consistency
issues between shell context menus and sync folder context menu.
2020-12-15 10:58:49 +01:00
Christian Kamm
021f994584
FolderWizard: Don't crash when typing invalid drive #7041
When the user typed "x:" where the drive x didn't exist, the validation
function would loop forever. Now it shows a "path doesn't exist" error.
2020-12-15 10:58:49 +01:00
Christian Kamm
21cb93e3ce
SyncEngine: Don't close db when done #7141
The db-close operation is likely a leftover from when the SyncEngine
owned its own db connection and serves no purpose anymore.

Closing the db causes the removal of the temporary wal and shm files.
These files are recreated when the db is opened again, which happens
almost immediately.

This is a problem for winvfs because the delete-recreate step wipes the
exclusion state on these files just after the sync is done. That meant
that the db temporaries permanently had a "needs sync" icon marker shown
in the explorer.

Avoiding reopening the db also reduces the number of log messages per
sync.
2020-12-15 10:58:48 +01:00
Christian Kamm
a4f357ee4b
Sqlite: Use FULL synchronous mode with non-WAL journal
According to the documentation DELETE+NORMAL isn't safe from corruption
on older file systems.
2020-12-15 10:58:48 +01:00
Christian Kamm
076e24f465
Log: Start logging immediately when "permanent logs" enabled #7146
Previously one would need to wait for the next sync run to create the
first log file.
2020-12-15 10:58:48 +01:00
Christian Kamm
772a210cc9
Mention selective sync when switching on vfs 2020-12-15 10:58:48 +01:00
Christian Kamm
cfbcdc01db
Discovery: Improvements to doc comments 2020-12-15 10:58:48 +01:00
Christian Kamm
fbe2dbf4ab
Discovery: Query data-fingerprint on root item
Previously the property wasn't queried, meaning the fingerprint logic
couldn't get triggered.
2020-12-15 10:58:48 +01:00
Christian Kamm
c50f041c5b
Discovery: 403 and 503 on root cause error
Previously these result codes during remote discovery of the sync root
would not cause an error and the discovery would get stuck.

Also extends RemoteDiscovery tests to check for errors on the root item.
2020-12-15 10:58:48 +01:00
Christian Kamm
7b96321df2
PropagateIgnore: Default to NormalError for INSTRUCTION_ERROR
Previously if one set the instruction to ERROR while forgetting to set
an error status, it'd propagate as FileIgnored. Now the default is
NormalError for INSTRUCTION_ERROR and FileIgnored for
INSTRUCTION_IGNORE.
2020-12-15 10:58:47 +01:00
Christian Kamm
da40e84aec
About: Add remark about vfs plugin that's in use #7137 2020-12-15 10:58:47 +01:00
Christian Kamm
2bffde2600
Fix windows build 2020-12-15 10:58:47 +01:00
Christian Kamm
22234e0e73
LogWindow: Remove output, add "go to log folder" button #6475 2020-12-15 10:58:47 +01:00
Christian Kamm
dcf34316fd
Vfs: Add 'availability', a simplified, user-facing pin state #7111
The idea is that the user's question is "is this folder's data available
offline?" and not "does this folder have AlwaysLocal pin state?".
The the answers to the two questions can differ: an always-local
folder can have subitems that are not always-local and are dehydrated.

The new availability enum intends to describe the answer to the user's
actual question and can be derived from pin states. If pin states aren't
stored in the database the way of calculating availability will depend
on the vfs plugin.
2020-12-15 10:58:47 +01:00
Christian Kamm
590db28541
Vfs: Clear up relationship between _type and pin state
The pin state is a per-item attribute that has an effect on _type:
AlwaysLocal dehydrated files will be marked for hydration and OnlineOnly
hydrated files will be marked for dehydration.

Where exactly this effect materializes depends on how the pin states are
stored. If they're stored in the db (suffix) the dbEntry._type is
changed during the discovery.

If the pin state is stored in the filesystem, the localEntry._type must
be adjusted by the plugin's stat callback.

This patch makes pin states behave more consistently between plugins.
Previously with suffix-vfs pin states only had an effect on new remote
files. Now the effect of pinning or unpinning files or directories is as
documented and similar to other plugins.
2020-12-15 10:58:46 +01:00
Christian Kamm
2738f110f2
Sqlite: Update to 3.27.2 2020-12-15 10:58:46 +01:00
Christian Kamm
cd10e3d28c
PropagateDirectory: Set initial dir mtime to server mtime #7119
It's still not synced in any way later.
2020-12-15 10:58:46 +01:00
Christian Kamm
69887c531e
PropagateDirectory: Remove dead code
1. The _firstJob is usually deleted by the time the PropagateDirectory
   finishes. (deleteLater() is called early)
2. The PropagateDirectory::_item and PropagateRemoteMkdir::_item point
   to the same SyncFileItem anyway. This code is a leftover from when
   each job had its own instance.
2020-12-15 10:58:46 +01:00
Christian Kamm
fd9b01981b
Detect missing server data during discovery #7112
This has two positive effects:
- We can put the error on the particular file that has missing data
- We can sync all other files
2020-12-15 10:58:46 +01:00
Christian Kamm
4bab93b246
Vfs: Better handling and more tests for suffix file renames
Previously removing the vfs suffix of a file always triggered a
conflict. Now it may just cause a file download.

This was done because users expected symmetry in the rename actions and
renaming foo -> foo.owncloud already triggers the "make the file
virtual" action. Now foo.owncloud -> foo triggers the "download the
contents" action.
2020-12-15 10:58:46 +01:00
Christian Kamm
5e5b0b3f76
Vfs suffix: Require suffix when creating placeholder files 2020-12-15 10:58:45 +01:00
Christian Kamm
3f55f9302e
Vfs: Hydrating a virtual is SYNC not NEW #7101
Previously it'd be NEW(ItemTypeFile), but now it has changed to be
SYNC(ItemTypeVirtualFileDownload) which allows better classification.
2020-12-15 10:58:45 +01:00
Christian Kamm
e0c8acc195
Notifications: Don't say "downloaded" for new files #7101
These files may very well just be new virtual files that were explicitly
*not* downloaded.
2020-12-15 10:58:45 +01:00
Christian Kamm
0c0049c908
Local discovery tracking: On success, also wipe _renameTarget
It's possible that the rename target was in the local discovery list.
2020-12-15 10:58:45 +01:00
Christian Kamm
04e0e22513
Protocol, Notifications: Show destination() instead of _file
destination() now consistently points to the file after the successful
sync operation. _file might be the place the item was moved from.
2020-12-15 10:58:45 +01:00
Christian Kamm
47f10fbf96
Fix and test _file and _renameTarget
There was a bunch of inconsistency around whether _file was set to
_renameTarget or not. This is now never done, passing on more
information.
2020-12-15 10:58:44 +01:00
Olivier Goffart
afbb580528
Test: fix compilation with GCC 4.9
It does not appear to support variadic lambda
2020-12-15 10:58:44 +01:00
Christian Kamm
0cf19123a7
VFS: Unbreak behavior for rename+hydrate #7001
Users can rename a file *and* add/remove the vfs suffix at the same time
leading to very complex sync actions. This patch doesn't add support for
them, but adds tests and makes sure these cases do not cause unintened
behavior.

The rename will be propagated, but the users's hydrate/dehydrate request
will be ignored.
2020-12-15 10:58:44 +01:00
Olivier Goffart
4346567a03
Async Poll: keep the size in the database
This was not required with 2.5 because a size of 0 was ignorted when comparing
size by the csync updater, to be compatible with very old version of the database.
But the we discovery will still think the file is changed if the database contains
a size of 0
2020-12-15 10:58:44 +01:00