Commit graph

15702 commits

Author SHA1 Message Date
Christian Kamm
bade7aedc6
Checksums: Add back QIODevice api
Because the winvfs plugin needs it. But be more careful about the
device's lifetime this time.
2020-12-15 10:58:55 +01:00
Christian Kamm
18e1098e38
Remove maxLogLines config option
It's no longer used.

For owncloud/docs#1365
2020-12-15 10:58:55 +01:00
Christian Kamm
ea829f96ca
Download: Don't trigger too many concurrent hash computations
Previously the job would only become "active" when the downloads
started. That meant that arbitrarily many hash computations could be
queued at the same time.

Since the the file was opened during future creation this could lead to
a "too many open files" problem if there were lots of new-new conflicts.

To change this:
- Make PropagateDownload become active when computing a hash
  asynchronously.
- Make the computation future open the file only once it gets run. This
  will make it less likely for this problem to occur even if thousands
  of these futures are queued.

For #7372
2020-12-15 10:58:55 +01:00
Christian Kamm
8a5a185752
Fix 'unused parameter name' warnings 2020-12-15 10:58:55 +01:00
Christian Kamm
c9dbe46542
Checksums: Fix crash due to threading issue
The checksum computation thread was potentially using a QFile that was
deleted in the gui thread.

For #7368
2020-12-15 10:58:54 +01:00
Christian Kamm
a587cd3a13
AccountSettings: Fetch subitems after wiping them
That helps avoid empty lists after account creation #7336
2020-12-15 10:58:54 +01:00
Christian Kamm
5761f4cd8a
SyncEngine: Don't duplicate fatal errors
Previously fatal error texts were duplicated: Once they entered the
SyncResult via the SyncFileItem and once via syncError().

syncError is intended for folder-wide sync issues that are not pinned
to particular files. Thus that duplicated path is removed.

For #5088
2020-12-15 10:58:54 +01:00
Christian Kamm
61972c35a8
Discovery: Parse etag to be consistent with RequestEtagJob
This avoids unnecessary sync runs.

For #7345
2020-12-15 10:58:54 +01:00
Christian Kamm
38c466efbf
Vfs: Require local discovery after disabling vfs
Without it local files aren't guaranteed to be downloaded #6936.
2020-12-15 10:58:54 +01:00
Christian Kamm
55ee3f440b
Vfs: Remove old db record when dehydrating via rename
For #7338
2020-12-15 10:58:54 +01:00
Christian Kamm
e6990bd04e
Vfs: Make move detection work with virtual files #7001
Previously a checksum computation could be done on a suffix-placeholder
file, making discovery believe that no move took place.
2020-12-15 10:58:54 +01:00
Christian Kamm
00dcf3ef59
Vfs: Ensure pins change with (de-)hydration
Previously an implicit hydration of a file in an online-only folder
would not change the pin state and cause a dehydration on the next
sync.
2020-12-15 10:58:53 +01:00
Christian Kamm
0df3b83bd2
Vfs: Add hook to allow update-metadata for unchanged files
Allows winvfs to convert files to placeholders when vfs is enabled. This
is required to mark files as in-sync #7329.
2020-12-15 10:58:53 +01:00
Christian Kamm
b34df4413b
Vfs: Improve strings for availability states 2020-12-15 10:58:53 +01:00
Christian Kamm
5bc2180478
Vfs: Preserve pin state on hydration
For #7322 and #7323
2020-12-15 10:58:53 +01:00
Christian Kamm
fc52c5d0cd
Vfs: Retain existing data when enabling vfs #7302
Previously all local data was deleted because the root folder was marked
as OnlineOnly.
2020-12-15 10:58:53 +01:00
Dominik Schmidt
92f6d866e1
Add missing OWNCLOUDSYNC_EXPORT 2020-12-15 10:58:52 +01:00
Christian Kamm
eb58352286
Vfs: Make test pass with different suffix #7279 2020-12-15 10:58:52 +01:00
Christian Kamm
5acb157a7e
VfsSuffix: Wipe stale pin states #7273
Previously the pin states of deleted files stayed in the 'flags'
database and could be inadvertently reused when a new file with the same
name appeared. Now they are deleted.

To make this work right, the meaning of the 'path' column in the 'flags'
table was changed: Previously it never had the .owncloud file suffix.
Now it's the same as in metadata.path.

This takes the safe parts from #7274 for inclusion in 2.6. The more
elaborate database schema changes (why use 'path' the join the two
tables in the first place?) shall go into master.
2020-12-15 10:58:52 +01:00
Christian Kamm
0e9f030b0f
RequestEtagJob: Consistently parse etags #7271
Previously RequestEtagJob did return the etag verbatim (including extra
quotes) while the db had the parsed form. That caused the etag
comparison during discovery move detection to always fail. The test
didn't catch it because the etags there didn't have quotes.

Now:
- RequestEtagJob will parse the etag, leading to a consistent format
- Tests have etags with quotes, detecting the problem
2020-12-15 10:58:52 +01:00
Christian Kamm
69fa1e4775
UploadDevice: Fix windows issues #7264
- Close the UploadDevice to close the QFile after the PUT job is done.
  This allows winvfs to get an oplock on the file later.

- Don't rely on QFile::fileName() to be valid after
  openAndSeekFileSharedRead() was called. The way it is openend on
  Windows makes it have an empty filename.
2020-12-15 10:58:52 +01:00
Christian Kamm
fb47419e84
Vfs: Move pin state if files move #7250
Previously renames of items didn't carry the pin state with them.
2020-12-15 10:58:52 +01:00
Christian Kamm
34dc5e4e88
Vfs: Don't let new local files start out unpinned #7250
If one adds a new file to an online-only folder the previous behavior
was to upload the file in one sync and dehydrate it in the next. Now
these new files get set to Unspecified pin state, making them retain
their data.
2020-12-15 10:58:52 +01:00
Markus Goetz
62d876b09a
OAuth2: Better error logging
This does not fix a bug, just was found while spotting a bug that was no bug.
For https://github.com/owncloud/enterprise/issues/2951
2020-12-15 10:58:51 +01:00
Christian Kamm
452ed56571
Upload: Read file chunks gradually #7226
Instead of all at once, to reduce peak memory use.

Changing UploadDevice in this way requires keeping the file open for the
duration of the upload. It also means changes to open(), seek(), close()
to ensure that uses of the device work right when a request needs to
be resent.
2020-12-15 10:58:51 +01:00
Christian Kamm
a7852e3aba
Http2: Resend requests on ContentReSend error #7174
Since Qt does not yet transparently resend HTTP2 requests in some cases
we do it manually.

The test showed a problem where the initial non-200 reply would close
the target temporary file and the follow-up request couldn't store any
data. Removing that close() call is safe because there also is a
_saveBodyToFile flag that guards writes to the target file.
2020-12-15 10:58:51 +01:00
Olivier Goffart
c335f69a65
Discovery: Do not abort the sync in case of error 404 (or 500)
Issue: #7199
2020-12-15 10:58:51 +01:00
Olivier Goffart
9f08636a4a
SyncEngine: Fix renaming a single file cause the "delete all file" popup
Possibly a regression, since the new discovery discovers rist the renamed
files as removed

Issue #7204
2020-12-15 10:58:51 +01:00
Olivier Goffart
2c975b1e70
SocketAPI: Fix string claiming a folder is a file
Issue #7206
2020-12-15 10:58:51 +01:00
Markus Goetz
46c336503a
Log Window: Create and open folder properly #7166 2020-12-15 10:58:50 +01:00
Christian Kamm
ee1078b768
owncloudcmd: log sync errors
Previously it was hard to debug some errors since the messages would not
show up in the output.
2020-12-15 10:58:50 +01:00
Christian Kamm
b974f579ae
Chunked upload: Fix percent encoding in If header #7176 2020-12-15 10:58:50 +01:00
Christian Kamm
9f3578b276
Tests: Fix DB locking issue in permissions test 2020-12-15 10:58:50 +01:00
Christian Kamm
a0457d5e1d
Tests: fix vfs availability test case 2020-12-15 10:58:50 +01:00
Christian Kamm
edad7ce7d7
Vfs: Mark sqlite temporaries excluded on db-open #7141
The previous patch ensured that the sqlite temporaries weren't deleted
and recreated for every sync run, but there was still time between
client startup and the first sync run where they would have the
"needs-sync" icon.
2020-12-15 10:58:50 +01:00
Christian Kamm
7774b8049e
Vfs: Distinguish availability error kinds #7143
Previously "no-availability" meant db-error and querying the
availability of a nonexistant path returned AllHydrated.

Now, the availability has a DbError and a NoSuchItem error case.
2020-12-15 10:58:49 +01:00
Christian Kamm
1e5ae77994
Fix logic for duration that an etag reply indicates connectivity
This got inverted accidentally when std::chrono was introduced.

For #7160
2020-12-15 10:58:49 +01:00
Christian Kamm
7f3f13fd97
Vfs: "free space" only shows when it has an effect #7143
To do this, introduce AllDehydrated availability and rename
SomeDehydrated to Mixed - it now guarantees there are also hydrated
items.
2020-12-15 10:58:49 +01:00
Christian Kamm
40d9fc4f4b
Vfs: Adjust and centralise action text #7143
Saying "Currently available locally" sounds more like an indicator than
"Availably locally" does. Centralizing translations avoids consistency
issues between shell context menus and sync folder context menu.
2020-12-15 10:58:49 +01:00
Christian Kamm
021f994584
FolderWizard: Don't crash when typing invalid drive #7041
When the user typed "x:" where the drive x didn't exist, the validation
function would loop forever. Now it shows a "path doesn't exist" error.
2020-12-15 10:58:49 +01:00
Christian Kamm
21cb93e3ce
SyncEngine: Don't close db when done #7141
The db-close operation is likely a leftover from when the SyncEngine
owned its own db connection and serves no purpose anymore.

Closing the db causes the removal of the temporary wal and shm files.
These files are recreated when the db is opened again, which happens
almost immediately.

This is a problem for winvfs because the delete-recreate step wipes the
exclusion state on these files just after the sync is done. That meant
that the db temporaries permanently had a "needs sync" icon marker shown
in the explorer.

Avoiding reopening the db also reduces the number of log messages per
sync.
2020-12-15 10:58:48 +01:00
Christian Kamm
a4f357ee4b
Sqlite: Use FULL synchronous mode with non-WAL journal
According to the documentation DELETE+NORMAL isn't safe from corruption
on older file systems.
2020-12-15 10:58:48 +01:00
Christian Kamm
076e24f465
Log: Start logging immediately when "permanent logs" enabled #7146
Previously one would need to wait for the next sync run to create the
first log file.
2020-12-15 10:58:48 +01:00
Christian Kamm
772a210cc9
Mention selective sync when switching on vfs 2020-12-15 10:58:48 +01:00
Christian Kamm
cfbcdc01db
Discovery: Improvements to doc comments 2020-12-15 10:58:48 +01:00
Christian Kamm
fbe2dbf4ab
Discovery: Query data-fingerprint on root item
Previously the property wasn't queried, meaning the fingerprint logic
couldn't get triggered.
2020-12-15 10:58:48 +01:00
Christian Kamm
c50f041c5b
Discovery: 403 and 503 on root cause error
Previously these result codes during remote discovery of the sync root
would not cause an error and the discovery would get stuck.

Also extends RemoteDiscovery tests to check for errors on the root item.
2020-12-15 10:58:48 +01:00
Christian Kamm
7b96321df2
PropagateIgnore: Default to NormalError for INSTRUCTION_ERROR
Previously if one set the instruction to ERROR while forgetting to set
an error status, it'd propagate as FileIgnored. Now the default is
NormalError for INSTRUCTION_ERROR and FileIgnored for
INSTRUCTION_IGNORE.
2020-12-15 10:58:47 +01:00
Christian Kamm
da40e84aec
About: Add remark about vfs plugin that's in use #7137 2020-12-15 10:58:47 +01:00
Christian Kamm
2bffde2600
Fix windows build 2020-12-15 10:58:47 +01:00