Commit graph

237 commits

Author SHA1 Message Date
Christian Kamm
e2eea24a03
Vfs: Send SyncFileStatusTracker data to vfs plugins 2020-12-15 10:58:37 +01:00
Christian Kamm
31394f14b5
Vfs: Make files that end up in db placeholders
Since 'placeholder' just means that it's an item of the special type
that the vfs plugin can deal with - no matter whether hydrated or
dehydrated - all done items should become placeholders. Even
directories.

Now every file that passes through updateMetadata() will be converted to
a placeholder if necessary.
2020-12-15 10:58:37 +01:00
Christian Kamm
fe27804afb
Move SyncFileStatus to libcommon
It'll be needed in vfs plugins so they can connect to the data coming
out of SyncFileStatusTracker.
2020-12-15 10:58:37 +01:00
Christian Kamm
22255e4976
vfs: Add vfs migration options to folder context menu
This allows enabling and disabling vfs.

To distinguish this operation from setting the root pin state, the
availability setting is adjusted as well to be similar to the
menu that shows in the shell extensions.
2020-12-15 10:58:33 +01:00
Christian Kamm
20ef0a0290
vfs: Allow retrieving of pin state paths and flags 2020-12-15 10:58:33 +01:00
Christian Kamm
7ef6e60660
Result: Add copy/move ctor/op=
It has a destructor and these operations make sense. Particularly the
move is important for code like:

Result<x, y> foo() { Result<x, y> v; return v; }

because the move-ctor will not autogenerate if x or y are not trivially
destructible.
2020-12-15 10:58:33 +01:00
Christian Kamm
4df101ed84
vfs: Allow (de-)hydrating the full sync folder 2020-12-15 10:58:33 +01:00
Christian Kamm
b4e73690b7
sqlite: Set exclusive locking_mode to avoid WAL issues #6881
Can be overridden with OWNCLOUD_SQLITE_LOCKING_MODE
2020-12-15 10:58:32 +01:00
Christian Kamm
aa382eda29
Db: Add wiping of pin state for subtrees 2020-12-15 10:58:30 +01:00
Christian Kamm
d956f518a8
vfs: Remove newFilesAreVirtual - use root PinState instead
This unifies how to deal with pin states.

Also enable reading a folders direct pin state vs its effective pin
state.
2020-12-15 10:58:30 +01:00
Christian Kamm
68126ac208
vfs: Introduce PinState db storage #6815
The idea is to allow folders (and later maybe files?) to be
- pinned to be available locally
- pinned to be online only
- inherit their pin from the parent

Where this pinning only controls the default for new files.
Subfolders may have a different pin state, and contained files
may be hydrated or dehydrated based on user actions.

This value is stored in a new 'flags' table. The idea is to store
data there that doesn't necessarily exist for each metadata entry.
The selective sync state could be migrated to this table.
2020-12-15 10:58:29 +01:00
Christian Kamm
69144566ce
Database: Change path for new dbs to .sync_* #5904
This is to avoid issues on OSX, where the ._ prefix has special meaning.

Originally (before 2.3.2) ._ was necessary to guarantee exclusion. But
since then the .sync_ prefix is excluded as well.

This does not affect existing database files.
2020-12-15 10:58:28 +01:00
Christian Kamm
bdfda460e6
Vfs plugins: Available plugins must load
A plugin that can't be loaded due to dependency issues should not be
considered as available.
2020-12-15 10:58:28 +01:00
Christian Kamm
d87c2b803d
vfs: Make Vfs ctor explicit 2020-12-15 10:58:27 +01:00
Christian Kamm
1e5e884805
vfs: Don't always load plugins, check metadata 2020-12-15 10:58:26 +01:00
Christian Kamm
b30f79edf6
vfs: Ensure SyncOptions::_vfs is never null
- Create a VfsOff derived class
- Make it a shared pointer shared with Folder::_vfs
2020-12-15 10:58:26 +01:00
Christian Kamm
fa2450cf11
vfs: Be more careful about Vfs instance ownership 2020-12-15 10:58:25 +01:00
Christian Kamm
77e5b956d1
Plugin: Drop PluginLoader 2020-12-15 10:58:25 +01:00
Christian Kamm
9bc28e3006
Plugin: style fixes 2020-12-15 10:58:25 +01:00
Christian Kamm
a5264f7740
Checksums: Make file ownership more explicit 2020-12-15 10:58:24 +01:00
Christian Kamm
9ced8dd836
vfs: Improve modeFromString() signature 2020-12-15 10:58:24 +01:00
Christian Kamm
f502a526fa
Generalize Result<> class, add Optional<>
To make it nicer to use outside of HTTP results.
2020-12-15 10:58:24 +01:00
Christian Kamm
e7e6b839c0
vfs: Add API docs 2020-12-15 10:58:24 +01:00
Dominik Schmidt
bfe136da7a
Fix export of vfs functions 2020-12-15 10:58:23 +01:00
Christian Kamm
fa6f3cd847
vfs: Fix plugin decision in wizards, sanitize loading 2020-12-15 10:58:23 +01:00
Christian Kamm
2b20985875
winvfs: initial work
Done by ckamm and dschmidt
2020-12-15 10:58:21 +01:00
Christian Kamm
22e08cf6ad
Checksums: Work on QIODevice*s
Needed for cfapi where we want to feed data through a custom device
which retrieves data from the windows api.
2020-12-15 10:58:21 +01:00
Christian Kamm
582803b6bd
cjhash: Fix build fix
See 17a4055e8d658f24b1fc47c44fd2daf66c7d5dbf
2020-12-15 10:58:20 +01:00
Olivier Goffart
175b064364
cjhash: use Q_FALLTHROUGH
This fixes the warning
   warning: unknown option after ‘#pragma GCC diagnostic’ kind [-Wpragmas]

Issue #6872
2020-12-15 10:58:20 +01:00
Olivier Goffart
d6a0290058
Optimisation: Add a cache SyncJournalDb::mapChecksumType
No need to do two sql query for something that's always the same and
there are very few checksum types
2020-12-15 10:58:19 +01:00
Olivier Goffart
69de2d5180
Sync: optimize by removing setFileRecordMetadata
Inh most case we already have a record from before, so avoid doing a useless
lookup in the database.
In owncloudpropagator.cpp, directories do not have a checksum so no need
to call a function that preserves it
2020-12-15 10:58:19 +01:00
Olivier Goffart
7e1840bb2f
OwnSql: Speedup by avoiding allocating an error string on success 2020-12-15 10:58:18 +01:00
Olivier Goffart
bb9ce8db21
Test that the sync behave well if there are errors while reading the database 2020-12-15 10:58:17 +01:00
Olivier Goffart
b820e46805
Fix compiler warning 2020-12-15 10:58:14 +01:00
Dominik Schmidt
5d9370594d
Disable autostartCheckBox if autostart is configured system wide 2020-12-15 10:58:14 +01:00
Christian Kamm
00edcf98a1
Discovery: Virtual file handling adjustments
- adjust virtual file path handing
- helpers for vfs suffix adding/removal
- helpers for isDirectory/isVirtual on SyncJournalRecords
- be clear about what PathTuple _local/_server mean
2020-12-15 10:58:14 +01:00
Olivier Goffart
58eaf9940a
Database: Add an index on the parent path
So we can quickly query the items in a parent directory

This uses a custom slite3 function, which means that when downgrading the client,
or using another tool to add entries in the database, any insertion in the metadata
table will produce an error: "unknown function: parent_hash()"
(This will crash the client 2.5)
2020-12-15 10:58:13 +01:00
Olivier Goffart
c009dae1ce
New discovery algorithm: fixups
Adapt reviews from ckamm in https://github.com/owncloud/client/pull/6738#pullrequestreview-164623532

- SyncJournalFileRecord: initialize everything inline
- Add more comments
- And some ENFORCE
2020-12-15 10:58:08 +01:00
Olivier Goffart
c8eff3da2d
New Discovery algorithm: Remove the sync cleanup phase
Since we do not recurse within some directories, many files are not seen.

The stale entry will cleanup by themself as the sync engine try to remove
the files that are already removed.
Should we need to actually do this cleanup, it should be dotected in the
discovery.
2020-12-15 10:58:08 +01:00
Christian Kamm
6b571b609c
Conflicts: Rename BasePath to InitialBasePath #6709 2020-12-15 10:58:05 +01:00
Olivier Goffart
d81ccbb0c1
Checksum: Add support for SHA256 and SHA3
In case, some day, the server also supports it
2020-12-15 10:58:05 +01:00
Olivier Goffart
bf6e4174c8
Move the checksum related routine from FileSystem to checksum, where they belong 2020-12-15 10:58:05 +01:00
Olivier Goffart
c3f745fa76
Conflict handling: add the OC-ConflictBasePath header
Issue #6702
2020-12-15 10:58:04 +01:00
Christian Kamm
be0fa72fcb
RemotePermissions: Fix empty vs null
Fixes two bugs that appeared since the introduction of the struct:
- when reading permissions from the journal, null ("") was read as
  empty-not-null
- when reading permissinos from the server, empty ("") was read as null

Addresses #4608
2020-12-15 10:58:03 +01:00
Olivier Goffart
7e36cc3fcb
New disco algorithm: Fix some moving
Fix TestSyncMove::testSelectiveSyncMovedFolder
2020-12-15 10:57:58 +01:00
Olivier Goffart
4ee244190b
Virtual Files: Allow to download a folder recursively from the socket API
Issue: #6466
2020-12-15 10:57:54 +01:00
Olivier Goffart
09cacc4cd4
Blacklist: remember the X-Request-ID
Issue #6420
Store the X-Request-ID in the SyncFileItem and also in the blacklist.
Note that for consistency reason, the X-Request-ID is also in the
SyncFileItem if the request succeeds.

Currently there is no UI to access it, but it can be queried with sql
commands
2020-12-15 10:57:53 +01:00
allexzander
29ca1404cc Fix Windows compilation broken after QML debugging fix.
Signed-off-by: allexzander <blackslayer4@gmail.com>
2020-11-24 16:50:16 +02:00
allexzander
92733e9069 Fix QML debugging by removing incorrect dependency
Signed-off-by: allexzander <blackslayer4@gmail.com>
2020-11-24 15:19:17 +02:00
Kevin Ottens
0b731eb516 Revert "moved default db sync file to Qt standard path AppDataLocation"
This reverts commit fc6b895f52.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-11-10 17:46:43 +00:00
Sergey Zolotarev
18c1bc0bd6 Misc fixes for Windows 7
Signed-off-by: Sergey Zolotarev <sryze@protonmail.com>
2020-10-28 06:24:53 +00:00
Christian Kamm
00e901f5a7
SocketAPI: Introduce conflict resolution actions #6252
For conflicts generally as well as new files in read-only directories
the context menu will now present delete and move options.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-10-22 16:40:10 +02:00
renyhp
e5b307c81f Update bookmarks location
Signed-off-by: renyhp <renyhp@disroot.org>
2020-10-12 15:02:00 +00:00
Kevin Ottens
836de0358d Add a function for a more friendly user agent string
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-10-08 05:55:47 +00:00
Jan Schmidt
3077a1378b syncjournaldb: index e2eMangledName column in metadata
Add an index on the e2eMangledName column in the metadata table
to speed up file sync by orders of magnitude on directories with
a large number of files.

Signed-off-by: Jan Schmidt <jan@centricular.com>
2020-09-03 22:56:14 +10:00
Kevin Ottens
80cc196f6f Enable bugprone-narrowing-conversions clang-tidy check
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-09-01 06:37:03 +00:00
Kevin Ottens
5cec1373ad Enable bugprone-macro-parentheses clang-tidy check
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-09-01 06:37:03 +00:00
Kevin Ottens
1a0945a973 Enable all the misc clang-tidy check except one
This flagged mostly unused parameters. Didn't enable the
misc-non-private-member-variables-in-classes check as we got a lot of
those. Hopefully we'll get to fix them at some point but that feels too
early and too much work for now.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-08-31 09:14:09 +02:00
Camila
71fbc65eb0 Minor fix in text of warning and debug message.
Signed-off-by: Camila <hello@camila.codes>
2020-08-18 14:00:21 +02:00
István Váradi
89d4b6e12a Include cstdint only when compiling C++ code
Signed-off-by: István Váradi <ivaradi@varadiistvan.hu>
2020-08-14 06:15:42 +02:00
Kevin Ottens
0e2af4b502 Enable the modernize-deprecated-headers check on clang-tidy
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-08-13 14:53:52 +02:00
Kevin Ottens
2f8c29082a Enable the modernize-use-equals-default check on clang-tidy
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-08-13 14:19:42 +02:00
Kevin Ottens
cdf6e7d72b Enable the modernize-user-nullptr check on clang-tidy
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-08-12 19:45:25 +02:00
Kevin Ottens
c50a968a1e
Add more raw string literals missed previously
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-08-12 09:48:29 +02:00
Kevin Ottens
7bb48a76c4 Add isE2eEncrypted column in the metadata table
This will allow to exploit the information of a directory being
encrypted or not during the discovery phase.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-06 05:27:14 +00:00
Kevin Ottens
9a2d04720a Also use mangled paths when looking for entries below a path
Otherwise on second sync we wouldn't find anything under a mangled path
and just end up thinking said files were removed.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-01 16:58:29 +00:00
Kevin Ottens
0f21b625ab Also check on mangled names when cleaning up the journal
Otherwise, after a first sync all the mangled entries would be removed
from the journal. On second sync it would be fine because we'd then have
seen the unmangled names because of the local files. Unfortunately
that'd mean reuploading them for nothing or trying to mkdir again on the
server for nothing... with a chance of using differently mangled names
(although I didn't spot it, I can't exclude it never happened).

This also led to weirdly getting stuck during sync when there was more
than one sync point.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-01 16:58:29 +00:00
Dominique Fuchs
f44df7f61a Major multi monitor improvements and rewrite of tray window positioning
Signed-off-by: Dominique Fuchs <32204802+DominiqueFuchs@users.noreply.github.com>
2020-06-15 12:32:25 +00:00
Michael Schuster
456c1eadbe
Simplify nullptr comparisons where appropriate
Make the codebase consistent, we already have a lot of implicit pointer comparisons.

Exception: Stay explicit on return's, example:
  return _db != nullptr;

Signed-off-by: Michael Schuster <michael@schuster.ms>
2020-06-11 00:27:30 +02:00
Rick van der Zwet
24177ec7e4 Change default autostart to hide gui
Signed-off-by: Rick van der Zwet <info@rickvanderzwet.nl>
2020-06-09 18:34:30 +02:00
Stephan Beyer
ea16804751 Replace NULL by nullptr in all C++ files
We keep NULL in the pure C files in src/csync/std and test/csync.

We also replace Doxygen documentation referring to "NULL" to
"\c nullptr" (formatted as code).

Signed-off-by: Stephan Beyer <s-beyer@gmx.net>
2020-06-08 20:07:50 +02:00
Nicolas Fella
bce93b052b Use APPLICATION_ICON_NAME for autostart icon
Signed-off-by: Nicolas Fella <nicolas.fella@gmx.de>
2020-06-08 20:06:00 +02:00
Kevin Ottens
e3e262e42e Use default member init when applicable
This also fixes a couple of warnings at places (out of order init for
instance) and a potential bug in the webflow credentials / qtkeychain
integration.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-06-03 16:10:39 +02:00
Camila San
3bae570f29 Do not declare local variables without an initial value.
Signed-off-by: Camila San <hello@camila.codes>
2020-06-03 07:50:40 +00:00
Kevin Ottens
a73a1f3927 Use = default for trivial ctors and dtors
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-05-25 21:33:24 +02:00
Stephan Beyer
04a24c14a8 Use default member initializers in OCC::SyncJournalErrorBlacklistRecord
This allows to remove its explicit default constructor definition.

Signed-off-by: Stephan Beyer <s-beyer@gmx.net>
2020-05-21 01:41:33 +02:00
Stephan Beyer
3015ea5715 Use default member initializers in OCC::SyncJournalDb sub-structs
This allows to remove their explicit default constructor definitions.

Signed-off-by: Stephan Beyer <s-beyer@gmx.net>
2020-05-21 01:32:16 +02:00
Kevin Ottens
5543e4d834 Don't use int literals for bools
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-05-20 13:52:58 +02:00
Kevin Ottens
712869db9a Use auto to avoiding repeating type names
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-05-20 03:54:41 +02:00
Kevin Ottens
dea6ebaa35 Use nullptr when appropriate
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-05-20 02:21:26 +02:00
Nicolas Fella
c1189421c0 Use qEnvironmentVariableIsEmpty
clazy suggests that it is more efficient since it doesn't allocate.

Signed-off-by: Nicolas Fella <nicolas.fella@gmx.de>
2020-03-21 01:06:15 +01:00
Dominique Fuchs
dac1c9b413 Avatar, full user name and server address now reflected in QML gui, provided by C++ backend
Signed-off-by: Dominique Fuchs <32204802+DominiqueFuchs@users.noreply.github.com>
2019-12-03 07:26:59 +01:00
Dominique Fuchs
478281d853 Restructures and refactorings: New systray class, handling of current user selection, integration of user information in tray window header 2019-12-02 22:45:14 +01:00
Michael Schuster
8e8858178b
Merge branch 'master' into move-rtfiles 2019-10-27 17:49:22 +01:00
Michael Schuster
9618500975
fix tab spacing
Signed-off-by: Michael Schuster <michael@schuster.ms>
2019-10-27 17:25:50 +01:00
Dominique Fuchs
3ad1c0e7a8 Created wrapper variable LINUX_APPLICATION_ID to clean up code
Signed-off-by: Dominique Fuchs <32204802+DominiqueFuchs@users.noreply.github.com>
2019-09-30 09:45:41 +02:00
Dominique Fuchs
83918619cf Corrected new .desktop file location in forgotten function
Signed-off-by: Dominique Fuchs <32204802+DominiqueFuchs@users.noreply.github.com>
2019-09-26 20:56:14 +02:00
Dominique Fuchs
fc6b895f52 moved default db sync file to Qt standard path AppDataLocation
Signed-off-by: Dominique Fuchs <32204802+DominiqueFuchs@users.noreply.github.com>
2019-09-26 17:09:24 +02:00
Dominique Fuchs
96da4674f9 Updated .desktop file naming scheme
Signed-off-by: Dominique Fuchs <32204802+DominiqueFuchs@users.noreply.github.com>
2019-09-26 12:36:05 +02:00
Dominique Fuchs
ab3d0141ec initialize _modtime to prevent undefined usage
Signed-off-by: Dominique Fuchs <32204802+DominiqueFuchs@users.noreply.github.com>
2019-09-09 19:23:30 +02:00
Dominique Fuchs
7473cdf184 Removed disabling of msvc warning to prevent generating a unknown option for other compilers in turn. Now detect specifically GCC in ifdef
Signed-off-by: Dominique Fuchs <32204802+DominiqueFuchs@users.noreply.github.com>
2019-09-09 13:46:46 +02:00
Dominique Fuchs
c585e81530 Moved macro definition due to timing issues while compiling when relying on header inheritance
Signed-off-by: Dominique Fuchs <32204802+DominiqueFuchs@users.noreply.github.com>
2019-09-08 19:46:08 +02:00
Dominique Fuchs
222b2d8645 Further testing of cond. include of Qt library > 5.9. Utilized different macro.
Signed-off-by: Dominique Fuchs <32204802+DominiqueFuchs@users.noreply.github.com>
2019-09-08 19:17:00 +02:00
Dominique Fuchs
f41eeaf6ec Fixed another logic error -> logical to bitwise OR for QTLEGACY
Signed-off-by: Dominique Fuchs <32204802+DominiqueFuchs@users.noreply.github.com>
2019-09-08 18:51:54 +02:00
Dominique Fuchs
48097801e8 Fixed logic error in QTLEGACY macro and added forgottin #if clause for header file
Signed-off-by: Dominique Fuchs <32204802+DominiqueFuchs@users.noreply.github.com>
2019-09-08 18:42:40 +02:00
Dominique Fuchs
dca83aad45 Added macro definition and compile-time condition to support Qt < 5.9
Signed-off-by: Dominique Fuchs <32204802+DominiqueFuchs@users.noreply.github.com>
2019-09-08 17:33:34 +02:00
Dominique Fuchs
be7a524557 Indeed, DWORD is a special snowflake - only when on _WIN32
Signed-off-by: Dominique Fuchs <32204802+DominiqueFuchs@users.noreply.github.com>
2019-09-08 16:22:08 +02:00
Dominique Fuchs
d60a216982 Use existing fct for RegKeyQuery instead of redundant subroutine
Signed-off-by: Dominique Fuchs <32204802+DominiqueFuchs@users.noreply.github.com>
2019-09-08 13:44:00 +02:00
Dominique Fuchs
d6af025a46 Numoerous safe conversions implemented. Added additional Utility::convertSizeToDWORD for windows builds.
Signed-off-by: Dominique Fuchs <32204802+DominiqueFuchs@users.noreply.github.com>
2019-09-08 11:33:20 +02:00
Dominique Fuchs
46e0a05078 Renamed conversion function to make intention more clear. Also defaulted to 'controlled truncation' to not stupidly crash. TBD/TODO: Better handling for such things.
Signed-off-by: Dominique Fuchs <32204802+DominiqueFuchs@users.noreply.github.com>
2019-09-08 02:20:24 +02:00