Commit graph

123 commits

Author SHA1 Message Date
Christian Kamm
f502a526fa
Generalize Result<> class, add Optional<>
To make it nicer to use outside of HTTP results.
2020-12-15 10:58:24 +01:00
Christian Kamm
e39d751b59
Fix big-folder detection when vfs is disabled 2020-12-15 10:58:22 +01:00
Christian Kamm
2b20985875
winvfs: initial work
Done by ckamm and dschmidt
2020-12-15 10:58:21 +01:00
Olivier Goffart
46510c2f39
Discovery phase: refactor some code in DiscoveryPhase::findAndCancelDeletedJob
Less code duplication
2020-12-15 10:58:09 +01:00
Olivier Goffart
1c2a3279bb
New Discovery Algorithm: more cleanups
- rename progress() to be more explicit
 - Make some more member of the discovery phase private
2020-12-15 10:58:09 +01:00
Olivier Goffart
2f8c77c54f
Fixup commit 835c9163374f42003aa2f7795ade3f4ff62c8877
The previous code considered the also HTTP 207 code without the
application/xml header to have this message.
httpCode 0 does not make much sense anyway.

This change the behavior to consider any 2xx without the xml header
to show this error message
2020-12-15 10:58:08 +01:00
Olivier Goffart
c009dae1ce
New discovery algorithm: fixups
Adapt reviews from ckamm in https://github.com/owncloud/client/pull/6738#pullrequestreview-164623532

- SyncJournalFileRecord: initialize everything inline
- Add more comments
- And some ENFORCE
2020-12-15 10:58:08 +01:00
Olivier Goffart
afed46afcc
New discovery algorithm: Parallel PROPFIND 2020-12-15 10:58:08 +01:00
Olivier Goffart
52dcfcb166
New Propagation algorithm: Fetch, and emit, the root etag
Remove the feature to concatenate etags as servers that don't
have a root etag are no longer suported
2020-12-15 10:58:07 +01:00
Olivier Goffart
d25d87e92c
New Discovery Algorithm: Ge tthe size of new folders
Also add a test that this works properly
2020-12-15 10:58:07 +01:00
Olivier Goffart
68ace415d4
Virtual files: don't check for new big folders
https://github.com/owncloud/client/issues/6724#issuecomment-417368475
2020-12-15 10:58:06 +01:00
Christian Kamm
be0fa72fcb
RemotePermissions: Fix empty vs null
Fixes two bugs that appeared since the introduction of the struct:
- when reading permissions from the journal, null ("") was read as
  empty-not-null
- when reading permissinos from the server, empty ("") was read as null

Addresses #4608
2020-12-15 10:58:03 +01:00
Christian Kamm
f6ca649308
Do not require server replies to contain an mtime
The check was added for #6317 in commit
13eb64584f.

We did see missing mtimes in replies in tests with live servers though.
Possibly those were old incomplete responses cached in the stat cache?
2020-12-15 10:58:02 +01:00
Olivier Goffart
9401273daf
New discovery algorithm: Call FolderDiscovered 2020-12-15 10:58:02 +01:00
Olivier Goffart
a36ed56f01
New Discovery algorithm: Refactor a bit the way the signal are emited 2020-12-15 10:58:02 +01:00
Olivier Goffart
b6487181c6
New Discovery Phase: fix a few FIXME
Some FIXME were already fixed.
2020-12-15 10:58:02 +01:00
Olivier Goffart
1bcbcd407c
New discovery phase: read the direct download URL and Cookie 2020-12-15 10:58:01 +01:00
Olivier Goffart
ca32eb1cf7
More cleanup of csync remains 2020-12-15 10:58:01 +01:00
Olivier Goffart
0a6d2d0f46
Remove csync_update and csync_reconcile
I guess some other csync utilities can also be remove dnow but
that will be for later
2020-12-15 10:58:00 +01:00
Olivier Goffart
57068b0fd9
New Discovery Algorithm: handle server errors 2020-12-15 10:58:00 +01:00
Olivier Goffart
a384a2d1cb
New Discovery algorithm: Handle of move within a moved directory 2020-12-15 10:57:58 +01:00
Olivier Goffart
e934f6b27b
New discovery algo
Make TestSyncEngine::testSelectiveSyncBug pass
2020-12-15 10:57:57 +01:00
Olivier Goffart
cf3846c565
csync: refactor csync_s::error_string to avoid valgrind error
The problem here is that we were sometimes allocating the error_string with
qstrdup, which need to be released with delete[] and not free().

Simplify the code by using QString instead.

```
==7230== Mismatched free() / delete / delete []
==7230==    at 0x4C2E10B: free (vg_replace_malloc.c:530)
==7230==    by 0x57C2321: csync_s::reinitialize() (csync.cpp:247)
==7230==    by 0x548130F: OCC::SyncEngine::finalize(bool) (syncengine.cpp:1212)
==7230==    by 0x5481223: OCC::SyncEngine::handleSyncError(csync_s*, char const*) (syncengine.cpp:746)
==7230==    by 0x5483E78: OCC::SyncEngine::slotDiscoveryJobFinished(int) (syncengine.cpp:965)
==7230==    by 0x5495E34: QtPrivate::FunctorCall<QtPrivate::IndexesList<0>, QtPrivate::List<int>, void, void (OCC::SyncEngine::*)(int)>::call(void (OCC::SyncEngine::*)(int), OCC::SyncEngine*, void**) (qobjectdefs_impl.h:134)
==7230==    by 0x5495D92: void QtPrivate::FunctionPointer<void (OCC::SyncEngine::*)(int)>::call<QtPrivate::List<int>, void>(void (OCC::SyncEngine::*)(int), OCC::SyncEngine*, void**) (qobjectdefs_impl.h:167)
==7230==    by 0x5495CB5: QtPrivate::QSlotObject<void (OCC::SyncEngine::*)(int), QtPrivate::List<int>, void>::impl(int, QtPrivate::QSlotObjectBase*, QObject*, void**, bool*) (qobjectdefs_impl.h:396)
==7230==    by 0xA9BF2E1: QObject::event(QEvent*) (in /usr/lib/libQt5Core.so.5.11.0)
==7230==    by 0x64BE983: QApplicationPrivate::notify_helper(QObject*, QEvent*) (in /usr/lib/libQt5Widgets.so.5.11.0)
==7230==    by 0x64C625A: QApplication::notify(QObject*, QEvent*) (in /usr/lib/libQt5Widgets.so.5.11.0)
==7230==    by 0xA994BC8: QCoreApplication::notifyInternal2(QObject*, QEvent*) (in /usr/lib/libQt5Core.so.5.11.0)
==7230==  Address 0x225b2640 is 0 bytes inside a block of size 50 alloc'd
==7230==    at 0x4C2DC6F: operator new[](unsigned long) (vg_replace_malloc.c:423)
==7230==    by 0xA7E8FC8: qstrdup(char const*) (in /usr/lib/libQt5Core.so.5.11.0)
==7230==    by 0x53F5750: OCC::DiscoveryJob::remote_vio_opendir_hook(char const*, void*) (discoveryphase.cpp:666)
==7230==    by 0x57E1278: csync_vio_opendir(csync_s*, char const*) (csync_vio.cpp:39)
==7230==    by 0x57D718F: csync_ftw(csync_s*, char const*, int (*)(csync_s*, std::unique_ptr<csync_file_stat_s, std::default_delete<csync_file_stat_s> >), unsigned int) (csync_update.cpp:674)
==7230==    by 0x57C1B05: csync_update(csync_s*) (csync.cpp:109)
==7230==    by 0x53F5BCC: OCC::DiscoveryJob::start() (discoveryphase.cpp:718)
==7230==    by 0x54B8F74: OCC::DiscoveryJob::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) (moc_discoveryphase.cpp:494)
==7230==    by 0xA9BF2E1: QObject::event(QEvent*) (in /usr/lib/libQt5Core.so.5.11.0)
==7230==    by 0x64BE983: QApplicationPrivate::notify_helper(QObject*, QEvent*) (in /usr/lib/libQt5Widgets.so.5.11.0)
==7230==    by 0x64C625A: QApplication::notify(QObject*, QEvent*) (in /usr/lib/libQt5Widgets.so.5.11.0)
==7230==    by 0xA994BC8: QCoreApplication::notifyInternal2(QObject*, QEvent*) (in /usr/lib/libQt5Core.so.5.11.0)
==7230==
```
2020-12-15 10:57:55 +01:00
Kevin Ottens
c8ea848596 Enable the bugprone-branch-clone clang-tidy check
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-09-01 06:37:03 +00:00
Kevin Ottens
712869db9a Use auto to avoiding repeating type names
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-05-20 03:54:41 +02:00
Brandon
d10bc1bb14 Correct wrong variable
Signed-off-by: Brandon <brandon.yeow@websparks.sg>
2020-01-15 23:07:14 +01:00
Brandon
18a88fcecf Correct wrong variable
Signed-off-by: Brandon <me@branbit.com>
Signed-off-by: Brandon <brandon.yeow@websparks.sg>
2020-01-15 23:07:14 +01:00
ritsute
c92f520423 Handle broken shared file error gracefully
Signed-off-by: Brandon <me@branbit.com>
Signed-off-by: Brandon <brandon.yeow@websparks.sg>
2020-01-15 23:07:14 +01:00
J-P Nurmi
1af9bf8abc libsync: run clang-tidy modernize-use-nullptr 2018-11-12 18:46:39 +01:00
Roeland Jago Douma
5cfdc71ca5
Merge branch 'master' into upstream/pr/6671 2018-10-31 21:13:10 +01:00
Olivier Goffart
08e4ba3363
Data-Fingerprint: Fix backup detection when fingerprint is empty
Add a test to test the data fingerprint feature make me realize it was broken.
The code was relying in the distinction between empty and null QByteArray,
but this was a bad idea as this difference is lost when going through QString.
2018-09-10 20:44:03 +02:00
Christian Kamm
5b228d1afd
Do not require server replies to contain an mtime
The check was added for #6317 in commit
13eb64584f.

We did see missing mtimes in replies in tests with live servers though.
Possibly those were old incomplete responses cached in the stat cache?
2018-09-10 17:33:53 +02:00
Olivier Goffart
5f30177bfc
CSync: Remove unused error codes
None of the removed ERRNO code were ever set.
So none of the removed error hapenned anymore
2018-06-04 15:41:38 +02:00
Roeland Jago Douma
f96206705a
Merge pull request #315 from nextcloud/upstream/pr/6379
Discovery: Error if properties are missing #6317
2018-06-03 19:58:21 +02:00
Christian Kamm
8077748c9d
slotFolderDiscovered: Fix 'local' always being false
It was unused, so no harm done.
2018-05-16 16:44:36 +02:00
Christian Kamm
687b6f5655
Discovery: Error if properties are missing #6317 2018-05-16 16:42:20 +02:00
Tomaz Canabrava
d24a1e542e Merge branch 'master' into clientSideEncryptionV3 2018-01-29 14:06:12 +01:00
Olivier Goffart
3ddd4b6f16 Get rid of csync_log
We use Qt's debugging code everywhere
2018-01-13 13:58:17 +01:00
Olivier Goffart
257d8142b1 Build system: Get rid of QtVersionAbstraction.cmake
Use modern cmake with target_link_libraries and Qt5:: that
automatically add the include path and compile flags
2018-01-13 13:58:17 +01:00
Daniel Nicoletti
a63d34f870 Prepend "nextcloud" for all logging categories
Thus making easier to exclude logging from kio, qt
and only enable "nextcloud.*"
2017-12-28 17:33:10 -02:00
Christian Kamm
8eebc53728 Unify item type enum
Previously, there was csync_ftw_type_e and SyncFileItem::Type. Having
two enums lead to a bug where Type::Unknown == Type::File that went
unnoticed for a good while.

This patch keeps only a single enum.
2017-12-15 18:02:04 +01:00
Christian Kamm
7ab127ad53 Excludes: Refactor for pending improvements
Make ExcludedFiles something that is instantiated outside of
the CSYNC context and then given to it as a hook.

ExcludedFiles still lives in csync_exclude and the internal
workings haven't been touched.
2017-12-07 14:38:21 +01:00
Christian Kamm
ceac18c554 Reconcile: Rename maps are consistent with update phase #6212
For duplicate file ids the update phase and reconcile phase determined
the rename mappings independently. If they disagreed (due to different
order of processing), complicated misbehavior would result.

This patch fixes it by letting reconcile try to use the mapping that the
update phase has computed first.
2017-12-06 16:42:11 +01:00
Olivier Goffart
480932a58a Checksum: Ignore unkown OC-Checksum header when downloading...
And if there are several checksums, pick the "best" one.

The case of several checksum was reported in
https://github.com/nextcloud/client_theming/issues/213
2017-11-20 10:51:15 +01:00
Markus Goetz
3a1a56c7ad Excludes: During directory traversal, use QRegularExpression
On Mac, this halves the time spent in csync_excluded_traversal
when using check_csync_excluded_performance. A similar performance
increase is seen on linux.
2017-10-05 10:19:48 +02:00
Olivier Goffart
95d23b1914 RemotePermissions: Store in a class rather than in a QByteArray to save memory
Create a specific type that parses the permissions so we can store
it in a short rather than in a QByteArray

Note: in RemotePermissions::toString, we make sure the string is not
empty by adding a space, this was already existing before commit
e8f7adc7ca where it was removed by mistake.
2017-09-22 09:29:08 +02:00
Olivier Goffart
02924ad83e DiscoveryPhase: use the new syntax in disconnect
Since we used the new syntax in connect, we need to use it in disconnect
2017-09-21 14:05:39 +02:00
Olivier Goffart
ff4213b59f Use the Qt5 connection syntax (automated with clazy)
This is motivated by the fact that QMetaObject::noralizeSignature takes 7.35%
CPU of the LargeSyncBench. (Mostly from ABstractNetworkJob::setupConnections and
PropagateUploadFileV1::startNextChunk). It could be fixed by using normalized
signature in the connection statement, but i tought it was a good oportunity
to modernize the code.

This commit only contains calls that were automatically converted with clazy.
2017-09-21 14:05:39 +02:00
Jocelyn Turcotte
a1f1775d15 Move SyncJournalDB to src/common 2017-09-18 14:00:52 +02:00
Jocelyn Turcotte
d66c2b5fae Replace the custom rbtree by a std::map 2017-09-18 11:46:09 +02:00