Commit graph

131 commits

Author SHA1 Message Date
Olivier Goffart
a52dff713b csync_rename: optimize lookup
Avoid many memory allocations in reconcile
2017-09-27 11:41:55 +02:00
Olivier Goffart
041c5e6d44 csync_reconcile: Optimize reconcile by avoiding alocations
Fix performence regression from commit d66c2b5fae
For every new file we would look up every parent directories. Allocating
a new QByteArreay for every parent riectory just to know if it is in the other
tree is wasting lots of CPU.
Use a ByteArrayRef trick, similar to QStringRef
2017-09-27 11:41:55 +02:00
Jocelyn Turcotte
54c522c861 SyncJournalDb: Reimplement the db_is_empty logic from csync
This reduces the initial sync local discovery time from
2.0 to 0.6 seconds in LargeSyncBench on my machine.
2017-09-26 11:03:23 +02:00
Jocelyn Turcotte
a034ee894c Use SyncJournalDb in csync
This gets rid of the csync_statedb sqlite layer and use
the same code and same connection as the rest of the SyncEngine.

Missing functions are added to SyncJournalDb and change a few minor
things (like changing SyncJournalFileRecord::_modtime to be an int64
instead of a QDateTime, like it was in csync).
2017-09-26 11:03:23 +02:00
Olivier Goffart
bf2b089c5c c_utf8_from_locale: optimize the UTF8 case on Linux
Avoid converting to a QString as an intermediate step
2017-09-25 10:35:28 +02:00
Olivier Goffart
3107e63040 csync: use a unordered_map for the FileMap
Saves more than 10% of csync_update on LargeSyncBench
2017-09-25 10:35:28 +02:00
Olivier Goffart
d8206ccbc7 csync_update: fix assert
Amend 1dc44c0f62
2017-09-25 00:17:47 +02:00
Olivier Goffart
35f80bd439 csync_vio_local: merge readdir and stat in the same call
For the remote this was already done long time ago
For windows, this was already partially done

The goal is to avoid needless conversion of the path to local encoding.
2017-09-22 16:18:18 +02:00
Olivier Goffart
1dc44c0f62 csync_update/reconcile: replace CSYNC_LOG with qDebug/qInfo
CSYNC_LOG has quite an overhead as it uses vsprintf and it might
be run even if the category is disabled
2017-09-22 11:12:19 +02:00
Olivier Goffart
aa322d41aa csync: remove unused type 2017-09-22 09:58:55 +02:00
Olivier Goffart
95d23b1914 RemotePermissions: Store in a class rather than in a QByteArray to save memory
Create a specific type that parses the permissions so we can store
it in a short rather than in a QByteArray

Note: in RemotePermissions::toString, we make sure the string is not
empty by adding a space, this was already existing before commit
e8f7adc7ca where it was removed by mistake.
2017-09-22 09:29:08 +02:00
Christian Kamm
53195b064a Conflicts: Upload them files if env variable says so
Set OWNCLOUD_UPLOAD_CONFLICT_FILES=1 to trigger this behavior.

Note that this is experimental and unsupported. The real feature is
likely to end up in 2.5.

Uploading conflict files is simply done by removing the pattern from
csync_exclude. The rest here deals with making the conflict notification
ui approximately work.

There are still some concerns about where an uploaded conflict file
appears in the sync protocol and issues list (it should be in both, but
is only in one of them currently!).

See #4557.
2017-09-19 11:45:38 +02:00
Jocelyn Turcotte
6e240a3e9c csync_update: Remove the check for csync_get_statedb_exists
This always returns true since _csync_statedb_is_empty also always
returns true. This function was initially intended to be a shortcut
in case the database doesn't contain any row, but has been broken since
27fb5d9128 and now won't return false
unless the statement fails.

The _last_db_return_error in _csync_detect_update would take care
of that code path in a more direct and clearer way anyway.
2017-09-18 14:00:52 +02:00
Jocelyn Turcotte
a1f1775d15 Move SyncJournalDB to src/common 2017-09-18 14:00:52 +02:00
Jocelyn Turcotte
bd25225045 Move non-csync-dependent parts of FileSystem to src/common
This keep the csync-dependent parts in src/libsync, slitting the namespace
over two files.
This will allow moving SyncJournalDB to src/common as well.
2017-09-18 14:00:52 +02:00
Jocelyn Turcotte
bfacac8683 Remove the usage of phash in csync
Only store the path since they represent the same thing, and do the
phash conversion during DB lookup like done in libsync.
We could get rid of everything since we also have an index on the path
column, but since it's the primary key this makes the migration non-trivial.
2017-09-18 11:46:09 +02:00
Jocelyn Turcotte
d66c2b5fae Replace the custom rbtree by a std::map 2017-09-18 11:46:09 +02:00
Jocelyn Turcotte
72e44ce3d7 csync: Keep the csync_file_stat_t instances created during discovery
Now that they use the same structure, avoid _csync_detect_update
having to recreate another instance and transfer everything manually.
Any instance created during discovery should now be used all the way
up to SyncEngine::treewalkFile.

This also makes sure that the path and types are properly set in that
object instead of having to pass everything as separate parameters.

This gets rid of csync_ftw_flags_e which was now converted from,
and to csync_ftw_type_e, already in the csync_file_stat_t.

Issue #1817
2017-09-18 11:46:09 +02:00
Christian Kamm
0b7ad2c804 Excludes: Allow escaping # #6012
Otherwise adding patterns that start with # are impossible to add, since
they get treated as comments. Also add this escaping for patterns added
in the ui.
2017-09-14 14:53:05 +02:00
Jocelyn Turcotte
a18a601291 csync_statedb: (really) Make sure children of non-synced items aren't read from the DB
No tests are covering this and I reversed the original logic.
2017-09-07 10:26:19 +02:00
Jocelyn Turcotte
92fc50d4b0 csync_statedb: Make sure children of non-synced items aren't read from the DB
Fix a regression during C++ conversion from 704ca366b3
2017-09-07 10:17:58 +02:00
Jocelyn Turcotte
715b8c4683 Fix build after C++ conversion of csync
Use the same macro as in SyncFileItem for bitfields on enums.

Fixes #5993
2017-09-06 13:08:23 +02:00
Jocelyn Turcotte
b1bb7ec1a8 Use a C++ contructor and destructor for CSYNC
Merge csync_create and csync_init into the constructor and
replace csync_destroy with the destructor.

Also use a QByteArray for csync_s::root_perms and flatten
csync_rename_s as a rename sub-struct of csync_s since it
can now handle C++ types.
2017-09-06 13:08:23 +02:00
Jocelyn Turcotte
f6136dd036 Remove csync_s::replica
The only difference with csync_s::current is that it's
assigned the value of csync_s::local::type and
csync_s::remote::type, which never change. So might as
well only use the "current" field with constants.
2017-09-06 13:08:23 +02:00
Jocelyn Turcotte
cb49635231 Replace TREE_WALK_FILE with csync_file_stat_t
Just expose csync_file_stat_t since we don't need an abstraction layer
anymore. Also pass the nodes of both trees directly to the visitor
function.

Issue #1817
2017-09-06 13:08:23 +02:00
Jocelyn Turcotte
e8f7adc7ca Replace csync_vio_file_stat_t with csync_file_stat_t
Also move csync_normalize_etag to common/utility since we
don't need the char* function anymore.

Remove the single space file_stat->remotePerm codepath since
this won't be used in csync anymore since
8de3bda0b1.

Issue #1817
2017-09-06 13:08:23 +02:00
Jocelyn Turcotte
704ca366b3 Make csync_file_stat_t public and partly convert to C++
This is the first commit trying to unify csync_file_stat_s,
csync_vio_file_stat_s and csync_tree_walk_file_s. Use QByteArray
and unique_ptr already since I'm not used to track memory allocations
and this will make the transition easier.

Issue #1817
2017-09-05 17:25:19 +02:00
Jocelyn Turcotte
cf15cbf0b3 Move Utility to a new common static library
Now that csync builds as C++, this will avoid having to implement
functionalities needed by csync mandatorily in csync itself.

This library is built as part of libocsync and symbols exported
through it.
This requires a relicense of Utility as LGPL. All classes moved into
this library from src/libsync will need to be relicensed as well.
2017-09-05 17:25:19 +02:00
Jocelyn Turcotte
4e5898efd7 Compile almost all of csync as C++
This will allow us to unify data structures between csync and libsync.

Utility functions like csync_time and c_std are still compiled as C
since we won't need to be coupled with Qt in the short term.
2017-08-22 15:22:56 +02:00
Jocelyn Turcotte
a512586140 Use .cpp for existing csync .cc files
Make this consistent with the rest of the client code.
2017-08-22 11:18:22 +02:00
Jocelyn Turcotte
28a0971cdf Move csync to src/csync 2017-08-22 11:18:22 +02:00