Commit graph

11091 commits

Author SHA1 Message Date
Jocelyn Turcotte
185cb2e39a
Use SyncJournalDb in csync
This gets rid of the csync_statedb sqlite layer and use
the same code and same connection as the rest of the SyncEngine.

Missing functions are added to SyncJournalDb and change a few minor
things (like changing SyncJournalFileRecord::_modtime to be an int64
instead of a QDateTime, like it was in csync).
2017-10-05 22:01:37 +02:00
Jocelyn Turcotte
c6610f6fbf
SyncJournalDB: Allow callers of getFileRecord if the query failed
The current implementation would return the same value whether the query failed
or if no row would be found. This is something that is currently checked by csync
and needs to be provided if we want to use SyncJournalDB there.

Adjusted all call sites to also check the return value even though they
could still just rely on rec.isValid(), but makes it more explicit as to what
happens for database errors in those cases, if we ever want to gracefully handle
them.
2017-10-05 22:01:37 +02:00
Jocelyn Turcotte
d76ecf015c
SyncJournalDB: Bind phash as a qlonglong instead of a QString
This is what csync does and we don't need to pass it as a string
to let sqlite convert it back to a uint64 later.
2017-10-05 22:01:36 +02:00
Jenkins for ownCloud
312f3a9bf3
[tx-robot] updated from transifex 2017-10-05 22:01:36 +02:00
Olivier Goffart
fd9662b803
bandwidthmanager: Fix unregistering devices on delete
from the destroyed signal, qobject_cast won't work because the object
is already destroyed. One must use reinterpret_cast then
2017-10-05 22:01:36 +02:00
Olivier Goffart
37810c0a19
PropagateUpload: Fix another non-normalized connect that appears while profiling 2017-10-05 22:01:36 +02:00
Olivier Goffart
38a76ac5bf
benchlargesync: Do several syncs
Make sure there is already a small sync before so the database is queried.
Make also a sync after to run an update with many files in the database.
2017-10-05 22:01:36 +02:00
Olivier Goffart
d100f9ac67
c_utf8_from_locale: optimize the UTF8 case on Linux
Avoid converting to a QString as an intermediate step
2017-10-05 22:01:36 +02:00
Olivier Goffart
5fc864adb6
csync: use a unordered_map for the FileMap
Saves more than 10% of csync_update on LargeSyncBench
2017-10-05 22:01:36 +02:00
Jenkins for ownCloud
2738d5d9ab
[tx-robot] updated from transifex 2017-10-05 22:01:36 +02:00
Olivier Goffart
0936cfee5d
chsck_vio: fix readdir test by removing it.
The code that was creating the files in the directory was removed in
commit 6906b8d30c. The directory is empty
so the result is expected to be null. It was passing before because the
code was returning an entry for . and ..,  but since commit
35f80bd439 this is no longer the case
2017-10-05 22:01:36 +02:00
Olivier Goffart
38962ca694
csync_update: fix assert
Amend 1dc44c0f62
2017-10-05 22:01:36 +02:00
Jenkins for ownCloud
7e677516ae
[tx-robot] updated from transifex 2017-10-05 22:01:36 +02:00
Markus Goetz
cabf7a79d5
macOS: Fix compile 2017-10-05 22:01:36 +02:00
Olivier Goffart
2b12990f19
Httpcreds: Fix double slash in the Request App Password url
Issue #6044
2017-10-05 22:01:36 +02:00
Olivier Goffart
3184d4bb32
OAuth2: Fix double slash in URL
We need to use concatPath to avoid possible double '/' in the URLs if the
account url() ends with '/'.

This has become even more of a problem since commit
d1b8370a4a which was resolving the url after
a redirect where most server actually add a '/' if the url is a folder
2017-10-05 22:01:36 +02:00
Jenkins for ownCloud
0cbbea9796
[tx-robot] updated from transifex 2017-10-05 22:01:35 +02:00
Olivier Goffart
b19677ea97
csync_vio_local: merge readdir and stat in the same call
For the remote this was already done long time ago
For windows, this was already partially done

The goal is to avoid needless conversion of the path to local encoding.
2017-10-05 22:01:35 +02:00
Olivier Goffart
dc0a0841fa
csync_update/reconcile: replace CSYNC_LOG with qDebug/qInfo
CSYNC_LOG has quite an overhead as it uses vsprintf and it might
be run even if the category is disabled
2017-10-05 22:01:35 +02:00
Christian Kamm
351e65309c
SyncEngine: Adjust free space messaging
Before, the way the info message was phrased implied a failure. And
since the large numbers can be hard to compare that can be confusing.
2017-10-05 22:01:35 +02:00
Olivier Goffart
e36c8678d9
csync: remove unused type 2017-10-05 22:01:35 +02:00
Olivier Goffart
f793de4629
RemotePermissions: Store in a class rather than in a QByteArray to save memory
Create a specific type that parses the permissions so we can store
it in a short rather than in a QByteArray

Note: in RemotePermissions::toString, we make sure the string is not
empty by adding a space, this was already existing before commit
e8f7adc7ca where it was removed by mistake.
2017-10-05 22:01:35 +02:00
Jenkins for ownCloud
cd4908ac6a
[tx-robot] updated from transifex 2017-10-05 22:01:35 +02:00
Olivier Goffart
0ad05aee0d
Port to new signal-slot syntax what cannot be done automatically
Some slot were protected or private but needed to be public.
Some needed a static_cast (can't use qOverload because it is in Qt 5.7)

This is not only a partial change.
2017-10-05 22:01:35 +02:00
Olivier Goffart
190f821d93
DiscoveryPhase: use the new syntax in disconnect
Since we used the new syntax in connect, we need to use it in disconnect
2017-10-05 22:01:35 +02:00
Olivier Goffart
7aca2352be
Use the Qt5 connection syntax (automated with clazy)
This is motivated by the fact that QMetaObject::noralizeSignature takes 7.35%
CPU of the LargeSyncBench. (Mostly from ABstractNetworkJob::setupConnections and
PropagateUploadFileV1::startNextChunk). It could be fixed by using normalized
signature in the connection statement, but i tought it was a good oportunity
to modernize the code.

This commit only contains calls that were automatically converted with clazy.
2017-10-05 22:01:35 +02:00
Olivier Goffart
3143b32aa5
test/stub: add some call required by the next patches
As we use the new connection syntax in folderman.cpp, some more symbol
need to be mocked
2017-10-05 22:01:35 +02:00
Olivier Goffart
e8a76859c1
Include <QAuthenticator> from code that connect to signal that uses it
The new connection syntax will need that
2017-10-05 22:01:35 +02:00
Olivier Goffart
c219ceea52
Gui: Use lambda instead of QSignalMapper
It's simpler, and QSignalMapper is deprecated in Qt 5.10
2017-10-05 22:01:34 +02:00
Olivier Goffart
608f5034ef
Remove settingsdialog.cpp.Rbt431 2017-10-05 22:01:34 +02:00
Jenkins for ownCloud
78458662a5
[tx-robot] updated from transifex 2017-10-05 22:01:34 +02:00
Markus Goetz
800dd485ef
ChangeLog: More 2.4.0 changes 2017-10-05 22:01:34 +02:00
Jenkins for ownCloud
f4ca43293b
[tx-robot] updated from transifex 2017-10-05 22:01:34 +02:00
Christian Kamm
6161b5519b
Fix compile with bad strrchr signature 2017-10-05 22:01:34 +02:00
Christian Kamm
2cac928810
Conflicts: Upload them files if env variable says so
Set OWNCLOUD_UPLOAD_CONFLICT_FILES=1 to trigger this behavior.

Note that this is experimental and unsupported. The real feature is
likely to end up in 2.5.

Uploading conflict files is simply done by removing the pattern from
csync_exclude. The rest here deals with making the conflict notification
ui approximately work.

There are still some concerns about where an uploaded conflict file
appears in the sync protocol and issues list (it should be in both, but
is only in one of them currently!).

See #4557.
2017-10-05 22:01:34 +02:00
Christian Kamm
05927b68a8
Private links: Retrieve link through propfind property #6020
* The sharing ui does a propfind anyway: use that to query the new
property as well!
* For the socket api, asynchronously query the server for the right url
when an action that needs it is triggered.

The old, manually generated URL will be used as fallback in case the
server doesn't support the new property or the property can't be
retrieved for some reason.

Depends on owncloud/core#29021
2017-10-05 22:01:34 +02:00
Jenkins for ownCloud
b3f90111e2
[tx-robot] updated from transifex 2017-10-05 22:01:34 +02:00
Jocelyn Turcotte
254f6291df
Fix the build (make install) 2017-10-05 22:01:34 +02:00
Olivier Goffart
faa7fd1b32
Gui: Allow to add multiple sync folder connection of the same folder
... or child folders

There is also no real reason to forbid the user from syncing the same
folder to multiple location on its hardrive.
A real use case is when the user uncheck a big directory using "choose
what to sync", but would still like to sync a folder within this disabled
tree. The user can now do this with the "add folder" feature

Since 2.3, we even support syncing the same local folder to multiple
remote folder, so why not allow syncing the same remote folder several
times?

Relates to issue #3645
2017-10-05 22:01:34 +02:00
Jocelyn Turcotte
76d95f8b87
SyncJournalDB: Only reset_and_clear_bindings once
We only need to do it once between each query execution and
that can be right before the first bind.
2017-10-05 22:01:34 +02:00
Jocelyn Turcotte
4e90bd7a5a
csync_update: Remove the check for csync_get_statedb_exists
This always returns true since _csync_statedb_is_empty also always
returns true. This function was initially intended to be a shortcut
in case the database doesn't contain any row, but has been broken since
27fb5d9128 and now won't return false
unless the statement fails.

The _last_db_return_error in _csync_detect_update would take care
of that code path in a more direct and clearer way anyway.
2017-10-05 22:01:33 +02:00
Jocelyn Turcotte
81e32e1a08
Move SyncJournalDB to src/common 2017-10-05 22:01:33 +02:00
Jocelyn Turcotte
98ef3050d8
Reverse the dependency between SyncJournalFileRecord and SyncFileItem
This will allow us to also use the SyncJournalDB in csync.
2017-10-05 22:01:33 +02:00
Jocelyn Turcotte
4c2e078eac
Move non-csync-dependent parts of FileSystem to src/common
This keep the csync-dependent parts in src/libsync, slitting the namespace
over two files.
This will allow moving SyncJournalDB to src/common as well.
2017-10-05 22:01:33 +02:00
Jocelyn Turcotte
dc3c3195b5
Update sync-algorithm.md
The rbtree was replaced by a std::map.
2017-10-05 22:01:33 +02:00
Christian Kamm
cda72effe2
Checksums: Clearer behavior and added testing 2017-10-05 22:01:33 +02:00
Jocelyn Turcotte
0f21ed6a5b
Remove SyncFileItem::_isDirectory
It's always equivalent to _type == SyncFileItem::Directory.
2017-10-05 22:01:33 +02:00
Jocelyn Turcotte
85a93efe51
Remove SyncFileItem::log
This remove the remaining "other" fields of the sync log to save a
bit of memory.

other_etag and other_fileId don't give much information to the users
and other_instruction will always be INST_NONE anyway.

other_modtime and other_size are kept since they are sometimes used.
They were renamed to have a bit more meaningful name.

SyncEngine::checkPermissions will now fetch its information from the
csync trees since they are now preserved until right after this point.

Fixes #3213
2017-10-05 22:01:33 +02:00
Jocelyn Turcotte
82dd1775eb
Remove SyncEngine::_remotePerms
Now that csync is using a more convenient data structure for
its file trees, wait a little bit longer before destroying them and
fetch the remote permissions from the remote tree there instead.
2017-10-05 22:01:33 +02:00
Jocelyn Turcotte
3bc1f63b0a
Remove the usage of phash in csync
Only store the path since they represent the same thing, and do the
phash conversion during DB lookup like done in libsync.
We could get rid of everything since we also have an index on the path
column, but since it's the primary key this makes the migration non-trivial.
2017-10-05 22:01:33 +02:00