Commit graph

288 commits

Author SHA1 Message Date
Olivier Goffart
95d23b1914 RemotePermissions: Store in a class rather than in a QByteArray to save memory
Create a specific type that parses the permissions so we can store
it in a short rather than in a QByteArray

Note: in RemotePermissions::toString, we make sure the string is not
empty by adding a space, this was already existing before commit
e8f7adc7ca where it was removed by mistake.
2017-09-22 09:29:08 +02:00
Olivier Goffart
c4e51247d8 test/stub: add some call required by the next patches
As we use the new connection syntax in folderman.cpp, some more symbol
need to be mocked
2017-09-21 14:05:39 +02:00
Jocelyn Turcotte
a1f1775d15 Move SyncJournalDB to src/common 2017-09-18 14:00:52 +02:00
Jocelyn Turcotte
bd25225045 Move non-csync-dependent parts of FileSystem to src/common
This keep the csync-dependent parts in src/libsync, slitting the namespace
over two files.
This will allow moving SyncJournalDB to src/common as well.
2017-09-18 14:00:52 +02:00
Christian Kamm
78212e03d6 Checksums: Clearer behavior and added testing 2017-09-18 12:52:09 +02:00
Jocelyn Turcotte
a8ea7b0858 Remove SyncFileItem::_isDirectory
It's always equivalent to _type == SyncFileItem::Directory.
2017-09-18 11:46:09 +02:00
Jocelyn Turcotte
0e97770c47 Remove SyncFileItem::log
This remove the remaining "other" fields of the sync log to save a
bit of memory.

other_etag and other_fileId don't give much information to the users
and other_instruction will always be INST_NONE anyway.

other_modtime and other_size are kept since they are sometimes used.
They were renamed to have a bit more meaningful name.

SyncEngine::checkPermissions will now fetch its information from the
csync trees since they are now preserved until right after this point.

Fixes #3213
2017-09-18 11:46:09 +02:00
Jocelyn Turcotte
bfacac8683 Remove the usage of phash in csync
Only store the path since they represent the same thing, and do the
phash conversion during DB lookup like done in libsync.
We could get rid of everything since we also have an index on the path
column, but since it's the primary key this makes the migration non-trivial.
2017-09-18 11:46:09 +02:00
Jocelyn Turcotte
d66c2b5fae Replace the custom rbtree by a std::map 2017-09-18 11:46:09 +02:00
Jocelyn Turcotte
72e44ce3d7 csync: Keep the csync_file_stat_t instances created during discovery
Now that they use the same structure, avoid _csync_detect_update
having to recreate another instance and transfer everything manually.
Any instance created during discovery should now be used all the way
up to SyncEngine::treewalkFile.

This also makes sure that the path and types are properly set in that
object instead of having to pass everything as separate parameters.

This gets rid of csync_ftw_flags_e which was now converted from,
and to csync_ftw_type_e, already in the csync_file_stat_t.

Issue #1817
2017-09-18 11:46:09 +02:00
Christian Kamm
3a55ff2119 PropagateUpload: Model of remote quota, avoid some uploads #5537
When we see a 507 error, assume that quota is < uploaded size.
2017-09-15 09:27:07 +02:00
Helmut K. C. Tessarek
709aa27031 remove qt4 code 2017-09-15 07:11:05 +02:00
Christian Kamm
0b7ad2c804 Excludes: Allow escaping # #6012
Otherwise adding patterns that start with # are impossible to add, since
they get treated as comments. Also add this escaping for patterns added
in the ui.
2017-09-14 14:53:05 +02:00
Markus Goetz
3d120983da ChunkingNgTest: CI debuggability #6015 2017-09-12 17:26:54 +02:00
Christian Kamm
4b686a4394 Fix compile errors
cmocka seems to #define fail to something inappropriate.
2017-09-07 11:02:05 +02:00
Jocelyn Turcotte
b1bb7ec1a8 Use a C++ contructor and destructor for CSYNC
Merge csync_create and csync_init into the constructor and
replace csync_destroy with the destructor.

Also use a QByteArray for csync_s::root_perms and flatten
csync_rename_s as a rename sub-struct of csync_s since it
can now handle C++ types.
2017-09-06 13:08:23 +02:00
Jocelyn Turcotte
f6136dd036 Remove csync_s::replica
The only difference with csync_s::current is that it's
assigned the value of csync_s::local::type and
csync_s::remote::type, which never change. So might as
well only use the "current" field with constants.
2017-09-06 13:08:23 +02:00
Jocelyn Turcotte
e8f7adc7ca Replace csync_vio_file_stat_t with csync_file_stat_t
Also move csync_normalize_etag to common/utility since we
don't need the char* function anymore.

Remove the single space file_stat->remotePerm codepath since
this won't be used in csync anymore since
8de3bda0b1.

Issue #1817
2017-09-06 13:08:23 +02:00
Jocelyn Turcotte
704ca366b3 Make csync_file_stat_t public and partly convert to C++
This is the first commit trying to unify csync_file_stat_s,
csync_vio_file_stat_s and csync_tree_walk_file_s. Use QByteArray
and unique_ptr already since I'm not used to track memory allocations
and this will make the transition easier.

Issue #1817
2017-09-05 17:25:19 +02:00
Jocelyn Turcotte
cf15cbf0b3 Move Utility to a new common static library
Now that csync builds as C++, this will avoid having to implement
functionalities needed by csync mandatorily in csync itself.

This library is built as part of libocsync and symbols exported
through it.
This requires a relicense of Utility as LGPL. All classes moved into
this library from src/libsync will need to be relicensed as well.
2017-09-05 17:25:19 +02:00
Christian Kamm
de7c301654 Add minor unittest for numeric id extraction #5933 2017-09-05 13:40:26 +02:00
Jocelyn Turcotte
48d2fc1599 StatusTracker: Fix different case paths not matching (#5981)
Use a custom std::map comparator functor to do all comparisons
on contained QStrings using Qt::CaseInsensitive on macOS and Windows.

Issue #5257
2017-08-31 13:32:00 +02:00
Jocelyn Turcotte
7a5a706397 Fix cmocka deprecation warnings
Just use cmocka_run_group_tests like we already use in
some of those tests.
2017-08-22 15:22:57 +02:00
Jocelyn Turcotte
59ea79d9e6 Enable csync logs in unit tests
Go through the Logger to enable the csync-QDebug bridge.
2017-08-22 15:22:56 +02:00
Jocelyn Turcotte
4e5898efd7 Compile almost all of csync as C++
This will allow us to unify data structures between csync and libsync.

Utility functions like csync_time and c_std are still compiled as C
since we won't need to be coupled with Qt in the short term.
2017-08-22 15:22:56 +02:00
Jocelyn Turcotte
28a0971cdf Move csync to src/csync 2017-08-22 11:18:22 +02:00
Jocelyn Turcotte
5c9827a33c Move csync/tests to test/csync 2017-08-22 11:18:22 +02:00
Jocelyn Turcotte
f0362b2332 Move csync/tests/ownCloud to test/scripts/txpl 2017-08-22 11:18:22 +02:00
Olivier Goffart
a831164d65 Revert "Revert "Discovery: consider also the "shared by me" as shared""
This reverts commit efa7821dd2.

This reverts the revert, but also add a check that the server version
is bigger than 10.0

Issue #4788
2017-07-25 12:11:33 +02:00
Olivier Goffart
2ba46e05ee Merge pull request #5326 from owncloud/cleanup_tests
Use OWNCLOUD_BIN_DIR directly in C++ code.
2017-07-15 14:48:25 +02:00
Christian Kamm
e7a0c1b6d0 SyncEngineTest: _size now correct for uploads #5855 2017-07-12 10:09:04 +02:00
Christian Kamm
da54b3d7e5 Merge remote-tracking branch 'origin/2.3' 2017-07-12 10:06:04 +02:00
Christian Kamm
a5ace5e71d Account/Credentials: Have identical lifetimes
The QNAM may continue to outlive both.

Rename Credentials::getQNAM() to createQNAM() while we're at it - it's
used to make a new QNAM that will subsequently be owned by the Account
object.

See d01065b9a1 for rationale.

Relates to
d40c56eda5
147cf798a6
2017-07-08 13:07:13 +02:00
Christian Kamm
0238a29c7c Introduce private link sharing #5023
* SocketAPI has COPL_LOCAL_LINK / EMAIL_LOCAL_LINK commands
* The nautilus and dolphing shell integrations show a submenu from which
  one can share as well as access the private link.
* The SocketAPI provides a new GET_STRINGS command to access localized
  strings.
* The private link can also be accessed from the user/group sharing
  dialog.
* The numeric file id is extracted from the full id to create the
  private link url.
2017-07-07 10:49:51 +02:00
Hefee
607287a9d6 Use OWNCLOUD_BIN_DIR directly in C++ code.
No need to do a STR replacement and the macro changing the path.
It actually broke test execution for Debian.

See https://bugs.debian.org/844937
2017-07-06 08:41:02 +02:00
Christian Kamm
0db095e02a SyncEngineTest: Fix test reliability
There was a rounding issue in the mtimes which sometimes resulted in an
off-by-one error. Caused by storing a full QDateTime in the FileInfo but
the mtime saved to the disk being truncated to seconds.
2017-07-04 09:57:22 +02:00
Markus Goetz
f49985697b TestSyncJournalDB: Fix concurrent usage on CI 2017-07-03 14:08:39 +02:00
Christian Kamm
851a3128e4 SyncEngine: Add unittest for SyncFileItem properties #5855
Checks instruction, direction, size, modtime for three common cases.
2017-07-03 12:54:24 +02:00
Markus Goetz
59bbbd5e66 Jenkinsfile: Some comments 2017-06-22 15:42:56 +02:00
Markus Goetz
1e98ba27c6 CSyncSqliteTest: Fix test 2017-06-22 15:11:49 +02:00
Markus Goetz
2244b35a6d test_journal.db: Add checksumtype table 2017-06-22 15:11:49 +02:00
Christian Kamm
8160963110 Compare the hash of files with identical mtime/size #5589
* For conflicts where mtime and size are identical:

  a) If there's no remote checksum, skip (unchanged)
  b) If there's a remote checksum that's a useful hash, create a
     PropagateDownload job and compute the local hash. If the hashes
     are identical, don't download the file and just update metadata.

* Avoid exposing the existence of checksumTypeId beyond the database
  layer. This makes handling checksums easier in general because they
  can usually be treated as a single blob.

  This change was prompted by the difficulty of producing file_stat_t
  entries uniformly from PROPFINDs and the database.
2017-06-15 13:54:16 +02:00
Christian Kamm
d50d8b86cf SyncEngineTest: Add network override
This is useful for monitoring what kind of network requests are
sent to the fake server. Such as "did this sync cause an upload?"
and "was there a propfind for this path?". It can also inject
custom replies.
2017-06-15 13:53:57 +02:00
Christian Kamm
b28e06608c SyncEngineTest: Send 'checksums' in FakePropfind 2017-06-15 13:53:57 +02:00
Christian Kamm
cd411e5fb1 SyncEngineTest: Fix date locale related bug
The client is very picky about date strings it accepts. If dates are
formatted with a non-C locale (such as localized weekday names), it
fails to parse it and tests fail in subtle ways.
2017-06-15 13:53:57 +02:00
Olivier Goffart
3314a472c3 SyncEngine: SelectiveSync: Remove local files of undelected folder despite other modified files
Issue #5783

When the directry that should be removed by selective sync contains changes,
we ignore the whole sub tree instead of only ignoreing new files.
We cannot ignore the whole directory, we need to ignore only the directory
that do not have files to remove
2017-06-08 14:53:54 +02:00
Christian Kamm
5eaae83a01 Merge remote-tracking branch 'origin/2.3' 2017-05-17 13:10:19 +02:00
Christian Kamm
c5a0ce5a43 Selective sync: Skip excluded folders when reading db
When a new folder becomes selective-sync excluded, we already mark it
and all its parent folders with _invalid_ etags to force rediscovery.

That's not enough however. Later calls to csync_statedb_get_below_path
could still pull data about the excluded files into the remote tree.

That lead to incorrect behavior, such as uploads happening for folders
that had been explicitly excluded from sync.

To fix the problem, statedb_get_below_path is adjusted to not read the
data about excluded folders from the database.

Currently we can't wipe this data from the database outright because we
need it to determine whether the files in the excluded folder can be
wiped away or not.

See owncloud/enterprise#1965
2017-05-16 13:58:45 +02:00
Jocelyn Turcotte
4d3212b31f Merge branch 2.3 into master 2017-05-09 18:23:01 +02:00
Jocelyn Turcotte
879dadba78 Use QTemporaryDir in TestOwnSql
This will improve the cleanup and possibly help for #5366.
2017-05-08 23:18:06 +02:00