Commit graph

296 commits

Author SHA1 Message Date
Olivier Goffart
253d197ccb fixup! [tx-robot] updated from transifex 2017-09-28 18:38:33 +02:00
Olivier Goffart
0ceb806f1a Test OAuth2
Include a test for PR #6057
2017-09-28 18:38:33 +02:00
Olivier Goffart
800b9cf167 SyncEngine: _hasNoneFiles should be set when there are INSTRUCTION_UPDATE_METADATA 2017-09-28 11:15:36 +02:00
Olivier Goffart
13e0cc6b50 Use QDateTime::currentDateTimeUtc instead of the non utc version
QDateTime::currentDateTime is terribly slow.
2017-09-26 16:14:17 +02:00
Jocelyn Turcotte
a034ee894c Use SyncJournalDb in csync
This gets rid of the csync_statedb sqlite layer and use
the same code and same connection as the rest of the SyncEngine.

Missing functions are added to SyncJournalDb and change a few minor
things (like changing SyncJournalFileRecord::_modtime to be an int64
instead of a QDateTime, like it was in csync).
2017-09-26 11:03:23 +02:00
Jocelyn Turcotte
6f46764daa SyncJournalDB: Allow callers of getFileRecord if the query failed
The current implementation would return the same value whether the query failed
or if no row would be found. This is something that is currently checked by csync
and needs to be provided if we want to use SyncJournalDB there.

Adjusted all call sites to also check the return value even though they
could still just rely on rec.isValid(), but makes it more explicit as to what
happens for database errors in those cases, if we ever want to gracefully handle
them.
2017-09-26 11:03:23 +02:00
Olivier Goffart
2953eed729 benchlargesync: Do several syncs
Make sure there is already a small sync before so the database is queried.
Make also a sync after to run an update with many files in the database.
2017-09-25 12:49:55 +02:00
Olivier Goffart
d113498c7b chsck_vio: fix readdir test by removing it.
The code that was creating the files in the directory was removed in
commit 6906b8d30c. The directory is empty
so the result is expected to be null. It was passing before because the
code was returning an entry for . and ..,  but since commit
35f80bd439 this is no longer the case
2017-09-25 00:17:47 +02:00
Olivier Goffart
95d23b1914 RemotePermissions: Store in a class rather than in a QByteArray to save memory
Create a specific type that parses the permissions so we can store
it in a short rather than in a QByteArray

Note: in RemotePermissions::toString, we make sure the string is not
empty by adding a space, this was already existing before commit
e8f7adc7ca where it was removed by mistake.
2017-09-22 09:29:08 +02:00
Olivier Goffart
c4e51247d8 test/stub: add some call required by the next patches
As we use the new connection syntax in folderman.cpp, some more symbol
need to be mocked
2017-09-21 14:05:39 +02:00
Jocelyn Turcotte
a1f1775d15 Move SyncJournalDB to src/common 2017-09-18 14:00:52 +02:00
Jocelyn Turcotte
bd25225045 Move non-csync-dependent parts of FileSystem to src/common
This keep the csync-dependent parts in src/libsync, slitting the namespace
over two files.
This will allow moving SyncJournalDB to src/common as well.
2017-09-18 14:00:52 +02:00
Christian Kamm
78212e03d6 Checksums: Clearer behavior and added testing 2017-09-18 12:52:09 +02:00
Jocelyn Turcotte
a8ea7b0858 Remove SyncFileItem::_isDirectory
It's always equivalent to _type == SyncFileItem::Directory.
2017-09-18 11:46:09 +02:00
Jocelyn Turcotte
0e97770c47 Remove SyncFileItem::log
This remove the remaining "other" fields of the sync log to save a
bit of memory.

other_etag and other_fileId don't give much information to the users
and other_instruction will always be INST_NONE anyway.

other_modtime and other_size are kept since they are sometimes used.
They were renamed to have a bit more meaningful name.

SyncEngine::checkPermissions will now fetch its information from the
csync trees since they are now preserved until right after this point.

Fixes #3213
2017-09-18 11:46:09 +02:00
Jocelyn Turcotte
bfacac8683 Remove the usage of phash in csync
Only store the path since they represent the same thing, and do the
phash conversion during DB lookup like done in libsync.
We could get rid of everything since we also have an index on the path
column, but since it's the primary key this makes the migration non-trivial.
2017-09-18 11:46:09 +02:00
Jocelyn Turcotte
d66c2b5fae Replace the custom rbtree by a std::map 2017-09-18 11:46:09 +02:00
Jocelyn Turcotte
72e44ce3d7 csync: Keep the csync_file_stat_t instances created during discovery
Now that they use the same structure, avoid _csync_detect_update
having to recreate another instance and transfer everything manually.
Any instance created during discovery should now be used all the way
up to SyncEngine::treewalkFile.

This also makes sure that the path and types are properly set in that
object instead of having to pass everything as separate parameters.

This gets rid of csync_ftw_flags_e which was now converted from,
and to csync_ftw_type_e, already in the csync_file_stat_t.

Issue #1817
2017-09-18 11:46:09 +02:00
Christian Kamm
3a55ff2119 PropagateUpload: Model of remote quota, avoid some uploads #5537
When we see a 507 error, assume that quota is < uploaded size.
2017-09-15 09:27:07 +02:00
Helmut K. C. Tessarek
709aa27031 remove qt4 code 2017-09-15 07:11:05 +02:00
Christian Kamm
0b7ad2c804 Excludes: Allow escaping # #6012
Otherwise adding patterns that start with # are impossible to add, since
they get treated as comments. Also add this escaping for patterns added
in the ui.
2017-09-14 14:53:05 +02:00
Markus Goetz
3d120983da ChunkingNgTest: CI debuggability #6015 2017-09-12 17:26:54 +02:00
Christian Kamm
4b686a4394 Fix compile errors
cmocka seems to #define fail to something inappropriate.
2017-09-07 11:02:05 +02:00
Jocelyn Turcotte
b1bb7ec1a8 Use a C++ contructor and destructor for CSYNC
Merge csync_create and csync_init into the constructor and
replace csync_destroy with the destructor.

Also use a QByteArray for csync_s::root_perms and flatten
csync_rename_s as a rename sub-struct of csync_s since it
can now handle C++ types.
2017-09-06 13:08:23 +02:00
Jocelyn Turcotte
f6136dd036 Remove csync_s::replica
The only difference with csync_s::current is that it's
assigned the value of csync_s::local::type and
csync_s::remote::type, which never change. So might as
well only use the "current" field with constants.
2017-09-06 13:08:23 +02:00
Jocelyn Turcotte
e8f7adc7ca Replace csync_vio_file_stat_t with csync_file_stat_t
Also move csync_normalize_etag to common/utility since we
don't need the char* function anymore.

Remove the single space file_stat->remotePerm codepath since
this won't be used in csync anymore since
8de3bda0b1.

Issue #1817
2017-09-06 13:08:23 +02:00
Jocelyn Turcotte
704ca366b3 Make csync_file_stat_t public and partly convert to C++
This is the first commit trying to unify csync_file_stat_s,
csync_vio_file_stat_s and csync_tree_walk_file_s. Use QByteArray
and unique_ptr already since I'm not used to track memory allocations
and this will make the transition easier.

Issue #1817
2017-09-05 17:25:19 +02:00
Jocelyn Turcotte
cf15cbf0b3 Move Utility to a new common static library
Now that csync builds as C++, this will avoid having to implement
functionalities needed by csync mandatorily in csync itself.

This library is built as part of libocsync and symbols exported
through it.
This requires a relicense of Utility as LGPL. All classes moved into
this library from src/libsync will need to be relicensed as well.
2017-09-05 17:25:19 +02:00
Christian Kamm
de7c301654 Add minor unittest for numeric id extraction #5933 2017-09-05 13:40:26 +02:00
Jocelyn Turcotte
48d2fc1599 StatusTracker: Fix different case paths not matching (#5981)
Use a custom std::map comparator functor to do all comparisons
on contained QStrings using Qt::CaseInsensitive on macOS and Windows.

Issue #5257
2017-08-31 13:32:00 +02:00
Jocelyn Turcotte
7a5a706397 Fix cmocka deprecation warnings
Just use cmocka_run_group_tests like we already use in
some of those tests.
2017-08-22 15:22:57 +02:00
Jocelyn Turcotte
59ea79d9e6 Enable csync logs in unit tests
Go through the Logger to enable the csync-QDebug bridge.
2017-08-22 15:22:56 +02:00
Jocelyn Turcotte
4e5898efd7 Compile almost all of csync as C++
This will allow us to unify data structures between csync and libsync.

Utility functions like csync_time and c_std are still compiled as C
since we won't need to be coupled with Qt in the short term.
2017-08-22 15:22:56 +02:00
Jocelyn Turcotte
28a0971cdf Move csync to src/csync 2017-08-22 11:18:22 +02:00
Jocelyn Turcotte
5c9827a33c Move csync/tests to test/csync 2017-08-22 11:18:22 +02:00
Jocelyn Turcotte
f0362b2332 Move csync/tests/ownCloud to test/scripts/txpl 2017-08-22 11:18:22 +02:00
Olivier Goffart
a831164d65 Revert "Revert "Discovery: consider also the "shared by me" as shared""
This reverts commit efa7821dd2.

This reverts the revert, but also add a check that the server version
is bigger than 10.0

Issue #4788
2017-07-25 12:11:33 +02:00
Olivier Goffart
2ba46e05ee Merge pull request #5326 from owncloud/cleanup_tests
Use OWNCLOUD_BIN_DIR directly in C++ code.
2017-07-15 14:48:25 +02:00
Christian Kamm
e7a0c1b6d0 SyncEngineTest: _size now correct for uploads #5855 2017-07-12 10:09:04 +02:00
Christian Kamm
da54b3d7e5 Merge remote-tracking branch 'origin/2.3' 2017-07-12 10:06:04 +02:00
Christian Kamm
a5ace5e71d Account/Credentials: Have identical lifetimes
The QNAM may continue to outlive both.

Rename Credentials::getQNAM() to createQNAM() while we're at it - it's
used to make a new QNAM that will subsequently be owned by the Account
object.

See d01065b9a1 for rationale.

Relates to
d40c56eda5
147cf798a6
2017-07-08 13:07:13 +02:00
Christian Kamm
0238a29c7c Introduce private link sharing #5023
* SocketAPI has COPL_LOCAL_LINK / EMAIL_LOCAL_LINK commands
* The nautilus and dolphing shell integrations show a submenu from which
  one can share as well as access the private link.
* The SocketAPI provides a new GET_STRINGS command to access localized
  strings.
* The private link can also be accessed from the user/group sharing
  dialog.
* The numeric file id is extracted from the full id to create the
  private link url.
2017-07-07 10:49:51 +02:00
Hefee
607287a9d6 Use OWNCLOUD_BIN_DIR directly in C++ code.
No need to do a STR replacement and the macro changing the path.
It actually broke test execution for Debian.

See https://bugs.debian.org/844937
2017-07-06 08:41:02 +02:00
Christian Kamm
0db095e02a SyncEngineTest: Fix test reliability
There was a rounding issue in the mtimes which sometimes resulted in an
off-by-one error. Caused by storing a full QDateTime in the FileInfo but
the mtime saved to the disk being truncated to seconds.
2017-07-04 09:57:22 +02:00
Markus Goetz
f49985697b TestSyncJournalDB: Fix concurrent usage on CI 2017-07-03 14:08:39 +02:00
Christian Kamm
851a3128e4 SyncEngine: Add unittest for SyncFileItem properties #5855
Checks instruction, direction, size, modtime for three common cases.
2017-07-03 12:54:24 +02:00
Markus Goetz
59bbbd5e66 Jenkinsfile: Some comments 2017-06-22 15:42:56 +02:00
Markus Goetz
1e98ba27c6 CSyncSqliteTest: Fix test 2017-06-22 15:11:49 +02:00
Markus Goetz
2244b35a6d test_journal.db: Add checksumtype table 2017-06-22 15:11:49 +02:00
Christian Kamm
8160963110 Compare the hash of files with identical mtime/size #5589
* For conflicts where mtime and size are identical:

  a) If there's no remote checksum, skip (unchanged)
  b) If there's a remote checksum that's a useful hash, create a
     PropagateDownload job and compute the local hash. If the hashes
     are identical, don't download the file and just update metadata.

* Avoid exposing the existence of checksumTypeId beyond the database
  layer. This makes handling checksums easier in general because they
  can usually be treated as a single blob.

  This change was prompted by the difficulty of producing file_stat_t
  entries uniformly from PROPFINDs and the database.
2017-06-15 13:54:16 +02:00