Commit graph

301 commits

Author SHA1 Message Date
Christian Kamm
19c6757a0f
Rename detection: File size must be equal
Comparison of file sizes for potential conflicts was added in
0eb9401c62, but did not extend to checking
the file size in case of potential local moves.

This commit adds this check and adds tests for various move+change
scenarios.
2017-10-05 22:01:42 +02:00
Christian Kamm
3636a59830
t9.pl: Make more reliable
Sometimes the 'touch' didn't change the mtime since too little time had
passed!
2017-10-05 22:01:42 +02:00
Christian Kamm
01bbbb7d08
Excludes: Fix bug with escapes in the regex, add tests 2017-10-05 22:01:42 +02:00
Christian Kamm
f609ea3bef
Excludes: optimize csync_is_windows_reserved_word 2017-10-05 22:01:42 +02:00
Markus Goetz
91223b1a3d
Excludes: During directory traversal, use QRegularExpression
On Mac, this halves the time spent in csync_excluded_traversal
when using check_csync_excluded_performance. A similar performance
increase is seen on linux.
2017-10-05 22:01:42 +02:00
Olivier Goffart
0a8c394ee9
fixup! [tx-robot] updated from transifex 2017-10-05 22:01:38 +02:00
Olivier Goffart
cd44d8ed77
Test OAuth2
Include a test for PR #6057
2017-10-05 22:01:38 +02:00
Olivier Goffart
cfc38753b4
SyncEngine: _hasNoneFiles should be set when there are INSTRUCTION_UPDATE_METADATA 2017-10-05 22:01:38 +02:00
Olivier Goffart
bbbd2319ed
Use QDateTime::currentDateTimeUtc instead of the non utc version
QDateTime::currentDateTime is terribly slow.
2017-10-05 22:01:37 +02:00
Jocelyn Turcotte
185cb2e39a
Use SyncJournalDb in csync
This gets rid of the csync_statedb sqlite layer and use
the same code and same connection as the rest of the SyncEngine.

Missing functions are added to SyncJournalDb and change a few minor
things (like changing SyncJournalFileRecord::_modtime to be an int64
instead of a QDateTime, like it was in csync).
2017-10-05 22:01:37 +02:00
Jocelyn Turcotte
c6610f6fbf
SyncJournalDB: Allow callers of getFileRecord if the query failed
The current implementation would return the same value whether the query failed
or if no row would be found. This is something that is currently checked by csync
and needs to be provided if we want to use SyncJournalDB there.

Adjusted all call sites to also check the return value even though they
could still just rely on rec.isValid(), but makes it more explicit as to what
happens for database errors in those cases, if we ever want to gracefully handle
them.
2017-10-05 22:01:37 +02:00
Olivier Goffart
38a76ac5bf
benchlargesync: Do several syncs
Make sure there is already a small sync before so the database is queried.
Make also a sync after to run an update with many files in the database.
2017-10-05 22:01:36 +02:00
Olivier Goffart
0936cfee5d
chsck_vio: fix readdir test by removing it.
The code that was creating the files in the directory was removed in
commit 6906b8d30c. The directory is empty
so the result is expected to be null. It was passing before because the
code was returning an entry for . and ..,  but since commit
35f80bd439 this is no longer the case
2017-10-05 22:01:36 +02:00
Olivier Goffart
f793de4629
RemotePermissions: Store in a class rather than in a QByteArray to save memory
Create a specific type that parses the permissions so we can store
it in a short rather than in a QByteArray

Note: in RemotePermissions::toString, we make sure the string is not
empty by adding a space, this was already existing before commit
e8f7adc7ca where it was removed by mistake.
2017-10-05 22:01:35 +02:00
Olivier Goffart
3143b32aa5
test/stub: add some call required by the next patches
As we use the new connection syntax in folderman.cpp, some more symbol
need to be mocked
2017-10-05 22:01:35 +02:00
Jocelyn Turcotte
81e32e1a08
Move SyncJournalDB to src/common 2017-10-05 22:01:33 +02:00
Jocelyn Turcotte
4c2e078eac
Move non-csync-dependent parts of FileSystem to src/common
This keep the csync-dependent parts in src/libsync, slitting the namespace
over two files.
This will allow moving SyncJournalDB to src/common as well.
2017-10-05 22:01:33 +02:00
Christian Kamm
cda72effe2
Checksums: Clearer behavior and added testing 2017-10-05 22:01:33 +02:00
Jocelyn Turcotte
0f21ed6a5b
Remove SyncFileItem::_isDirectory
It's always equivalent to _type == SyncFileItem::Directory.
2017-10-05 22:01:33 +02:00
Jocelyn Turcotte
85a93efe51
Remove SyncFileItem::log
This remove the remaining "other" fields of the sync log to save a
bit of memory.

other_etag and other_fileId don't give much information to the users
and other_instruction will always be INST_NONE anyway.

other_modtime and other_size are kept since they are sometimes used.
They were renamed to have a bit more meaningful name.

SyncEngine::checkPermissions will now fetch its information from the
csync trees since they are now preserved until right after this point.

Fixes #3213
2017-10-05 22:01:33 +02:00
Jocelyn Turcotte
3bc1f63b0a
Remove the usage of phash in csync
Only store the path since they represent the same thing, and do the
phash conversion during DB lookup like done in libsync.
We could get rid of everything since we also have an index on the path
column, but since it's the primary key this makes the migration non-trivial.
2017-10-05 22:01:33 +02:00
Jocelyn Turcotte
2276df1eec
Replace the custom rbtree by a std::map 2017-10-05 22:01:33 +02:00
Jocelyn Turcotte
ec52807e29
csync: Keep the csync_file_stat_t instances created during discovery
Now that they use the same structure, avoid _csync_detect_update
having to recreate another instance and transfer everything manually.
Any instance created during discovery should now be used all the way
up to SyncEngine::treewalkFile.

This also makes sure that the path and types are properly set in that
object instead of having to pass everything as separate parameters.

This gets rid of csync_ftw_flags_e which was now converted from,
and to csync_ftw_type_e, already in the csync_file_stat_t.

Issue #1817
2017-10-05 22:01:33 +02:00
Christian Kamm
0e8ce9c3db
PropagateUpload: Model of remote quota, avoid some uploads #5537
When we see a 507 error, assume that quota is < uploaded size.
2017-10-05 22:01:32 +02:00
Helmut K. C. Tessarek
e8f17f5b0b
remove qt4 code 2017-10-05 22:01:32 +02:00
Christian Kamm
72131ff4ce
Excludes: Allow escaping # #6012
Otherwise adding patterns that start with # are impossible to add, since
they get treated as comments. Also add this escaping for patterns added
in the ui.
2017-10-05 22:01:06 +02:00
Markus Goetz
7ca1107ea0
ChunkingNgTest: CI debuggability #6015 2017-10-05 22:01:05 +02:00
Christian Kamm
8a094d2b0e
Fix compile errors
cmocka seems to #define fail to something inappropriate.
2017-10-05 22:01:04 +02:00
Jocelyn Turcotte
e0fa5723eb
Use a C++ contructor and destructor for CSYNC
Merge csync_create and csync_init into the constructor and
replace csync_destroy with the destructor.

Also use a QByteArray for csync_s::root_perms and flatten
csync_rename_s as a rename sub-struct of csync_s since it
can now handle C++ types.
2017-10-05 22:01:04 +02:00
Jocelyn Turcotte
4947dc22ab
Remove csync_s::replica
The only difference with csync_s::current is that it's
assigned the value of csync_s::local::type and
csync_s::remote::type, which never change. So might as
well only use the "current" field with constants.
2017-10-05 22:01:04 +02:00
Jocelyn Turcotte
7fad474863
Replace csync_vio_file_stat_t with csync_file_stat_t
Also move csync_normalize_etag to common/utility since we
don't need the char* function anymore.

Remove the single space file_stat->remotePerm codepath since
this won't be used in csync anymore since
8de3bda0b1.

Issue #1817
2017-10-05 22:01:04 +02:00
Jocelyn Turcotte
61d949730b
Make csync_file_stat_t public and partly convert to C++
This is the first commit trying to unify csync_file_stat_s,
csync_vio_file_stat_s and csync_tree_walk_file_s. Use QByteArray
and unique_ptr already since I'm not used to track memory allocations
and this will make the transition easier.

Issue #1817
2017-10-05 22:01:03 +02:00
Jocelyn Turcotte
bfc7ea562d
Move Utility to a new common static library
Now that csync builds as C++, this will avoid having to implement
functionalities needed by csync mandatorily in csync itself.

This library is built as part of libocsync and symbols exported
through it.
This requires a relicense of Utility as LGPL. All classes moved into
this library from src/libsync will need to be relicensed as well.
2017-10-05 22:01:03 +02:00
Christian Kamm
6d8c98f6b4
Add minor unittest for numeric id extraction #5933 2017-10-05 22:01:03 +02:00
Jocelyn Turcotte
4aca1b9697
StatusTracker: Fix different case paths not matching (#5981)
Use a custom std::map comparator functor to do all comparisons
on contained QStrings using Qt::CaseInsensitive on macOS and Windows.

Issue #5257
2017-10-05 22:01:03 +02:00
Jocelyn Turcotte
7280359c66
Fix cmocka deprecation warnings
Just use cmocka_run_group_tests like we already use in
some of those tests.
2017-10-05 22:01:03 +02:00
Jocelyn Turcotte
b30c3202f0
Enable csync logs in unit tests
Go through the Logger to enable the csync-QDebug bridge.
2017-10-05 22:01:03 +02:00
Jocelyn Turcotte
ac4be17192
Compile almost all of csync as C++
This will allow us to unify data structures between csync and libsync.

Utility functions like csync_time and c_std are still compiled as C
since we won't need to be coupled with Qt in the short term.
2017-10-05 22:01:03 +02:00
Jocelyn Turcotte
f897a91354
Move csync to src/csync 2017-10-05 22:01:03 +02:00
Jocelyn Turcotte
5948f26884
Move csync/tests to test/csync 2017-10-05 22:01:03 +02:00
Jocelyn Turcotte
95579b3de1
Move csync/tests/ownCloud to test/scripts/txpl 2017-10-05 22:00:57 +02:00
Olivier Goffart
a831164d65 Revert "Revert "Discovery: consider also the "shared by me" as shared""
This reverts commit efa7821dd2.

This reverts the revert, but also add a check that the server version
is bigger than 10.0

Issue #4788
2017-07-25 12:11:33 +02:00
Olivier Goffart
2ba46e05ee Merge pull request #5326 from owncloud/cleanup_tests
Use OWNCLOUD_BIN_DIR directly in C++ code.
2017-07-15 14:48:25 +02:00
Christian Kamm
e7a0c1b6d0 SyncEngineTest: _size now correct for uploads #5855 2017-07-12 10:09:04 +02:00
Christian Kamm
da54b3d7e5 Merge remote-tracking branch 'origin/2.3' 2017-07-12 10:06:04 +02:00
Christian Kamm
a5ace5e71d Account/Credentials: Have identical lifetimes
The QNAM may continue to outlive both.

Rename Credentials::getQNAM() to createQNAM() while we're at it - it's
used to make a new QNAM that will subsequently be owned by the Account
object.

See d01065b9a1 for rationale.

Relates to
d40c56eda5
147cf798a6
2017-07-08 13:07:13 +02:00
Christian Kamm
0238a29c7c Introduce private link sharing #5023
* SocketAPI has COPL_LOCAL_LINK / EMAIL_LOCAL_LINK commands
* The nautilus and dolphing shell integrations show a submenu from which
  one can share as well as access the private link.
* The SocketAPI provides a new GET_STRINGS command to access localized
  strings.
* The private link can also be accessed from the user/group sharing
  dialog.
* The numeric file id is extracted from the full id to create the
  private link url.
2017-07-07 10:49:51 +02:00
Hefee
607287a9d6 Use OWNCLOUD_BIN_DIR directly in C++ code.
No need to do a STR replacement and the macro changing the path.
It actually broke test execution for Debian.

See https://bugs.debian.org/844937
2017-07-06 08:41:02 +02:00
Christian Kamm
0db095e02a SyncEngineTest: Fix test reliability
There was a rounding issue in the mtimes which sometimes resulted in an
off-by-one error. Caused by storing a full QDateTime in the FileInfo but
the mtime saved to the disk being truncated to seconds.
2017-07-04 09:57:22 +02:00
Markus Goetz
f49985697b TestSyncJournalDB: Fix concurrent usage on CI 2017-07-03 14:08:39 +02:00