Commit graph

11048 commits

Author SHA1 Message Date
Jocelyn Turcotte
4c2e078eac
Move non-csync-dependent parts of FileSystem to src/common
This keep the csync-dependent parts in src/libsync, slitting the namespace
over two files.
This will allow moving SyncJournalDB to src/common as well.
2017-10-05 22:01:33 +02:00
Jocelyn Turcotte
dc3c3195b5
Update sync-algorithm.md
The rbtree was replaced by a std::map.
2017-10-05 22:01:33 +02:00
Christian Kamm
cda72effe2
Checksums: Clearer behavior and added testing 2017-10-05 22:01:33 +02:00
Jocelyn Turcotte
0f21ed6a5b
Remove SyncFileItem::_isDirectory
It's always equivalent to _type == SyncFileItem::Directory.
2017-10-05 22:01:33 +02:00
Jocelyn Turcotte
85a93efe51
Remove SyncFileItem::log
This remove the remaining "other" fields of the sync log to save a
bit of memory.

other_etag and other_fileId don't give much information to the users
and other_instruction will always be INST_NONE anyway.

other_modtime and other_size are kept since they are sometimes used.
They were renamed to have a bit more meaningful name.

SyncEngine::checkPermissions will now fetch its information from the
csync trees since they are now preserved until right after this point.

Fixes #3213
2017-10-05 22:01:33 +02:00
Jocelyn Turcotte
82dd1775eb
Remove SyncEngine::_remotePerms
Now that csync is using a more convenient data structure for
its file trees, wait a little bit longer before destroying them and
fetch the remote permissions from the remote tree there instead.
2017-10-05 22:01:33 +02:00
Jocelyn Turcotte
3bc1f63b0a
Remove the usage of phash in csync
Only store the path since they represent the same thing, and do the
phash conversion during DB lookup like done in libsync.
We could get rid of everything since we also have an index on the path
column, but since it's the primary key this makes the migration non-trivial.
2017-10-05 22:01:33 +02:00
Jocelyn Turcotte
2276df1eec
Replace the custom rbtree by a std::map 2017-10-05 22:01:33 +02:00
Jocelyn Turcotte
ec52807e29
csync: Keep the csync_file_stat_t instances created during discovery
Now that they use the same structure, avoid _csync_detect_update
having to recreate another instance and transfer everything manually.
Any instance created during discovery should now be used all the way
up to SyncEngine::treewalkFile.

This also makes sure that the path and types are properly set in that
object instead of having to pass everything as separate parameters.

This gets rid of csync_ftw_flags_e which was now converted from,
and to csync_ftw_type_e, already in the csync_file_stat_t.

Issue #1817
2017-10-05 22:01:33 +02:00
Jenkins for ownCloud
7f88a3e1cd
[tx-robot] updated from transifex 2017-10-05 22:01:33 +02:00
Jenkins for ownCloud
74049e3e92
[tx-robot] updated from transifex 2017-10-05 22:01:33 +02:00
Jenkins for ownCloud
d01c7dc7db
[tx-robot] updated from transifex 2017-10-05 22:01:33 +02:00
Christian Kamm
b92de38c31
AbstractNetworkJob: Allow relative redirect
Previously all redirects were considered absolute.
2017-10-05 22:01:32 +02:00
Christian Kamm
78b082a23c
Reduce timeout for some admin jobs
The oauth token jobs and the wizard redirect check job shouldn't have
5min timeouts.
2017-10-05 22:01:32 +02:00
Christian Kamm
1aa1e7f833
Wizard: Handle url-shortener redirects #5954
Grab any permanent redirects from the base url the user entered
before attempting to connect to a modified url (with status.php
added).
2017-10-05 22:01:32 +02:00
Christian Kamm
4d2261420d
HttpCreds: Remove migration code from 1.7 2017-10-05 22:01:32 +02:00
Christian Kamm
23153a4866
Credentials: Use per-account keychain entries #5830
This requires a lot of migration code: the old entries need to be read,
saved to the new locations and then deleted.
2017-10-05 22:01:32 +02:00
Christian Kamm
7f75a533d1
Journal: Second attempt at journal mode fallback #5723
Some filesystems, vms or other limitations make using the WAL journal
mode impossible. We are notified of this problem through an sqlite
IOERR for SHMMAP. In that case We want to attempt to fall back to the
DELETE journal mode.
2017-10-05 22:01:32 +02:00
Christian Kamm
e47b8ffaca
OAuth: Don't use implicit POST bodies
The query args of POST requests become the request body. If there's a
redirect, the redirected url will therefore not contain the query
arguments. Use an explicit request body to make the redirection work.
2017-10-05 22:01:32 +02:00
Christian Kamm
a79b0a3791
OAuth: Pass client auth in header instead of url
To play more nicely with redirects.
2017-10-05 22:01:32 +02:00
Christian Kamm
ce51ea34b9
OAuth: Use redirectable jobs for oauth token management 2017-10-05 22:01:32 +02:00
Christian Kamm
0e8ce9c3db
PropagateUpload: Model of remote quota, avoid some uploads #5537
When we see a 507 error, assume that quota is < uploaded size.
2017-10-05 22:01:32 +02:00
Christian Kamm
76faf4937b
SyncOptions/Propagator: Parallelism as an option
Very useful for changing it on a per-test basis.
2017-10-05 22:01:32 +02:00
Christian Kamm
df05042e7f
Make DetailError different from BlacklistedError
It's quite different in regard to blacklist handling and overall
sync failure changes.
2017-10-05 22:01:32 +02:00
Helmut K. C. Tessarek
e8f17f5b0b
remove qt4 code 2017-10-05 22:01:32 +02:00
Jenkins for ownCloud
7d1285d313
[tx-robot] updated from transifex 2017-10-05 22:01:31 +02:00
Christian Kamm
72131ff4ce
Excludes: Allow escaping # #6012
Otherwise adding patterns that start with # are impossible to add, since
they get treated as comments. Also add this escaping for patterns added
in the ui.
2017-10-05 22:01:06 +02:00
Jenkins for ownCloud
c03d53e1dd
[tx-robot] updated from transifex 2017-10-05 22:01:05 +02:00
Jenkins for ownCloud
bc5a6527f9
[tx-robot] updated from transifex 2017-10-05 22:01:05 +02:00
Markus Goetz
7ca1107ea0
ChunkingNgTest: CI debuggability #6015 2017-10-05 22:01:05 +02:00
Christian Kamm
4acb70e79d
Journal: Fall back to DELETE on IO error #5723
Also add logging of extended error codes for this IO error, maybe we can
become more specific about which situations should trigger a journal
mode switch.
2017-10-05 22:01:05 +02:00
Christian Kamm
755b18db61
Folder: Check etag again after active sync #4116
Maybe more things were happening on the server?
2017-10-05 22:01:05 +02:00
Jenkins for ownCloud
bf8151a8ad
[tx-robot] updated from transifex 2017-10-05 22:01:05 +02:00
Christian Kamm
b810ce7768
Update server url in case of permanent redirection #5972
This is the first time the account url may update outside of
account setup.

Summary of redirection handling:
1. During account setup (wizard)
   - status.php gets permanently redirected -> adjust url
   - authed PROPFIND gets *any* redirection -> adjust url
2. During connectivity ping (ConnectionValidator)
   - status.php gets permanently redirected -> adjust url (new!)

All other redirections should be followed transparently and
don't update the account url in the settings.
2017-10-05 22:01:05 +02:00
Christian Kamm
1c0d80c20d
Use DetermineAuthTypeJob in HttpCredentials
* Move it to networkjobs
* Minor adjustments to its logic
* Fixes redirect handling for oauth/basic http auth check #6003
2017-10-05 22:01:05 +02:00
Markus Goetz
ed9977a2b3
Add clean_tarball.sh #6005 2017-10-05 22:01:05 +02:00
Jenkins for ownCloud
546c7c420a
[tx-robot] updated from transifex 2017-10-05 22:01:05 +02:00
Jocelyn Turcotte
bdf3f1e271
Windows: Make sure that the folder and Desktop.ini have the right attributes (#6018)
When created from scratch, the owncloud icon won't be picked-up by
explorer unless the right attributes are set on both the sync root and the
Desktop.ini file.

Issue #2446
2017-10-05 22:01:04 +02:00
rockihack
4e90179065
Enable DEP & ASLR on Windows 2017-10-05 22:01:04 +02:00
Markus Goetz
43f85a787b
Remove qtkeychain submodule #6004 2017-10-05 22:01:04 +02:00
Jocelyn Turcotte
a8c33885bb
Fix the macOS Sparkle build 2017-10-05 22:01:04 +02:00
Christian Kamm
3591849279
Http credentials: Fix behavior for bad password #5989
When the GET request from askFromUser is scheduled on the QNAM inside
the slot that handles the QNetworkReply::finished signal, it seems to
not get processed at all.

This workaround moves the sending of the new GET to the event loop,
sidestepping the problem.
2017-10-05 22:01:04 +02:00
Christian Kamm
8a094d2b0e
Fix compile errors
cmocka seems to #define fail to something inappropriate.
2017-10-05 22:01:04 +02:00
Jocelyn Turcotte
deb123b80d
csync_statedb: (really) Make sure children of non-synced items aren't read from the DB
No tests are covering this and I reversed the original logic.
2017-10-05 22:01:04 +02:00
Jocelyn Turcotte
7a97dc1783
csync_statedb: Make sure children of non-synced items aren't read from the DB
Fix a regression during C++ conversion from 704ca366b3
2017-10-05 22:01:04 +02:00
Jocelyn Turcotte
41f215fd8d
Fix build after C++ conversion of csync
Use the same macro as in SyncFileItem for bitfields on enums.

Fixes #5993
2017-10-05 22:01:04 +02:00
Jocelyn Turcotte
e0fa5723eb
Use a C++ contructor and destructor for CSYNC
Merge csync_create and csync_init into the constructor and
replace csync_destroy with the destructor.

Also use a QByteArray for csync_s::root_perms and flatten
csync_rename_s as a rename sub-struct of csync_s since it
can now handle C++ types.
2017-10-05 22:01:04 +02:00
Jocelyn Turcotte
4947dc22ab
Remove csync_s::replica
The only difference with csync_s::current is that it's
assigned the value of csync_s::local::type and
csync_s::remote::type, which never change. So might as
well only use the "current" field with constants.
2017-10-05 22:01:04 +02:00
Jocelyn Turcotte
157e933ae5
Replace TREE_WALK_FILE with csync_file_stat_t
Just expose csync_file_stat_t since we don't need an abstraction layer
anymore. Also pass the nodes of both trees directly to the visitor
function.

Issue #1817
2017-10-05 22:01:04 +02:00
Jocelyn Turcotte
7fad474863
Replace csync_vio_file_stat_t with csync_file_stat_t
Also move csync_normalize_etag to common/utility since we
don't need the char* function anymore.

Remove the single space file_stat->remotePerm codepath since
this won't be used in csync anymore since
8de3bda0b1.

Issue #1817
2017-10-05 22:01:04 +02:00