Commit graph

5365 commits

Author SHA1 Message Date
Christian Kamm
9c7ee6ef85 Reconcile: Rename handling fixes: duplicate file ids
When users share the same tree several times (say A/ and A/B/ are both
shared) the remote tree can have several entries that have the same
file id. This needs to be respected in rename detection.

Also adds several tests and fixes for issues noticed during testing.

See #6096
2017-10-24 10:54:23 +02:00
Christian Kamm
e85a339d94 Local discovery: Use db instead of filesystem
We mostly trust the file watchers meaning that we don't re-scan the
local tree if we have done that recently and no file watcher events
have arrived. If the file watchers invalidate a subtree, we rescan
only that subtree.

Since we're not entirely sure the file watchers are reliable, we still
do full local discoveries regularly (1h by default). There is a config
file setting as well as an environment variable to control the interval.
2017-10-24 10:00:20 +02:00
Christian Kamm
66f0ce6616 FolderWatcher: reliability and data-loss signal
Add state and signal to catch the following two known problems:
* Linux: inotify user watch pool is exhausted. Folder watcher becomes
unreliable.
* Windows: buffer is too small, some notifications are lost but watching
stays reliable.
2017-10-24 10:00:20 +02:00
Christian Kamm
fb2d31c9ce Folder: Move FolderWatcher to Folder
This makes it unnecessary for FolderMan to manage the list and removes
the need for some forwarders.

This is done in preparation for follow-up commits that want to add
diagnostics to FolderWatcher that shall be available from within Folder.
2017-10-24 10:00:20 +02:00
Christian Kamm
f3ea375083 Wizard: Resolve url/ redirects only if url/status.php not found
Unfortunately checking the base-url for redirects in all cases lead
to incorrect behavior in some SAML/OAuth2 edge cases.

This new iteration checks the base url for redirects only if the
standard CheckServerJob can't reach the server. That way the 2.3
behavior is only changed in cases that would have lead to errors.

See #5954
2017-10-24 09:42:08 +02:00
Tomaz Canabrava
476fe66043 [CSE] Add beginnign of the code to deal with the metadata 2017-10-23 21:06:26 +02:00
Tomaz Canabrava
bb53c2586f [CSE] Add Nlohmann's json library
Because nobody deservers to deal with Qt's JSON API
2017-10-23 21:06:26 +02:00
Tomaz Canabrava
23f5bb7ed9 [CSE] Move network jobs out of networkjobs.h
Since those networkjobs are all about client side
encryption, mvoe them to clientsideencryption.h
This will help with fewer conflicts with the origin.
2017-10-23 21:06:26 +02:00
Olivier Goffart
f41c9fbb7f owncloudsetupwizard: Fix "add new account" if the wizard is already visible
Clicking on the "Add new Account" from the systray menu should raise
the wizard, even if it is already running.

Relates to issue #6105
2017-10-20 12:41:48 +02:00
Olivier Goffart
a0e50670de Shibolleth: raise the browser when clicking on the tray
Issue #6105

Dynamically find the browser trough topLevelWidgets instead of
forwarding the call to the relevant page as it would require to break
many abstration layers (OwncloudSetupWizard -> OwncloudWizard ->
AbstractCredentialsWizardPage -> OwncloudShibbolethCredsPage)
And considering that we want to phase shibboleth down, I tought is
was not worth adding an interface for this.

The OAuth page don't have this problem because it shows a label and
allow the user to re-open the browser.
2017-10-20 10:51:35 +02:00
Olivier Goffart
a9761a8976 Use qEnvironmentVariable* instead of qgetenv when appropriate
Now that we use Qt5, we should do that. It is slightly more efficient
and declares the intent.
(Modified using clazy)
2017-10-19 13:57:49 +02:00
Tomaz Canabrava
1a47052aa3
[CSE] Remove unused function
This is already being called from somwhere else.
2017-10-18 21:51:39 +02:00
Tomaz Canabrava
e0988f482c
[CSE] Build fix
- change the name of one Qt Message Handler
- changed parameter from int to QString
2017-10-18 21:51:39 +02:00
Tomaz Canabrava
d83e8819ce
[CSE] SetEncryptionFlagApiJob
This new job sets the Encryption Flag in a folder.
2017-10-18 21:51:39 +02:00
Tomaz Canabrava
623eb29845
[CSE] Helper functions for the URL's 2017-10-18 21:51:39 +02:00
Tomaz Canabrava
1436d5bac1
[CSE] Better debug output 2017-10-18 21:51:39 +02:00
Tomaz Canabrava
0b50afe915
[CSE] Correctly Set's the client as connected
If we manage to encrypt and upload the private key
to the server, the client is in it's connected state.
2017-10-18 21:51:39 +02:00
Tomaz Canabrava
cfb6e3be8c
[CSE] Send the Private Key to the server 2017-10-18 21:51:38 +02:00
Tomaz Canabrava
78136a10b0
[CSE] Debug statements to help find the encrypt error 2017-10-18 21:51:38 +02:00
Tomaz Canabrava
08dfe86f37
[CSE] Adjust the calls to ent-to-end encryption
We changed the name.
2017-10-18 21:51:38 +02:00
Tomaz Canabrava
0f1480728e
[cse] Use fake data to verify the encryption
beware, this is broken.
2017-10-18 21:51:38 +02:00
Tomaz Canabrava
ba3d2a61d5
[cse] Call the encrypt and decrypt functions on fake data 2017-10-18 21:51:38 +02:00
Tomaz Canabrava
e0d368cbb3
[cse] Update the encrypt and decrypt algorithm to GCM 2017-10-18 21:51:38 +02:00
Tomaz Canabrava
4f7265c04f
[aes] Create the Decrypt function. 2017-10-18 21:51:38 +02:00
Tomaz Canabrava
6b53b4f257
[cse] Add the crypt method, based on OpenSSL Wiki 2017-10-18 21:51:38 +02:00
Tomaz Canabrava
77c0309e02
[cse] use PKCS5_PBKDF2_HMAC_SHA1 on the random-word passphrase 2017-10-18 21:51:38 +02:00
Tomaz Canabrava
5395fc56b1
[cse] Start the encryption algorithm for the Private Key 2017-10-18 21:51:38 +02:00
Tomaz Canabrava
fd00e180f5
[aes] fix out of bounds access. 2017-10-18 21:51:38 +02:00
Tomaz Canabrava
60729f1fc4
[cse] Enable OpenSSL for encryption 2017-10-18 21:51:38 +02:00
Tomaz Canabrava
418401a33c
[cse] new method, getUnifiedString
Enter a string list, return a string.
2017-10-18 21:51:38 +02:00
Tomaz Canabrava
2111aeaac1
[cse] Add wordlist & related methods. 2017-10-18 21:51:37 +02:00
Tomaz Canabrava
69c709714d
[cse] Save signed key on disk 2017-10-18 21:51:37 +02:00
Tomaz Canabrava
090336c928
[cse] Correctly send the CSR
finally.

Signed-off-by: Tomaz Canabrava <tcanabrava@kde.org>
2017-10-18 21:51:37 +02:00
Tomaz Canabrava
307dfd195c
[cse] Call the CSR job.
There's something wrong on the CSR job that I need to discover.
2017-10-18 21:51:37 +02:00
Tomaz Canabrava
304231811d
[cse] Start the job to sign the public key. 2017-10-18 21:51:37 +02:00
Tomaz Canabrava
ecb05020a9
[cse] Do not save the certificate on disk
Store it on memory, and discard it as soon as no longer
needed.
2017-10-18 21:51:37 +02:00
Tomaz Canabrava
d2992d92ba
[cse] Generate the CSE
I still need to send it to the server. It's been a long
learning with the OpenSSL library.
2017-10-18 21:51:37 +02:00
Tomaz Canabrava
c358980448
[cse] Link Against OpenSSL
For some reason, this was working untill I added a call
to X509_REQ_get_subject_name, then the linking suddenly
stopped working (even tougth I'm using a ton of other
OpenSSL calls)

Force to link against 1.0
2017-10-18 21:51:37 +02:00
Tomaz Canabrava
42a3098595
[cse] s/scr/csr 2017-10-18 21:51:37 +02:00
Tomaz Canabrava
58e2e6b30b
[cse] Generate the public / private keys and store locally
Now I need to understand what the hell I need to do
to send this to the server.
2017-10-18 21:51:37 +02:00
Tomaz Canabrava
8c342cb1dd
[cse] Generate the KeyPair
Not stored anywhere yet, but it's correctly running.
2017-10-18 21:51:37 +02:00
Tomaz Canabrava
17693a75e5
[cse] Request public key from server
This is the first step needed to properly communicate.
Next, get private key.
2017-10-18 21:51:37 +02:00
Tomaz Canabrava
f4bbec1019
[bugfix] Return the error code, don't discard it. 2017-10-18 21:51:37 +02:00
Tomaz Canabrava
29b64640fa
[cse] Start to fetch the basics to fetch the key from the server 2017-10-18 21:51:37 +02:00
Tomaz Canabrava
3f4d915a17
[cse] Add files to handle client side encryption
This will be the responsible for encryption,
decryption and talking with the server.
2017-10-18 21:51:36 +02:00
Tomaz Canabrava
7e2085375f
[cse] Invalid read of 1 2017-10-18 21:51:36 +02:00
Tomaz Canabrava
1c85f94b00
[cse] Use server capabilities to query for cse 2017-10-18 21:51:36 +02:00
Tomaz Canabrava
920047fa70
[CSE] Remove uneeded includes 2017-10-18 21:51:36 +02:00
Tomaz Canabrava
f7e74f520b
[CSE] Call encrypt and unencrypt from the GUI
If the server allows client side encryption, show
the user the menu with the encrypt and unencrupt options
2017-10-18 21:51:36 +02:00
Tomaz Canabrava
5b51346e83
[CSE] New Network Job: DeleteApiJob
This network job does a DELETE http request on a URL. It's the
second class that does basically the same, but this one returns
the http return code, and it's set to do a api call.
2017-10-18 21:51:36 +02:00
Tomaz Canabrava
e2091bb0a3
[CSE] Add FileIdRole for the FolderStatusModel
This way we can actually request the id from outside of the model.
2017-10-18 21:51:36 +02:00
Tomaz Canabrava
9870f39dcb
[CSE] Fetch file-id for subfolders
File id is a must if we want to call any API.

Signed-off-by: Tomaz Canabrava <tcanabrava@kde.org>
2017-10-18 21:51:36 +02:00
Tomaz Canabrava
9318c487b9
[CSE] Display menu for encrytp / decryot folders
If the server supports client syde encryption, display
a menu on right click that should display encrypt and decrypt.
ideally it would show the encrypt if the folder is decrypted, and
decrypt if the folder is encrypted but currently there's no way
for the client to know that.
2017-10-18 21:51:36 +02:00
Tomaz Canabrava
1c32f6bb2e
[CSE] Check the server for client side encryption
Call the apps api and check if the response has
the client_side_encryption key.

Signed-off-by: Tomaz Canabrava <tcanabrava@kde.org>
2017-10-18 21:51:36 +02:00
Roeland Jago Douma
bc7e65b395
Merge branch 'master' of https://github.com/owncloud/client 2017-10-18 21:03:15 +02:00
Markus Goetz
9866010b4c SettingsDialogMac: Fix account display name #6078 2017-10-18 10:41:01 +02:00
Markus Goetz
a3c1052cae owncloudcmd: Fix timestamps, Fix --logdebug
We did not set a log handler so there were no timestamps.
The --debug didn't have an effect, let's use --logdebug like in GUI version.
(Command line always outputs some log)

Found in owncloud/documentation#3436
2017-10-18 09:16:05 +02:00
Christian Kamm
bf39343920 Sync: Add capability for invalid filename regexes #6092 2017-10-17 17:15:49 +02:00
Markus Goetz
d339b68715 Checksums: Use addData function
Our implementation had bad error handling.
This one now uses a new Qt5 addData function.

owncloud/enterprise#2252
2017-10-17 13:26:54 +02:00
Christian Kamm
17b1c83ae5 Activity: Allow sorting of issues and protocol #6086
The issues tab uses custom ordering where overall and summary sync
issues are displayed first. This ordering is preserved by creating
special sorting logic for the "time" column.

It needed special handling anyway, since sorting by time-string would
have yielded incorrect results.
2017-10-17 09:47:57 +02:00
Christian Kamm
b2a8ffc577 Abort: Fix crash with early aborts
_chunkCount could be 0, leading to a floating point exception

I also added initializers for several uninitialized integers in the
upload jobs.
2017-10-17 09:44:52 +02:00
Christian Kamm
e2711224ed Propagator: Avoid duplicate async abort 2017-10-17 09:44:52 +02:00
Christian Kamm
111bb485ec UploadNG: Avoid div-by-zero for super fast uploads 2017-10-17 09:44:52 +02:00
Piotr Mrowczynski
e10775d34f Fix paused sync file move issue #5949
Dont abort final chunk immedietally

Use sync and async aborts
2017-10-17 09:44:52 +02:00
Christian Kamm
f598ac89ac HttpCreds: Fix retry after wrong password #5989
This is an ugly solution.
2017-10-13 14:24:37 +02:00
Christian Kamm
3f7b3ca962 Checksums: Improve logging and add global disable #5017 2017-10-13 13:08:20 +02:00
Christian Kamm
01c2ffe2ae PropagateDownload: Read Content-md5 header #6088 2017-10-11 09:06:23 +02:00
Christian Kamm
92e90f9c55 Context menu: More detailed status messages
Previously it could only display synchronization progress or "up to
date". Now it also communicates the same overall state that the icon
shows.

See owncloud/enterprise#2134
2017-10-09 14:14:31 +02:00
Olivier Goffart
b8f7c6daae CMakeLists.txt: Put -pie as a linker flag were it belongs
Fixes warning introduced by PR #6040
2017-10-06 14:35:56 +02:00
Markus Goetz
cd07865da6 Settings Dialog: Fix display name compilation for macOS #6078 2017-10-06 13:34:40 +02:00
Thomas Müller
315e38e814 Use display-name from the ocs call in the settings dialog 2017-10-06 10:59:20 +02:00
Christian Kamm
19c6757a0f
Rename detection: File size must be equal
Comparison of file sizes for potential conflicts was added in
0eb9401c62, but did not extend to checking
the file size in case of potential local moves.

This commit adds this check and adds tests for various move+change
scenarios.
2017-10-05 22:01:42 +02:00
Christian Kamm
01bbbb7d08
Excludes: Fix bug with escapes in the regex, add tests 2017-10-05 22:01:42 +02:00
Christian Kamm
f609ea3bef
Excludes: optimize csync_is_windows_reserved_word 2017-10-05 22:01:42 +02:00
Markus Goetz
91223b1a3d
Excludes: During directory traversal, use QRegularExpression
On Mac, this halves the time spent in csync_excluded_traversal
when using check_csync_excluded_performance. A similar performance
increase is seen on linux.
2017-10-05 22:01:42 +02:00
Markus Goetz
f202593ce4
sqlite: Update bundled version to 3.20.1
For OS X and Windows.
2017-10-05 22:01:42 +02:00
rockihack
2bfd85e3c7
Linux Hardening
see: https://wiki.debian.org/Hardening#User_Space
2017-10-05 22:01:38 +02:00
Jürgen Weigert
09c3043c85
support old sqlite before 3.7.7
quickfix: define the symbol for backwards compat
or FIXME: better solution #ifdef SQLITE_IOERR_SHMMAP where needed...
2017-10-05 22:01:38 +02:00
Olivier Goffart
cd44d8ed77
Test OAuth2
Include a test for PR #6057
2017-10-05 22:01:38 +02:00
Olivier Goffart
cfc38753b4
SyncEngine: _hasNoneFiles should be set when there are INSTRUCTION_UPDATE_METADATA 2017-10-05 22:01:38 +02:00
Olivier Goffart
c424dec7d4
OAuth: fix compilation with old gcc
We need to used QPointer::data in the signal slot connection

Relates to pr #6065
2017-10-05 22:01:38 +02:00
Markus Goetz
36117336e7
Updater: Rudimentary support for beta channel
So we can get people update from our alpha to the beta and rc
2017-10-05 22:01:38 +02:00
Olivier Goffart
d59c146488
Fix encoding of version string 2017-10-05 22:01:38 +02:00
Olivier Goffart
6583ebdcd9
csync_rename: optimize lookup
Avoid many memory allocations in reconcile
2017-10-05 22:01:38 +02:00
Olivier Goffart
917e2e43c6
csync_reconcile: Optimize reconcile by avoiding alocations
Fix performence regression from commit d66c2b5fae
For every new file we would look up every parent directories. Allocating
a new QByteArreay for every parent riectory just to know if it is in the other
tree is wasting lots of CPU.
Use a ByteArrayRef trick, similar to QStringRef
2017-10-05 22:01:37 +02:00
Dominik Schmidt
bdd3dc743b
Set CMAKE_INSTALL_RPATH as RPATH for installed executables 2017-10-05 22:01:37 +02:00
Christian Kamm
96a6d64500
Packaging: Make check for theme.qrc more specific #5957
Extracted and adjusted from packaging.diff by @jnweiger and @dschmidt
2017-10-05 22:01:37 +02:00
Christian Kamm
f64519bfbd
Move the --version output text to Theme #5957
Extracted and adjusted from packaging.diff by @jnweiger and @dschmidt
2017-10-05 22:01:37 +02:00
Olivier Goffart
bbbd2319ed
Use QDateTime::currentDateTimeUtc instead of the non utc version
QDateTime::currentDateTime is terribly slow.
2017-10-05 22:01:37 +02:00
Olivier Goffart
79c775bcd8
OAuth: Fix crash when closing the browser while identifying
To reproduce, log in and click "authorize" on the browser, then close
the browser before the client has replied, (but after redirected to localhost,
i.e. when the client is asking the server for the token)

The problem is that socket can be destroyed so we don't need to answer on a
destroyed socket.
2017-10-05 22:01:37 +02:00
Olivier Goffart
a3b1597d17
OwnSQL: don't parse the error message when we are done 2017-10-05 22:01:37 +02:00
Jocelyn Turcotte
ecb4e96794
SyncJournalDb: Reimplement the db_is_empty logic from csync
This reduces the initial sync local discovery time from
2.0 to 0.6 seconds in LargeSyncBench on my machine.
2017-10-05 22:01:37 +02:00
Jocelyn Turcotte
185cb2e39a
Use SyncJournalDb in csync
This gets rid of the csync_statedb sqlite layer and use
the same code and same connection as the rest of the SyncEngine.

Missing functions are added to SyncJournalDb and change a few minor
things (like changing SyncJournalFileRecord::_modtime to be an int64
instead of a QDateTime, like it was in csync).
2017-10-05 22:01:37 +02:00
Jocelyn Turcotte
c6610f6fbf
SyncJournalDB: Allow callers of getFileRecord if the query failed
The current implementation would return the same value whether the query failed
or if no row would be found. This is something that is currently checked by csync
and needs to be provided if we want to use SyncJournalDB there.

Adjusted all call sites to also check the return value even though they
could still just rely on rec.isValid(), but makes it more explicit as to what
happens for database errors in those cases, if we ever want to gracefully handle
them.
2017-10-05 22:01:37 +02:00
Jocelyn Turcotte
d76ecf015c
SyncJournalDB: Bind phash as a qlonglong instead of a QString
This is what csync does and we don't need to pass it as a string
to let sqlite convert it back to a uint64 later.
2017-10-05 22:01:36 +02:00
Olivier Goffart
fd9662b803
bandwidthmanager: Fix unregistering devices on delete
from the destroyed signal, qobject_cast won't work because the object
is already destroyed. One must use reinterpret_cast then
2017-10-05 22:01:36 +02:00
Olivier Goffart
37810c0a19
PropagateUpload: Fix another non-normalized connect that appears while profiling 2017-10-05 22:01:36 +02:00
Olivier Goffart
d100f9ac67
c_utf8_from_locale: optimize the UTF8 case on Linux
Avoid converting to a QString as an intermediate step
2017-10-05 22:01:36 +02:00
Olivier Goffart
5fc864adb6
csync: use a unordered_map for the FileMap
Saves more than 10% of csync_update on LargeSyncBench
2017-10-05 22:01:36 +02:00
Olivier Goffart
38962ca694
csync_update: fix assert
Amend 1dc44c0f62
2017-10-05 22:01:36 +02:00