Commit graph

4875 commits

Author SHA1 Message Date
Christian Kamm
4c1fdf1dee Double check usage of asserts #5429
A few are supposed to be fatal.
2017-02-08 15:15:19 +01:00
Christian Kamm
18e25122db Performance: reserve() _remotePerm #5496 2017-02-08 15:06:09 +01:00
Olivier Goffart
439e688906 Folder: Change the wording for the massive deletion message
Issue #5503
2017-02-08 14:30:14 +01:00
ckamm
4198d9f420 Settings: Don't migrate settings on access error #5499 (#5523)
Previously, we'd try migrating from legacy settings if reading
the settings failed with an error. Now, we try again after a
couple of seconds and eventually give up.
2017-02-08 14:28:50 +01:00
Olivier Goffart
06579a5c70 AccountManager: Fix small leak in the account migration code
The QSettings object was not deleted in every possible code paths
2017-02-08 12:31:07 +01:00
Markus Goetz
df773ea8bb CentOS: More build fixes 2017-02-08 11:39:28 +01:00
Markus Goetz
e7be4faac8 CentOS: Another attempt 2017-02-08 11:15:23 +01:00
Markus Goetz
1102ac20ac Improve previous fix
After comment on 78a798eef3
2017-02-07 19:40:32 +01:00
Markus Goetz
6a4ae63f14 CentOS: Another fix for old compiler version (2) 2017-02-07 19:22:04 +01:00
Markus Goetz
78a798eef3 Old Qt: Another build fix 2017-02-07 18:54:40 +01:00
Markus Goetz
19dd656989 CentOS: Another fix for old compiler version 2017-02-07 18:32:34 +01:00
Markus Goetz
d9fac50e9b CentOS: Fix build for old compiler version 2017-02-07 18:05:29 +01:00
Markus Goetz
9614a94035 Revert "Disable RTTI"
This reverts commit 85d3de1589.
This breaks our old Qt on CentOS.

We can re-introduce this with #5470
2017-02-07 17:35:04 +01:00
Olivier Goffart
0d16cf41fe Merge remote-tracking branch 'origin/master' into 2.3
We can do that because the only changes that were in master but not in 2.3 were the
translations change and documentation change, and the support for the 'M' permission
which we want in 2.3.
2017-01-27 17:45:05 +01:00
Olivier Goffart
65e4afedc4 Account Wizard: don't whitelist everything on the first sync if we still want to confirm big folder
Issue: https://github.com/owncloud/client/pull/5340#issuecomment-274564441
2017-01-27 15:59:59 +01:00
Olivier Goffart
b76a9654cc General Settings: Make sure to reload the settings when an account is added
The Size limit, or confirmation checkboxes might have changed.

We need to guard against saving if the control changes while we are loading

Issue: https://github.com/owncloud/client/pull/5340#issuecomment-274878023
2017-01-27 15:59:59 +01:00
Olivier Goffart
38cf459b3e Confirm External Storage: adjust the notification message
We need to forward the information that the folder is an external storage
for the notification message.

Issue: https://github.com/owncloud/client/pull/5340#issuecomment-274878023
2017-01-27 15:59:59 +01:00
Olivier Goffart
ed7416098e Wizard: Increase the size
The advanced page has become quite complex and does not fit on the screen
anymore if the fonts are too big
2017-01-27 15:59:59 +01:00
Olivier Goffart
517623e457 FolderStatusModel: Different icon for external storages
Issue: https://github.com/owncloud/client/pull/5340#issuecomment-274564441
2017-01-27 15:59:59 +01:00
Olivier Goffart
f89bc09fd1 Discovery: Filter 'M' out of permissions for non root storage
The sync engine rely on the 'M' in premission to ask for confirmation
(As requested in issue #5340)
But we only want to ask the premission for the 'root' of the mounting point and not
for every subfolders within it.
So we change the discovery phase in a way that it does not keep the 'M' for
children within the external storage.
2017-01-27 15:59:59 +01:00
Olivier Goffart
f854c5263b Wizard: Add options to ask confirmation for external storage
Added two checkboxes in the Account Wizard in the advanced page to change the first options.
Also added a checkbox in the general settings to ask for confirmation for external storages.

Theme options allow to hide the checkboxes in the wizard.

As described in issue #5340
2017-01-27 15:59:59 +01:00
Olivier Goffart
70da607e06 Discovery: Add branding option to disable default sync of 'M' directories
Issue #5331 and https://github.com/owncloud/enterprise/issues/1594
2017-01-27 15:59:59 +01:00
Jocelyn Turcotte
8884fe7236 Remove PropagateDirectory::_runningNow
This variable isn't used anymore except in a stray Q_ASSERT.
2017-01-26 17:13:54 +01:00
Jocelyn Turcotte
bb3efc5988 Clear SyncEngine::_remotePerms after the sync
Also make reuse the SyncFileItem::_remotePerm QByteArray allocation
instead of redoing the copy from the csync char*.
2017-01-26 17:13:54 +01:00
Jocelyn Turcotte
f6665ccc81 Pack boolean members in PropagateUploadFileCommon #5457 2017-01-26 17:13:54 +01:00
Jocelyn Turcotte
f1e9be4fa8 Don't use the StopWatch for each job in production builds #5456
This accounted for more than 10% of the peak memory usage while only
providing value through the debugging log.
2017-01-26 17:13:54 +01:00
Jocelyn Turcotte
9db23d4df1 Remove SyncFileItem::_requestDuration #5456
This will save a QElapsedTimer in each job that was used only for
.owncloudsync.log (for which the request duration doesn't bring much
value).
2017-01-26 17:13:54 +01:00
Jocelyn Turcotte
9d16ad844c Don't reallocate a QByteArray for each contentChecksumType call #5481 2017-01-26 17:13:54 +01:00
Jocelyn Turcotte
b0700ebbab Remove duplicate SyncFileItem entries for the log
The accuracy of that log isn't as important as the few bytes those
fields take as hostage for the whole sync.
2017-01-26 17:13:54 +01:00
Markus Goetz
2bda55be81 Don't create legacy 'folders/' subdirectory 2017-01-26 12:29:58 +01:00
Markus Goetz
e859d220be Cookies: Use different DB for different accounts (#5490)
This is a follow up to #5469
2017-01-26 10:54:03 +01:00
Piotr Mrówczyński
c3ae5123cb Delete finished propagation jobs in PropagateDirectory #5269 (#5400) 2017-01-26 10:03:22 +01:00
Jocelyn Turcotte
ee211d7609 Release SyncFileItem objects with their job
We now delete subjobs as their propagation is complete. This allows us
to also release the item by making sure that nothing else is holding a
reference to it.

Remove the stored SyncFileItemVector from SyncEngine and SyncResult
and instead gather the needed info progressively as each itemCompleted
signal is emitted.

This frees some holes on the heap as propagation goes, allowing many
memory allocations without the need of requesting more virtual memory
from the OS, preventing the memory usage from increasingly growing.
2017-01-25 23:26:23 +01:00
Jocelyn Turcotte
1fc5a76622 Pass the SyncFileItem as SyncFileItemPtr in itemCompleted
This will allow us to keep a reference on the items in connected slots.
2017-01-25 23:26:23 +01:00
Jocelyn Turcotte
a764d7eb86 Don't pass the PropagatorJob in itemCompleted
This was to catch duplicate emissions for PropagateDirectory but we
don't emit this signal anymore from there.
This fixes a warning about PropagatorJob not being a registered metatype.

This reverts commit fe42c1a818.
2017-01-25 23:26:23 +01:00
Jocelyn Turcotte
92e86641d1 Bring back the error message if the sync-exclude.lst file is missing
We unconditionnally add the file on startup, but don't check for errors.
During sync we check for errors, but only try loading the file if it
exists.
2017-01-25 23:26:23 +01:00
Markus Goetz
c8cfb3160e Merge remote-tracking branch 'origin/selective-sync-big-folder' into 2.3 2017-01-25 19:45:20 +01:00
Olivier Goffart
268fc97a71 Merge pull request #5480 from owncloud/chunking-ng-cleanup
Chunking ng cleanup
2017-01-25 19:19:48 +01:00
Christian Kamm
59c1fdbe05 Shib: Use different keychain entry per account #5469
Previously shib multiaccount didn't work at all because the
session cookie was stored in the same keychain entry.
2017-01-24 13:14:11 +01:00
Christian Kamm
26234dbf6c Folders: Adjust ui when account is disconnected #5477
Previously if you paused/unpaused a folder for a disconnected account
they would prepare to sync and thus display the 'Waiting...' text. With
this change, folders that can't possibly sync don't show text like
this.

When account connectivity changes, all unpaused folders will be
scheduled anyway.
2017-01-24 10:26:03 +01:00
Olivier Goffart
d6fdda8efa ChunkingNG: add '0' padding on the filename (#5476)
The server sorts the chunk by name alphabetically. So if we want to keep
the chunk in order, we need to add a few '0' in front of the chunk name
2017-01-20 16:03:50 +01:00
Olivier Goffart
a63d970e5e ChunkingNG: remove stale chunks when cleaning the uploadInfo table
Stale chunks might be there because a file was removed or would just not
be uploaded, for any reason.
We just start the DeleteJob but we don't care if it success or not.

Relates to https://github.com/owncloud/core/issues/26981

One of the test is testing the case where the file is modified on the server
during the upload. So this test the precondition failed error.
The FakeGetReply logic was modified because resizing a 150MB big QByteArray
by increment of 16k just did not scale when downloading a big file.
2017-01-20 14:48:53 +01:00
Olivier Goffart
f6c77fad17 ChunkingNG: delete stale chunks if the file was changed locally
Relates to https://github.com/owncloud/core/issues/26981

We do not track the success or error of the DeleteJob because it does not
matter. If it fails, it might be because the chunks were already removed.
If not, the chunks will be stale, but the server must anyway do a few
cleanup from time to time because we do not always remove the chunks
2017-01-20 14:48:53 +01:00
Jocelyn Turcotte
b68a28de8d StatusTracker: Emit OK for the last child before parents (#5467)
The current logic tried to avoid a DB lookup just to fetch whether
the file is shared or not since that info is already in the
SyncFileItem. The implementation would however need to decrease the
sync count for itself (and parents) before emitting the new status,
thus emitting the OK status for parents before that last child that
ended the propagation for that folder.

Change the implementation to achieve what we want: give the
possibility to decSyncCount to use a pre-fetched sharing state while
still doing the emission for all involved files. This ensures that
the leaf file also gets its status emitted before its parents.

Issue #4797
2017-01-18 15:40:52 +01:00
Jocelyn Turcotte
d3a0608bd5 SocketAPI: Only push status for files requested by the shell #5361
We currently push the SYNC status for all files that will be propagated,
and then the OK status when those files are propagated.
On top of this, we send those statuses to all clients connected, even
if the socket is kept open by an application that only needed to show
a file open dialog. On macOS we're also using an NSConnection which
means that we have to wait for the RPC call to return from the
extension, which makes bulk status changes possibly heavy.

Reduce the time spent needlessly sending status pushes by limiting
them to files requested through that socket since it connected.
To limit the data to store, only remember the parent directory of
files requested, and store those in a bloom filter.

Note that this adds a requirement to shell extensions: they should
make sure that the status cache only contains entries that have been
requested through the socket API. In other words, the status cache
must be empty when each socket client connects to the socket API.
Otherwise the cached icon type will be shown to the user, and the
SocketAPI won't push new status for that file if it didn't receive
a RETRIEVE_FILE_STATUS.
2017-01-18 12:15:52 +01:00
Jocelyn Turcotte
c25b599bbb SocketApi::slotReadSocket: Small optimizations
- Use the looked-up method index also for the invocation
- Do the method name concatenation already on QByteArray since we'll
  convert anyway
- Use staticMetaObject instead of metaObject()
2017-01-18 12:15:52 +01:00
Jocelyn Turcotte
a5ff9e58e3 SocketAPI: Remove unecessary null-checks
Those methods are private and there is only one call-site, we can
be sure that this parameter won't be null already.
2017-01-18 12:15:52 +01:00
Olivier Goffart
9b96899d75 Sync Engine: Specify what chars are not syncable in the error message #1733 (#5449) 2017-01-18 11:45:46 +01:00
Olivier Goffart
b046cca010 Fix more compilation errors with Qt 4
Ammand last commit which was a fixup to the change from pull request #5468
for issue #5453
2017-01-18 10:19:51 +01:00
Olivier Goffart
edf0a99a8a Fix compilation with Qt 4
Fixup to the change from pull request #5468 for issue #5453
2017-01-18 09:57:36 +01:00
Christian Kamm
352f168313 Folder: Remove unused variable 2017-01-17 14:53:09 +01:00
Markus Goetz
85d3de1589 Disable RTTI
Shrinks owncloud binary by 24 KB and libowncloudsync by 14 KB.
I don't know if it has influence on memory usage or runtime speed though.

Was worth a try.
2017-01-17 14:52:10 +01:00
Markus Goetz
588a88fb63 Updater: Don't create instance for invalid URL
Also don't use dynamic_cast
For enterprise/#689
2017-01-17 14:52:10 +01:00
Christian Kamm
cd9e88ad22 PropagatorJob: Remove unneccessary pointer #5453 2017-01-17 14:47:38 +01:00
Christian Kamm
29b39acfbe PropagatorJob: Access propagator through function #5453 2017-01-17 14:47:38 +01:00
Markus Goetz
9d7425b201 DB: Also migrate -shm and -wal to new name #3764 #5045 (#5459) 2017-01-16 15:42:11 +01:00
Markus Goetz
7eb43d7f7e PropagateUploadFileNG: Delete unused map contents to free memory 2017-01-13 19:00:04 +01:00
ckamm
d76e0ec6d8 Reset stuck chunked uploads eventually #5344 (#5443)
Previously this wasn't happening for errors that were not
NormalErrors because they don't end up in the blacklist.

This revises the resetting logic to be independent of the
error blacklist and make use of UploadInfo::errorCount
instead.

412 errors should reset chunked uploads because they might be
indicative of a checksum error.

Additionally, server bugs might require that additional
errors cause an upload reset. To allow that, a new capability
is added that can be used to advise the client about this.
2017-01-13 14:44:45 +01:00
Olivier Goffart
0249a68420 SelectiveSync: adjust the position of the "Loading" label
It should be parented to the treeview to be sure that is shows inside of it.
2017-01-13 10:55:45 +01:00
Christian Kamm
aa6f041c36 wizardSelectiveSyncDefaultNothing applies to folder wizard #4581
Previous the Theme option only applied to the account setup wizard.
2017-01-13 10:52:39 +01:00
Christian Kamm
dce3f8c4f6 Selective sync: Adjust widget common to folder/account wizards 2017-01-13 10:52:10 +01:00
Christian Kamm
ffbf34cb97 Log the auth type being used for accounts
There have been hard to track down bugs related to the wrong auth
type being used.
2017-01-11 09:30:59 +01:00
Markus Goetz
76fde49282 sqlite: Update bundled version to 3.16.1
For OS X and Windows.
2017-01-05 17:27:46 +01:00
Christian Kamm
adbe5ecf55 ownsql: qFatal on prepare errors #5357 2017-01-04 15:15:06 +01:00
Christian Kamm
084146756b Account: Fix displayName() #5413
This displayName() seemed to be based on Account::user() which used
to call _credentials->user(). But then we repurposed user() to be
davUser() and this usage wasn't updated to point back to the username
used for the credentials.
2017-01-03 13:54:59 +01:00
ckamm
e1f5a49c21 Don't blacklist 507 Insufficent Storage #5346 (#5424) 2017-01-03 13:48:32 +01:00
Christian Kamm
e286bb1b64 Wizard: Don't accidentally pre-fill url field #5412
slotUrlEditFinished() is called when the wizard is triggered in
some situations.
2017-01-03 11:53:40 +01:00
Olivier Goffart
d433f0e08e HTTPCreds: remove all cookies when logging out (#5383)
Some custom server use persistent cookies with the auth token. So we should
clear all the cookies when disconnecting.
Account::clearCookieJar is only called from the HTTPCredentials. This funciton
is not used for shibboleth.
There is probably no reasons to keep the HTTP cookie anyway.

Issue #5370
2017-01-03 11:39:10 +01:00
Christian Kamm
fa9b36f829 Wizard: Let the user know if url is invalid #5398 2017-01-03 11:28:56 +01:00
Christian Kamm
6835429a28 Sql: Allow prepare fail for PRAGMA quick_check #5357
This actually happens in low-disk situations.
2017-01-03 10:43:29 +01:00
Christian Kamm
22135f9f57 Fixup broken authType on load #5408 2017-01-03 10:38:20 +01:00
Christian Kamm
473dcb0947 Only create a FolderWatcher if the path exists
Also retry creating it before each sync.

See #5317
2017-01-02 08:52:26 +01:00
Christian Kamm
e306f4611c Reschedule a folder regularly with some delay for some errors
Like "folder doesn't exist" - such that we will detect when the folder
becomes available and start syncing.

See #5317
2017-01-02 08:52:26 +01:00
Markus Goetz
c6f4f44619 Fix up SSL client certificates #5213 #69 (#5289)
The re-enables the UI, uses Qt API for importing and
stores the certificate/key in the system keychain.
People who had set up client certs need to re-setup the account. This is ok
since it was an undocumented feature anyway.
2017-01-02 08:34:02 +01:00
Christian Kamm
3b7887ca35 Wizard: Detach saved account from wizard
Fixes #5408 #5407.

The problem was that cleanup of the credentials page set the
credentials of the account back to dummy, thereby overriding
things like shib usernames.

This should be broken since a932eac832.
2016-12-21 15:31:12 +01:00
Piotr Mrówczyński
d781e63fab Add capability to disable parallel chunked upload #5364 - technical review (#5403) 2016-12-21 15:08:45 +01:00
Christian Kamm
030e3a5d4a Increase initial bandwidth estimates #4428 #5390
To 2 MB/s and 10 files/s.
2016-12-21 08:53:47 +01:00
Christian Kamm
3912dba33a Move qDeclareMetatype from Folder to SyncEngine 2016-12-20 11:43:39 +01:00
Christian Kamm
d8c479ab1e Don't wipe db if check fails while disk space is low #5357
An experimental fix for the issue, merged because it's low risk and
makes testing easier.
2016-12-20 09:33:59 +01:00
Olivier Goffart
775a1c9ad8 FolderStatusModel: Fix insert/remove items when there are labels
Otherwise it might happen that the model is inconsistant and this can
lead to crash in the worst case.

(For example, if there was a "fetching" label, and we hide it because it
was a 404. In this case, we would not call begin/endRemoveRows, so the
view could still call the model with an index of row 0, that used to be
for the label, but now correspond to the first element of _subs. And
because _subs is empty, this could lead to crashes)
2016-12-14 14:13:47 +01:00
Olivier Goffart
ac95844ebd Propagator: make sure every network job has a parent
This could make sure that the network job gets deleted if the parent job gets
deleted, and would avoid crashes like:

Crash: EXCEPTION_ACCESS_VIOLATION_READ at 0xffffffff8b008a04
  File "qiodevice.cpp", line 1617, in QIODevice::errorString
  File "propagatedownload.cpp", line 264, in OCC::GETFileJob::slotReadyRead
  File "moc_propagatedownload.cpp", line 85, in OCC::GETFileJob::qt_static_metacall
  File "qobject.cpp", line 3716, in QMetaObject::activate
  File "moc_qiodevice.cpp", line 154, in QIODevice::readyRead
  File "qnetworkreplyhttpimpl.cpp", line 1045, in QNetworkReplyHttpImplPrivate::replyDownloadData

(#5329)
2016-12-14 14:13:47 +01:00
Olivier Goffart
c5b90d9507 Account::slotHandleSslErrors: add guards in case the reply gets deleted in the dialog event loop
Since slotHandleSslErrors will show a dialog with an eventloop, it could be
That the reply gets deleted. Guard against that.

(#5329)
2016-12-14 14:13:47 +01:00
Christian Kamm
d631f2e070 Sync now: Disable on disconnect, rename while running #2158 2016-12-13 15:28:52 +01:00
Jocelyn Turcotte
f985111b62 Fix the log window not showing csync logs
The csync log level was only set up on startup, and for log files.
Fix the issue by making Logger::isNoop rely on being explicitly activated
for the log window instead of relying on the presence of a connected
signal, and move the csync log level logic in Logger.
2016-12-09 10:03:46 +01:00
Jocelyn Turcotte
e8d734b1c2 Fix the sync direction being wrong on MSVC
The compiler seems to use signed enums and we need to reserve an extra
bit for the sign to avoid the 2 value to overflow and being interpreted
as -2 when read, and thus not being correctly compared to the full enum
value.
2016-12-09 10:03:46 +01:00
ckamm
5bef1aa402 Merge pull request #5045 from owncloud/dbjournal_per_account
Allow a folder to be synced to several accounts. This changes the path of the sync journal file!
2016-12-06 10:55:58 +01:00
Olivier Goffart
4ac2b764a4 libsync: Clear the upload info after several retries (#5345)
Issue #5344
2016-12-05 17:21:36 +01:00
Olivier Goffart
19d2677db1 libsync: use the new webdav url if the server reports it
The rules to select the webdav url are now:

 - If the server reports that the new chunking algorithm is working,
   always use remote.php/dav/files/<username>
   This capability can be overriden with an environment variable

 - Otherwise, use the dav path provided by the theme, which defaults to
   remote.php/webdav

This means that with the newer server, the branding can no longer override
the webdav URL. If there is still an usecase for the branding to do so, we
need to find another way to override it. But it is now more complicated to
configure as might need include the username and need different endpoint
depending on the operations (chunks or not)

Issue #4007
2016-11-29 16:16:20 +01:00
Christian Kamm
a139d1a279 FolderWizard: pass AccountPtr by const&
For review of #5045
2016-11-29 10:39:20 +01:00
Christian Kamm
acb155503b Ensure the 'Sync paused' state is used after pausing #5290 2016-11-29 09:56:45 +01:00
Olivier Goffart
109056e676 owncloudcmd: Don't put the password in the URL
As the URL might be print on the logs.
Also don't change the scheme from http to owncloud.

This was required before when we were using neon through csync, but now that
we use QNAM for everything we don't need it. The credentials from the account
are used.
2016-11-26 12:57:44 +01:00
Christian Kamm
68f99bcc27 AccountSettings: Add a "Force sync now" context menu option 2016-11-25 14:23:56 +01:00
Christian Kamm
045498c71a Account: Rename user() to davUser()
Avoids potential confusion with credentials->user()
2016-11-23 17:08:17 +01:00
Christian Kamm
49f8143f00 Bugfixes for sync journal name generation and usage
* Use 'user' value for journal name generation
* Save journal name in settings
* Make owncloudcmd choose the right db
2016-11-23 16:48:15 +01:00
Christian Kamm
bea7241910 Don't wipe each journal on each start 2016-11-23 16:48:15 +01:00
Christian Kamm
61b4da944c Split folders configuration locations for backwards compatibility 2016-11-23 16:48:15 +01:00
Christian Kamm
6fe1868693 Always migrate .csync_journal to ._sync_xxx 2016-11-23 16:48:15 +01:00
Christian Kamm
9641c7a1e7 Rename sync journal to ._sync_xxx.db.
The added underscore means that older clients will also ignore the
file.
2016-11-23 16:48:15 +01:00
Christian Kamm
3e59a9b316 Merge branch 'master' into dbjournal_per_account 2016-11-23 16:47:56 +01:00
Markus Goetz
679ac0d26a Wizard: Show if server asks for client certificate #2799 #69 (#5261)
This also nicely displays the 'Untrusted domain' message of oC.
The link to add a trusted domain (via web browser) is clickable.
2016-11-23 11:43:15 +01:00