Commit graph

10548 commits

Author SHA1 Message Date
Jocelyn Turcotte
8884fe7236 Remove PropagateDirectory::_runningNow
This variable isn't used anymore except in a stray Q_ASSERT.
2017-01-26 17:13:54 +01:00
Jocelyn Turcotte
bb3efc5988 Clear SyncEngine::_remotePerms after the sync
Also make reuse the SyncFileItem::_remotePerm QByteArray allocation
instead of redoing the copy from the csync char*.
2017-01-26 17:13:54 +01:00
Jocelyn Turcotte
f6665ccc81 Pack boolean members in PropagateUploadFileCommon #5457 2017-01-26 17:13:54 +01:00
Jocelyn Turcotte
f1e9be4fa8 Don't use the StopWatch for each job in production builds #5456
This accounted for more than 10% of the peak memory usage while only
providing value through the debugging log.
2017-01-26 17:13:54 +01:00
Jocelyn Turcotte
9db23d4df1 Remove SyncFileItem::_requestDuration #5456
This will save a QElapsedTimer in each job that was used only for
.owncloudsync.log (for which the request duration doesn't bring much
value).
2017-01-26 17:13:54 +01:00
Jocelyn Turcotte
9d16ad844c Don't reallocate a QByteArray for each contentChecksumType call #5481 2017-01-26 17:13:54 +01:00
Jocelyn Turcotte
b0700ebbab Remove duplicate SyncFileItem entries for the log
The accuracy of that log isn't as important as the few bytes those
fields take as hostage for the whole sync.
2017-01-26 17:13:54 +01:00
Markus Goetz
2bda55be81 Don't create legacy 'folders/' subdirectory 2017-01-26 12:29:58 +01:00
Markus Goetz
e859d220be Cookies: Use different DB for different accounts (#5490)
This is a follow up to #5469
2017-01-26 10:54:03 +01:00
Piotr Mrówczyński
c3ae5123cb Delete finished propagation jobs in PropagateDirectory #5269 (#5400) 2017-01-26 10:03:22 +01:00
Jenkins for ownCloud
a1a5111a8a [tx-robot] updated from transifex 2017-01-26 02:18:40 +01:00
Jocelyn Turcotte
ee211d7609 Release SyncFileItem objects with their job
We now delete subjobs as their propagation is complete. This allows us
to also release the item by making sure that nothing else is holding a
reference to it.

Remove the stored SyncFileItemVector from SyncEngine and SyncResult
and instead gather the needed info progressively as each itemCompleted
signal is emitted.

This frees some holes on the heap as propagation goes, allowing many
memory allocations without the need of requesting more virtual memory
from the OS, preventing the memory usage from increasingly growing.
2017-01-25 23:26:23 +01:00
Jocelyn Turcotte
1fc5a76622 Pass the SyncFileItem as SyncFileItemPtr in itemCompleted
This will allow us to keep a reference on the items in connected slots.
2017-01-25 23:26:23 +01:00
Jocelyn Turcotte
a764d7eb86 Don't pass the PropagatorJob in itemCompleted
This was to catch duplicate emissions for PropagateDirectory but we
don't emit this signal anymore from there.
This fixes a warning about PropagatorJob not being a registered metatype.

This reverts commit fe42c1a818.
2017-01-25 23:26:23 +01:00
Jocelyn Turcotte
605a18ff73 Tests: Add a large sync benchmark
This simulates a ~50k files sync that can be used to measure memory
usage without having to wait for a server.
2017-01-25 23:26:23 +01:00
Jocelyn Turcotte
92e86641d1 Bring back the error message if the sync-exclude.lst file is missing
We unconditionnally add the file on startup, but don't check for errors.
During sync we check for errors, but only try loading the file if it
exists.
2017-01-25 23:26:23 +01:00
Markus Goetz
c8cfb3160e Merge remote-tracking branch 'origin/selective-sync-big-folder' into 2.3 2017-01-25 19:45:20 +01:00
Olivier Goffart
268fc97a71 Merge pull request #5480 from owncloud/chunking-ng-cleanup
Chunking ng cleanup
2017-01-25 19:19:48 +01:00
Jenkins for ownCloud
de2458d892 [tx-robot] updated from transifex 2017-01-25 02:18:34 +01:00
Matthew Setter
d75b838897 Add extra note about deleting .csync_journal.db
This is added as per the recommendation from @guruz in #5252.
2017-01-24 16:18:21 +01:00
Carla Schroder
4df7cab72a add note about deleting .csync_journal.db for "CSync unknown error" 2017-01-24 16:18:21 +01:00
Christian Kamm
59c1fdbe05 Shib: Use different keychain entry per account #5469
Previously shib multiaccount didn't work at all because the
session cookie was stored in the same keychain entry.
2017-01-24 13:14:11 +01:00
Christian Kamm
26234dbf6c Folders: Adjust ui when account is disconnected #5477
Previously if you paused/unpaused a folder for a disconnected account
they would prepare to sync and thus display the 'Waiting...' text. With
this change, folders that can't possibly sync don't show text like
this.

When account connectivity changes, all unpaused folders will be
scheduled anyway.
2017-01-24 10:26:03 +01:00
Jenkins for ownCloud
05a254b527 [tx-robot] updated from transifex 2017-01-24 02:18:28 +01:00
Jenkins for ownCloud
d282a8380e [tx-robot] updated from transifex 2017-01-23 11:11:23 +01:00
Jenkins for ownCloud
83dfbb933b [tx-robot] updated from transifex 2017-01-22 02:18:27 +01:00
Jenkins for ownCloud
100603fdc0 [tx-robot] updated from transifex 2017-01-21 02:18:28 +01:00
Olivier Goffart
d6fdda8efa ChunkingNG: add '0' padding on the filename (#5476)
The server sorts the chunk by name alphabetically. So if we want to keep
the chunk in order, we need to add a few '0' in front of the chunk name
2017-01-20 16:03:50 +01:00
Markus Goetz
faedaa5e09 Update issue_template.md 2017-01-20 16:00:50 +01:00
Olivier Goffart
8a70d22af7 ChunkingNG: Add some tests
- Test that we recover correctly if the chunks were removed on the server.
 - Test changing the file localy while uploading.
2017-01-20 14:48:53 +01:00
Olivier Goffart
a63d970e5e ChunkingNG: remove stale chunks when cleaning the uploadInfo table
Stale chunks might be there because a file was removed or would just not
be uploaded, for any reason.
We just start the DeleteJob but we don't care if it success or not.

Relates to https://github.com/owncloud/core/issues/26981

One of the test is testing the case where the file is modified on the server
during the upload. So this test the precondition failed error.
The FakeGetReply logic was modified because resizing a 150MB big QByteArray
by increment of 16k just did not scale when downloading a big file.
2017-01-20 14:48:53 +01:00
Olivier Goffart
f6c77fad17 ChunkingNG: delete stale chunks if the file was changed locally
Relates to https://github.com/owncloud/core/issues/26981

We do not track the success or error of the DeleteJob because it does not
matter. If it fails, it might be because the chunks were already removed.
If not, the chunks will be stale, but the server must anyway do a few
cleanup from time to time because we do not always remove the chunks
2017-01-20 14:48:53 +01:00
Jenkins for ownCloud
661c7f0558 [tx-robot] updated from transifex 2017-01-20 02:18:34 +01:00
Jenkins for ownCloud
a7efe144fc [tx-robot] updated from transifex 2017-01-19 02:18:40 +01:00
Jocelyn Turcotte
b68a28de8d StatusTracker: Emit OK for the last child before parents (#5467)
The current logic tried to avoid a DB lookup just to fetch whether
the file is shared or not since that info is already in the
SyncFileItem. The implementation would however need to decrease the
sync count for itself (and parents) before emitting the new status,
thus emitting the OK status for parents before that last child that
ended the propagation for that folder.

Change the implementation to achieve what we want: give the
possibility to decSyncCount to use a pre-fetched sharing state while
still doing the emission for all involved files. This ensures that
the leaf file also gets its status emitted before its parents.

Issue #4797
2017-01-18 15:40:52 +01:00
Samuel Alfageme
72a7b7ca42 Reference to build Qt5 with qtwebkit support in the docs #5129 (#5471)
This addresses the issue only on OS X. It should be stated though, that qtwebkit is not part of the qt5core lib and therefore a requirement for the projectto build in the rest of the OSS
2017-01-18 14:46:47 +01:00
Jocelyn Turcotte
166ef85a51 shell/Windows: Update binaries to build 44 2017-01-18 13:34:02 +01:00
Matthew Setter
1f60c61f87 Update the MacOS automatic update instructions
As requested in #5431, the automatic update instructions for MacOS X
have been simplified; removing the older, incorrect instructions.
2017-01-18 12:55:23 +01:00
Matthew Setter
30ef794fa1 Fix several typos in the client documentation 2017-01-18 12:53:33 +01:00
Jocelyn Turcotte
47fbfbc006 shell/Windows: Avoid memory allocations and copies when querying icons
IsMemberOf is called for every file (in the ownCloud directory or not) and
with every instance of OCOverlay (we have 5) when displaying a list of
files in Explorer.

Refactor the code to avoid copying the list of watched directories, as
well as creating a wstring from a PWWSTR for files outside watched
directories.

Also change some calls of begin_with to use isDescendantOf since it
properly handles parent paths not ending with a backslash, which could
lead to SocketAPI queries for sibling folders with a name that starts with
a watched folder name.
2017-01-18 12:15:52 +01:00
Jocelyn Turcotte
e131c142ff shell/Windows: Fix the view not being updated on StateError
Since StateError == 0, if this was the status used when the path
isn't in the map already, the view would not be updated since the
new state would be the same as the default-constructed state in the
map. Fix by explicitly inserting in that case, this also avoid aving
to do two lookups in the map when a path already has an entry.
2017-01-18 12:15:52 +01:00
Jocelyn Turcotte
1b99ff2e91 shell/Windows: Remove the UPDATE_VIEW message handling
The client now tries to only push STATUS messages to connections
unless they previously requested it with a RETRIEVE_FILE_STATUS,
but this means that we now have to make sure that a new connection
will rerequest every icon that the user will see on files from that
point on.

To that end, we now send a SHChangeNotify for every file in the cache
when we lose the connection (or on UNREGISTER_PATH) to make sure that
what's on the screen matches what's in the cache. We also remove the
_oldCache logic that made this more difficult to enforce.

The client has been able to reliably push status updates on macOS
for a few versions now, and we don't need it on Windows either. The
_oldCache mechanism was to avoid sending to many update requests
when receiving an UPDATE_VIEW.

Also fixes #4766
2017-01-18 12:15:52 +01:00
Jocelyn Turcotte
3e85d47a57 shell/Windows: Convert tabs to space
It was a mix of both and the rest of the code uses spaces.
2017-01-18 12:15:52 +01:00
Jocelyn Turcotte
e709d75d56 shell/Windows: Remove unused files
The real OCContextMenu.cpp is in the OCContextMenu folder.
2017-01-18 12:15:52 +01:00
Jocelyn Turcotte
cb5cfb8cf6 shell/macOS: Make sure the root folder's cache is cleared on disconnect
The SocketAPI now only sends status pushes for paths requested by the
shell. We have to make sure that Finder doesn't just show the entry
from its cache after a reconnect.
2017-01-18 12:15:52 +01:00
Jocelyn Turcotte
ebd2a15711 shell/macOS: Remove backslashes from the extension display name
Tell cmake not to escape shell strings adding backslashes before
spaces by passing the VERBATIM optiton. Also add the quotes to
whole command line arguments to prevent quotes from appearing on the
XCode side, because of the VERBATIM option.

This only affects themed clients since the default theme doesn't
have space in its APPLICATION_NAME.
2017-01-18 12:15:52 +01:00
Jocelyn Turcotte
d3a0608bd5 SocketAPI: Only push status for files requested by the shell #5361
We currently push the SYNC status for all files that will be propagated,
and then the OK status when those files are propagated.
On top of this, we send those statuses to all clients connected, even
if the socket is kept open by an application that only needed to show
a file open dialog. On macOS we're also using an NSConnection which
means that we have to wait for the RPC call to return from the
extension, which makes bulk status changes possibly heavy.

Reduce the time spent needlessly sending status pushes by limiting
them to files requested through that socket since it connected.
To limit the data to store, only remember the parent directory of
files requested, and store those in a bloom filter.

Note that this adds a requirement to shell extensions: they should
make sure that the status cache only contains entries that have been
requested through the socket API. In other words, the status cache
must be empty when each socket client connects to the socket API.
Otherwise the cached icon type will be shown to the user, and the
SocketAPI won't push new status for that file if it didn't receive
a RETRIEVE_FILE_STATUS.
2017-01-18 12:15:52 +01:00
Jocelyn Turcotte
c25b599bbb SocketApi::slotReadSocket: Small optimizations
- Use the looked-up method index also for the invocation
- Do the method name concatenation already on QByteArray since we'll
  convert anyway
- Use staticMetaObject instead of metaObject()
2017-01-18 12:15:52 +01:00
Jocelyn Turcotte
a5ff9e58e3 SocketAPI: Remove unecessary null-checks
Those methods are private and there is only one call-site, we can
be sure that this parameter won't be null already.
2017-01-18 12:15:52 +01:00
Olivier Goffart
9b96899d75 Sync Engine: Specify what chars are not syncable in the error message #1733 (#5449) 2017-01-18 11:45:46 +01:00