Commit graph

1024 commits

Author SHA1 Message Date
Piotr M
34c59ba9ed reduce number of loops 2016-10-31 19:45:24 +01:00
Olivier Goffart
c8014a0afd ChunkingNG: Add Test 2016-10-31 15:16:53 +01:00
Olivier Goffart
456d82715e Fix compile after merge 2016-10-31 11:29:33 +01:00
Olivier Goffart
8ca3eb7883 Merge remote-tracking branch 'origin/master' into chunking-ng 2016-10-31 11:09:12 +01:00
Christian Kamm
5f47c01346 Account: Rearrange for readability 2016-10-25 13:20:23 +02:00
Christian Kamm
10644d3568 Move concatUrl and settingsWithGroup to Utility
There was little reason to keep them cluttering Account.
2016-10-25 12:05:28 +02:00
Christian Kamm
9ee3144358 Account: Remove wasMigrated/setMigrated
It was unused since early in the multi-account work:
a932eac832
2016-10-25 11:43:06 +02:00
Christian Kamm
cf48ea2e00 Remove unused functions
Account::changed and AbstractCredentials::changed have not been needed
in a long while.
2016-10-25 11:33:38 +02:00
Christian Kamm
db24f60ae3 License: Adjust license of GPLv2 source files to GPLv2+
See #5180
2016-10-25 11:06:54 +02:00
Olivier Goffart
e33b89c222 Chunking-NG: Enable if the server supports it 2016-10-21 16:42:27 +02:00
Olivier Goffart
273590fdfc ChunkingNG: Use the 'If' header
As discussed in https://github.com/owncloud/core/pull/26368
2016-10-20 11:16:06 +02:00
Olivier Goffart
0960058842 Merge remote-tracking branch 'origin/master' into chunking-ng 2016-10-20 09:25:03 +02:00
Christian Kamm
69a1e46d0c Progress: Reset lastCompletedItem #5256
Otherwise progress listeners think it's still the last-completed
item when the next sync starts. This lead to spurious entries in
the "Recent Changes" list.
2016-10-18 12:29:52 +02:00
Christian Kamm
e020a5327e Force a remote discovery when upgrading to this version #5242
A remote discovery will fix up any lingering problems caused by #5190.
2016-10-18 11:21:01 +02:00
Olivier Goffart
36a19703db SyncEngine: Permission error in subfolder of a folder that cannnot be added is a SoftError
Issue #5059
2016-10-17 12:22:04 +02:00
Markus Goetz
d600fdc89b OS X: Use monochrome tray icon by default (#5229) 2016-10-14 14:26:31 +02:00
Markus Goetz
8c5ea8dc90 Propagator: Also upload more in parallel (#5230)
Fix for https://github.com/owncloud/client/issues/4986#issuecomment-227071801

This is before we have bundling at some point.
2016-10-14 14:23:55 +02:00
Klaas Freitag
27d23edacc Utility: Add a function to check if two filenames are equal plus test.
It calls canonical path always and works with the correct case preserving
depending on the platform.
2016-10-12 14:48:00 +02:00
Olivier Goffart
d80d5a8ee4 SyncEngine: Fix renaming a folder should keep the selective sync state
Issue #5224

Two problems:

 - In the discovery phase, we need to check the selective sync entries of
   the source path in case of renames.

 - When the rename is done, we need to actually update the black list in the
   database.
2016-10-11 14:38:25 +02:00
Olivier Goffart
b6079bfe4f Remove usused variable 2016-10-11 12:12:28 +02:00
Olivier Goffart
f75106fd8e SyncJournalDB: check the return value of the prepare statements in checkConnect
Relates to #5199
2016-10-11 11:35:26 +02:00
Olivier Goffart
57fe7b800f Theme: Fix compiler warning
Relates #5227
2016-10-11 10:31:47 +02:00
Klaas Freitag
e1a48e3c33 Move the journal file name generation to the syncjournaldb class.
As requested by Olivier.
2016-10-10 16:59:17 +02:00
Olivier Goffart
86522cbbf0 OwnSQL: avoid crashing in SqlQuery::reset_and_clear_bindings
The crash reporter shows a lot of crashes in sqlite3_clear_bindings
which seems to indicate that _stmt is null. We should guard against
a null value in order to avoid crashing.

This should only happen if the prepare call fails. We don't usually
check the return value of the prepare call, but if _stmt is null, the
exec call should return false, not true. We check the result of the
exec call, so this should then abort the sync with an error, rather
than crashing.
2016-10-10 13:27:12 +02:00
Tommy Sparber
67ecca492b Improve macOS SysTray/StatusBar monochrome icon (#5227)
- Use a white icon if the context menu is visible.
- Enable `QIcon::setIsMask` if compiled on Qt >= 5.6 to allow automatic
  macOS color handling.
- No changes if the colored icons are used.
2016-10-06 17:18:51 +02:00
Markus Goetz
84c925dc58 503: Also parse other server error message (#5228)
From https://github.com/owncloud/core/pull/26145
2016-10-06 14:45:04 +02:00
Klaas Freitag
ed6a708460 SyncJournalDb: Migrate the old csync journal to the new name.
Migration strategy to the new filename.
2016-09-29 16:41:11 +02:00
Christian Kamm
598a7cec5d Merge remote-tracking branch 'origin/2.2' 2016-09-28 12:07:45 +02:00
Markus Goetz
95c479aa07 Wizard: Do the system proxy lookup in a thread (#5160)
This is analogous to the code in ConnectionValidator.
2016-09-26 15:45:30 +02:00
Olivier Goffart
bc04f79959 Missing Inc -> GmbH (#5181) 2016-09-22 15:32:12 +02:00
Olivier Goffart
85b8ab178e SyncEngine: Fix renaming of folder when file are changed (#5195)
Two bugs:
 - The change filed are not considered as move, they are re-downloaded
   but the old file was not removed from the database. The change in
   owncloudpropagator.cpp takes care of removing the old entries.

 - Next sync would then remove the file in the server in the old folder
   This was not a problem until we start reusing the sync engine, and
   that the _renamedFolders map is not cleared. We were before deleting
   a non-existing file. But now we delete the actual file.

Also improve the tests to be able to do move on the server.
This include support for file id.

Issue #5192
2016-09-22 09:02:47 +02:00
Olivier Goffart
98bfa8d5de Propagator: properly send the success flag (#5189)
In case of the root directory, it may happen that the _item
is empty and the _item->_status is NoStatus.  But we still need to report
the proper success or error of the whole propagation. We should really
use _hasError for that. However, _hasError is also defined to NoStatus
if there was no error, so in that case we need to set Success.

This fixes the problem in which the data-fingerprint is not saved on the
database because the SyncEngine think that the sync failed. (Issue #5185)
2016-09-21 16:35:42 +02:00
Olivier Goffart
ff701bd473 ConnectionValidator: properly handle error in status.php (#5188)
We wwer enot connecting to the right signal from the check server
job, and therefore we were not catching the condition in which the
json was invalid. We would then never terminate the ConnectionValidator job.

Note that instanceNotFound is also emited if there is a network error.

The log looked like this:

10:25:51.247 OCC::CheckServerJob::finished: status.php from server is not valid JSON!
10:25:51.248 OCC::CheckServerJob::finished: status.php returns:  QMap()   QNetworkReply::NetworkError(NoError)  Reply:  QNetworkReplyHttpImpl(0x2b6a790)
10:25:51.248 OCC::CheckServerJob::finished: No proper answer on  QUrl("http://localhost/~owncloud/status.php")
10:26:23.235 OCC::AccountState::checkConnectivity: ConnectionValidator already running, ignoring "owncloud@localhost"
10:26:55.235 OCC::AccountState::checkConnectivity: ConnectionValidator already running, ignoring "owncloud@localhost"
[...]
2016-09-20 11:55:43 +02:00
Jocelyn Turcotte
2b274e9460 Fix the MSVC build
- Replace functions that are provided by MinGW with a Win32-based
  implementation
- Explicitly export needed symbols from ocsync.dll
- Rename share.h to sharemanager.h since the name clashes with one
  of the Windows headers and get included from there
- Remove the timestamp from the fallback csync stderr logging, it's
  not used since we always provide a log callback
2016-09-16 17:43:39 +02:00
Olivier Goffart
da26e59770 Chunking-NG: add some headers when uploading chunks
These are not understood by owncloud yet, but were requested for CernBox

OC-Total-Length in the MKCOL: The full lenght of the file
OC-Chunk-Offset in the PUT: The offset within the file in which this chunk belongs
OC-Checksum in the MOVE: The transission checksum
2016-09-16 16:36:46 +02:00
Olivier Goffart
3c24d5a148 Chunking-NG: The MOVE will return the code 204 if the file was already there 2016-09-16 16:20:40 +02:00
Olivier Goffart
c222793525 Chunking-NG: use OC-If-Destination-Match instread of If-Match
For the MOVE command, because If-Match in Webdav relates to the source, not
the destination
2016-09-16 16:15:09 +02:00
Olivier Goffart
28018e8590 Chunking-NG: Fix destination URL 2016-09-16 16:14:53 +02:00
Olivier Goffart
4c79ce2ae6 ConnectionValidator: fetch the account name.
This is needed for the new webdav path used by the new chunking.
The user might not be the same as the one used to connect
2016-09-16 15:49:43 +02:00
Christian Kamm
3e1690ff7a Improvements to file recall 2016-09-14 15:42:19 +02:00
Olivier Goffart
6f1b03c560 SyncEngine: Fix renaming of folder when file are changed (#5195)
Two bugs:
 - The change filed are not considered as move, they are re-downloaded
   but the old file was not removed from the database. The change in
   owncloudpropagator.cpp takes care of removing the old entries.

 - Next sync would then remove the file in the server in the old folder
   This was not a problem until we start reusing the sync engine, and
   that the _renamedFolders map is not cleared. We were before deleting
   a non-existing file. But now we delete the actual file.

Also improve the tests to be able to do move on the server.
This include support for file id.

Issue #5192
(cherry picked from commit 85b8ab178e)
2016-09-22 09:05:09 +02:00
Olivier Goffart
a810d69daa ConnectionValidator: properly handle error in status.php (#5188)
We wwer enot connecting to the right signal from the check server
job, and therefore we were not catching the condition in which the
json was invalid. We would then never terminate the ConnectionValidator job.

Note that instanceNotFound is also emited if there is a network error.

The log looked like this:

10:25:51.247 OCC::CheckServerJob::finished: status.php from server is not valid JSON!
10:25:51.248 OCC::CheckServerJob::finished: status.php returns:  QMap()   QNetworkReply::NetworkError(NoError)  Reply:  QNetworkReplyHttpImpl(0x2b6a790)
10:25:51.248 OCC::CheckServerJob::finished: No proper answer on  QUrl("http://localhost/~owncloud/status.php")
10:26:23.235 OCC::AccountState::checkConnectivity: ConnectionValidator already running, ignoring "owncloud@localhost"
10:26:55.235 OCC::AccountState::checkConnectivity: ConnectionValidator already running, ignoring "owncloud@localhost"
[...]
(cherry picked from commit ff701bd473)
2016-09-21 14:30:36 +02:00
ckamm
dfb121236c Excludes: Consider files in hidden folders excluded #5163 (#5173)
Previously, we only checked the hiddenness of the target file and
ignored the hiddenness of the containing folders. This lead to
undesired behavior when people synced their home folders and there
was a folder watcher notification for a non-hidden file in one of
the hidden folders.

I'm not fully sure why, but sometimes notifications for .foo/bar were
already ignored, but notifications for .foo/bar/car were not. This may
be because of how we set up the FolderWatchers on Linux.

The new behavior is to check all path components for hiddenness, up
until the base path (but excluding the base path, so using a hidden
folder as a sync folder will work).
2016-09-13 14:14:27 +02:00
Christian Kamm
5ffaf3a90f Set discovery thread priority to low #5017
(cherry picked from commit bd7ec19644)
2016-09-12 14:09:13 +02:00
Olivier Goffart
1040e7b57b ExcludeFiles: Fix when the folder casing is not the same in the settings and in the FS
If the folder has different case in the settings and in the FS, we should
not ignore all the files. This is important for the files system watcher.

(cherry picked from commit 98268d102f)
2016-09-12 14:09:12 +02:00
Olivier Goffart
65c49e1de6 CleanupPollsJob: Fix possible leak
Missing deleteLater when the CleanupPollsJob aborts.
This is only a problem if the SyncEngine is kept alive a long time. Which is
usually not the case in the configuration where poll jobs are used.

(cherry picked from commit 3465024898)
2016-09-12 14:09:12 +02:00
Christian Kamm
4abaee7736 Recall: Copy instead of move recalled file #5150
That was an unintentional change in
2662203fb7

Also expand the test case to cover this.

(cherry picked from commit af9c4d0e2f)
2016-09-12 14:09:12 +02:00
Daniel Molkentin
37adaa7872 Update owner to be ownCloud GmbH
Fixes #5079

(cherry picked from commit a83c5598e8)
2016-09-12 14:09:12 +02:00
Olivier Goffart
98268d102f ExcludeFiles: Fix when the folder casing is not the same in the settings and in the FS
If the folder has different case in the settings and in the FS, we should
not ignore all the files. This is important for the files system watcher.
2016-09-11 12:12:10 +02:00
Olivier Goffart
7c75a39bc1 Chunking-NG: Some fixup after feedback from the pull request #5102 2016-09-10 12:30:14 +02:00
Christian Kamm
af9c4d0e2f Recall: Copy instead of move recalled file #5150
That was an unintentional change in
2662203fb7

Also expand the test case to cover this.
2016-09-06 10:42:59 +02:00
Klaas Freitag
84ede3f01f Make sync journal name generating a method of SyncJournal.
Before it was in Folder, however, the command line client does not
have the Folder class. To not duplicate code, the function to generate
the sync journal name went to SyncEngine class.
2016-09-02 16:19:10 +02:00
Klaas Freitag
3b651b2da9 folderman: Clean some comments 2016-09-02 16:19:04 +02:00
Klaas Freitag
e46fad52bb Make the sync journal file name a method of the Folder class.
The sync journal name has a dependency on the remote url now.
2016-09-02 12:29:21 +02:00
Olivier Goffart
c41f6ed76b Chunking-NG: use new dav URL for the move 2016-08-31 12:12:34 +02:00
Olivier Goffart
818b5854ce Chunking-NG: Qt4 compile 2016-08-31 10:28:44 +02:00
Olivier Goffart
79abb8b4e3 ChunkingNg: enable depending on an environment variable 2016-08-31 10:28:15 +02:00
Olivier Goffart
fad387b6b8 Chunking-Ng: Resume 2016-08-31 10:28:15 +02:00
Olivier Goffart
a1558100b8 WIP: new chunking algorithm
Current limitations of this WiP
 - No resuming implemented yet
 - No parallel chunks
 - Hackish way to get the webdav paths
2016-08-31 10:28:15 +02:00
Olivier Goffart
4f3f642da6 Upload: refactor the upload in two classes so the new chuning can be implemented 2016-08-31 10:28:15 +02:00
Olivier Goffart
86eab48981 Qt4 Compile 2016-08-31 10:27:20 +02:00
Olivier Goffart
d2bde5489f Propagator, remove the QNAM in the name as the alternative has been removed a long time ago 2016-08-31 10:25:06 +02:00
Markus Goetz
6ecda6e7f4 Merge branch '2.2' 2016-08-24 11:39:12 +02:00
Jocelyn Turcotte
2ff7b63551 Merge pull request #5122 from jturcotte/syncfilestatustrackerTests 2016-08-17 16:04:02 +02:00
Jocelyn Turcotte
efb6b8c2c9 [overlays] Fix folders appearing as OK even though children are still syncing #4797
This would happen if the directory would first need to be created
through an mkdir propagation job. This job's itemCompleted signal
would trigger the directory to show as SYNC even though its children
are still propagating.

Fix the issue by tracking the sync count for each file, affecting
its parents. This allows us to get rid of the O(n) vector lookup
for each status query, and properly track the hierachical sync
status of a directory.

This also removes the itemCompleted signal emission from the
PropagateDirectory job. Since we only needed for overlay icons, and
since this job doesn't do any direct propagation, we can remove it
to ensure that we won't call itemCompleted twice for the item attached
to Propagate*Mkdir jobs (since the PropagateDirectory is backed by
the same SyncFileItem, instruction and status).
2016-08-17 15:40:02 +02:00
Jocelyn Turcotte
e974771796 csync: Use an explicit instruction for should_update_metadata
The current way of tracking the need to update the metadata without
propagation using a separate flag makes it difficult to track
priorities between the local and remote tree. The logic is also
difficult to logically cover since the possibilities matrix isn't
100% covered, leaving the flag only used in a few situations
(mostly involving folders, but not only).

The reason we need to change this is to be able to track the sync
state of files for overlay icons. The instruction alone can't be
used since CSYNC_INSTRUCTION_SYNC is used for folders even though
they won't be propagated. Removing this logic is however not possible
without using something else than CSYNC_INSTRUCTION_NONE since too
many codepath interpret (rightfully) this as meaning "nothing to do".

This patch adds a new CSYNC_INSTRUCTION_UPDATE_METADATA instruction
to let the update and reconcile steps tell the SyncEngine to update
the metadata of a file without any propagation. Other flags are left
to be interpretted by the implementation as implicitly needing
metadata update or not, as this was already the case for most file
propagation jobs. For example, CSYNC_INSTRUCTION_NEW for directories
now also implicitly update the metadata.

Since it's not impossible for folders to emit CSYNC_INSTRUCTION_SYNC
or CSYNC_INSTRUCTION_CONFLICT, the corresponding code paths in the
sync engine have been removed.

Since the reconcile step can now know if the local tree needs metadata
update while the remote side might want propagation, the
localMetadataUpdate logic in SyncEngine::treewalkFile now simply use
a CSYNC_INSTRUCTION_UPDATE_METADATA for the local side, which is now
implemented as a different database query.
2016-08-17 15:39:31 +02:00
Jocelyn Turcotte
82ef1bcfe0 Make sure that we invalidate parents on blacklisted items
Add a missing call that we currently only do in slotItemCompleted.
This would normally only affect the first sync and would have
gotten properly update at the end of the sync anyway.
2016-08-17 15:39:30 +02:00
Jocelyn Turcotte
b7ff4a76e8 Add TestSyncEngine and TestSyncFileStatusTracker auto tests
To be able to test the SyncEngine efficiently, a set of server
mocking classes have been implemented on top of QNetworkAccessManager.

The local disk side hasn't been mocked since this would require adding
a large abstraction layer in csync. The SyncEngine is instead pointed
to a different temporary dir in each test and we test by interacting
with files in this directory instead.

The FakeFolder object wraps the SyncEngine with those abstractions
and allow controlling the local files, and the fake remote state
through the FileModifier interface, using a FileInfo tree structure
for the remote-side implementation as well as feeding and comparing
the states on both side in tests.

Tests run fast and require no setup to be run, but each server feature
that we want to test on the client side needs to be implemented in
this fake objects library. For example, the OC-FileId header isn't
set as of this commit, and we can't test the file move logic properly
without implementing it first.

The TestSyncFileStatusTracker tests already contain a few QEXPECT_FAIL
for what I esteem being issues that need to be fixed in order to catch
up on our test coverage without making this patch too huge.
2016-08-17 15:39:30 +02:00
ckamm
7b26e6b8f9 Progress: Don't display unlikely estimates #5046 (#5066) 2016-08-15 13:36:53 +02:00
Daniel Molkentin
bb5c2cbfa5 Always follow redirects in network jobs (#4905)
This is a move away from the original policy where jobs
would only follow redirects in special cases.

Two restrictions are in place:

1. We do not allow protocol downgrades (https -> http)
2. We stop redirects after we find them looping (e.g. old = new url, or
indirectly when looping 10 times).

This is closer to RFC conforming behavior, although currently
we will treat 301 replies like they were 302. This is for a separate
commit.

Error handling (and display) also needs improvement.

Addresses #2791
2016-08-09 16:01:29 +02:00
Olivier Goffart
4a7f3cb486 SyncEngine: Fix detection of backup (#5104)
Once upon a time, the SyncEngine was instantiated once per sync. But now that
the SyncEngine is kept between sync, we need to reset all these variable between
syncs.
2016-08-04 16:06:10 +02:00
Jocelyn Turcotte
b307f2b65c Revert "Fix a deadlock when shutting down during discovery" (#5100)
Reverts commit 622017adcf

Could be the cause of #5092 and the cost is higher than the benefit if this is the case.
A network request taking more than 30 seconds isn't something unlikely in this world
and shouldn't be a good reason to abort. We should try to untangle the threads
dependencies to properly fix this if possible instead.
2016-08-03 16:35:24 +02:00
Olivier Goffart
73a6939b70 ownsql: handle QByteArray without converting to QString (#5097)
QByteArray is used for checksum
2016-08-02 11:39:08 +02:00
Olivier Goffart
88e5a9411a SyncEngine: Reads the data-fingerprint property (#5056)
When it changes, assume a backup was recovered, and keep conflict files.

Issues: #2325 and https://github.com/owncloud/enterprise/issues/966
2016-08-02 10:30:49 +02:00
Klaas Freitag
5ac434a740 Logs: Add the name of the platform we're running on to the log. (#5082) 2016-07-28 16:30:40 +02:00
Daniel Molkentin
a83c5598e8 Update owner to be ownCloud GmbH
Fixes #5079
2016-07-26 16:48:08 +02:00
Olivier Goffart
3465024898 CleanupPollsJob: Fix possible leak
Missing deleteLater when the CleanupPollsJob aborts.
This is only a problem if the SyncEngine is kept alive a long time. Which is
usually not the case in the configuration where poll jobs are used.
2016-07-14 09:23:22 +02:00
Olivier Goffart
7c671756e6 Merge remote-tracking branch 'origin/2.2'
Conflicts:
	src/libsync/syncfilestatustracker.cpp
2016-07-12 17:38:15 +02:00
Olivier Goffart
cb64ba22b3 Fix Qt4 build 2016-07-11 12:27:59 +02:00
Klaas Freitag
37fc4e4332 SyncJournalDb: Cleanup of Constructor interface.
The parameter path is not longer needed.
2016-07-10 13:04:29 +02:00
Klaas Freitag
4ceee86c66 SyncEngine: Calculate a uniq journal name using the remote account.
That should allow to sync the same local folder to multiple remote destinations.

see #3764
2016-07-10 12:56:43 +02:00
Klaas Freitag
a1bc01d3b1 SyncJournalDb: Add a method to set the name of the db file. 2016-07-10 12:51:42 +02:00
Christian Kamm
bd7ec19644 Set discovery thread priority to low #5017 2016-07-05 13:53:57 +02:00
Daniel Molkentin
8a1a2eb5b3 Only accept notification API Capability if endpoint is OCS-enabled
Fixes #5033
2016-07-04 18:33:02 +02:00
Jocelyn Turcotte
622017adcf Fix a deadlock when shutting down during discovery (#4993)
Since the SyncEngine now quits and waits for the discovery thread,
the main thread can enter a deadlock where the discovery thread waits
for its directory result.

Add a 2 seconds timer to the discovery thread wait condition
to limit the deadlock time.
2016-06-27 16:12:39 +02:00
Christian Kamm
7601783553 Fix Qt4 build 2016-06-22 15:47:33 +02:00
Christian Kamm
6e9df8673f Merge remote-tracking branch 'origin/2.2' 2016-06-22 15:41:37 +02:00
Jocelyn Turcotte
5d71ad83ec Fix the Qt4 build
QSharedPointer::reset doesn't exist in Qt4.
2016-06-21 12:04:26 +02:00
Markus Goetz
ab24980001 Merge pull request #4979 from jturcotte/2.2
Memory improvements during propagation
2016-06-21 10:00:41 +02:00
ckamm
43a2aec92b Performance: Don't redundantly add the same exclude files #4967 (#4988)
Excludes: Don't redundantly add the same exclude files #4967, Never accept duplicate exclude patterns #4967
2016-06-20 15:14:13 +02:00
Jocelyn Turcotte
edfd75949d Reduce the memory usage of the StopWatch during propagation
Use a QMap to avoid using a full hashtable for only a few entries, and
clear the QMap once we're done with the measuring. This saves a few
hundred bytes per job during propagation that would otherwise only be
freed at the end of the sync.
2016-06-17 19:20:53 +02:00
Jocelyn Turcotte
7fb134b4e0 Delete the ComputeChecksum object when the job is finished
Each object takes almost 1k and we don't need it once the propagation is
done.
2016-06-17 19:20:53 +02:00
Jocelyn Turcotte
3342ebfcc5 SocketAPI: Push new status of dirty files regardless when not synced (#4970)
The FolderWatcher inserts files to be marked as SYNC and we
currently assume that all file statuses will be updated by the
following sync. It's however possible that the FolderWatcher
notify us of a change that csync won't consider necessary to
propagate, in which case a new status wouldn't be pushed and
the file manager would continue showing this file as syncing.

Re-push the file status when emptying the dirty files list
before propagating to avoid this issue, most likely the OK
status.
2016-06-16 15:27:10 +02:00
Olivier Goffart
b4900d60b7 Checksum speedup. (#4968)
No need to allocate (and initialize to 0) a 10 MiB buffer for each files, even
when most files are much smaller than that.
So make sure the buffer that we allocate is not bigger than the file size.
And Also 10 MiB is a bit big for a buffer. 500 KiB should be more than enough.
(Too big allocations can cause problem because of memory fragmentation and such)
2016-06-16 08:28:30 +02:00
Olivier Goffart
92309013da SyncEngine: make abort and closing more robust
We first need to set the abort flag to csync and then aborting the discovery
job, otherwise, the discovery thread could start a new job in the mean time.

We also need to make sure that the thread has existed before we destroy the
exclude list.
2016-06-16 08:26:33 +02:00
Jocelyn Turcotte
cb50fae0a7 SocketAPI: Move the normalization closer to the data entrance (#4972)
Follow-up for #4884
2016-06-15 21:03:21 +02:00
Jocelyn Turcotte
05d199881e Merge pull request #4971 from owncloud/fixAccountHandleSslErrorsCrash
Attempt to fix the Account::slotHandleSslErrors crash
2016-06-15 18:53:47 +02:00
Jocelyn Turcotte
147cf798a6 Attempt to fix the Account::slotHandleSslErrors crash
Events from the crash reporter suggest that the QNAM and its
child replies might get deleted before returning from this method
and the only possible cause we can see is that the inner event
loop has something to do with it.

Try keeping a ref on the QNAM while in this method to make sure
that it won't get deleted by the inner event loop.
2016-06-15 18:12:02 +02:00
Jocelyn Turcotte
d40c56eda5 Account: Use smart pointers to hold its credentials and QNAM objects
Also make sure that both use QObject::deleteLater as their delete
function.
2016-06-15 18:12:01 +02:00
Olivier Goffart
6dade0b114 PropagateUpload: fix crash in PUTFileJob::slotTimeout
Same fix as in commit 60c101d9

From the crash reporter:

Crash
EXCEPTION_ACCESS_VIOLATION_READ at 0x4
qnetworkreply.cpp in QNetworkReply::request at line 476
propagateupload.cpp in OCC::PUTFileJob::slotTimeout at line 100
moc_abstractnetworkjob.cpp in OCC::AbstractNetworkJob::qt_static_metacall at line 98
qobject.cpp in QMetaObject::activate at line 3716
moc_qtimer.cpp in QTimer::timeout at line 192
qtimer.cpp in QTimer::timerEvent at line 247
qobject.cpp in QObject::event at line 1267
qapplication.cpp in QApplicationPrivate::notify_helper at line 3722
qapplication.cpp in QApplication::notify at line 3505
qcoreapplication.cpp in QCoreApplication::notifyInternal at line 932
2016-06-15 14:32:25 +02:00
Christian Kamm
87b4693a9d Add error message for trailing spaces on Windows #4878 2016-06-09 12:41:02 +02:00
ckamm
28c12a3ca0 Move touched-files tracking to SyncEngine #4927 (#4946) 2016-06-09 12:07:18 +02:00
Christian Kamm
66f041f74c Fix Qt4 build 2016-06-07 10:05:57 +02:00
Christian Kamm
48efbcb8ca Merge remote-tracking branch 'origin/2.2' 2016-06-07 09:39:44 +02:00
Olivier Goffart
d072e4a678 Fix compilation on windows 2016-06-03 13:28:53 +02:00
Markus Goetz
d88ab4653b Connectivity: Delete job on timeout #4275
This was not done if there was no reply
2016-05-30 18:15:41 +02:00
Robin Geuze
d45d6ca9da Fix Qt4 build 2016-05-30 09:55:45 +02:00
Daniel Molkentin
a2b238e2e5 Undo regression caused by 727e73d
normalization to NFC is still required. Mac OS API will not take
care of that by default.

Resolves #4884
2016-05-27 16:03:59 +02:00
ckamm
0e2c16e827 Certs: Re-ask for different cert after rejection #4898 (#4911)
Previously rejecting any kind of certificate meant that the user
was never asked again, even if the certificate changed.

Now we keep track of which certificates were rejected and ask again
if the ones mentioned in the ssl errors change.

mitmproxy is excellent for testing this.
2016-05-27 12:08:42 +02:00
ckamm
b9fdae6d67 Progress: Don't count items without propagation jobs #4856 (#4910)
* Progress: Don't count dirs without propagation jobs #4856

These directory SyncFileItems are necessary for bookkeeping
but should not influence the progress display at all.

* Progress: Skip ignored files #4856
2016-05-27 10:52:00 +02:00
Olivier Goffart
275ad1e157 Utility: Better fix for the translation of minutes, seconds, ...
The plural was not translated because of QTBUG-3945. Work around it.
Issue #4855
2016-05-26 12:17:16 +02:00
Olivier Goffart
159535fe90 Merge branch '2.2' 2016-05-25 16:30:17 +02:00
Olivier Goffart
fff5c280b3 SyncEngine: cleanup setting isDirectory 2016-05-25 16:27:40 +02:00
Olivier Goffart
e960b265a8 Merge remote-tracking branch '2.1' into 2.2 2016-05-25 16:23:58 +02:00
Olivier Goffart
f6b35e5d58 SyncEngine: invalid the blacklist entry when the rename destination change
The problem in this case is if we rename the file "xxx" to "invalid\file".
The rename will fail because the new filename constains a slash, and it
will be blacklisted.
But then if the user re-rename the file to "valid_name", then we should
invalidate the blacklist entry and retry to upload. But we did not do
that because renaming don't change the mtime and we did not store the
rename target in the database

IL issue 558
2016-05-25 15:32:45 +02:00
ckamm
bf02ccc1e8 Remember to upload files that are locked during a sync run (#4865)
See owncloud/enterprise#1342
2016-05-23 09:03:03 +02:00
Markus Goetz
552ba94c41 SyncEngine: Set isDirectory before syncItemDiscovered
(cherry picked from commit fc1933803e)
2016-05-20 17:00:22 +02:00
Markus Goetz
fc1933803e SyncEngine: Set isDirectory before syncItemDiscovered 2016-05-20 16:58:44 +02:00
Markus Goetz
46e4ec3183 Checksums: Use SHA1 like in >=2.2 2016-05-20 16:31:47 +02:00
Olivier Goffart
03e3b3bf50 Upload: put the job in the active list while doing the checksum computation.
This fixes an issue in which too many jobs are started un parallel
while uploading many files, which could cause too much memory usage as the
chunks are stored in memory.

Probably the fix for #4611
2016-05-20 16:28:27 +02:00
Christian Kamm
9aed8dbce8 Checksums: Compute content checksum on download #4375
Cherry-picked from d6d35029
2016-05-20 16:06:30 +02:00
ckamm
c6794cd338 Progress info: Reset between syncs #4856 (PR #4872) 2016-05-20 15:07:54 +02:00
ckamm
567fe87e98 Use natural sorting for filenames if available #4859 (#4864) 2016-05-19 15:36:46 +02:00
Markus Goetz
a038c99232 Upload: Remove bandwidth quota debug message #4837 2016-05-18 16:18:13 +02:00
Olivier Goffart
30c8fa1c93 Utility: Fix translation of second, minute, hour
Issue #4855

A typo in the context string made the translation lookup fail.
But also the %Ln was not recognized as a plural form by transifex, so only
the singular was translated
2016-05-18 10:52:22 +02:00
Olivier Goffart
5676685f58 SyncEngine: Add a compile option so we rename to restoring a move we don't have the permission to do
IL issue 550
2016-05-12 13:15:30 +02:00
Jocelyn Turcotte
edc58c045f Fix assert on restore after propagation (#4823)
The assert was there to make sure that this case wasn't happening
to eventually be properly tested. Remove the assert for now but this
codepath should eventually be unit tested using this specific situation.
2016-05-11 18:16:46 +02:00
Jocelyn Turcotte
727e73d640 [shell/windows] Fix the windows status push not working (#4784)
Since the windows implementation first does cache lookups using the
path string, directories need to be passed identically as through
RETRIEVE_FILE_STATUS.

Change the convention to never have a trailing slash for directories
in the protocol. This allows the convention to be applied without
having to access the disk (since we'd need to know if the path is
represented by a directory) and also matches the convention of the
rest of the sync engine. Individual file manager plugins are then
responsible of handling pushed paths as not ending with a trailing
slash.

This also:
- Moves the trailing slash removal logic from the SyncFileStatusTracker
  to the SocketApi class
- Remove the unneeded QString::normalized call in fileStatus, since
  this should already be done by the FolderWatcher and plugins
2016-05-06 12:32:01 +02:00
Jocelyn Turcotte
7c2fdee78b Avoid a SyncFileStatusTracker private overload with the same name
Having an overload as a private function in the same class makes the
code harder to follow. Rename the private fileStatus to syncFileItemStatus.
2016-05-03 13:01:51 +02:00
Jocelyn Turcotte
7bfe46962f Simplify the root status logic
Go through fileStatus like other cases to make sure that all use
cases go through the same code path. This also makes sure to use
lookupProblem which will use lower_bound which is more efficient
for larger sets of sync problems.

This also fixes the issue with lookupProblem that prevented it to
properly match an empty pathToMatch, caused by the fact that the
problem map contains relative paths not starting with a slash.
2016-05-03 13:01:51 +02:00
Jocelyn Turcotte
32b3023a8e Fix the root item sync status #4682
Make sure that we push the new status when the status of the SyncEngine
changed. SyncEngine::started comes a bit late, only when the propagation
starts, although it's better in this case since child folders will
only switch to Sync in aboutToPropagate.

Also fix an issue with SyncEngine::findSyncItem when using an empty
fileName; this would match and return the wrong item, even though
not currently happening with the code since fileStatus won't call
it with an empty fileName anymore.
2016-05-03 13:01:51 +02:00
Jocelyn Turcotte
a5df44c757 Fix the sync status push for parent directories #4682
As before, we rely on metadata-update SyncFileItem entries for parent
directories to notify us that a directory contains files to propagate,
and to know when all children were propagated through its itemCompleted
signal.

Those metadata SyncFileItems however have a None direction and we need
to add a explicit directory check to show them as Sync.
This fix also handles new files as well as existing ones, so no need
to keep a separate logic for new files.
2016-05-03 13:01:51 +02:00
Olivier Goffart
a67173610d Propagator: fix qt4 build
propagatedownload.cpp:712:35: error: 'seenLockedFile' is a protected member of 'OCC::OwncloudPropagator'

Signals are protected in Qt4 but public in Qt5, mark the class accessing it
as friend when compiling with Qt4
2016-05-02 17:34:21 +02:00
ckamm
e6b937f508 LockWatcher: Keep an eye on Windows file locks (#4758)
When a conflict-rename or a temporary-rename fails, notify the
LockWatcher. It'll regularly check whether the file has become
accesible again. When it has, another sync is triggered.

owncloud/enterprise#1288
2016-04-29 16:14:18 +02:00
Olivier Goffart
be466b47b7 Merge remote-tracking branch 'origin/2.1' into 2.2 2016-04-29 08:52:17 +02:00
Olivier Goffart
11b144957b PropagateDownload: Throw an error if the file is empty while it should not have been (#4753)
If the downloaded file is empty but the PROPFIND previously announced it
should not have been empty, this might mean the file was somehow corrupted
because of a bug on the server and that we should therefore not accept
the file.

Normaly we accept a change between the actual size of the file and what we
got during discovery because the file might have been updated to a new version
inbetween. But after this patch we won't accept the file if it was replaced
by an empty file.

Will help for issue #4583
Also requested by IL for issue 548
2016-04-29 08:49:27 +02:00
ckamm
38bad564a0 Merge pull request #4739 from ckamm/trackdirty
Overlay icons: Track touched files #4730
2016-04-28 07:17:17 +02:00
Christian Kamm
c2fa3fb4c8 Overlay icons: Track touched files #4730
This uses the file watcher to keep track of files that were modified
in order to assign them the blue icon.

This is transient state that's not persisted across restarts.
2016-04-28 07:16:49 +02:00
Markus Goetz
f7c6efb391 ConnectionValidator: Also set short timeout for capabilities job
Default timeout was 5 min.
2016-04-27 12:19:49 +02:00
Christian Kamm
09eea7f5f2 Checksums: Use the first supported type if nothing is preferred 2016-04-26 13:10:53 +02:00
Olivier Goffart
0711abbf56 Theme: missing virtual keyword 2016-04-22 10:09:07 +02:00
Olivier Goffart
84ac2e64e0 Quota: Add branding option for the base folder (#4714)
As discussed on issue ##4460
Having the quote to be queried on subfolder is wrong in the generic case,
so add a branding option to configure it.

This partially reverts commit ff4cdc3161
2016-04-22 10:05:50 +02:00
Klaas Freitag
e5a5b95b9a Merge pull request #4689 from owncloud/overlays_handle_root
SyncFileStatusTracker: Add a method to handle the root folder of the sync.

Also, handle new files put in the sync in the statustracker logic.
2016-04-21 15:01:11 +02:00
Klaas Freitag
7acdf50a2c SocketAPI: Code simplifications as asked for in the review. 2016-04-21 14:54:13 +02:00
Klaas Freitag
361ebf5464 SyncFileStatusTracker: Remember the state of new files.
In the before-propagate slot, new files that wait to be
pushed to the server are remembered in the _syncProblems
map. That way, the parents show a sync icon properly as
asked for in #4682.

After the item has been transfered properly, the item is
removed from the map again because success is the default.
2016-04-21 13:56:08 +02:00
Olivier Goffart
522f7afa9e Merge pull request #4663 from 'ckamm/disabletransmissionchecksum'
Disable validation of transmission checksums by default
2016-04-21 11:46:01 +02:00
Olivier Goffart
fa69d089cf Checksums: remove the downloadChecksumValidationEnabled option
Added in previous commit from pull request #4663

As discussed, we do not need this option so no need to introduce
a new dependency on the config file in the sync engine
2016-04-21 11:43:44 +02:00
Klaas Freitag
bd37eab3ad SocketAPI: Display a warning on the directory if there are errors
Errors with individual files within a directory now result in
a warning label on the parent directories.
2016-04-19 18:18:00 +02:00
Markus Goetz
a17f37d8ae Discovery: Be more explicit about files with unknown size #4573
Also related https://github.com/owncloud/core/issues/23468
2016-04-19 15:31:16 +02:00
Daniel Molkentin
c0ec481436 No need for WebKitWidgets in libsync anymore 2016-04-16 10:58:12 +02:00