Commit graph

261 commits

Author SHA1 Message Date
Olivier Goffart
520923b5a7 HTTP/2 Support
We need Qt 5.9 for HTTP2 because, even if Qt 5.8 already has support
for it, there is some critical bug in the HTTP2 implementation which
make it unusable [ https://codereview.qt-project.org/186050 and
https://codereview.qt-project.org/186066 ]

When using HTTP2, we can use many more parallel network request, this
is especially good for small file handling

Lower the priority of the GET and PUT propagation jobs, so the quota
or selective sync ui PROPFIND will not be blocked by them
2017-07-17 08:20:17 +02:00
Christian Kamm
2cdf5517cb Conflicts: Detect and show in issues tab
Incidentally fixes a potential issue where conflicts were silently-
ignored and thus deleted if the parent folder was deleted.
2017-07-13 11:48:10 +02:00
Christian Kamm
da54b3d7e5 Merge remote-tracking branch 'origin/2.3' 2017-07-12 10:06:04 +02:00
Christian Kamm
971abaea80 IssuesWidget: Add button to retry 507 errors #5537
Since these errors are blacklisted, it can take up to 24h to retry items
that had a 507 error for a while. This way users can intervene and cause
an upload attempt immediately.
2017-07-12 09:04:27 +02:00
Christian Kamm
cd1b89475c PropagateUpload: Better messaging for 507 #5537
It now produces a summary error message indicating the problem.

Adjust blacklist database table to contain 'errorCategory'. This is
useful for two things:
  - Reestablishing summary messages based on blacklisted errors. For
    example if we don't retry a 507ed file, we still want to show the
    message about space on the server
  - Selectively wiping the blacklist: When we have ui for something like
    "I deleted some files, please retry all files now!", we want to
    delete all blacklist entries of a specific category only.
2017-07-12 09:04:27 +02:00
Christian Kamm
c3cb186952 ProgressInfo: Carry a sync status
* A bunch of code was determining sync status by ad-hoc comparing some
  progress info fields. It can now just check the status, making it
  easier to comprehend.
* There's a clear indication for "a new sync is starting", which helps
  wiping the issues tab at the right time.
2017-07-12 09:04:27 +02:00
Christian Kamm
5ca743dd25 SyncEngine: Introduce overall errors that are not tied to a file #5746
For now we use them for:
* csync errors: This allows them to appear in the sync issues tab
* insufficient local disk space, as a summary of individual file errors

Insufficient remote space will use them too, as might other issues that
are bigger than a single sync item.
2017-07-12 09:04:27 +02:00
Christian Kamm
b1aaf055b1 Blacklist: Don't let errors become warnings #5516
Before, blacklisted errors were set to FileIgnored status and hence
displayed as warnings. Now, they have their own BlacklistedError
category which allows them to appear as errors in the issues list and in
the shell integration icons.
2017-07-04 13:07:51 +02:00
Markus Goetz
f4d1a07b72 SyncEngine: Keep local size in SyncItem #5855 2017-07-04 11:42:44 +02:00
Jocelyn Turcotte
5d24a2dc0f Remove debug messages by default from sync.engine
Other categories were already set properly.
2017-06-15 20:10:30 +02:00
Christian Kamm
8160963110 Compare the hash of files with identical mtime/size #5589
* For conflicts where mtime and size are identical:

  a) If there's no remote checksum, skip (unchanged)
  b) If there's a remote checksum that's a useful hash, create a
     PropagateDownload job and compute the local hash. If the hashes
     are identical, don't download the file and just update metadata.

* Avoid exposing the existence of checksumTypeId beyond the database
  layer. This makes handling checksums easier in general because they
  can usually be treated as a single blob.

  This change was prompted by the difficulty of producing file_stat_t
  entries uniformly from PROPFINDs and the database.
2017-06-15 13:54:16 +02:00
Olivier Goffart
3314a472c3 SyncEngine: SelectiveSync: Remove local files of undelected folder despite other modified files
Issue #5783

When the directry that should be removed by selective sync contains changes,
we ignore the whole sub tree instead of only ignoreing new files.
We cannot ignore the whole directory, we need to ignore only the directory
that do not have files to remove
2017-06-08 14:53:54 +02:00
Christian Kamm
c8d0f788e0 Apply clang-format 2017-05-17 12:26:27 +02:00
Jocelyn Turcotte
7fd2f292e7 Promote a few more logs to info/warning 2017-05-11 17:22:59 +02:00
Jocelyn Turcotte
b7553d5bdf Upgrade some qCDebug to qCInfo or qCWarning
Use qCInfo for anything that has general value for support and
development. Use qCWarning for any recoverable error and qCCritical
for anything that could result in data loss or would identify a serious
issue with the code.

Issue #5647
2017-05-11 17:22:59 +02:00
Jocelyn Turcotte
4ad190a558 Use Qt logging categories for logging
This gives more insight about the logs and allow setting fine-tuned
logging rules. The categories are set to only output Info by default
so this allows us to provide more concise logging while keeping the
ability to extract more information for a specific category when
developping or debugging customer issues.

Issue #5647
2017-05-11 17:22:59 +02:00
Olivier Goffart
13705999f7 Discovery: Increase the MAX_DEPTH and show deep folder as ignored
Before this patch, to deep folder would just be ignored, without any feedback.
This patch makes it so deep folder are properly shown as ignored in the UI.

Also increase the MAX_DEPTH

Issue: #1067
2017-05-08 11:29:51 +02:00
Olivier Goffart
bb244d527c Better words for the error message when one cannot create the database.
Issue #5291
2017-05-03 17:53:46 +02:00
Christian Kamm
e86499d990 dynamic chunking: cleanup, fixes, improvements
* make target duration a client option instead of a capability
* simplify algorithm for determining chunk size significantly
* preserve chunk size for the whole propagation, not just per upload
* move options to SyncOptions to avoid depending on ConfigFile
  in the propagator
* move chunk-size adjustment to after a chunk finishes, not when
  a new chunk starts
2017-03-28 11:32:10 +02:00
Olivier Goffart
0ed929f24b SyncEngine: fix test regression in TestChunkingNG::testCreateConflictWhileSyncing
In 8ef11a38c9, we started blacklisting
SoftError for 0 seconds.  But if the two sync happen with less than
1s interval, we would still prevent them to happen.
So make sure we expire if 0 seconds have expired
2017-03-16 10:17:08 +01:00
Christian Kamm
b98876e265 Account server version: Helper to create versions
Hex literals don't work well with version 10: 0x100000 doesn't do
the right thing.
2017-03-15 16:30:08 +01:00
Christian Kamm
4d7dd0c909 Recent Changes: Fix duplicates #5256
The transmissionProgress emission when the whole sync finishes
shouldn't look exactly like the one for the last file.
2017-02-16 15:10:46 +01:00
ckamm
7879c470b3 Merge pull request #5518 from ckamm/asserts
Improve usage of asserts
2017-02-08 15:25:02 +01:00
Christian Kamm
4c1fdf1dee Double check usage of asserts #5429
A few are supposed to be fatal.
2017-02-08 15:15:19 +01:00
Christian Kamm
18e25122db Performance: reserve() _remotePerm #5496 2017-02-08 15:06:09 +01:00
Olivier Goffart
0d16cf41fe Merge remote-tracking branch 'origin/master' into 2.3
We can do that because the only changes that were in master but not in 2.3 were the
translations change and documentation change, and the support for the 'M' permission
which we want in 2.3.
2017-01-27 17:45:05 +01:00
Olivier Goffart
38cf459b3e Confirm External Storage: adjust the notification message
We need to forward the information that the folder is an external storage
for the notification message.

Issue: https://github.com/owncloud/client/pull/5340#issuecomment-274878023
2017-01-27 15:59:59 +01:00
Olivier Goffart
f854c5263b Wizard: Add options to ask confirmation for external storage
Added two checkboxes in the Account Wizard in the advanced page to change the first options.
Also added a checkbox in the general settings to ask for confirmation for external storages.

Theme options allow to hide the checkboxes in the wizard.

As described in issue #5340
2017-01-27 15:59:59 +01:00
Jocelyn Turcotte
bb3efc5988 Clear SyncEngine::_remotePerms after the sync
Also make reuse the SyncFileItem::_remotePerm QByteArray allocation
instead of redoing the copy from the csync char*.
2017-01-26 17:13:54 +01:00
Jocelyn Turcotte
b0700ebbab Remove duplicate SyncFileItem entries for the log
The accuracy of that log isn't as important as the few bytes those
fields take as hostage for the whole sync.
2017-01-26 17:13:54 +01:00
Jocelyn Turcotte
ee211d7609 Release SyncFileItem objects with their job
We now delete subjobs as their propagation is complete. This allows us
to also release the item by making sure that nothing else is holding a
reference to it.

Remove the stored SyncFileItemVector from SyncEngine and SyncResult
and instead gather the needed info progressively as each itemCompleted
signal is emitted.

This frees some holes on the heap as propagation goes, allowing many
memory allocations without the need of requesting more virtual memory
from the OS, preventing the memory usage from increasingly growing.
2017-01-25 23:26:23 +01:00
Jocelyn Turcotte
1fc5a76622 Pass the SyncFileItem as SyncFileItemPtr in itemCompleted
This will allow us to keep a reference on the items in connected slots.
2017-01-25 23:26:23 +01:00
Jocelyn Turcotte
a764d7eb86 Don't pass the PropagatorJob in itemCompleted
This was to catch duplicate emissions for PropagateDirectory but we
don't emit this signal anymore from there.
This fixes a warning about PropagatorJob not being a registered metatype.

This reverts commit fe42c1a818.
2017-01-25 23:26:23 +01:00
Olivier Goffart
a63d970e5e ChunkingNG: remove stale chunks when cleaning the uploadInfo table
Stale chunks might be there because a file was removed or would just not
be uploaded, for any reason.
We just start the DeleteJob but we don't care if it success or not.

Relates to https://github.com/owncloud/core/issues/26981

One of the test is testing the case where the file is modified on the server
during the upload. So this test the precondition failed error.
The FakeGetReply logic was modified because resizing a 150MB big QByteArray
by increment of 16k just did not scale when downloading a big file.
2017-01-20 14:48:53 +01:00
Olivier Goffart
9b96899d75 Sync Engine: Specify what chars are not syncable in the error message #1733 (#5449) 2017-01-18 11:45:46 +01:00
Christian Kamm
e306f4611c Reschedule a folder regularly with some delay for some errors
Like "folder doesn't exist" - such that we will detect when the folder
becomes available and start syncing.

See #5317
2017-01-02 08:52:26 +01:00
Christian Kamm
3912dba33a Move qDeclareMetatype from Folder to SyncEngine 2016-12-20 11:43:39 +01:00
Christian Kamm
3e59a9b316 Merge branch 'master' into dbjournal_per_account 2016-11-23 16:47:56 +01:00
Olivier Goffart
a1dc4069c9 libsync: Don't store the remote URI in the csync or in the SyncEngine
We are going to change the webdav path depending on the capabilities.
But the SyncEngine and csync might have been created before the capabilities
are retrieved.

The main raison why we gave the path to the sync engine was to pass it to csync.
But the thing is that csync don't need anymore this url as everything is done by the
discovery classes in libsync that use the network jobs that use the account for the urls.
So csync do not need the remote URI.

shortenFilename in folderstatusmodel.cpp was useless because the string is the
_file of a SyncFileItem which is the relative file name, that name never
starts with owncloud://.

All the csync test creates the folder because csync use to check if the folder
exists. But we don't need to do that anymore
2016-11-21 08:09:11 +01:00
Christian Kamm
e485c5c008 Always send a progress message at the end of a sync #5290
Some listeners detect whether a sync is finished by checking
for isUpdatingEstimates and completedFiles >= totalFiles. But
if a sync didn't transfer any files we never sent signal
with these values. Now we do.
2016-11-18 12:33:44 +01:00
ckamm
36d61ef3a9 Doc: Add sync algorithm overview and comments (#5277)
* Doc: Add sync algorithm overview and comments
2016-11-08 16:10:55 +01:00
Olivier Goffart
e3a4c3989a SyncEngine: disable heuristics for backup restoration for server >= 9.1
The ownCloud 9.1 server has a data-fingerprint property that the admin must
change in case of backup restoration. When this change, the client understands
that a backup was restored, and will generate conflict files and re-upload
new files.

The heuristics based system checks that there is at least two files wose mtime
is put back in the past and no files that goes forward. In that case we ask the
user before creating the conflicts.

This commit disable the heuristics for newer server that have the data-fingerpint.
And change the heuristics to two hours because we want to avoid false positive due
to some clock error, and that 2 hours of lost due to backup restoration is probably
not so bad.

We only ask the user in the heuristics based aproach so in practice this mean that
the "backup-detected" dialog will no longer appear with newer server.

Relates issues #5260, #5109
2016-11-04 16:30:58 +01:00
Olivier Goffart
36a19703db SyncEngine: Permission error in subfolder of a folder that cannnot be added is a SoftError
Issue #5059
2016-10-17 12:22:04 +02:00
Klaas Freitag
e1a48e3c33 Move the journal file name generation to the syncjournaldb class.
As requested by Olivier.
2016-10-10 16:59:17 +02:00
Olivier Goffart
85b8ab178e SyncEngine: Fix renaming of folder when file are changed (#5195)
Two bugs:
 - The change filed are not considered as move, they are re-downloaded
   but the old file was not removed from the database. The change in
   owncloudpropagator.cpp takes care of removing the old entries.

 - Next sync would then remove the file in the server in the old folder
   This was not a problem until we start reusing the sync engine, and
   that the _renamedFolders map is not cleared. We were before deleting
   a non-existing file. But now we delete the actual file.

Also improve the tests to be able to do move on the server.
This include support for file id.

Issue #5192
2016-09-22 09:02:47 +02:00
Klaas Freitag
84ede3f01f Make sync journal name generating a method of SyncJournal.
Before it was in Folder, however, the command line client does not
have the Folder class. To not duplicate code, the function to generate
the sync journal name went to SyncEngine class.
2016-09-02 16:19:10 +02:00
Klaas Freitag
e46fad52bb Make the sync journal file name a method of the Folder class.
The sync journal name has a dependency on the remote url now.
2016-09-02 12:29:21 +02:00
Markus Goetz
6ecda6e7f4 Merge branch '2.2' 2016-08-24 11:39:12 +02:00
Jocelyn Turcotte
efb6b8c2c9 [overlays] Fix folders appearing as OK even though children are still syncing #4797
This would happen if the directory would first need to be created
through an mkdir propagation job. This job's itemCompleted signal
would trigger the directory to show as SYNC even though its children
are still propagating.

Fix the issue by tracking the sync count for each file, affecting
its parents. This allows us to get rid of the O(n) vector lookup
for each status query, and properly track the hierachical sync
status of a directory.

This also removes the itemCompleted signal emission from the
PropagateDirectory job. Since we only needed for overlay icons, and
since this job doesn't do any direct propagation, we can remove it
to ensure that we won't call itemCompleted twice for the item attached
to Propagate*Mkdir jobs (since the PropagateDirectory is backed by
the same SyncFileItem, instruction and status).
2016-08-17 15:40:02 +02:00
Jocelyn Turcotte
e974771796 csync: Use an explicit instruction for should_update_metadata
The current way of tracking the need to update the metadata without
propagation using a separate flag makes it difficult to track
priorities between the local and remote tree. The logic is also
difficult to logically cover since the possibilities matrix isn't
100% covered, leaving the flag only used in a few situations
(mostly involving folders, but not only).

The reason we need to change this is to be able to track the sync
state of files for overlay icons. The instruction alone can't be
used since CSYNC_INSTRUCTION_SYNC is used for folders even though
they won't be propagated. Removing this logic is however not possible
without using something else than CSYNC_INSTRUCTION_NONE since too
many codepath interpret (rightfully) this as meaning "nothing to do".

This patch adds a new CSYNC_INSTRUCTION_UPDATE_METADATA instruction
to let the update and reconcile steps tell the SyncEngine to update
the metadata of a file without any propagation. Other flags are left
to be interpretted by the implementation as implicitly needing
metadata update or not, as this was already the case for most file
propagation jobs. For example, CSYNC_INSTRUCTION_NEW for directories
now also implicitly update the metadata.

Since it's not impossible for folders to emit CSYNC_INSTRUCTION_SYNC
or CSYNC_INSTRUCTION_CONFLICT, the corresponding code paths in the
sync engine have been removed.

Since the reconcile step can now know if the local tree needs metadata
update while the remote side might want propagation, the
localMetadataUpdate logic in SyncEngine::treewalkFile now simply use
a CSYNC_INSTRUCTION_UPDATE_METADATA for the local side, which is now
implemented as a different database query.
2016-08-17 15:39:31 +02:00
Jocelyn Turcotte
b7ff4a76e8 Add TestSyncEngine and TestSyncFileStatusTracker auto tests
To be able to test the SyncEngine efficiently, a set of server
mocking classes have been implemented on top of QNetworkAccessManager.

The local disk side hasn't been mocked since this would require adding
a large abstraction layer in csync. The SyncEngine is instead pointed
to a different temporary dir in each test and we test by interacting
with files in this directory instead.

The FakeFolder object wraps the SyncEngine with those abstractions
and allow controlling the local files, and the fake remote state
through the FileModifier interface, using a FileInfo tree structure
for the remote-side implementation as well as feeding and comparing
the states on both side in tests.

Tests run fast and require no setup to be run, but each server feature
that we want to test on the client side needs to be implemented in
this fake objects library. For example, the OC-FileId header isn't
set as of this commit, and we can't test the file move logic properly
without implementing it first.

The TestSyncFileStatusTracker tests already contain a few QEXPECT_FAIL
for what I esteem being issues that need to be fixed in order to catch
up on our test coverage without making this patch too huge.
2016-08-17 15:39:30 +02:00
Olivier Goffart
4a7f3cb486 SyncEngine: Fix detection of backup (#5104)
Once upon a time, the SyncEngine was instantiated once per sync. But now that
the SyncEngine is kept between sync, we need to reset all these variable between
syncs.
2016-08-04 16:06:10 +02:00
Olivier Goffart
88e5a9411a SyncEngine: Reads the data-fingerprint property (#5056)
When it changes, assume a backup was recovered, and keep conflict files.

Issues: #2325 and https://github.com/owncloud/enterprise/issues/966
2016-08-02 10:30:49 +02:00
Klaas Freitag
5ac434a740 Logs: Add the name of the platform we're running on to the log. (#5082) 2016-07-28 16:30:40 +02:00
Klaas Freitag
4ceee86c66 SyncEngine: Calculate a uniq journal name using the remote account.
That should allow to sync the same local folder to multiple remote destinations.

see #3764
2016-07-10 12:56:43 +02:00
Christian Kamm
bd7ec19644 Set discovery thread priority to low #5017 2016-07-05 13:53:57 +02:00
Christian Kamm
6e9df8673f Merge remote-tracking branch 'origin/2.2' 2016-06-22 15:41:37 +02:00
Olivier Goffart
92309013da SyncEngine: make abort and closing more robust
We first need to set the abort flag to csync and then aborting the discovery
job, otherwise, the discovery thread could start a new job in the mean time.

We also need to make sure that the thread has existed before we destroy the
exclude list.
2016-06-16 08:26:33 +02:00
Christian Kamm
87b4693a9d Add error message for trailing spaces on Windows #4878 2016-06-09 12:41:02 +02:00
ckamm
28c12a3ca0 Move touched-files tracking to SyncEngine #4927 (#4946) 2016-06-09 12:07:18 +02:00
Olivier Goffart
fff5c280b3 SyncEngine: cleanup setting isDirectory 2016-05-25 16:27:40 +02:00
Olivier Goffart
e960b265a8 Merge remote-tracking branch '2.1' into 2.2 2016-05-25 16:23:58 +02:00
Olivier Goffart
f6b35e5d58 SyncEngine: invalid the blacklist entry when the rename destination change
The problem in this case is if we rename the file "xxx" to "invalid\file".
The rename will fail because the new filename constains a slash, and it
will be blacklisted.
But then if the user re-rename the file to "valid_name", then we should
invalidate the blacklist entry and retry to upload. But we did not do
that because renaming don't change the mtime and we did not store the
rename target in the database

IL issue 558
2016-05-25 15:32:45 +02:00
Markus Goetz
552ba94c41 SyncEngine: Set isDirectory before syncItemDiscovered
(cherry picked from commit fc1933803e)
2016-05-20 17:00:22 +02:00
Markus Goetz
fc1933803e SyncEngine: Set isDirectory before syncItemDiscovered 2016-05-20 16:58:44 +02:00
ckamm
c6794cd338 Progress info: Reset between syncs #4856 (PR #4872) 2016-05-20 15:07:54 +02:00
Olivier Goffart
5676685f58 SyncEngine: Add a compile option so we rename to restoring a move we don't have the permission to do
IL issue 550
2016-05-12 13:15:30 +02:00
Jocelyn Turcotte
32b3023a8e Fix the root item sync status #4682
Make sure that we push the new status when the status of the SyncEngine
changed. SyncEngine::started comes a bit late, only when the propagation
starts, although it's better in this case since child folders will
only switch to Sync in aboutToPropagate.

Also fix an issue with SyncEngine::findSyncItem when using an empty
fileName; this would match and return the wrong item, even though
not currently happening with the code since fileStatus won't call
it with an empty fileName anymore.
2016-05-03 13:01:51 +02:00
ckamm
e6b937f508 LockWatcher: Keep an eye on Windows file locks (#4758)
When a conflict-rename or a temporary-rename fails, notify the
LockWatcher. It'll regularly check whether the file has become
accesible again. When it has, another sync is triggered.

owncloud/enterprise#1288
2016-04-29 16:14:18 +02:00
Klaas Freitag
d433c24186 Check if the record returned from getFileRecord is valid.
Handle database fails properly.
2016-04-11 16:04:20 +02:00
Klaas Freitag
ee58cc3b66 SyncEngine: Close the sync journal after the sync run has finished. 2016-04-11 11:31:54 +02:00
Klaas Freitag
e4604b406f SyncEngine: finalize properly on error with syncjournal 2016-04-06 17:20:48 +02:00
Klaas Freitag
7b1f02fcda SelectiveSync: Verify if the list could be read from journal.
If there is a read error from the database while trying to get
the list from database, make sure to not behave badly because
the list is empty.
2016-04-06 15:01:28 +02:00
Olivier Goffart
3334067d9f Merge branch '2.1'
Conflicts:
	src/gui/socketapi.cpp
2016-04-04 16:58:49 +02:00
Olivier Goffart
434d16941b SyncEngine: fixed restoring files when they are moved in a new directory
Imagine tgus scenario on a read only share that you move file from
one location to a new directory in the read only share.
Creating the read only directory fails for permission error.
But we should also restore the files that have been moved.

IL issue 542
2016-04-04 10:41:12 +02:00
Jocelyn Turcotte
82190eaa81 Refactor the overlay icon logic to show errors as a warning for parent folders #3634
This also remove all smartness from the SocketApi about the status
of a file and solely use info from the current and last sync.
This simplifies the logic a lot and prevents any discrepancy between
the status shown in the activity log and the one displayed on the
overlay icon of a file.

The main benefit of the additional simplicity is that we are able
to push all new status of a file reliably (including warnings for
parent folders) to properly update the icon on overlay implementations
that don't allow us invalidating the status cache, like on OS X.

Both errors and warning from the last sync are now kept in a set,
which is used to also affect parent folders of an error.

To make sure that errors don't become warning icons on a second
sync, SyncFileItem::_hasBlacklistEntry is also interpreted as an error.
This also renames StatusIgnore to StatusWarning to match this semantic.

SyncEngine::aboutToPropagate is used in favor of SyncEngine::syncItemDiscovered
since the latter is emitted before file permission warnings are set on the
SyncFileItem. SyncEngine::finished is not used since we have all the
needed information in SyncEngine::itemCompleted.
2016-03-28 14:29:48 +02:00
Jocelyn Turcotte
69aa39f1f6 Don's use SyncFileStatus for Folder::createGuiLog
SyncFileStatus' purpose is to track overlay icon status.
Instead of putting comments and default: clauses in switch
on both sides about unused enums, use different enums.

This also remove STATUS_NEW which is the equivalent of
STATUS_SYNC in all shell extension implementations, and
remove STATUS_UPDATED and STATUS_STAT_ERROR which have
the same semantic as STATUS_UPTODATE and STATUS__ERROR.
2016-03-28 14:29:47 +02:00
Jocelyn Turcotte
da7b9916e5 Move the SocketApi business logic to a libsync SyncFileStatusTracker class
This will allow testing this code and avoid going through too many
layers to get notified and a file status changed.
2016-03-28 14:29:47 +02:00
Jocelyn Turcotte
2d2c7bc9b8 Move the SyncEngine construction to the Folder constructor
The SyncEngine is now created only once, at construction of the
Folder, instead of being reconstructed on each sync.
2016-03-28 13:26:38 +02:00
Luca Niccoli
6735126c09 Fix crash due to destruction order 2016-03-18 16:41:48 +01:00
Olivier Goffart
80b5f3f43d Never overwrite the size from the db when updating the metadata
the size on the server might be different from the size on the client
with certain backend so it should be ignored.

(cherry picked from commit 9222db6df9b19a21e1bea5a238d745d96a6385e3)
2016-03-17 10:48:49 +01:00
Markus Goetz
9460aa7f21 SyncEngine: Also emit item in other code path
Else a user of this library cannot build a complete list of items.
2016-03-14 18:12:07 +01:00
Jocelyn Turcotte
e91a5c85ff Move the Logger reference back into src/gui 2016-03-03 20:26:48 +01:00
Jocelyn Turcotte
b8227afcaa Move the csync_context creation in SyncEngine
The creation doesn't need to be separated from the SyncEngine anymore.
This allows the SyncEngine to be created in fewer steps if we want to
use it in tests.

This moves most of the direct csync code from Folder into the SyncEngine.
The exclude file logic for the context has been wrapped using the
existing ExcludedFiles class as well.
2016-02-25 20:53:13 +01:00
Olivier Goffart
b3d57f3c7c Cleanup syncengine after the new option not to ask confirmation when everything is removed
Cleanup after pull reuqest  #4389

Do not make the SyncEngine depends on the ConfigFile
2016-02-22 16:14:22 +01:00
Markus Goetz
a76ba06817 Merge pull request #4389 from Bottswana/deleteprompt
Add option to disable the delete all files prompt
2016-02-22 15:44:50 +01:00
Markus Goetz
5d7aa792e7 SyncEngine: Improve error message for ignored files/folders #4143 2016-02-22 11:13:29 +01:00
Olivier Goffart
c93ecfbfb5 Merge branch restore_backup 2016-02-10 17:47:06 +01:00
James Botting
d8af949a6a Alter sync engine to check new setting before displaying prompt 2016-01-24 18:45:29 +00:00
Olivier Goffart
47710d167a SyncEngine: fix inode after move if the file has moved on the file system
This is the fix for issue #4370

Step to reproduce the bug:
 1) have lots of files in directory "dir1"
 2) do mkdir dir2 && mv dir1/* dir2
 3) DURING the sync (which takes time because of the many moves) do mkdir dir3 && mv dir2/* dir3/
 4) observe that files are PUT in the next sync

The problem is that SyncJournalFileRecord::SyncJournalFileRecord will fail to
get the inode after the forst move because the files are already moved on the
filesystem.  Normaly it should use the inode from the discovery phase in that
case but that is not working because it comes from the remote node in case of
moves, so the code in SyncEngine::treewalkFile would not set the inode.

Test in https://github.com/owncloud/smashbox/pull/143
2016-01-20 13:49:12 +01:00
Olivier Goffart
bbedeed1c5 SyncEngine: For server older than 8.1, ignore invalid char in new directories
Server older than 8.1 cannot cope with invalid char in the filename
so we must not send them from the client. We were already checking
for new files, but not for renames or new directories.

https://github.com/owncloud/enterprise/issues/1009
2016-01-13 17:53:12 +01:00
Christian Kamm
abf5a5ad1e Propagation: Fix dir <-> file changes propagating to server #4302
* Ensure every time a file becomes a directory or the other way around
  the item is flagged as INSTRUCTION_TYPE_CHANGE.
* Delete the badly-typed entity if necessary in the propagation jobs.
2016-01-08 05:38:08 +01:00
Daniel Molkentin
9c7066ac47 Translation cleanups
- Disambiguation
- Remove potentially illigitmate concatenations
2016-01-06 16:50:59 +01:00
Jocelyn Turcotte
62df938465 Rename SyncFileStatus::STATUS_SYNC to STATUS_UPTODATE
Also use STATUS_UPDATED instead to match SyncFileItem::Success in the SocketAPI.
2016-01-05 15:41:15 +01:00
Jocelyn Turcotte
6e3809528e Fix OK statuses not being broadcasted during a sync #3944
Since the presence of any path in SyncEngine::_syncedItems
would translate in a SYNC status, platforms that don't refresh
all their status cache after an UPDATE_VIEW message like OS X
or Windows would keep displaying that status even after all
files are successfully synchronized.

- Read SyncFileItem::_status to determine the status to display mid-sync
- Match moved paths also to _renameTarget since this might be the
  path to match
- Make sure that PropagateDirectory jobs also set SyncFileItem::_status
  properly
2016-01-05 15:08:26 +01:00
Christian Kamm
2662203fb7 Windows: Fix deleting and replacing of read-only files #4308 2016-01-05 13:15:59 +01:00
Olivier Goffart
1bb76f5343 Attempt to recover from backup restoration on the server
If all the files bring us to past timestamp, it is possibly a backup
restoration in the server. In which case we want don't want to just
overwrite newer files with the older ones.

Issue #2325
2016-01-05 12:14:18 +01:00
Christian Kamm
2458f07ca1 Checksums: Reuse the discovery checksum where possible 2015-11-23 14:13:10 +01:00
Christian Kamm
6cf5fc7f7d Permissions: Keep more user permissions
t4.pl discovered that it'd needlessly override permissions sometimes
2015-11-23 13:24:05 +01:00
Christian Kamm
a25f094c4c Checksums: Don't reupload if size and checksum are unchanged #3235
* Compute the content checksum (in addition to the optional
  transmission checksum) during upload (.eml files only)

* Add hook to compute and compare the checksum in csync_update

* Add content checksum to database, remove transmission checksum
2015-11-23 11:59:56 +01:00