Commit graph

34 commits

Author SHA1 Message Date
Olivier Goffart
1ed4eb46f2 Merge remote-tracking branch 'origin/2.3'
Conflicts:
	VERSION.cmake
2017-04-13 10:19:35 +02:00
Christian Kamm
35af03b2e5 Improve http error messages; cleanup
By default QNetworkReply::errorString() often produces messages like
   "Error downloading <url> - server replied: <reason>"
but the "downloading" part invariably confuses people since the
error might very well have been produced by a PUT request.

This commit produces clearer error messages for HTTP errors.

Additionally:
* Remove some unnecessary null checks from slots connected to
  network job signals and document that these signals never send
  null replies.
* There was a bug where AbstractNetworkJob::_timedout wasn't
  set when derived classes overrode slotTimeout. We now ensure
  it's always set by disallowing overrides of slotTimeout.
  Instead it now calls onTimedOut, which allows custom handling.
* Several subclasses declared errorString, isTimedOut. Move
  these to AbstractNetworkJob.
* Unify handling of OC-ErrorString (via the new, general
  Job::errorString)
* Add documentation in various places.
2017-04-04 09:27:37 +02:00
Markus Goetz
28e9c56a4b Chunking NG: Also send OC-Total-Length on MOVE
For https://github.com/owncloud/core/issues/26988
2017-03-29 19:51:02 +02:00
Christian Kamm
e86499d990 dynamic chunking: cleanup, fixes, improvements
* make target duration a client option instead of a capability
* simplify algorithm for determining chunk size significantly
* preserve chunk size for the whole propagation, not just per upload
* move options to SyncOptions to avoid depending on ConfigFile
  in the propagator
* move chunk-size adjustment to after a chunk finishes, not when
  a new chunk starts
2017-03-28 11:32:10 +02:00
Piotr M
88e0f97798 dynamic chunking prototype 2017-03-27 13:43:39 +02:00
Jocelyn Turcotte
19bf5e2ff1 Reduce the connection data used by PropagateDirectory objects
Avoid using connections to report up the job tree for signals
that we can directly communicate to the OwncloudPropagator.
This slightly reduces the memory usage and avoid passing those calls
through the whole parent chain.
2017-02-23 17:54:33 +01:00
Christian Kamm
4c1fdf1dee Double check usage of asserts #5429
A few are supposed to be fatal.
2017-02-08 15:15:19 +01:00
Jocelyn Turcotte
f1e9be4fa8 Don't use the StopWatch for each job in production builds #5456
This accounted for more than 10% of the peak memory usage while only
providing value through the debugging log.
2017-01-26 17:13:54 +01:00
Jocelyn Turcotte
9db23d4df1 Remove SyncFileItem::_requestDuration #5456
This will save a QElapsedTimer in each job that was used only for
.owncloudsync.log (for which the request duration doesn't bring much
value).
2017-01-26 17:13:54 +01:00
Olivier Goffart
268fc97a71 Merge pull request #5480 from owncloud/chunking-ng-cleanup
Chunking ng cleanup
2017-01-25 19:19:48 +01:00
Olivier Goffart
d6fdda8efa ChunkingNG: add '0' padding on the filename (#5476)
The server sorts the chunk by name alphabetically. So if we want to keep
the chunk in order, we need to add a few '0' in front of the chunk name
2017-01-20 16:03:50 +01:00
Olivier Goffart
f6c77fad17 ChunkingNG: delete stale chunks if the file was changed locally
Relates to https://github.com/owncloud/core/issues/26981

We do not track the success or error of the DeleteJob because it does not
matter. If it fails, it might be because the chunks were already removed.
If not, the chunks will be stale, but the server must anyway do a few
cleanup from time to time because we do not always remove the chunks
2017-01-20 14:48:53 +01:00
Olivier Goffart
b046cca010 Fix more compilation errors with Qt 4
Ammand last commit which was a fixup to the change from pull request #5468
for issue #5453
2017-01-18 10:19:51 +01:00
Christian Kamm
29b39acfbe PropagatorJob: Access propagator through function #5453 2017-01-17 14:47:38 +01:00
Markus Goetz
7eb43d7f7e PropagateUploadFileNG: Delete unused map contents to free memory 2017-01-13 19:00:04 +01:00
ckamm
d76e0ec6d8 Reset stuck chunked uploads eventually #5344 (#5443)
Previously this wasn't happening for errors that were not
NormalErrors because they don't end up in the blacklist.

This revises the resetting logic to be independent of the
error blacklist and make use of UploadInfo::errorCount
instead.

412 errors should reset chunked uploads because they might be
indicative of a checksum error.

Additionally, server bugs might require that additional
errors cause an upload reset. To allow that, a new capability
is added that can be used to advise the client about this.
2017-01-13 14:44:45 +01:00
Olivier Goffart
ac95844ebd Propagator: make sure every network job has a parent
This could make sure that the network job gets deleted if the parent job gets
deleted, and would avoid crashes like:

Crash: EXCEPTION_ACCESS_VIOLATION_READ at 0xffffffff8b008a04
  File "qiodevice.cpp", line 1617, in QIODevice::errorString
  File "propagatedownload.cpp", line 264, in OCC::GETFileJob::slotReadyRead
  File "moc_propagatedownload.cpp", line 85, in OCC::GETFileJob::qt_static_metacall
  File "qobject.cpp", line 3716, in QMetaObject::activate
  File "moc_qiodevice.cpp", line 154, in QIODevice::readyRead
  File "qnetworkreplyhttpimpl.cpp", line 1045, in QNetworkReplyHttpImplPrivate::replyDownloadData

(#5329)
2016-12-14 14:13:47 +01:00
Olivier Goffart
19d2677db1 libsync: use the new webdav url if the server reports it
The rules to select the webdav url are now:

 - If the server reports that the new chunking algorithm is working,
   always use remote.php/dav/files/<username>
   This capability can be overriden with an environment variable

 - Otherwise, use the dav path provided by the theme, which defaults to
   remote.php/webdav

This means that with the newer server, the branding can no longer override
the webdav URL. If there is still an usecase for the branding to do so, we
need to find another way to override it. But it is now more complicated to
configure as might need include the username and need different endpoint
depending on the operations (chunks or not)

Issue #4007
2016-11-29 16:16:20 +01:00
Christian Kamm
045498c71a Account: Rename user() to davUser()
Avoids potential confusion with credentials->user()
2016-11-23 17:08:17 +01:00
Olivier Goffart
a1dc4069c9 libsync: Don't store the remote URI in the csync or in the SyncEngine
We are going to change the webdav path depending on the capabilities.
But the SyncEngine and csync might have been created before the capabilities
are retrieved.

The main raison why we gave the path to the sync engine was to pass it to csync.
But the thing is that csync don't need anymore this url as everything is done by the
discovery classes in libsync that use the network jobs that use the account for the urls.
So csync do not need the remote URI.

shortenFilename in folderstatusmodel.cpp was useless because the string is the
_file of a SyncFileItem which is the relative file name, that name never
starts with owncloud://.

All the csync test creates the folder because csync use to check if the folder
exists. But we don't need to do that anymore
2016-11-21 08:09:11 +01:00
Olivier Goffart
5377d1e283 Chunking-NG: code cleanup after review 2016-11-14 10:42:07 +01:00
Olivier Goffart
15f2b911d9 ChunkingNG: remove stale files when resuming 2016-11-04 16:43:01 +01:00
Olivier Goffart
c8014a0afd ChunkingNG: Add Test 2016-10-31 15:16:53 +01:00
Olivier Goffart
456d82715e Fix compile after merge 2016-10-31 11:29:33 +01:00
Olivier Goffart
273590fdfc ChunkingNG: Use the 'If' header
As discussed in https://github.com/owncloud/core/pull/26368
2016-10-20 11:16:06 +02:00
Olivier Goffart
da26e59770 Chunking-NG: add some headers when uploading chunks
These are not understood by owncloud yet, but were requested for CernBox

OC-Total-Length in the MKCOL: The full lenght of the file
OC-Chunk-Offset in the PUT: The offset within the file in which this chunk belongs
OC-Checksum in the MOVE: The transission checksum
2016-09-16 16:36:46 +02:00
Olivier Goffart
3c24d5a148 Chunking-NG: The MOVE will return the code 204 if the file was already there 2016-09-16 16:20:40 +02:00
Olivier Goffart
c222793525 Chunking-NG: use OC-If-Destination-Match instread of If-Match
For the MOVE command, because If-Match in Webdav relates to the source, not
the destination
2016-09-16 16:15:09 +02:00
Olivier Goffart
28018e8590 Chunking-NG: Fix destination URL 2016-09-16 16:14:53 +02:00
Olivier Goffart
7c75a39bc1 Chunking-NG: Some fixup after feedback from the pull request #5102 2016-09-10 12:30:14 +02:00
Olivier Goffart
c41f6ed76b Chunking-NG: use new dav URL for the move 2016-08-31 12:12:34 +02:00
Olivier Goffart
818b5854ce Chunking-NG: Qt4 compile 2016-08-31 10:28:44 +02:00
Olivier Goffart
fad387b6b8 Chunking-Ng: Resume 2016-08-31 10:28:15 +02:00
Olivier Goffart
a1558100b8 WIP: new chunking algorithm
Current limitations of this WiP
 - No resuming implemented yet
 - No parallel chunks
 - Hackish way to get the webdav paths
2016-08-31 10:28:15 +02:00