Commit graph

55 commits

Author SHA1 Message Date
Matthieu Gallien
c194605c35 implement bulk upload
add PutMultiFileJob to send many files at once

use it in BulkPropagatorJob to implement bulk upload feature

Signed-off-by: Matthieu Gallien <matthieu.gallien@nextcloud.com>
2021-11-24 16:17:47 +01:00
Felix Weilbach
8a8d488454 Add dialog to set user status
Signed-off-by: Felix Weilbach <felix.weilbach@nextcloud.com>
2021-09-09 11:18:22 +02:00
Felix Weilbach
031b2e0683 Remove assert that caues crashes
This assert does not add value. Utility::concatUrlPath() takes care of
appending paths correctly.

Signed-off-by: Felix Weilbach <felix.weilbach@nextcloud.com>
2021-09-06 15:54:44 +00:00
Matthieu Gallien
22a5fadc13 remove broken assert that is not needed and break account setup
it has been also removed from the original branch so no problem removing
it

Signed-off-by: Matthieu Gallien <matthieu.gallien@nextcloud.com>
2021-08-23 14:13:58 +02:00
Hannah von Reth
acd83a2998 Ensure unit test are using absolute paths 2021-08-23 09:44:34 +02:00
Hannah von Reth
1eee5c849e Add assert to ensure we used _remotePath as base 2021-08-23 09:44:34 +02:00
Hannah von Reth
9d9eadba8e
Use time the request was send,..
not when it was processed by the client, to determine the quality of the connection.
2020-12-15 11:01:08 +01:00
Hannah von Reth
6c2c544713
Log HTTP requests and responses
Issue: #7873
2020-12-15 10:59:13 +01:00
Markus Goetz
62d876b09a
OAuth2: Better error logging
This does not fix a bug, just was found while spotting a bug that was no bug.
For https://github.com/owncloud/enterprise/issues/2951
2020-12-15 10:58:51 +01:00
Christian Kamm
a7852e3aba
Http2: Resend requests on ContentReSend error #7174
Since Qt does not yet transparently resend HTTP2 requests in some cases
we do it manually.

The test showed a problem where the initial non-200 reply would close
the target temporary file and the follow-up request couldn't store any
data. Removing that close() call is safe because there also is a
_saveBodyToFile flag that guards writes to the target file.
2020-12-15 10:58:51 +01:00
Olivier Goffart
7c1871ae65
Account: Fix crash when deleting an account
We should not have request parented to the account, otherwise we might get
a loop on the deletion order.

Issue #6893
2020-12-15 10:58:21 +01:00
Olivier Goffart
35967fc2d6
OAuth2: Refresh the token without aborting the sync
OAuth2 access token typically only has a token valid for 1 hour.
Before this patch, when the token was timing out during the sync, the
sync was aborted, and the ConnectionValidator was then requesting a new
token, so the sync can be started over.
If the discovery takes longer than the oauth2 validity, this means that
the sync can never proceed, as it would be always restarted from scratch.

With this patch, we try to transparently renew the OAuth2 token and restart
the jobs that failed because the access token was invalid.

Note that some changes were required in the GETFile job because it handled
the error itself and so it was erroring the jobs before its too late.

Issue #6814
2020-12-15 10:58:14 +01:00
Olivier Goffart
09cacc4cd4
Blacklist: remember the X-Request-ID
Issue #6420
Store the X-Request-ID in the SyncFileItem and also in the blacklist.
Note that for consistency reason, the X-Request-ID is also in the
SyncFileItem if the request succeeds.

Currently there is no UI to access it, but it can be queried with sql
commands
2020-12-15 10:57:53 +01:00
Kevin Ottens
b1dc14977f
Get TestDownload to pass
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-12-15 10:57:51 +01:00
Michael Schuster
ccc409dbd0
Merge pull request #2024 from nextcloud/use_raw_string_literals
Use raw string literals when appropriate
2020-05-27 05:54:54 +02:00
Kevin Ottens
f5350db078 Use raw string literals when appropriate
This is especially nice for Windows paths.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-05-26 20:13:51 +02:00
Kevin Ottens
86b1821251 Remove redundant initializer
This is the same value than the initializer value in the header, no need
to repeat it.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-05-25 20:12:09 +02:00
Michael Schuster
5131463644
Fix: Apply http2 patch from owncloud #1573 - only with Qt >= 5.8
Drone builds failed with Qt 5.7 and we introduce a new ifdef here
to avoid patching specifically for Ubuntu Xenial only.

Signed-off-by: Michael Schuster <michael@schuster.ms>
2019-11-14 02:39:06 +01:00
XNG
cf1532acf1 Apply http2 patch from owncloud
Signed-off-by: XNG <Milokita@users.noreply.github.com>
2019-11-01 09:35:10 +08:00
J-P Nurmi
1af9bf8abc libsync: run clang-tidy modernize-use-nullptr 2018-11-12 18:46:39 +01:00
Markus Goetz
367d0c39e8
Logging: Avoid the word "Error"
The old code printed "QNetworkReply::NoError"
2018-06-06 20:24:32 +02:00
Daniel Nicoletti
a63d34f870 Prepend "nextcloud" for all logging categories
Thus making easier to exclude logging from kio, qt
and only enable "nextcloud.*"
2017-12-28 17:33:10 -02:00
Olivier Goffart
02283fc686
Merge pull request #6224 from owncloud/deprecated
Remove use of deprecated Qt API
2017-12-12 11:48:04 +01:00
Olivier Goffart
3df65460f5 Remove usage of QString::null
Replaces by "QString()"
2017-12-08 16:15:22 +01:00
Olivier Goffart
ac844a2a45 AbstractNetworkJob: move the httpTimeout from the propagator to the network job
Remove one dependency from the config file for the sync engine.
Part of issue #6213
2017-12-07 17:39:16 +01:00
Christian Kamm
bff24ffd2f PropagateDownload: Fix GET with redirects #6159
The GET jobs were redirected, but the custom incremental handling
in readyRead didn't propagate to the follow-up job.
2017-11-15 10:11:15 +01:00
Christian Kamm
9af6e29f42 DetermineAuthType: Adjustments for tight firewalls #6135
With some firewalls we can't GET /remote.php/webdav/. Here we keep the
GET request to detect shibboleth through the redirect pattern but then
use PROPFIND to figure out the http auth method.

Currently we prefer OAuth to Shibboleth to Basic auth.

This also restores the fallback behavior of assuming basic auth
when no auth type can be determined.
2017-11-06 13:09:10 +01:00
Olivier Goffart
ff4213b59f Use the Qt5 connection syntax (automated with clazy)
This is motivated by the fact that QMetaObject::noralizeSignature takes 7.35%
CPU of the LargeSyncBench. (Mostly from ABstractNetworkJob::setupConnections and
PropagateUploadFileV1::startNextChunk). It could be fixed by using normalized
signature in the connection statement, but i tought it was a good oportunity
to modernize the code.

This commit only contains calls that were automatically converted with clazy.
2017-09-21 14:05:39 +02:00
Olivier Goffart
1218a38f2c Include <QAuthenticator> from code that connect to signal that uses it
The new connection syntax will need that
2017-09-21 14:05:39 +02:00
Christian Kamm
be5d5aea6d AbstractNetworkJob: Allow relative redirect
Previously all redirects were considered absolute.
2017-09-15 15:25:10 +02:00
Christian Kamm
e05f5fc50d OAuth: Don't use implicit POST bodies
The query args of POST requests become the request body. If there's a
redirect, the redirected url will therefore not contain the query
arguments. Use an explicit request body to make the redirection work.
2017-09-15 09:28:03 +02:00
Christian Kamm
7d075cdcb7 OAuth: Use redirectable jobs for oauth token management 2017-09-15 09:28:03 +02:00
Helmut K. C. Tessarek
709aa27031 remove qt4 code 2017-09-15 07:11:05 +02:00
Christian Kamm
09173fb727 Update server url in case of permanent redirection #5972
This is the first time the account url may update outside of
account setup.

Summary of redirection handling:
1. During account setup (wizard)
   - status.php gets permanently redirected -> adjust url
   - authed PROPFIND gets *any* redirection -> adjust url
2. During connectivity ping (ConnectionValidator)
   - status.php gets permanently redirected -> adjust url (new!)

All other redirections should be followed transparently and
don't update the account url in the settings.
2017-09-11 19:27:36 +02:00
Olivier Goffart
5100a2daf1 OAuth: cleanup debug messages
- Add category to the all messages (they did not have it was merged right after
the patch to add category everywhere, but this code did not have it.)

- Make sure there is no warnings in the normal flow. (The wizard does a request
without authentication to determine the auth type)
2017-06-13 12:08:24 +02:00
Christian Kamm
c8d0f788e0 Apply clang-format 2017-05-17 12:26:27 +02:00
Jocelyn Turcotte
7fd2f292e7 Promote a few more logs to info/warning 2017-05-11 17:22:59 +02:00
Jocelyn Turcotte
b7553d5bdf Upgrade some qCDebug to qCInfo or qCWarning
Use qCInfo for anything that has general value for support and
development. Use qCWarning for any recoverable error and qCCritical
for anything that could result in data loss or would identify a serious
issue with the code.

Issue #5647
2017-05-11 17:22:59 +02:00
Jocelyn Turcotte
4ad190a558 Use Qt logging categories for logging
This gives more insight about the logs and allow setting fine-tuned
logging rules. The categories are set to only output Info by default
so this allows us to provide more concise logging while keeping the
ability to extract more information for a specific category when
developping or debugging customer issues.

Issue #5647
2017-05-11 17:22:59 +02:00
Christian Kamm
2598579d84 Switch JsonApiJob to Qt5's QJson #5710 2017-05-08 11:50:33 +02:00
Christian Kamm
35af03b2e5 Improve http error messages; cleanup
By default QNetworkReply::errorString() often produces messages like
   "Error downloading <url> - server replied: <reason>"
but the "downloading" part invariably confuses people since the
error might very well have been produced by a PUT request.

This commit produces clearer error messages for HTTP errors.

Additionally:
* Remove some unnecessary null checks from slots connected to
  network job signals and document that these signals never send
  null replies.
* There was a bug where AbstractNetworkJob::_timedout wasn't
  set when derived classes overrode slotTimeout. We now ensure
  it's always set by disallowing overrides of slotTimeout.
  Instead it now calls onTimedOut, which allows custom handling.
* Several subclasses declared errorString, isTimedOut. Move
  these to AbstractNetworkJob.
* Unify handling of OC-ErrorString (via the new, general
  Job::errorString)
* Add documentation in various places.
2017-04-04 09:27:37 +02:00
Christian Kamm
4a1a5fa076 AbstractNetworkJob: Improve redirect handling #5555
* For requests:
  - reuse the original QNetworkRequest, so headers and attributes
    are the same as in the original request
  - determine the original http method from the reply and the request
    attributes
  - keep the original request body around such that it can be sent
    again in case the request is redirected

* Simplify the interface that is used for creating new requests in
  AbstractNetworkJob.
2017-03-07 13:18:01 +01:00
Christian Kamm
21a09df7d1 Wizard: Handle PROPFIND redirects #5553
By default, followRedirects is true for all requests, to transparently
handle redirections. In the wizard, we have special redirect-handling
code though and that was being skipped.

Setting the flag to false allows the wizard to be aware of redirects
and to handle them in the correct way. Tested with the server described
in
https://github.com/owncloud/administration/tree/master/redirectServer

There's a second bug here, where followRedirects always converts
redirected requests to the GET verb. That means redirected PROPFINDs
will never have worked. This change un-breaks them for the wizard only.
There should be no case that previously worked that stops working now.
2017-02-22 13:24:13 +01:00
Jocelyn Turcotte
9db23d4df1 Remove SyncFileItem::_requestDuration #5456
This will save a QElapsedTimer in each job that was used only for
.owncloudsync.log (for which the request duration doesn't bring much
value).
2017-01-26 17:13:54 +01:00
Daniel Molkentin
bb5c2cbfa5 Always follow redirects in network jobs (#4905)
This is a move away from the original policy where jobs
would only follow redirects in special cases.

Two restrictions are in place:

1. We do not allow protocol downgrades (https -> http)
2. We stop redirects after we find them looping (e.g. old = new url, or
indirectly when looping 10 times).

This is closer to RFC conforming behavior, although currently
we will treat 301 replies like they were 302. This is for a separate
commit.

Error handling (and display) also needs improvement.

Addresses #2791
2016-08-09 16:01:29 +02:00
Markus Goetz
d88ab4653b Connectivity: Delete job on timeout #4275
This was not done if there was no reply
2016-05-30 18:15:41 +02:00
Klaas Freitag
6b0d535120 Merge branch 'notifications' 2016-04-04 10:40:33 +02:00
Christian Kamm
4b19cdeca0 Propfind: Treat broken XML response as failure #4575
Soldiering on with a broken or incomplete response could lead to
incorrect sync behavior.

Since discovery uses LsCol jobs which already handle errors
correctly, this should not have a significant impact.
2016-03-17 11:32:19 +01:00
Klaas Freitag
0eb1041290 AbstractNetworkJob: Add a delete job.
It is needed to easily send delete requests which happen
through the notify API.
2016-03-10 17:22:36 +01:00
Markus Goetz
634dad033f Debug: Attempt to make log more useful 2016-01-05 15:12:00 +01:00