nextcloud-desktop/src/libsync
Olivier Goffart a7847a4e82
Upload: Store the size in the UploadInfo, and compare it when resolving potential conflict
This is about the conflicts that happens when the file has been uploaded
correctly to the server, but the etag was not recieved because the connection
was closed before we got the reply.

We used to compare only the mtime when comparing the uploaded file and the
existing file.  However, to be perfectly correct, we also should check the
size.

This was found because TestChunkingNG::connectionDroppedBeforeEtagRecieved is
flaky. Example of faillure found in https://drone.owncloud.com/owncloud/client/481/5
while testing PR #6626

(very trimmed log:)

06-29 07:58:02:015 [ info sync.csync.csync ]:	## Starting local discovery ##
06-29 07:58:02:016 [ info sync.csync.updater ]:	Database entry found, compare: 1530259082 <-> 1530259051, etag:  <-> 1644a8c8750, inode: 1935629 <-> 1935629, size: 301 <-> 300, perms: 0 <-> ff, type: 0 <-> 0, checksum:  <-> SHA1:cc9adedebe27a6259efb8d6ed09f4f2eff559ad1, ignore: 0
06-29 07:58:02:016 [ info sync.csync.updater ]:	file: A/a0, instruction: INSTRUCTION_EVAL <<=
06-29 07:58:02:972 [ warning sync.networkjob ]:	QNetworkReply::NetworkError(OperationCanceledError) "Connection timed out" QVariant(Invalid)
.. next sync...
06-29 07:58:02:980 [ info sync.engine ]:	#### Discovery start ####################################################
06-29 07:58:02:981 [ info sync.csync.csync ]:	## Starting local discovery ##
06-29 07:58:02:983 [ info sync.csync.updater ]:	Database entry found, compare: 1530259082 <-> 1530259051, etag:  <-> 1644a8c8750, inode: 1935629 <-> 1935629, size: 302 <-> 300, perms: 0 <-> ff, type: 0 <-> 0, checksum:  <-> SHA1:cc9adedebe27a6259efb8d6ed09f4f2eff559ad1, ignore: 0
06-29 07:58:02:983 [ info sync.csync.updater ]:	file: A/a0, instruction: INSTRUCTION_EVAL <<=
06-29 07:58:02:985 [ info sync.csync.csync ]:	## Starting remote discovery ##
06-29 07:58:02:985 [ info sync.networkjob ]:	OCC::LsColJob created for "http://localhost/owncloud" + "" "OCC::DiscoverySingleDirectoryJob"
06-29 07:58:02:987 [ info sync.csync.updater ]:	Database entry found, compare: 1530259082 <-> 1530259051, etag: 1644a8c8b26 <-> 1644a8c8750, inode: 0 <-> 1935629, size: 301 <-> 300, perms: ff <-> ff, type: 0 <-> 0, checksum: SHA1:5adcdac9608ae0811247f07f4cf1ab0a2ef99154 <-> SHA1:cc9adedebe27a6259efb8d6ed09f4f2eff559ad1, ignore: 0
06-29 07:58:02:987 [ info sync.csync.updater ]:	file: A/a0, instruction: INSTRUCTION_EVAL <<=
06-29 07:58:02:989 [ info sync.csync.csync ]:	Update detection for remote replica took 0.004 seconds walking 13 files
06-29 07:58:02:990 [ info sync.engine ]:	#### Discovery end ####################################################  9 ms
06-29 07:58:02:990 [ info sync.database ]:	Updating file record for path: "A/a0" inode: 1935629 modtime: 1530259082 type: 0 etag: "1644a8c8b26" fileId: "16383ea4" remotePerm: "WDNVCKR" fileSize: 301 checksum: "SHA1:cc9adedebe27a6259efb8d6ed09f4f2eff559ad1"
06-29 07:58:02:990 [ info sync.csync.reconciler ]:	INSTRUCTION_UPDATE_METADATA    client file: A/a0
06-29 07:58:02:990 [ info sync.csync.csync ]:	Reconciliation for local replica took  0 seconds visiting  13  files.
06-29 07:58:02:990 [ info sync.csync.reconciler ]:	INSTRUCTION_UPDATE_METADATA    server dir:  A
06-29 07:58:02:990 [ info sync.csync.csync ]:	Reconciliation for remote replica took  0 seconds visiting  13  files.
06-29 07:58:02:990 [ info sync.engine ]:	#### Reconcile end ####################################################  9 ms
06-29 07:58:02:990 [ info sync.database ]:	Updating local metadata for: "A/a0" 1530259082 302 1935629
FAIL!  : TestChunkingNG::connectionDroppedBeforeEtagRecieved(small file) '!fakeFolder.syncOnce()' returned FALSE. ()
2020-08-13 16:46:08 +02:00
..
creds Enable the modernize-use-equals-default check on clang-tidy 2020-08-13 14:19:42 +02:00
abstractnetworkjob.cpp Merge pull request #2024 from nextcloud/use_raw_string_literals 2020-05-27 05:54:54 +02:00
abstractnetworkjob.h Fix: Apply http2 patch from owncloud #1573 - only with Qt >= 5.8 2019-11-14 02:39:06 +01:00
accessmanager.cpp Fix build with older Qt: Disable http2 for now due to Qt bug, add env var (upstream) 2020-03-02 01:47:00 +01:00
accessmanager.h libsync: run clang-tidy modernize-use-nullptr 2018-11-12 18:46:39 +01:00
account.cpp Fix issue #2108: Default system proxy not working with Account Wizard 2020-06-22 11:59:13 +02:00
account.h Enable the modernize-use-equals-default check on clang-tidy 2020-08-13 14:19:42 +02:00
accountfwd.h Enable the modernize-use-using check on clang-tidy 2020-08-12 18:18:57 +02:00
bandwidthmanager.cpp Simplify nullptr comparisons where appropriate 2020-06-11 00:27:30 +02:00
bandwidthmanager.h bandwidthmanager: Fix unregistering devices on delete 2017-09-25 12:56:20 +02:00
capabilities.cpp Update our E2E API requirement 2020-07-15 18:39:29 +02:00
capabilities.h Enable the modernize-user-nullptr check on clang-tidy 2020-08-12 19:45:25 +02:00
clientproxy.cpp Fix libsync exports for proxy changes in #1886 2020-06-18 21:05:11 +02:00
clientproxy.h Fix libsync exports for proxy changes in #1886 2020-06-18 21:05:11 +02:00
clientsideencryption.cpp Add support for BASE64 encoded '|' when decrypting 2020-08-10 13:13:19 +02:00
clientsideencryption.h Move the private key salt handling in its own function 2020-08-10 13:13:19 +02:00
clientsideencryptionjobs.cpp E2EE RFC has been updated to use e2e-token instead of token 2020-07-09 16:22:54 +02:00
clientsideencryptionjobs.h Use QHash instead of QMap 2020-06-30 11:29:08 +02:00
CMakeLists.txt Move KeychainChunk class from gui to libsync 2020-07-06 21:51:39 +02:00
configfile.cpp Cleanup auto pointers and qobject casts, refactor KeychainChunk 2020-07-06 21:51:40 +02:00
configfile.h Enable the modernize-use-using check on clang-tidy 2020-08-12 18:18:57 +02:00
cookiejar.cpp Enable the modernize-loop-convert check on clang-tidy 2020-08-13 12:55:14 +00:00
cookiejar.h libsync: run clang-tidy modernize-use-nullptr 2018-11-12 18:46:39 +01:00
discoveryphase.cpp Use auto to avoiding repeating type names 2020-05-20 03:54:41 +02:00
discoveryphase.h Use default member init when applicable 2020-06-03 16:10:39 +02:00
encryptfolderjob.cpp Delete a space character 2020-07-01 14:10:49 +02:00
encryptfolderjob.h Move the encrypt folder logic in a reusable job class 2020-06-30 11:29:08 +02:00
filesystem.cpp Remove unused c_path 2018-01-13 13:58:17 +01:00
filesystem.h Move non-csync-dependent parts of FileSystem to src/common 2017-09-18 14:00:52 +02:00
libsync.md Structure developer documentation 2015-06-29 18:43:21 +02:00
logger.cpp Have a filter rule capturing all of nextcloud debug 2020-07-01 15:39:52 +02:00
logger.h [Logger] Ensure log file is flushed before crash 2020-07-01 15:13:20 +02:00
networkjobs.cpp Allow to pass extraHeaders to a path based MkColJob 2020-07-09 16:22:54 +02:00
networkjobs.h Allow to pass extraHeaders to a path based MkColJob 2020-07-09 16:22:54 +02:00
nextcloudtheme.cpp Replace "https://..." hint by "https://host:port" 2020-05-23 18:32:18 +02:00
nextcloudtheme.h Show URL placeholder hint in wizard 2020-05-23 18:32:15 +02:00
owncloudpropagator.cpp Enable the modernize-loop-convert check on clang-tidy 2020-08-13 12:55:14 +00:00
owncloudpropagator.h Enable the modernize-use-equals-default check on clang-tidy 2020-08-13 14:19:42 +02:00
owncloudpropagator_p.h Add error category for http file lock error status 423. 2019-06-11 23:32:10 +02:00
progressdispatcher.cpp Use = default for trivial ctors and dtors 2020-05-25 21:33:24 +02:00
progressdispatcher.h Use default member init when applicable 2020-06-03 16:10:39 +02:00
propagatedownload.cpp Deal properly with encrypted item updates accross connections 2020-07-01 16:58:29 +00:00
propagatedownload.h libsync: run clang-tidy modernize-use-nullptr 2018-11-12 18:46:39 +01:00
propagatedownloadencrypted.cpp Deal properly with encrypted item updates accross connections 2020-07-01 16:58:29 +00:00
propagatedownloadencrypted.h Take care of mangled paths on download as well 2020-06-30 11:29:08 +02:00
propagateremotedelete.cpp Prevent parallelism when deleting encrypted files 2020-07-09 16:22:54 +02:00
propagateremotedelete.h Prevent parallelism when deleting encrypted files 2020-07-09 16:22:54 +02:00
propagateremotedeleteencrypted.cpp Stop unlocking from within PropagateRemoteDeleteEncrypted 2020-07-09 16:22:54 +02:00
propagateremotedeleteencrypted.h Stop unlocking from within PropagateRemoteDeleteEncrypted 2020-07-09 16:22:54 +02:00
propagateremotemkdir.cpp Disable parallelism if we think the parent is e2e 2020-07-15 13:05:54 +02:00
propagateremotemkdir.h Allow to override the parallelism 2020-07-15 13:05:11 +02:00
propagateremotemove.cpp Enable the modernize-loop-convert check on clang-tidy 2020-08-13 12:55:14 +00:00
propagateremotemove.h libsync: run clang-tidy modernize-use-nullptr 2018-11-12 18:46:39 +01:00
propagateupload.cpp Pass e2e-token to PropagateUpload subjobs 2020-07-09 16:22:54 +02:00
propagateupload.h Override parallelism() for uploads 2020-07-01 17:11:57 +00:00
propagateuploadencrypted.cpp Recover from missing metadata on uploads 2020-07-09 16:22:54 +02:00
propagateuploadencrypted.h Fix parenting issues of PropagateUploadEncrypted 2020-06-30 11:29:08 +02:00
propagateuploadng.cpp Upload: Store the size in the UploadInfo, and compare it when resolving potential conflict 2020-08-13 16:46:08 +02:00
propagateuploadv1.cpp Upload: Store the size in the UploadInfo, and compare it when resolving potential conflict 2020-08-13 16:46:08 +02:00
propagatorjobs.cpp Enable the modernize-deprecated-headers check on clang-tidy 2020-08-13 14:53:52 +02:00
propagatorjobs.h Take care of mangled paths on download as well 2020-06-30 11:29:08 +02:00
syncengine.cpp Enable the modernize-loop-convert check on clang-tidy 2020-08-13 12:55:14 +00:00
syncengine.h Enable the modernize-deprecated-headers check on clang-tidy 2020-08-13 14:53:52 +02:00
syncfileitem.cpp Remove table for e2e and add an e2eMangledName column (#169) 2018-02-12 12:50:51 +01:00
syncfileitem.h Enable the modernize-use-using check on clang-tidy 2020-08-12 18:18:57 +02:00
syncfilestatus.cpp Enable the modernize-use-equals-default check on clang-tidy 2020-08-13 14:19:42 +02:00
syncfilestatus.h Use default member init when applicable 2020-06-03 16:10:39 +02:00
syncfilestatustracker.cpp Enable the modernize-loop-convert check on clang-tidy 2020-08-13 12:55:14 +00:00
syncfilestatustracker.h Enable the modernize-use-using check on clang-tidy 2020-08-12 18:18:57 +02:00
syncoptions.h Use std::chrono::milliseconds to represent milliseconds 2018-06-06 17:26:53 +02:00
syncresult.cpp Enable the modernize-use-equals-default check on clang-tidy 2020-08-13 14:19:42 +02:00
syncresult.h Use default member init when applicable 2020-06-03 16:10:39 +02:00
theme.cpp Use PNGs for branded clients 2020-07-28 09:20:34 +00:00
theme.h Let Theme tell us if we're branded or not 2020-07-28 09:20:34 +00:00
wordlist.cpp Enable the modernize-loop-convert check on clang-tidy 2020-08-13 12:55:14 +00:00
wordlist.h [cse] new method, getUnifiedString 2017-10-18 21:51:38 +02:00