Commit graph

59 commits

Author SHA1 Message Date
Christian Kamm
17d174e6fa
Conflicts: Add user name to conflict file name #6325
For the case of uploading conflict files only.
2018-05-15 22:38:35 +02:00
Tomaz Canabrava
7432d6a814 Merge branch 'master' into clientSideEncryptionV4 2018-04-10 16:18:58 +02:00
Daniel Nicoletti
d93f7269b8 Properly mark encrypted files for removal 2018-04-09 13:23:52 +02:00
Daniel Nicoletti
d3b6aacf3f Fix merging remote and local trees when e2e files are involved
When populating the tree from the filesystem we need to fill
the e2eMangledName from DB and we when trying to find another
match on the oposite tree we need to take in account that names
don't match and search with the mangled name information
2018-04-09 13:23:52 +02:00
Camila San
e33d2bf7f7
Removes (s) from time strings.
Signed-off-by: Camila San <hello@camila.codes>
2018-02-25 23:55:04 +01:00
Daniel Nicoletti
99117078f3 Remove table for e2e and add an e2eMangledName column (#169)
If the code was not complex enough syncing two tables
already started to give UNIQUE constrains errors on
simple sync operations, this also adds initial support
remote delete of an encrypted file
2018-02-12 12:50:51 +01:00
Tomaz Canabrava
d24a1e542e Merge branch 'master' into clientSideEncryptionV3 2018-01-29 14:06:12 +01:00
Tomaz Canabrava
7b123650da [CSE] Search for name, not mangled name. 2018-01-28 23:48:25 +01:00
Tomaz Canabrava
0c94142c97 [CSE] More debug output 2018-01-28 23:47:58 +01:00
Tomaz Canabrava
a003d216a5 [CSE] Implementation of the setE2eRelation
This call creates a new entry on the database for e2e.
2018-01-28 18:05:28 +01:00
Tomaz Canabrava
32eaaa0601 [CSE] GetMangledName implementation 2018-01-28 17:50:10 +01:00
Tomaz Canabrava
be817748f8 [CSE] Reset the new Queries on close 2018-01-28 17:46:20 +01:00
Tomaz Canabrava
b909bb6977 [CSE] Change the Database to handle Encrypted Files
Create a table to hold the information between the
real file name and the fake file name
2018-01-28 16:41:07 +01:00
Olivier Goffart
776bbbf7b1 Don't include sqlite3.h from headers
So that sqlite is not part of the public interface of csync
(and that the sqlite include path don't need to be passed when compiling
libsync or gui)
2018-01-13 13:58:17 +01:00
Dominik Schmidt
dbf15fb8dc Make setLaunchOnStartup debug output more meaningful 2018-01-11 16:11:23 +01:00
Christian Kamm
3988f3ad77 GCC: Fix implicit-fallthrough warnings 2018-01-05 09:43:53 +01:00
Daniel Nicoletti
a63d34f870 Prepend "nextcloud" for all logging categories
Thus making easier to exclude logging from kio, qt
and only enable "nextcloud.*"
2017-12-28 17:33:10 -02:00
Christian Kamm
a7c0cfc8eb Upload conflict files #4557
If the server has the 'uploadConflictFiles' capability conflict
files will be uploaded instead of ignored.

Uploaded conflict files have the following headers set during upload
  OC-Conflict: 1
  OC-ConflictBaseFileId: 172489174instanceid
  OC-ConflictBaseMtime: 1235789213
  OC-ConflictBaseEtag: myetag
when the data is available. Downloads accept the same headers in return
when downloading a conflict file.

In the absence of server support clients will identify conflict files
through the file name pattern and attempt to deduce the base fileid.
Base etag and mtime can't be deduced though.

The upload job for a new conflict file will be triggered directly from
the job that created the conflict file now. No second sync run is
necessary anymore.

This commit does not yet introduce a 'username' like identifier that
automatically gets added to conflict file filenames (to name the files
foo_conflict-Fred-1345.txt instead of just foo_conflict-1345.txt).
2017-12-15 18:03:53 +01:00
Christian Kamm
8eebc53728 Unify item type enum
Previously, there was csync_ftw_type_e and SyncFileItem::Type. Having
two enums lead to a bug where Type::Unknown == Type::File that went
unnoticed for a good while.

This patch keeps only a single enum.
2017-12-15 18:02:04 +01:00
Olivier Goffart
4dc49ff3b0 SyncEngine: Recover when the PUT reply (or chunkin's MOVE) is lost
This can happen if the upload of a file is finished, but we just got
disconnected right before recieving the reply containing the etag.
So nothing was save din the DB, and we are not sure if the server
recieved the file properly or not. Further local update of the file
will cause a conflict.

In order to fix this, store the checksum of the uploading file in
the uploadinfo table of the local db (even if there is no chunking
involved).  And when we have a conflict, check that it is not because
of this situation by checking the entry in the uploadinfo table.

Issue #5106
2017-12-14 11:56:12 +01:00
Christian Kamm
e5ed8fc90a Merge remote-tracking branch 'origin/2.4' 2017-12-13 11:03:24 +01:00
Christian Kamm
dcf0baa9de Journal: 64bit inodes, fix storing uint64s generally
In addition to using the right function when retrieving inodes this
*also* fixes a more general bug ownsql had with storing uint64 values
that didn't fit into an int64.
2017-12-12 10:30:54 +01:00
Olivier Goffart
1a503d54f6 Remove usage of deprecated qSort 2017-12-08 16:15:47 +01:00
Olivier Goffart
263668b1cb Remove use of deprecated translate call 2017-12-08 16:15:38 +01:00
Olivier Goffart
3df65460f5 Remove usage of QString::null
Replaces by "QString()"
2017-12-08 16:15:22 +01:00
Olivier Goffart
74672d493d Utility: use QUrlQuery
For QUrl::setQuery is deprecated in Qt5
2017-12-08 16:15:17 +01:00
Olivier Goffart
ed5385faa3 Merge remote-tracking branch 'origin/2.4'
Conflicts:
	src/libsync/connectionvalidator.cpp
     	src/libsync/networkjobs.cpp
2017-12-05 15:58:13 +01:00
Markus Goetz
6276d9290d utility_mac: Check for NULL from LSSharedFileListItemResolve #6123
Possibly also #6074
2017-11-28 13:35:15 +01:00
Christian Kamm
aafe7c4732 Merge remote-tracking branch 'origin/2.4' 2017-11-20 11:54:56 +01:00
Olivier Goffart
480932a58a Checksum: Ignore unkown OC-Checksum header when downloading...
And if there are several checksums, pick the "best" one.

The case of several checksum was reported in
https://github.com/nextcloud/client_theming/issues/213
2017-11-20 10:51:15 +01:00
Christian Kamm
820ebf4b6e Sql: db close() failing isn't fatal
The assert was made fatal when looking at asserts for #5429 #5518
without having a particular problem in mind. Recent reports weakly
suggest that this might lead to occasional crashes for users when
sqlite_close fails in ways that look ignorable.
2017-11-14 12:10:19 +01:00
Christian Kamm
3608114ec1 Journal: Fix low-disk space errors
Fixes a regression from 4dbe9d4113,
low disk space can lead to IOERR not just CANTOPEN.
2017-11-03 10:44:47 +01:00
Christian Kamm
4dbe9d4113 Journal: Don't crash if the db file is readonly #6050
Surprisingly sqlite3_open_v2() returns ok on readonly files with
SQLITE_OPEN_READWRITE. Probably due to the journal mode?
2017-11-02 16:36:51 +01:00
Christian Kamm
05c1bfb6cf Merge remote-tracking branch 'origin/2.4' 2017-10-27 10:29:31 +02:00
Jocelyn Turcotte
56e38e6f80 Windows: Add sync folders to Explorer's navigation pane
This is only the navigation pane, the SyncRootManager entries aren't handled yet.

This follows the instructions from:
https://msdn.microsoft.com/en-us/library/windows/desktop/dn889934%28v=vs.85%29.aspx

Issue #5295
2017-10-24 16:10:53 +02:00
Christian Kamm
9c7ee6ef85 Reconcile: Rename handling fixes: duplicate file ids
When users share the same tree several times (say A/ and A/B/ are both
shared) the remote tree can have several entries that have the same
file id. This needs to be respected in rename detection.

Also adds several tests and fixes for issues noticed during testing.

See #6096
2017-10-24 10:54:23 +02:00
Christian Kamm
e85a339d94 Local discovery: Use db instead of filesystem
We mostly trust the file watchers meaning that we don't re-scan the
local tree if we have done that recently and no file watcher events
have arrived. If the file watchers invalidate a subtree, we rescan
only that subtree.

Since we're not entirely sure the file watchers are reliable, we still
do full local discoveries regularly (1h by default). There is a config
file setting as well as an environment variable to control the interval.
2017-10-24 10:00:20 +02:00
Olivier Goffart
a9761a8976 Use qEnvironmentVariable* instead of qgetenv when appropriate
Now that we use Qt5, we should do that. It is slightly more efficient
and declares the intent.
(Modified using clazy)
2017-10-19 13:57:49 +02:00
Markus Goetz
d339b68715 Checksums: Use addData function
Our implementation had bad error handling.
This one now uses a new Qt5 addData function.

owncloud/enterprise#2252
2017-10-17 13:26:54 +02:00
Christian Kamm
3f7b3ca962 Checksums: Improve logging and add global disable #5017 2017-10-13 13:08:20 +02:00
Jürgen Weigert
28fe903aee support old sqlite before 3.7.7
quickfix: define the symbol for backwards compat
or FIXME: better solution #ifdef SQLITE_IOERR_SHMMAP where needed...
2017-09-28 19:01:55 +02:00
Olivier Goffart
13e0cc6b50 Use QDateTime::currentDateTimeUtc instead of the non utc version
QDateTime::currentDateTime is terribly slow.
2017-09-26 16:14:17 +02:00
Olivier Goffart
278129b286 OwnSQL: don't parse the error message when we are done 2017-09-26 11:03:23 +02:00
Jocelyn Turcotte
54c522c861 SyncJournalDb: Reimplement the db_is_empty logic from csync
This reduces the initial sync local discovery time from
2.0 to 0.6 seconds in LargeSyncBench on my machine.
2017-09-26 11:03:23 +02:00
Jocelyn Turcotte
a034ee894c Use SyncJournalDb in csync
This gets rid of the csync_statedb sqlite layer and use
the same code and same connection as the rest of the SyncEngine.

Missing functions are added to SyncJournalDb and change a few minor
things (like changing SyncJournalFileRecord::_modtime to be an int64
instead of a QDateTime, like it was in csync).
2017-09-26 11:03:23 +02:00
Jocelyn Turcotte
6f46764daa SyncJournalDB: Allow callers of getFileRecord if the query failed
The current implementation would return the same value whether the query failed
or if no row would be found. This is something that is currently checked by csync
and needs to be provided if we want to use SyncJournalDB there.

Adjusted all call sites to also check the return value even though they
could still just rely on rec.isValid(), but makes it more explicit as to what
happens for database errors in those cases, if we ever want to gracefully handle
them.
2017-09-26 11:03:23 +02:00
Jocelyn Turcotte
e1d482ea7c SyncJournalDB: Bind phash as a qlonglong instead of a QString
This is what csync does and we don't need to pass it as a string
to let sqlite convert it back to a uint64 later.
2017-09-26 11:03:23 +02:00
Olivier Goffart
95d23b1914 RemotePermissions: Store in a class rather than in a QByteArray to save memory
Create a specific type that parses the permissions so we can store
it in a short rather than in a QByteArray

Note: in RemotePermissions::toString, we make sure the string is not
empty by adding a space, this was already existing before commit
e8f7adc7ca where it was removed by mistake.
2017-09-22 09:29:08 +02:00
Olivier Goffart
ff4213b59f Use the Qt5 connection syntax (automated with clazy)
This is motivated by the fact that QMetaObject::noralizeSignature takes 7.35%
CPU of the LargeSyncBench. (Mostly from ABstractNetworkJob::setupConnections and
PropagateUploadFileV1::startNextChunk). It could be fixed by using normalized
signature in the connection statement, but i tought it was a good oportunity
to modernize the code.

This commit only contains calls that were automatically converted with clazy.
2017-09-21 14:05:39 +02:00
Christian Kamm
30843cd368 Fix compile with bad strrchr signature 2017-09-19 12:44:59 +02:00