Commit graph

99 commits

Author SHA1 Message Date
Tomaz Canabrava
32eaaa0601 [CSE] GetMangledName implementation 2018-01-28 17:50:10 +01:00
Tomaz Canabrava
be817748f8 [CSE] Reset the new Queries on close 2018-01-28 17:46:20 +01:00
Tomaz Canabrava
b909bb6977 [CSE] Change the Database to handle Encrypted Files
Create a table to hold the information between the
real file name and the fake file name
2018-01-28 16:41:07 +01:00
Olivier Goffart
776bbbf7b1 Don't include sqlite3.h from headers
So that sqlite is not part of the public interface of csync
(and that the sqlite include path don't need to be passed when compiling
libsync or gui)
2018-01-13 13:58:17 +01:00
Dominik Schmidt
dbf15fb8dc Make setLaunchOnStartup debug output more meaningful 2018-01-11 16:11:23 +01:00
Christian Kamm
3988f3ad77 GCC: Fix implicit-fallthrough warnings 2018-01-05 09:43:53 +01:00
Daniel Nicoletti
a63d34f870 Prepend "nextcloud" for all logging categories
Thus making easier to exclude logging from kio, qt
and only enable "nextcloud.*"
2017-12-28 17:33:10 -02:00
Christian Kamm
a7c0cfc8eb Upload conflict files #4557
If the server has the 'uploadConflictFiles' capability conflict
files will be uploaded instead of ignored.

Uploaded conflict files have the following headers set during upload
  OC-Conflict: 1
  OC-ConflictBaseFileId: 172489174instanceid
  OC-ConflictBaseMtime: 1235789213
  OC-ConflictBaseEtag: myetag
when the data is available. Downloads accept the same headers in return
when downloading a conflict file.

In the absence of server support clients will identify conflict files
through the file name pattern and attempt to deduce the base fileid.
Base etag and mtime can't be deduced though.

The upload job for a new conflict file will be triggered directly from
the job that created the conflict file now. No second sync run is
necessary anymore.

This commit does not yet introduce a 'username' like identifier that
automatically gets added to conflict file filenames (to name the files
foo_conflict-Fred-1345.txt instead of just foo_conflict-1345.txt).
2017-12-15 18:03:53 +01:00
Christian Kamm
8eebc53728 Unify item type enum
Previously, there was csync_ftw_type_e and SyncFileItem::Type. Having
two enums lead to a bug where Type::Unknown == Type::File that went
unnoticed for a good while.

This patch keeps only a single enum.
2017-12-15 18:02:04 +01:00
Olivier Goffart
4dc49ff3b0 SyncEngine: Recover when the PUT reply (or chunkin's MOVE) is lost
This can happen if the upload of a file is finished, but we just got
disconnected right before recieving the reply containing the etag.
So nothing was save din the DB, and we are not sure if the server
recieved the file properly or not. Further local update of the file
will cause a conflict.

In order to fix this, store the checksum of the uploading file in
the uploadinfo table of the local db (even if there is no chunking
involved).  And when we have a conflict, check that it is not because
of this situation by checking the entry in the uploadinfo table.

Issue #5106
2017-12-14 11:56:12 +01:00
Christian Kamm
e5ed8fc90a Merge remote-tracking branch 'origin/2.4' 2017-12-13 11:03:24 +01:00
Christian Kamm
dcf0baa9de Journal: 64bit inodes, fix storing uint64s generally
In addition to using the right function when retrieving inodes this
*also* fixes a more general bug ownsql had with storing uint64 values
that didn't fit into an int64.
2017-12-12 10:30:54 +01:00
Olivier Goffart
1a503d54f6 Remove usage of deprecated qSort 2017-12-08 16:15:47 +01:00
Olivier Goffart
263668b1cb Remove use of deprecated translate call 2017-12-08 16:15:38 +01:00
Olivier Goffart
3df65460f5 Remove usage of QString::null
Replaces by "QString()"
2017-12-08 16:15:22 +01:00
Olivier Goffart
74672d493d Utility: use QUrlQuery
For QUrl::setQuery is deprecated in Qt5
2017-12-08 16:15:17 +01:00
Olivier Goffart
ed5385faa3 Merge remote-tracking branch 'origin/2.4'
Conflicts:
	src/libsync/connectionvalidator.cpp
     	src/libsync/networkjobs.cpp
2017-12-05 15:58:13 +01:00
Markus Goetz
6276d9290d utility_mac: Check for NULL from LSSharedFileListItemResolve #6123
Possibly also #6074
2017-11-28 13:35:15 +01:00
Christian Kamm
aafe7c4732 Merge remote-tracking branch 'origin/2.4' 2017-11-20 11:54:56 +01:00
Olivier Goffart
480932a58a Checksum: Ignore unkown OC-Checksum header when downloading...
And if there are several checksums, pick the "best" one.

The case of several checksum was reported in
https://github.com/nextcloud/client_theming/issues/213
2017-11-20 10:51:15 +01:00
Christian Kamm
820ebf4b6e Sql: db close() failing isn't fatal
The assert was made fatal when looking at asserts for #5429 #5518
without having a particular problem in mind. Recent reports weakly
suggest that this might lead to occasional crashes for users when
sqlite_close fails in ways that look ignorable.
2017-11-14 12:10:19 +01:00
Christian Kamm
3608114ec1 Journal: Fix low-disk space errors
Fixes a regression from 4dbe9d4113,
low disk space can lead to IOERR not just CANTOPEN.
2017-11-03 10:44:47 +01:00
Christian Kamm
4dbe9d4113 Journal: Don't crash if the db file is readonly #6050
Surprisingly sqlite3_open_v2() returns ok on readonly files with
SQLITE_OPEN_READWRITE. Probably due to the journal mode?
2017-11-02 16:36:51 +01:00
Christian Kamm
05c1bfb6cf Merge remote-tracking branch 'origin/2.4' 2017-10-27 10:29:31 +02:00
Jocelyn Turcotte
56e38e6f80 Windows: Add sync folders to Explorer's navigation pane
This is only the navigation pane, the SyncRootManager entries aren't handled yet.

This follows the instructions from:
https://msdn.microsoft.com/en-us/library/windows/desktop/dn889934%28v=vs.85%29.aspx

Issue #5295
2017-10-24 16:10:53 +02:00
Christian Kamm
9c7ee6ef85 Reconcile: Rename handling fixes: duplicate file ids
When users share the same tree several times (say A/ and A/B/ are both
shared) the remote tree can have several entries that have the same
file id. This needs to be respected in rename detection.

Also adds several tests and fixes for issues noticed during testing.

See #6096
2017-10-24 10:54:23 +02:00
Christian Kamm
e85a339d94 Local discovery: Use db instead of filesystem
We mostly trust the file watchers meaning that we don't re-scan the
local tree if we have done that recently and no file watcher events
have arrived. If the file watchers invalidate a subtree, we rescan
only that subtree.

Since we're not entirely sure the file watchers are reliable, we still
do full local discoveries regularly (1h by default). There is a config
file setting as well as an environment variable to control the interval.
2017-10-24 10:00:20 +02:00
Olivier Goffart
a9761a8976 Use qEnvironmentVariable* instead of qgetenv when appropriate
Now that we use Qt5, we should do that. It is slightly more efficient
and declares the intent.
(Modified using clazy)
2017-10-19 13:57:49 +02:00
Markus Goetz
d339b68715 Checksums: Use addData function
Our implementation had bad error handling.
This one now uses a new Qt5 addData function.

owncloud/enterprise#2252
2017-10-17 13:26:54 +02:00
Christian Kamm
3f7b3ca962 Checksums: Improve logging and add global disable #5017 2017-10-13 13:08:20 +02:00
Jürgen Weigert
28fe903aee support old sqlite before 3.7.7
quickfix: define the symbol for backwards compat
or FIXME: better solution #ifdef SQLITE_IOERR_SHMMAP where needed...
2017-09-28 19:01:55 +02:00
Olivier Goffart
13e0cc6b50 Use QDateTime::currentDateTimeUtc instead of the non utc version
QDateTime::currentDateTime is terribly slow.
2017-09-26 16:14:17 +02:00
Olivier Goffart
278129b286 OwnSQL: don't parse the error message when we are done 2017-09-26 11:03:23 +02:00
Jocelyn Turcotte
54c522c861 SyncJournalDb: Reimplement the db_is_empty logic from csync
This reduces the initial sync local discovery time from
2.0 to 0.6 seconds in LargeSyncBench on my machine.
2017-09-26 11:03:23 +02:00
Jocelyn Turcotte
a034ee894c Use SyncJournalDb in csync
This gets rid of the csync_statedb sqlite layer and use
the same code and same connection as the rest of the SyncEngine.

Missing functions are added to SyncJournalDb and change a few minor
things (like changing SyncJournalFileRecord::_modtime to be an int64
instead of a QDateTime, like it was in csync).
2017-09-26 11:03:23 +02:00
Jocelyn Turcotte
6f46764daa SyncJournalDB: Allow callers of getFileRecord if the query failed
The current implementation would return the same value whether the query failed
or if no row would be found. This is something that is currently checked by csync
and needs to be provided if we want to use SyncJournalDB there.

Adjusted all call sites to also check the return value even though they
could still just rely on rec.isValid(), but makes it more explicit as to what
happens for database errors in those cases, if we ever want to gracefully handle
them.
2017-09-26 11:03:23 +02:00
Jocelyn Turcotte
e1d482ea7c SyncJournalDB: Bind phash as a qlonglong instead of a QString
This is what csync does and we don't need to pass it as a string
to let sqlite convert it back to a uint64 later.
2017-09-26 11:03:23 +02:00
Olivier Goffart
95d23b1914 RemotePermissions: Store in a class rather than in a QByteArray to save memory
Create a specific type that parses the permissions so we can store
it in a short rather than in a QByteArray

Note: in RemotePermissions::toString, we make sure the string is not
empty by adding a space, this was already existing before commit
e8f7adc7ca where it was removed by mistake.
2017-09-22 09:29:08 +02:00
Olivier Goffart
ff4213b59f Use the Qt5 connection syntax (automated with clazy)
This is motivated by the fact that QMetaObject::noralizeSignature takes 7.35%
CPU of the LargeSyncBench. (Mostly from ABstractNetworkJob::setupConnections and
PropagateUploadFileV1::startNextChunk). It could be fixed by using normalized
signature in the connection statement, but i tought it was a good oportunity
to modernize the code.

This commit only contains calls that were automatically converted with clazy.
2017-09-21 14:05:39 +02:00
Christian Kamm
30843cd368 Fix compile with bad strrchr signature 2017-09-19 12:44:59 +02:00
Christian Kamm
53195b064a Conflicts: Upload them files if env variable says so
Set OWNCLOUD_UPLOAD_CONFLICT_FILES=1 to trigger this behavior.

Note that this is experimental and unsupported. The real feature is
likely to end up in 2.5.

Uploading conflict files is simply done by removing the pattern from
csync_exclude. The rest here deals with making the conflict notification
ui approximately work.

There are still some concerns about where an uploaded conflict file
appears in the sync protocol and issues list (it should be in both, but
is only in one of them currently!).

See #4557.
2017-09-19 11:45:38 +02:00
Christian Kamm
dca2664707 Private links: Retrieve link through propfind property #6020
* The sharing ui does a propfind anyway: use that to query the new
property as well!
* For the socket api, asynchronously query the server for the right url
when an action that needs it is triggered.

The old, manually generated URL will be used as fallback in case the
server doesn't support the new property or the property can't be
retrieved for some reason.

Depends on owncloud/core#29021
2017-09-19 10:51:03 +02:00
Jocelyn Turcotte
5e332983e2 SyncJournalDB: Only reset_and_clear_bindings once
We only need to do it once between each query execution and
that can be right before the first bind.
2017-09-18 14:00:52 +02:00
Jocelyn Turcotte
a1f1775d15 Move SyncJournalDB to src/common 2017-09-18 14:00:52 +02:00
Jocelyn Turcotte
bd25225045 Move non-csync-dependent parts of FileSystem to src/common
This keep the csync-dependent parts in src/libsync, slitting the namespace
over two files.
This will allow moving SyncJournalDB to src/common as well.
2017-09-18 14:00:52 +02:00
Helmut K. C. Tessarek
709aa27031 remove qt4 code 2017-09-15 07:11:05 +02:00
Jocelyn Turcotte
2d7a4bdb06 Windows: Make sure that the folder and Desktop.ini have the right attributes (#6018)
When created from scratch, the owncloud icon won't be picked-up by
explorer unless the right attributes are set on both the sync root and the
Desktop.ini file.

Issue #2446
2017-09-11 15:32:42 +02:00
Jocelyn Turcotte
e8f7adc7ca Replace csync_vio_file_stat_t with csync_file_stat_t
Also move csync_normalize_etag to common/utility since we
don't need the char* function anymore.

Remove the single space file_stat->remotePerm codepath since
this won't be used in csync anymore since
8de3bda0b1.

Issue #1817
2017-09-06 13:08:23 +02:00
Jocelyn Turcotte
cf15cbf0b3 Move Utility to a new common static library
Now that csync builds as C++, this will avoid having to implement
functionalities needed by csync mandatorily in csync itself.

This library is built as part of libocsync and symbols exported
through it.
This requires a relicense of Utility as LGPL. All classes moved into
this library from src/libsync will need to be relicensed as well.
2017-09-05 17:25:19 +02:00