Commit graph

252 commits

Author SHA1 Message Date
Markus Goetz
61e72ef50b CSync: statedb sanity check changes 2014-10-21 18:09:23 +02:00
Markus Goetz
24d4840c93 SyncJournalDB: Set WAL files to hidden 2014-10-21 15:58:56 +02:00
Olivier Goffart
56316bc980 csync_reconcile: remove useless debug output 2014-10-21 11:47:18 +02:00
Olivier Goffart
86d8079739 Reconcile: Make sure not to loose new files added in a directory removed on the server
Found while investigating #2296

The problem is that we should not remove a directory locally if it contains
modified files.
But the modification time of the directory is not necessarily chaning (so
the instruction of the directory may still be NONE)

We have to move the child_modified test a bit down to be recursive
2014-10-21 10:38:21 +02:00
Markus Goetz
5d048c18bc Sqlite3: Switch to WAL
This is faster.
2014-10-20 23:09:28 +02:00
Markus Goetz
0fb779d363 SyncEngine: De-initialize recursive PROPFIND value correctly 2014-10-20 22:24:28 +02:00
Markus Goetz
d33b68ccc9 CSync: Error out if DB error 2014-10-20 19:32:06 +02:00
Klaas Freitag
b5736fb5a7 Sql: Handle SQLITE_BUSY properly for sqlite3_step and sqlite3_prepare.
Repeat the statements a couple of times and sleep in between.
2014-10-18 16:16:29 +02:00
Klaas Freitag
7822a6b000 csync_statedb: Finalize the precompiled statements after discovery.
As described in http://www.sqlite.org/cvstrac/wiki?p=MultiThreading precompiled
statements should not be used across thread borders. However, the reconcile
phase would reuse the statements if defined (it calls statedb function from
a different thread) so it is saver to finalize them at the end of the
update run.
2014-10-18 14:18:11 +02:00
Markus Goetz
1d5b3aadea sqlite3: Bundle 3.8.6 for Mac and Windows
it is linked into ocsync.
2014-10-16 17:38:45 +02:00
Markus Goetz
b285e98988 SQlite: Print versions we use 2014-10-11 17:33:35 +02:00
Olivier Goffart
7e898cf60c SelectiveSync: rename blacklist to selectiveSyncBlackList in some places
Issue #2301
2014-10-11 16:24:45 +02:00
Klaas Freitag
2f5cea0e73 owncloudcmd: Add a custom ssl verification callback for neon.
That fixes the SSL related problems we had with ci.owncloud.org
2014-10-11 15:55:37 +02:00
Markus Goetz
75dbf12ae6 CSync: Remove dead code 2014-10-10 11:44:40 +02:00
Olivier Goffart
661fe5df66 csync_statedb: fix inverted logic in testing if the sqlite was compiled with threads 2014-10-10 10:12:54 +02:00
Klaas Freitag
edf7cd29dd csync statedb: Set PRAGMA synchronous to NORMAL for read only db. 2014-10-09 10:05:20 +02:00
Klaas Freitag
27318dded6 csync statedb: Issue a warning if sqlite is not compiled thread safe. 2014-10-09 10:05:20 +02:00
Klaas Freitag
11fe0c5b4b csync statedb: Open the statedb in read only mode and with mutex set.
For that, sqlite3_open_v2 is used with the appropiate flags.
2014-10-09 10:05:20 +02:00
Olivier Goffart
fe1653ede6 csync_update: don't recurse into ignored directory.
This save some time when selective sync deselected a large tree
2014-09-26 12:35:56 +02:00
Markus Goetz
6d94acee8f permissions: Fix root properties if no permissions used 2014-09-25 10:24:51 +02:00
Klaas Freitag
2e12acdf32 doc: add a memo about the issue that defines the permission flags. 2014-09-24 16:35:50 +02:00
Olivier Goffart
1cd5681967 permissions: Remember the permissions for the root folder 2014-09-19 10:58:52 +02:00
Klaas Freitag
0cc2bbf5d9 csync exclude: Export the csync_exclude_add function in debug mode 2014-09-18 15:34:24 +02:00
Christian Kamm
a88ac00115 SyncEngine: Better reporting when ignoring hardlinks. #2217
Also touch csync to actually let us know it's ignoring the file
due to it having hard links.
2014-09-18 11:32:29 +02:00
Christian Kamm
0eb9401c62 File change detection: Also look at file sizes.
See #2167 and #2169.

* csync_detect_update: Reupload a file if the sizes differ, similar
  to how we reupload if the mtimes differ.
* PropagateUploadFileQNAM::slotPutFinished(): Abort an upload if the
  size changed, similar to how we abort an upload if the mtimes changed.
2014-09-17 09:05:13 +02:00
Olivier Goffart
76f0f3cbf2 csync_vio_local on windows: fix overflow when computing the size of file > 4GiB
MAXDWORD is 0xFFFFFFFF, so if we add one, it overflows and is 0.
We need to cast it to 64 bits before
2014-09-11 11:10:58 +02:00
Daniel Molkentin
2dfbc2058f Win32: ignore files ending in '.' or at least one space
Fixes #416
Fixes #2176
2014-09-10 15:21:19 +02:00
Olivier Goffart
79ee6c2f3b csync_vio_local on windows: fix overflow when computing the size of file > 4GiB
MAXDWORD is 0xFFFFFFFF, so if we add one, it overflows and is 0.
We need to cast it to 64 bits before
2014-09-10 13:21:31 +02:00
Olivier Goffart
fa962b7f53 csync_update: fix compilation warning 2014-09-10 11:08:00 +02:00
Markus Goetz
1909b465dc CSync: Show file size and inode during update traversal
With the previous code, we saw those values only when the file/dir was also in the local DB
2014-09-05 15:23:33 +02:00
Olivier Goffart
a73fbccf8c t4.pl: add a test to test that ignored file are not deleted 2014-08-28 19:33:52 +02:00
Olivier Goffart
d4bbd28757 csync: remove ignored_cleanup: it's not used anymore 2014-08-28 17:16:51 +02:00
Olivier Goffart
98b7248df0 reconcile/update: don't remove directory that contains ignored files 2014-08-28 17:16:17 +02:00
Olivier Goffart
79d13d9242 Selective sync: deselecting a folder removes it from the local file system 2014-08-28 15:53:49 +02:00
Markus Goetz
449abace66 CSync: Ignore OS X files with hidden flag 2014-08-21 16:43:24 +02:00
Klaas Freitag
dc8f437b31 Excludes: Add an context free implementation of exclude file loading.
So that we are more thread safe.
2014-08-21 12:43:04 +02:00
Olivier Goffart
936252765a Merge branch 'master' into selective_sync
Conflicts:
	src/mirall/syncengine.cpp
	src/mirall/syncengine.h
2014-08-18 14:44:43 +02:00
Klaas Freitag
ddbe181e48 Update phase progress: Check if callback is defined. 2014-08-18 11:51:45 +02:00
Markus Goetz
64a7025522 CSync: Add a 30 second connect (not read) timeout 2014-08-15 15:54:13 +02:00
Markus Goetz
ce2741cebc SyncEngine & UI: Progress notifications for update phase
For each directory (local and remote, we have UI update throtting code)
a signal is emitted.
It is used by the settings dialog and the tray menu.
2014-08-15 15:28:35 +02:00
Olivier Goffart
c1831f4946 Selective sync: use a black list instead of a white list 2014-08-15 12:29:10 +02:00
Olivier Goffart
4b2c1bacd1 Merge remote-tracking branch 'origin/master' into selective_sync 2014-08-15 10:28:50 +02:00
Klaas Freitag
b09498d852 csync journal: Improve get_below_path query.
Add another index on the pathlen column. Use that column to deselect
all rows that are shorter than the path to search files below. That
shrinks the amount of rows to examine using LIKE tremendously by
a cheaply to query for criteria.
2014-08-14 13:52:44 +02:00
Klaas Freitag
8bc0d9acd3 Updater: Added a bit of useful logging. 2014-08-14 11:07:31 +02:00
Klaas Freitag
73325bcd41 csync statedb: Fixed handling of sqlite reply value. 2014-08-14 11:07:31 +02:00
Klaas Freitag
1b5bbfdad3 csync statedb: Set sqlite3_busy_timeout to 5 seconds. 2014-08-14 11:07:31 +02:00
Olivier Goffart
4ff0e7e0a2 Silent some warnings 2014-08-12 14:03:04 +02:00
Olivier Goffart
12459bf07e selective sync: fix whitelist matching 2014-08-12 11:43:42 +02:00
Olivier Goffart
7e009667a2 Selective sync: ignore the files that are not in the selective sync white list 2014-08-11 18:42:24 +02:00
Klaas Freitag
12f5e4b46e csync oC Util: set field flag correctly to avoid etag memleak.
The fields varialbe should contain or'ed flags of the fields set
in the csync_vio_file_stat_t struct. The problem was that the field
for CSYNC_VIO_FILE_STAT_FIELDS_PERM was assigned rather than or'ed
which makes the release function for the struct not freeing the
etag memory => memleak.
2014-08-07 23:14:21 +02:00
Olivier Goffart
6827f9977c Merge branch 'il'
Conflicts:
	doc/ocdoc
2014-08-07 12:15:50 +02:00
Markus Goetz
22c1629dd3 SyncEngine: Keep csync_journal with proper values for fileId and remotePerm
Before this patch, we had a lot of empty rows because we created
the SyncFileItems with the wrong(=local) data.
2014-08-07 10:14:14 +02:00
Klaas Freitag
77e3480b2f csync oC Util: set field flag correctly to avoid etag memleak.
The fields varialbe should contain or'ed flags of the fields set
in the csync_vio_file_stat_t struct. The problem was that the field
for CSYNC_VIO_FILE_STAT_FIELDS_PERM was assigned rather than or'ed
which makes the release function for the struct not freeing the
etag memory => memleak.
2014-07-21 13:19:36 +02:00
Olivier Goffart
336e74b992 csync_owncloud: fix the name of the permissions property 2014-07-18 16:52:04 +02:00
Olivier Goffart
2f284209d8 Permissions: When moving is not allowed, fallback to delete and upload
We decided that we never want to rename a directory behind the
back of the user as the user may be using files in the directory
during the sync.
If moving is not allowed, we just erase the inode form the database so
the next sync will try to do an upload and delete and recover from there
using normal resolution.

This also add some code to update the inode back to the db when it is detected
as changed.
2014-06-27 15:30:08 +02:00
Olivier Goffart
8de3bda0b1 csync_update: update the permission in the db when they change
The current code only update the permissions in the DB when
the permission becomes non-empty.
Now we update the permission each time they change.

That way the code is the same for file id and permission so it is
simpler.
2014-06-24 11:10:50 +02:00
Olivier Goffart
9c0a21a5fb Permission: keep a space if the permission is empty
To distinguish no permission present to nothing is allowed.
That was the intention of the old code but it did not work as
the first if was always taken
2014-06-24 10:52:42 +02:00
Olivier Goffart
02704cdf74 Merge remote-tracking branch 'origin/1.6' into il
Conflicts:
	VERSION.cmake
	csync/src/csync_statedb.c
	src/mirall/syncengine.cpp
	src/mirall/syncfileitem.h
2014-06-23 12:48:34 +02:00
Klaas Freitag
041066a252 Exclude: Add a missing free in case of empty lines.
This fixes Coverity CLT 12893
2014-06-20 15:15:35 +02:00
Olivier Goffart
b29a757b18 Revert "csync file util: Remove compare file function, not needed anymore."
This break the test.
And the function is aleady gone in master anyway

This reverts commit 407b3bebfe.
2014-06-20 14:29:43 +02:00
Klaas Freitag
db3d2eed5f csync core: Remove logically dead code: Can never be reached.
This fixes Coverity CID 12881
2014-06-20 13:08:03 +02:00
Olivier Goffart
b34afa1afc reconcile: use the proper enum type
thankfully the value hapenned to be the same, but the type
is of CSYNC_FTW_..  and not CSYNC_VIO_FILE_TYPE
Detected by coverity (CID 12887)
2014-06-20 11:51:07 +02:00
Olivier Goffart
b7c9fa6d5f csync_owncloud: silent CID 12883 2014-06-20 11:51:07 +02:00
Klaas Freitag
407b3bebfe csync file util: Remove compare file function, not needed anymore.
This also fixes Coverity CID 12890 and CID 12898
2014-06-20 11:39:28 +02:00
Olivier Goffart
f04c80dd0e httpbf: silent coverity issue 12905 2014-06-20 11:33:29 +02:00
Klaas Freitag
b144a5bbf9 csync_exclude: Add a missing free of path components.
This fixes Coverity CID 12895
2014-06-20 11:14:50 +02:00
Olivier Goffart
5225fe07e0 csync_owncloud_recursive_propfind: "fix" possible memory leak
This was only leaking memory if ne_path_parent returns 0, which
should never happen

This fixes coverity issue 12897
2014-06-20 11:12:45 +02:00
Klaas Freitag
85cdbd1f1d stateDB: Close the file descriptor even if stat failed.
This fixes Coverity CID 12897
2014-06-20 11:05:40 +02:00
Klaas Freitag
e4f8a136f1 StateDB: Free locale string also if attribs are invalid.
This fixes Coverity CID 12898
2014-06-20 11:05:40 +02:00
Daniel Molkentin
c263c38cdf statedb.c: fix potential memory leak on win32
This fixes coverity issue 12898
2014-06-20 10:58:03 +02:00
Daniel Molkentin
df8553e878 httpbf.c: Fix resource leak
This fixes Coverity CID 12902
2014-06-20 10:58:03 +02:00
Daniel Molkentin
bec66c85d4 Fix potential memory leak
This fixes Coverity CID 12893
2014-06-20 10:58:03 +02:00
Daniel Molkentin
f9710cc1d5 c_time: Fix resource leak in error case
This fixes Coverity CID 12903
2014-06-20 10:58:03 +02:00
Klaas Freitag
d0c992c991 Updater: Free tmp variable that might point to temp malloced memory
This fixes Coverity CID 12900
2014-06-20 10:41:01 +02:00
Klaas Freitag
9ee86cf06b Fix resource leak in win32 code path, added free of locale filename.
This fixes Coverity CID 12901
2014-06-20 10:25:00 +02:00
Olivier Goffart
38254125c9 csync: fix the size in strncpy for the remote perms
We must only do strncpy with size one smaller than
the size of the buffer in order to leave at least
one '\0' at the end
2014-06-18 15:56:13 +02:00
Olivier Goffart
e5b3363ecf csync_statedb: remove noisy output 2014-06-18 15:40:26 +02:00
Olivier Goffart
0a953b91f9 csync_vio_local: fix memory leak on windows 2014-06-17 16:40:38 +02:00
Markus Goetz
a5967e4ecd CSync: Fix win32 compile 2014-06-16 16:53:10 +02:00
Markus Goetz
319cf76417 CSync: Move compiler ifdef outwards 2014-06-16 16:26:00 +02:00
Markus Goetz
be7b08b50a CMake: Fix token auth compile 2014-06-16 15:34:46 +02:00
Markus Goetz
61999a67cd CSync: Give ctx to proxy callback
This fixes a crash.
2014-06-13 15:36:00 +02:00
Markus Goetz
8dbfcd782b CSync: Fetch permissions from server for whole tree on server update 2014-06-06 17:10:07 +02:00
Markus Goetz
70ff928381 CSync & statedb: Parse 'perm' from server
ownCloud 6 sends this.
2014-06-06 15:24:17 +02:00
Markus Goetz
b48ab79a92 CSync & statedb: Remove uid/gid
Columns are still in the DB as we don't want to break any compatibility.
2014-06-06 13:41:16 +02:00
Markus Goetz
9a7fbd4f71 csync_owncloud: Parse properties in one function 2014-06-06 13:41:16 +02:00
Olivier Goffart
6dd248e527 csync: fix warnings
warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
2014-06-05 15:48:53 +02:00
Olivier Goffart
7d00c3646a csync: remove unused variable 2014-06-05 15:45:15 +02:00
Olivier Goffart
e355e12385 Fix crash in tests 2014-06-05 15:43:27 +02:00
Daniel Molkentin
7fcf6f9f79 Merge branch '1.6' 2014-06-05 12:01:50 +02:00
Markus Goetz
4e28ba73bb Windows: Fix build harder. 2014-06-05 10:13:20 +02:00
Markus Goetz
d4d630b2e9 Windows: Fix build 2014-06-04 12:47:17 +02:00
Markus Goetz
2f81167164 CSync: Remove more unused things 2014-06-04 11:19:09 +02:00
Markus Goetz
6897c5d41f CSync: Always have statedb enabled 2014-06-04 11:06:41 +02:00
Markus Goetz
233450d850 CSync: Remove config dir setting 2014-06-04 10:56:56 +02:00
Markus Goetz
7428a8fa63 CSync: Remove unused std functions 2014-06-04 10:33:19 +02:00
Markus Goetz
18359d7871 StateDB: Remove _csync_win32_hide_file and use other implementation 2014-06-04 09:46:31 +02:00
Olivier Goffart
6605a89990 csync_statedb: Fix wrong % code in debug output
%ld is for long, but int64_t is larger than long on windows
2014-06-03 18:28:31 +02:00
Olivier Goffart
806ab8ea46 csync: remove the local mode
It is no longer used for a long time
2014-06-03 17:52:07 +02:00
Olivier Goffart
582a8fe7fd Merge branch '1.6' 2014-06-03 17:27:12 +02:00